B2G/QA/Automation/UI/Strategy/Integration vs End to end: Difference between revisions

Separate concerns for Continuous Integration (CI) vs. User Acceptance Testing (UAT) so that test processes and automation harnesses can optimize for their specific purposes and minimize risk and dependency.

* Development and QA have different approaches and needs from UI testing  

* Team has been blocked too much by cross-team dependencies  

To date, UI automation has been treated as a single type of testing, with the differences expressed largely as to whether it was running on TBPL or not, was written in JS or Python, who sheriffed, and so forth. And so much discussion, debate, and analysis paralysis has involved whether the automation should be run, written, or treated a particular way. It has been all too common to reach an impasse based on differing opinions.

However, these are weaknesses in perception. When used for their own specific purposes, each type of automation provides great value. Further, the compromises each makes are compensated for by the other.

The solution is to separate the concerns. And since each type has a different primary stakeholder, ownership should be separated as well.

* Continuous Integration

** Primarily runs on-device to test full stack

The main differences are:

* It is more important that UATs be complete than solid. All acceptance criteria must be tested to accept a build.

Ownership is separated because these differences can lead to variant needs in terms of breadth and depth of verification.  

Via skillful code reuse, each suite can be treated as an independent target without increasing maintenance unacceptably.

The time is now.  

QA has defined and will own UATs as an aid to its acceptance mandate, development will continue to own CI as an aid to its own processes.

None.