B2G/QA/Automation/UI/Best Practices - Revision history

Gmealer: /* Tests that check functional behavior */

2015-07-02T23:15:49Z

Tests that check functional behavior

← Older revision		Revision as of 23:15, 2 July 2015
Line 49:		Line 49:
	* Should not be concerned with the UI or API details of how they perform their actions or verify results.		* Should not be concerned with the UI or API details of how they perform their actions or verify results.
	* Should not call or verify UI controls or back-end APIs directly in test code.		* Should not call or verify UI controls or back-end APIs directly in test code.
	* Should ~~use a functional abstraction model like .~~		* Should encapsulate UI actions and checks in [http://martinfowler.com/bliki/PageObject.html Page (aka App or Region) Objects]. The Page Object should be the only place UI is used directly.
	* Encapsulate UI actions and checks in [http://martinfowler.com/bliki/PageObject.html Page (aka App or Region) Objects]. The Page Object should be the only place UI is used directly.
	* Encapsulate API details in objects or reusable functions as well.		* Encapsulate API details in objects or reusable functions as well.
	* Should favor verifications round trip through UI over verifications of back end APIs. They should look for what the user would look for.		* Should favor verifications round trip through UI over verifications of back end APIs. They should look for what the user would look for.

Gmealer: /* Programming Best Practices */

2015-03-23T17:31:12Z

Programming Best Practices

← Older revision		Revision as of 17:31, 23 March 2015
Line 90:		Line 90:
	* Do share ownership of anything that is not a test detail, including (and especially) almost any particular task done in setup and cleanup. Know your level of [[#Abstraction\|abstraction]].		* Do share ownership of anything that is not a test detail, including (and especially) almost any particular task done in setup and cleanup. Know your level of [[#Abstraction\|abstraction]].
	* Sometimes setup/cleanup needs a function and the test needs a macro that do the same thing. Sharing code for these is risky. It's usually better to let setup/cleanup use a highly robust function, preferably without involving UI, that you can change at will to make faster or more robust later without risking test coverage.		* Sometimes setup/cleanup needs a function and the test needs a macro that do the same thing. Sharing code for these is risky. It's usually better to let setup/cleanup use a highly robust function, preferably without involving UI, that you can change at will to make faster or more robust later without risking test coverage.
	* Balance SPOT with your organizational structure. If it's not natural for two groups to communicate, those groups should not have shared ownership of code. ~~Spin~~ shared code into a module ~~owned~~ by a ~~single group~~ or ~~copy~~. SPOT only helps when you all talk. Keep in mind more than a few people ~~talking~~ to make a given change is an [http://en.wikipedia.org/wiki/The_Mythical_Man-Month#The_mythical_man-month antipattern].		* Balance SPOT with your organizational structure. If it's not natural for two groups to communicate, those groups should not have shared ownership of code. Either refactor shared code into a module with single ownership and consumed by the others as a version-pinned library, or both groups should maintain their own copies. SPOT only helps when you all talk. Keep in mind more than a few people needing to talk to make a given change is an [http://en.wikipedia.org/wiki/The_Mythical_Man-Month#The_mythical_man-month antipattern].

	= Test Runs =		= Test Runs =

Gmealer: /* Tests that check functional behavior */

2015-03-23T17:20:28Z

Tests that check functional behavior

← Older revision		Revision as of 17:20, 23 March 2015
Line 49:		Line 49:
	* Should not be concerned with the UI or API details of how they perform their actions or verify results.		* Should not be concerned with the UI or API details of how they perform their actions or verify results.
	* Should not call or verify UI controls or back-end APIs directly in test code.		* Should not call or verify UI controls or back-end APIs directly in test code.
	* Should use a functional abstraction model like [http://martinfowler.com/bliki/PageObject.html Page ~~Object Model~~].		* Should use a functional abstraction model like .
	* Encapsulate UI actions and checks in Page ~~Objects. The model~~ should be the only place UI is used directly.		* Encapsulate UI actions and checks in [http://martinfowler.com/bliki/PageObject.html Page (aka App or Region) Objects]. The Page Object should be the only place UI is used directly.
	* Encapsulate API details in ~~object~~ as well.		* Encapsulate API details in objects or reusable functions as well.
	* Should favor verifications through UI over verifications of back end APIs.		* Should favor verifications round trip through UI over verifications of back end APIs. They should look for what the user would look for.
	* Will consist of a string of functions at high-level, such as calendar.createEntry(...) and calendar.getEntry() == defaultEntry.		* Will consist of a string of functions at high-level, such as calendar.createEntry(...) and calendar.getEntry() == defaultEntry.
	* Will be longer and more complex, so need to be at a higher level of abstraction to promote reuse and keep clarity.		* Will be longer and more complex, so need to be at a higher level of abstraction to promote reuse and keep clarity.

Gmealer: /* Logic */

2015-03-23T17:11:47Z

Logic

← Older revision		Revision as of 17:11, 23 March 2015
Line 68:		Line 68:
	= Logic =		= Logic =

	* Generally, minimize logic in tests. Tests do one thing and should require branching.		* Generally, minimize logic in tests. Tests do one thing and should rarely require branching on decisions.
	* However, there are exceptions. A much larger discussion on this can be found [https://github.com/geoelectric/ui_auto_best_practice/blob/master/uiabp.md here].		* However, there are exceptions. A much larger discussion on this can be found [https://github.com/geoelectric/ui_auto_best_practice/blob/master/uiabp.md here].

Gmealer: /* Tests that check UI mechanics */

2015-03-23T17:10:21Z

Tests that check UI mechanics

← Older revision		Revision as of 17:10, 23 March 2015
Line 28:		Line 28:
	* Should therefore be kept very short and isolated, with no extended sequences. Ideally, one changed assumption = one broken test.		* Should therefore be kept very short and isolated, with no extended sequences. Ideally, one changed assumption = one broken test.
	* Should still abstract selectors as these change frequently and aren't directly important to the test.		* Should still abstract selectors as these change frequently and aren't directly important to the test.
	* Will not generally correspond with a traditionally-scripted QA test. They'll be much shorter.		* Will not generally correspond with a traditionally-scripted QA test. They'll be much shorter and simpler, more like a single test step.
	* Should have almost no logic whatsoever.		* Should have almost no logic whatsoever.

Gmealer: /* Tests that check UI mechanics */

2015-03-23T17:09:55Z

Tests that check UI mechanics

← Older revision		Revision as of 17:09, 23 March 2015
Line 25:		Line 25:
	* Will consist of a string of UI calls such as calendar.newEntryButton.tap() and calendar.summary.text == ...		* Will consist of a string of UI calls such as calendar.newEntryButton.tap() and calendar.summary.text == ...
	* Should not delegate strings of UI calls used in a test to functional-style Page Objects or similar high level abstraction mechanisms. They own that sequence.		* Should not delegate strings of UI calls used in a test to functional-style Page Objects or similar high level abstraction mechanisms. They own that sequence.
	* Should, however, abstract selectors as these change frequently.
	* Will be very fragile, as any change in UI assumption will require changes to all tests making that assumption.		* Will be very fragile, as any change in UI assumption will require changes to all tests making that assumption.
	* Should therefore be kept very short and isolated, with no extended sequences. Ideally, one changed assumption = one broken test.		* Should therefore be kept very short and isolated, with no extended sequences. Ideally, one changed assumption = one broken test.
			* Should still abstract selectors as these change frequently and aren't directly important to the test.
	* Will not generally correspond with a traditionally-scripted QA test. They'll be much shorter.		* Will not generally correspond with a traditionally-scripted QA test. They'll be much shorter.
	* Should have almost no logic whatsoever.		* Should have almost no logic whatsoever.

Gmealer: /* Tests that check UI mechanics */

2015-03-23T17:09:08Z

Tests that check UI mechanics

← Older revision		Revision as of 17:09, 23 March 2015
Line 26:		Line 26:
	* Should not delegate strings of UI calls used in a test to functional-style Page Objects or similar high level abstraction mechanisms. They own that sequence.		* Should not delegate strings of UI calls used in a test to functional-style Page Objects or similar high level abstraction mechanisms. They own that sequence.
	* Should, however, abstract selectors as these change frequently.		* Should, however, abstract selectors as these change frequently.
	* Will be very fragile, as any change in UI assumption will ~~break~~ all tests making that assumption.		* Will be very fragile, as any change in UI assumption will require changes to all tests making that assumption.
	* Should therefore be kept very short and isolated, with no extended sequences. Ideally, one changed assumption = one broken test.		* Should therefore be kept very short and isolated, with no extended sequences. Ideally, one changed assumption = one broken test.
	* Will not generally correspond with a traditionally-scripted QA test. They'll be much shorter.		* Will not generally correspond with a traditionally-scripted QA test. They'll be much shorter.

Gmealer: /* Tests that check functional behavior */

2015-03-23T17:08:35Z

Tests that check functional behavior

← Older revision		Revision as of 17:08, 23 March 2015
Line 50:		Line 50:
	* Should not call or verify UI controls or back-end APIs directly in test code.		* Should not call or verify UI controls or back-end APIs directly in test code.
	* Should use a functional abstraction model like [http://martinfowler.com/bliki/PageObject.html Page Object Model].		* Should use a functional abstraction model like [http://martinfowler.com/bliki/PageObject.html Page Object Model].
	* Encapsulate UI actions and checks in Page Objects. The model is the only place UI is used directly.		* Encapsulate UI actions and checks in Page Objects. The model should be the only place UI is used directly.
	* Should favor verifications through UI over verifications of ~~the~~ back end.		* Encapsulate API details in object as well.
			* Should favor verifications through UI over verifications of back end APIs.
	* Will consist of a string of functions at high-level, such as calendar.createEntry(...) and calendar.getEntry() == defaultEntry.		* Will consist of a string of functions at high-level, such as calendar.createEntry(...) and calendar.getEntry() == defaultEntry.
	* Will be longer and more complex, so need to be at a higher level of abstraction to promote reuse and keep clarity.		* Will be longer and more complex, so need to be at a higher level of abstraction to promote reuse and keep clarity.

Gmealer: /* Synchronization */

2015-03-23T17:06:52Z

Synchronization

← Older revision		Revision as of 17:06, 23 March 2015
Line 9:		Line 9:

	* Waits should almost always be conditional, using Marionette [http://marionette-client.readthedocs.org/en/latest/reference.html#wait Wait().until()] constructs.		* Waits should almost always be conditional, using Marionette [http://marionette-client.readthedocs.org/en/latest/reference.html#wait Wait().until()] constructs.
	* Conversely, any sort of hard sleep() type construct is discouraged. These cause tests to have a longer best-case execution time.		* Conversely, any sort of hard sleep() type construct is discouraged. These cause tests to have a longer best-case execution time, instead of adapting their runtime to the best possible allowed by the AUT.

	= Abstraction =		= Abstraction =

Gmealer: /* Synchronization */

2015-03-23T17:06:18Z

Synchronization

← Older revision		Revision as of 17:06, 23 March 2015
Line 8:		Line 8:
	= Synchronization =		= Synchronization =

	* Waits should almost always be conditional, using [http://marionette-client.readthedocs.org/en/latest/reference.html#wait ~~Marionette~~ Wait().until() constructs].		* Waits should almost always be conditional, using Marionette [http://marionette-client.readthedocs.org/en/latest/reference.html#wait Wait().until()] constructs.
	* Conversely, any sort of hard sleep() type construct is discouraged. These cause tests to have a longer best-case execution time.		* Conversely, any sort of hard sleep() type construct is discouraged. These cause tests to have a longer best-case execution time.