B2G/QA/Automation/Style Guide/Best Practices

Use External Parameters

Using test variable file when running gaiatest can avoid defining variables inside the script. The testvars template is located here.
Make sure to fill in the appropriate section if you're planning to use it, and supply the name and location of the .json file as the parameter to the gaiatest command with --testvars= option.

If you want to access the varable value defined in the .json file, you can do as the following example:

test_phone_number = self.testvars['remote_phone_number']

If you need to access the sub-variable, consider below example as well:

self.testvars['plivo']['auth_id'],
self.testvars['plivo']['auth_token'],
self.testvars['plivo']['phone_number']

Make sure that you are not including your testvars.json file in your PR request, as the testvars file for jenkins is managed separately.

Use sleep() calls only if you have no other choice

There are several ways to wait for something to happen:

# Good
## Wait for an element to be displayed
Wait(self.marionette).until(expected.element_displayed(By, locator))

## Wait for an element to disappear
Wait(self.marionette).until(expected.element_not_displayed(By, locator))

## Wait for a transition to be over
Wait(self.marionette).until(lambda m: element.rect['y'] == expected_y_position)

# Bad
import time
time.sleep(seconds)

sleep() calls should be used only when there is no other way to delay the action of Marionette. Using sleep() instead of Wait() is bad for following reasons:

• sleep() does not care about the UI changes in app. If you're using sleep() to just 'wait enough', you'll run into problems when the app behavior changes and requires more/less time to wait.
• sleep() does not care about the phone performance. If the speed of the execution changes because of the changes in memory allocation or running on a newer/older devices, it will still wait for specified time.

If have to use the sleep() call, make sure to put in the comment explaining why other methods won't work.

Limit the Use of Conditionals

• Methods should not contain logic that depends on properties of the page. The logic and expectations should be within the test, and adding this to the page object could guard your tests against genuine failures.

# Good
def click_login(self)
    self.selenium.find_element(*self._login_locator).click()

# Bad
def click_login(self)
    if not self.is_user_logged_in:
        self.selenium.find_element(*self._login_locator).click()
    else:
        pass

PageRegions

In some circumstances, for example where a header or a list element is common across the app, we will use a page region. The page region is a child class of the base Base object, which is inherited by all page objects. This means that the navigation can be reached from any page object and herein lies the DRY!

A brief example:

class MyAppBase():

    @property
    def header(self):
        return MyApp.HeaderRegion(self.marionette)
    
    class HeaderRegion(PageRegion):

        _login_link = (By.ID, "home")

        def tap_login(self):
            self.marionette.find_element(*self._login_link).tap()

Referring to this page region with a property makes it very readable and concise from within the test. Clicking login during a test would be performed like this:

my_page.header.tap_login()

Another example where this might be used is on a search results page, the page region being the search results element.

One possible issue of using PageRegion object is that changing frames after instantiating PageRegion object might make the root_element stale. This will cause an error when trying methods that needs access to self.root_element. The workaround in such case would be to refresh the root_element again, by locating it again with the locator.

The Page Object Pattern

TBD - jlorenzo

• where to call marionette's APIs
• why a page class shouldn't import data from the test
• All page objects should inherit from Page in page.py.
• Page objects should not do asserts. This should be done within the test.
• Each page should be grouped within one module.
• If using multiple words to describe a module separate them with underscores '_'
• Single quotes (') should be used instead of double (") throughout.
• Methods should have a single purpose.

Assertions

• Tests should handle the asserts -- not the page objects.
• Tests should use Python's native assert statement.
• When doing equivalency assertions, put the expected value first, followed by the actual value, for example:

# Good
a = some_function()
assert 'expected result' == a

# Bad
a = some_function()
assert a == 'expected result'

How to Use GaiaHeader

GaiaHeader class is mainly used to tap the back button on the gaia header of the app page. This helps to standardize the method to return to the previous page on most of the app page.

  # ID of the gaia header tag
  _header_locator = (By.ID, 'header') 
  
  # specify the locator, and in this case, going back actually exits the app 
  # if it is not a fullscreen app (status bar is visible), then statusbar=True
  def tap_back_button(self):
        GaiaHeader(self.marionette, self._header_locator).go_back(exit_app=True, app=self)

How to use GaiaBinaryControl

GaiaBinaryControl is a derived class of BinaryControl class, which can be used to control the gaia-switch UI.
In some rare cases where the switch UI is using the older <switch> tag, one has to use the HtmlBinaryControl class, but most of the switches were recently converted to gaia-switch.

        _cell_broadcast_switch_locator = (By.CSS_SELECTOR, '#menuItem-cellBroadcast gaia-switch')
        _emerg_alert_switch_locator = (By.CSS_SELECTOR, '#menuItem-emergencyAlert gaia-switch')

        cell_broadcast_switch = GaiaBinaryControl(self.marionette, self._cell_broadcast_switch_locator)
        if cell_broadcast_switch.is_checked:
            emerg_alert_switch = GaiaBinaryControl(self.marionette, self._emerg_alert_switch_locator)
            Wait(self.marionette).until(lambda m: emerg_alert_switch.is_checked)

app.py and regions/helper.py

In gaiatest/apps folder, each subfolder contains the helper python method files. The rule of thumb is as follows:

• app.py: Contains the class for the main screen when the app is instantiated. There should be a method that instantiates the subpage objects.
• regions/*.py: Contains the classes for the each subpage of the app.

Meaningful Custom Assert() Messages

Put a custom error message in Assert() call only when it provides more information than the one given by default. `self.asserEqual()` by itself is a nice assertion method: if it fails, it'll give you the expected value and the actual.

For example,

self.assertEqual(displayed_phone_number, expected_phone_number)

will fail with:

 AssertionError: u'+34...' != u'+33...'
 Stacktrace:

But if you provide an error message as follows,

 self.assertEqual(displayed_phone_number, expected_phone_number, msg='Phone numbers are not the same')

then you'll have something like:

 AssertionError: Phone numbers are not the same
 Stacktrace:

In other words, by adding more context, you might actually remove some useful debug data.

Simulate End-user Check

• Check what an end-user would check (i.e., presence of dialogs, texts, icons). There is little need to check the value of internal state inaccessible to the user

Clean Up Afterwards

If your script have changed the data settings or other settings that are not reverted by resetting B2G, it is recommended to revert your setting on the teardown() method. tearDown() method gets invoked at the end of the script execution, regardless of the test result.

    def tearDown(self):
        self.marionette.switch_to_frame()

        # turn off the cell data that was enabled during the test
        self.data_layer.disable_cell_data()
       
        # don't forget call the super method as well
        GaiaTestCase.tearDown(self)

Update Manifest File

After the script is done, make sure the corresponding manifest.ini file is updated with the right flags.

• If the test should not run on a particular device, use skip-if (and provide explanation)

[test_browser_bookmark.py]
# Bug 1178859 - test_browser_bookmark.py: "IOError: Connection to Marionette server is lost."
skip-if = device == "flame"

• If the test should be failed under particular device, use fail-if (and provide explanation)

fail-if = device == "desktop"

Following are commonly used flags:

• external (using external components)
• lan (local area network)
• smoketest (part of smoketest suite)
• sanity (part of sanity test suite)
• dogfood (part of dogfood test suite)
• carrier (needs carrier connection)
• stable (set to false for intermittent failures)
• online (internet)
• antenna (FM antenna)
• sdcard (sd card)
• wifi ( wifi connection)
• bluetooth (bluetooth connection)