B2G/QA/Automation/Style Guide

The goal of this style guide is to provide rules to write ui-automation code that is clear and effective. It is a fork of this webqa page.

It is also recommended to review Geo's draft of automation best practices, which gives a theoretical overview of writing automation tests.

How-Tos

Using WebIDE to Define Locators <njpark>

WebIDE can connect to firefox OS devices, and lets the user to view the locators of Firefox OS app. For basics of WebIDE, please refere to this page.

Prerequisites:

• From the phone, go to Settings -> Developer and select "ADB and DevTools" under "Debugging via USB"
• Connect the phone to PC via USB.
• From WebIDE, click 'Select Runtime' and select the phone under "USB Devices".
• From the device, allow USB connection.
• From the upper right corner, select the app that you wish to view. If no apps are shown in the list, go to Runtime -> Runtime Info drop-down menu, and click "Request Higher Privileges" button. The phone will restart, and you should be able to see all apps loaded on the phone.

CSS Selectors

Use following references as guides.

• W3
• Locator Cheat Sheet

CSS Selectors

Use following references as guides.

• W3
• Locator Cheat Sheet

Tests - Style

Naming

• Module names should be called test_ and then behavioral areas.

   test_search.py

• Test method names should always show the intent of the test case.

# Good
def test_that_advanced_search_does_not_find_item(self):

# Bad
def test_advanced_search(self):

File Headers

• Each file should have a completed copy of the MPL2 license block, immediately followed by an empty line.
• Each file should pass PEP8 except for line length, see below.

# Good
def method(self, parameter) 

# Bad
def method(self,parameter)

• Lines should try not to have more than 100 characters.
• Docstrings should conform to PEP0257 and should be on a single line wherever possible.

# Good
def click_login():
"""Clicks the login link."""

# Bad
def click_login():
"""
Clicks the login link.
"""

Where not possible, the first line should be a summary.

# Good
def login():
"""Logs in.

Clicks the login link and then waits for the home page to load.

"""

# Bad
def login():
"""Logs in.
Clicks the login link and then waits for the home page to load."""

• Indenting should be a soft tab (4 spaces) as common with in Python. Do not mix tabs and spaces!
• There should be no whitespace at the end of the file (as per PEP8).
• Comments should be on the line above. Remember to update comments when changing code so that code matches the comments.
• Comments should be used carefully. Try not to put comments that don't provide any additional value
• Class names should be in Pascal style as this is Python idiomatic.

# Good
class TestThisSite:
    
# Bad
class test_this_site:

Actions

• Methods that perform actions on the page should indicate the action in the method name.

# Good
def click_report_with_length(length)

# Bad
def report_length(length)

• Actions should wait for the appropriate action to complete. This could be an implicit or explicit wait. For example, clicking a login button might explicitly wait for a username field to be visible.

Use testvars.json

TBD - njpark

Using high-level methods

TBD - jlorenzo

• In a test, make sure to only have steps that are high level (for instance: messages.send_an_sms_to_yourself() instead of detailing every single tap and click)

Avoid sleep() calls

TBD - njpark

Variable naming

TBD - jlorenzo

• Name your variables with units. For example: timeout_in_seconds, width_in_pixels
• Don't shorten variable names. A bad example: self.t_out_err_tim = 1000 instead of self.time_out_error_time

app.py and regions/helper.py

TBD - njpark

How to create filename strings

TBD - njpark http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/apps/settings/app.py#296

Meaningful custom Assert() messages

TBD - njpark

https://bugzilla.mozilla.org/show_bug.cgi?id=1198449#c2

Clean up afterwards

TBD - njpark

Avoid code duplication

TBD - jlorenzo

• Use parameterized()
• Centralize workarounds

Simulate end-user check

TBD - jlorenzo

• Check what an end-user would check.
• Make sure manifest.ini is updated with right flags

Making test multi-locale

TBD - njpark

• we check the raw text in order to verify the input from the user (for instance a phone number put in the dialer, we verify it appears in the call log)
• we test the l10n-id for any string that comes from Gaia only (like an error message)

Be aware of outside consumers of ui-test

TBD - mwargers

• Be aware if you change the Gaia UI test API, that outside consumers (mtbf, etc) might get broken

Logic

Handling shadow DOM

TBD - njpark

Switching frames and system frame

TBD - mwargers

Handling browser instances

TBD - mwargers

Returning the page object after completing an action

TBD - njpark

   When writing methods that is doing some action on the device, for instance opening an app or opening a subpage, make sure that the resulting action is finished and return the object of the resulting action, e.g. the app or the subpage (PageRegion)

Use of Libraries

Limited use of conditionals

• Methods should not contain logic that depends on properties of the page. The logic and expectations should be within the test, and adding this to the page object could guard your tests against genuine failures.

# Good
def click_login(self)
    self.selenium.find_element(*self._login_locator).click()

# Bad
def click_login(self)
    if not self.is_user_logged_in:
        self.selenium.find_element(*self._login_locator).click()
    else:
        pass

Locators

TBD - jlorenzo

• Locator variables should be prefixed with _ to show that it is private.
• Variables should be descriptive of the area and not clash with any properties.
• Should have a suffix of _locator.
• Accessing locators should be done through a property or method as this keeps the locator as read-only.

@property
def search_term(self):
    return self.selenium.find_element(*self._search_box_locator).value

• We should use locators in the following order of preference (there will be exceptions):
  • ID
  • Name
  • Class name
  • CSS selector
  • XPath
• CSS locators should use whitespace for readability when using direct descendants.

# Good
_my_locator = "css=#content > p > a"
    
# Bad
_my_locator = "css=#content>p>a"

• Use Python tuples to define locators:

# Good
_my_locator = (By.ID, "content")

PageRegions

TBD - jlorenzo

• How to deal with stale root_elements

In some circumstances, for example where a header/navigation is common across the website, we will use a page region. The page region is a child class of the base Page object, which is inherited by all page objects. This means that the navigation can be reached from any page object and herein lies the DRY!

A brief example:

class BasePage(Page):

    @property
    def header(self):
        return BasePage.HeaderRegion(self.testsetup)
    
    class HeaderRegion(Page):

        _login_link = (By.ID, "home")

        @def click_login(self):
            self.selenium.find_element(*self._login_link).click()

Referring to this page region with a property makes it very readable and concise from within the test. Clicking login during a test would be performed like this:

my_page.header.click_login()

Another example where this might be used is on a search results page, the page region being the search results element.

Assertions

TBD - jlorenzo

• Tests should handle the asserts -- not the page objects.
• Tests should use Python's native assert statement.
• When doing equivalency assertions, put the expected value first, followed by the actual value, for example:

# Good
a = some_function()
assert 'expected result' == a

# Bad
a = some_function()
assert a == 'expected result'

How to use GaiaHeader and GaiaBinaryControl

TBD - jlorenzo

Submitting and Reviewing Patches

Submission

To make sure that we can review your patch as quickly and efficiently as possibly we would like patches to have a single test in them and the necessary changes to the page objects. This also limits the chances of merge conflicts later.
Please refer to this page for gaia patch submission steps.

Review

First of all, make sure above mentioned guidelines are not violated, and check for the following:

Look for transitions that are not waited on

TBD - jlorenzo

Check for change/removal of methods

In order to make sure that the change is cascaded, `git grep` is your friend. TBD - jlorenzo

Are the locators correct? Could they be improved?

TBD - jlorenzo

Are the locators/methods used that were added?

TBD - mwargers A linter would also help to enforce it, see: https://bugzilla.mozilla.org/show_bug.cgi?id=1186388

Are workarounds explained in the code file?

TBD - jlorenzo