B2G/UpdateTesting
From MozillaWiki
< B2G
This document is a WIP for the upcoming B2G update test automation in bug 821412
Overview
When a system update is applied in B2G, it causes either a process restart (OTA) or a device reboot (FOTA), requiring special high level support from a testing framework to automate testing of the various stages of the update.
To solve this problem, specialized Marionette based frontends have been written to solve two different use cases:
- A script based frontend for use by developers, and eventually TBPL
- A "continuous build" smoke testing frontend designed to automatically test updates between builds in a Continuous Integration environment.
Smoke Tests
Setup
To get started, just make a directory that will store updates and flash scripts for each build, and inside it create a smoketest-config.json.
$ mkdir $B2G_UPDATE_TESTDATA
Example smoketest-config.json:
{ "devices": { "unagi": { "system_fs_type": "ext4", "system_location": "/dev/block/mmcblk0p19", "data_fs_type": "ext4", "data_location": "/dev/block/mmcblk0p22", "sdcard": "/mnt/sdcard", "sdcard_recovery": "/sdcard", "serials": ["full_unagi"] } }, "public_key": "/Users/mculpepper/Code/B2G-dev/build/target/product/security/testkey.x509.pem", "private_key": "/Users/mculpepper/Code/B2G-dev/build/target/product/security/testkey.pk8" }
CI integration
- Each CI build should have its update staged for each test device:
$ testing/marionette/update-smoketests/stage-update.py unagi $B2G_UPDATE_TESTDATA
- Once at least 2 builds are staged, the smoketests can be run by passing a list of build IDs to test between:
$ testing/marionette/update-smoketests/run-smoketests.py --build-dir $B2G_UPDATE_TESTDATA --run-dir $B2G_UPDATE_RUNDATA 20121219101023 20121220172647