Marionette Improvements
From MozillaWiki
Contents
Project Description
This project involved fixing the Marionette related problems preventing tests from being run on devices at the Bitbar device farm with results returned to Treeherder. Work ended when Firefox OS was sunset.
Requirements:
Removal of all of python from the Marionette harness
* New Marionette promises driver * Create tests for new Marionette client promises driver * Automate conversion of existing tests onto new API * Remove Sockit-to-me from Marionette
Device testing with TaskCluster / Treeherder using Bitbar
* Automate the running of Marionette.js tests on Bitbar devices
MJS stability
* Compile and clean up marionette-js-runner codebase with babel * Remove wrapper and use Marionette-Mocha directly * Missing stacktrace errors fix
Sandbox environment stability
* Gaia integration with latest version of node * Update to Mocha 2
Program Status
| Milestone | Due Date | Status | Notes |
|---|---|---|---|
| Removal of python from the Marionette harness | 02/26/16 | ON TARGET | New marionette-client promises driver developed. Fixed TC related race condition and now working on demonstrating the writing of Gij tests using the new promises api. |
| Device testing on TaskCluster/Treeherder with Bitbar | ON HOLD | Pending new direction and its test requirements. | |
| Marionette stability | 05/25/16 | ON TARGET | Bugs have been entered and work is slotted for Q1. |
| Sandbox environment stability | 05/25/16 | ON TARGET | Work is slotted for Q1. |
| * MJS Intermittents Related Issues Fixed | DONE | ||
| * Gaia integration with latest version of node | DONE | ||
| * Update to Mocha 2 | DONE | ||
| * Documentation | DONE |
Status Key
| Color | Status | Key |
|---|---|---|
| On Target | The project or deliverable is expected to meet its due date. | |
| Challenged | The project or deliverable is facing an issue that might cause it to miss its due date, but a “get well” plan has been developed to get it back on track. | |
| At Risk or Late | The project or deliverable is blocked or facing an issue that might cause it to miss its due date, and there’s no “get well” plan to get it back on track, or it is already late. | |
| Done | The project or deliverable has been completed. | |
| On Hold | The project or deliverable has been placed on hold. |
Program Timeline
Project Scope
Python Removal from Marionette Harness
| ID | Summary | Status | Target milestone | Resolution | Assigned to | Depends on | Blocks | Whiteboard | Feature-b2g |
|---|---|---|---|---|---|---|---|---|---|
| 1104285 | Frequent Gij failures stemming from "TypeError: Cannot call method 'send' of undefined" errors | RESOLVED | --- | FIXED | Gareth Aye [:gaye] (back from PTO) | 1091680 | 1161727, 1179595 | [itermittents] [MJS] [CI] | No feature-b2g |
| 1155116 | [fxos-automation] marionette-js-client should have contextMenuClick() method to compliment other touch and click events | RESOLVED | --- | FIXED | Aus Lacroix [:aus] | 1155110, 1161727 | [MJS] [CI] | No feature-b2g | |
| 1161727 | [gaia-automation] Marionette JS Features & Bugs | RESOLVED | --- | WORKSFORME | Aus Lacroix [:aus] | 1104285, 1140344, 1155116, 1159200, 1175116, 1179012, 1186235, 1188648, 1224630, 1227184 | [MJS] [CI] | No feature-b2g | |
| 1175116 | Stacktrace missing from test report because "Crash detected but error running stackwalk" although no crash occurred | RESOLVED | --- | FIXED | Aus Lacroix [:aus] | 950943, 1161727, 1174901, 1231773, 1239338 | [MJS] [CI] | No feature-b2g | |
| 1227577 | Develop a new marionette-client promises driver | RESOLVED | --- | FIXED | André Natal | 1239470 | 1227580, 1227583, 1227584, 1227585, 1227597, 1231773 | [MJS] [CI] | No feature-b2g |
| 1227580 | (META_Removal_of_python_from_the_Marionette_harness) [META] Removal of python from the Marionette harness | RESOLVED | 2.6 S5 - 1/15 | INCOMPLETE | Gareth Aye [:gaye] (back from PTO) | 1227577, 1227583, 1227584, 1227585, 1227597, 1231775 | 1231773 | [MJS] [CI] | No feature-b2g |
| 1227583 | Demonstrate writing Gij tests using the new promises api | RESOLVED | --- | INCOMPLETE | Gareth Aye [:gaye] (back from PTO) | 1227577 | 1227580 | [MJS] [CI] | No feature-b2g |
| 1227584 | Automate converting as many existing tests as possible onto the new API | RESOLVED | --- | WONTFIX | André Natal | 1227577 | 1227580 | [MJS] [CI] | No feature-b2g |
| 1227585 | Remove sockit-to-me from marionettejs | RESOLVED | --- | WONTFIX | André Natal | 1227577 | 1227580 | [MJS] [CI] | No feature-b2g |
| 1227597 | Create new tests for the new marionettejs Promises driver | RESOLVED | --- | DUPLICATE | André Natal | 1227577 | 1227580 | [MJS] [CI] | No feature-b2g |
| 1230275 | [gatt] [device-ci] Investigate current state of BitBar Device Worker for TaskCluster | RESOLVED | --- | WONTFIX | 1224630 | [TC] [MJS] [CI] | No feature-b2g | ||
| 1231773 | JSMarionette Q4 2015 & Q1 2016 improvements | RESOLVED | --- | INCOMPLETE | 1175116, 1227577, 1227580, 1231775, 1231779, 1231786, 1231789, 1234601 | [MJS] [CI] | No feature-b2g | ||
| 1231775 | Move marionette-js-runner onto fxos-device-service | RESOLVED | --- | INCOMPLETE | Gareth Aye [:gaye] (back from PTO) | 1235451 | 1212617, 1227580, 1231773 | [MJS] [CI] | No feature-b2g |
| 1231779 | Remove gaia-marionette | RESOLVED | --- | INCOMPLETE | 1231773 | [MJS] [CI] | No feature-b2g | ||
| 1231786 | Compile and clean up marionette-js-runner codebase with babel | RESOLVED | --- | INCOMPLETE | 1231773 | [MJS] [CI] | No feature-b2g | ||
| 1231789 | Investigate speaking directly to fxos-device-service from mocha in marionette-js-runner | RESOLVED | --- | INCOMPLETE | 1231773 | [MJS] [CI] | No feature-b2g | ||
| 1234601 | Error while trying to run make test-integration-test: TypeError: Cannot read property 'getAppClass' of undefined | RESOLVED | --- | FIXED | Gareth Aye [:gaye] (back from PTO) | 1231773 | [MJS] [CI] | No feature-b2g | |
| 1235451 | [fxos-device-service] Support for uploading and downloading profiles as archives | RESOLVED | --- | FIXED | Gareth Aye [:gaye] (back from PTO) | 1231775 | [MJS] [CI] | No feature-b2g | |
| 1236376 | [Gij] RangeError: port should be >= 0 and < 65536: 65537, RangeError: port should be >= 0 and < 65536: 65536 ... Not connected. To write data you must call connect first | RESOLVED | 2.6 S5 - 1/15 | FIXED | Michael Henretty [:mikehenrty][:mhenretty] | [systemsfe][MJS] [CI] | No feature-b2g | ||
| 1236378 | Test fails, but does not specify which test, only "Tests exited with return code 2: harness failures" | RESOLVED | --- | FIXED | Aus Lacroix [:aus] | 1174901 | [MJS] [CI] | No feature-b2g | |
| 1240854 | AssertionError: deleteSession called on client w/o driver! | RESOLVED | --- | INCOMPLETE | Gareth Aye [:gaye] (back from PTO) | [MJS] [CI] | No feature-b2g |
21 Total; 0 Open (0%); 21 Resolved (100%); 0 Verified (0%);
Marionette Stability
| ID | Summary | Status | Target milestone | Resolution | Assigned to | Depends on | Blocks | Whiteboard | Feature-b2g |
|---|---|---|---|---|---|---|---|---|---|
| 1104285 | Frequent Gij failures stemming from "TypeError: Cannot call method 'send' of undefined" errors | RESOLVED | --- | FIXED | Gareth Aye [:gaye] (back from PTO) | 1091680 | 1161727, 1179595 | [itermittents] [MJS] [CI] | No feature-b2g |
| 1155116 | [fxos-automation] marionette-js-client should have contextMenuClick() method to compliment other touch and click events | RESOLVED | --- | FIXED | Aus Lacroix [:aus] | 1155110, 1161727 | [MJS] [CI] | No feature-b2g | |
| 1161727 | [gaia-automation] Marionette JS Features & Bugs | RESOLVED | --- | WORKSFORME | Aus Lacroix [:aus] | 1104285, 1140344, 1155116, 1159200, 1175116, 1179012, 1186235, 1188648, 1224630, 1227184 | [MJS] [CI] | No feature-b2g | |
| 1175116 | Stacktrace missing from test report because "Crash detected but error running stackwalk" although no crash occurred | RESOLVED | --- | FIXED | Aus Lacroix [:aus] | 950943, 1161727, 1174901, 1231773, 1239338 | [MJS] [CI] | No feature-b2g | |
| 1227577 | Develop a new marionette-client promises driver | RESOLVED | --- | FIXED | André Natal | 1239470 | 1227580, 1227583, 1227584, 1227585, 1227597, 1231773 | [MJS] [CI] | No feature-b2g |
| 1227580 | (META_Removal_of_python_from_the_Marionette_harness) [META] Removal of python from the Marionette harness | RESOLVED | 2.6 S5 - 1/15 | INCOMPLETE | Gareth Aye [:gaye] (back from PTO) | 1227577, 1227583, 1227584, 1227585, 1227597, 1231775 | 1231773 | [MJS] [CI] | No feature-b2g |
| 1227583 | Demonstrate writing Gij tests using the new promises api | RESOLVED | --- | INCOMPLETE | Gareth Aye [:gaye] (back from PTO) | 1227577 | 1227580 | [MJS] [CI] | No feature-b2g |
| 1227584 | Automate converting as many existing tests as possible onto the new API | RESOLVED | --- | WONTFIX | André Natal | 1227577 | 1227580 | [MJS] [CI] | No feature-b2g |
| 1227585 | Remove sockit-to-me from marionettejs | RESOLVED | --- | WONTFIX | André Natal | 1227577 | 1227580 | [MJS] [CI] | No feature-b2g |
| 1227597 | Create new tests for the new marionettejs Promises driver | RESOLVED | --- | DUPLICATE | André Natal | 1227577 | 1227580 | [MJS] [CI] | No feature-b2g |
| 1230275 | [gatt] [device-ci] Investigate current state of BitBar Device Worker for TaskCluster | RESOLVED | --- | WONTFIX | 1224630 | [TC] [MJS] [CI] | No feature-b2g | ||
| 1231773 | JSMarionette Q4 2015 & Q1 2016 improvements | RESOLVED | --- | INCOMPLETE | 1175116, 1227577, 1227580, 1231775, 1231779, 1231786, 1231789, 1234601 | [MJS] [CI] | No feature-b2g | ||
| 1231775 | Move marionette-js-runner onto fxos-device-service | RESOLVED | --- | INCOMPLETE | Gareth Aye [:gaye] (back from PTO) | 1235451 | 1212617, 1227580, 1231773 | [MJS] [CI] | No feature-b2g |
| 1231779 | Remove gaia-marionette | RESOLVED | --- | INCOMPLETE | 1231773 | [MJS] [CI] | No feature-b2g | ||
| 1231786 | Compile and clean up marionette-js-runner codebase with babel | RESOLVED | --- | INCOMPLETE | 1231773 | [MJS] [CI] | No feature-b2g | ||
| 1231789 | Investigate speaking directly to fxos-device-service from mocha in marionette-js-runner | RESOLVED | --- | INCOMPLETE | 1231773 | [MJS] [CI] | No feature-b2g | ||
| 1234601 | Error while trying to run make test-integration-test: TypeError: Cannot read property 'getAppClass' of undefined | RESOLVED | --- | FIXED | Gareth Aye [:gaye] (back from PTO) | 1231773 | [MJS] [CI] | No feature-b2g | |
| 1235451 | [fxos-device-service] Support for uploading and downloading profiles as archives | RESOLVED | --- | FIXED | Gareth Aye [:gaye] (back from PTO) | 1231775 | [MJS] [CI] | No feature-b2g | |
| 1236376 | [Gij] RangeError: port should be >= 0 and < 65536: 65537, RangeError: port should be >= 0 and < 65536: 65536 ... Not connected. To write data you must call connect first | RESOLVED | 2.6 S5 - 1/15 | FIXED | Michael Henretty [:mikehenrty][:mhenretty] | [systemsfe][MJS] [CI] | No feature-b2g | ||
| 1236378 | Test fails, but does not specify which test, only "Tests exited with return code 2: harness failures" | RESOLVED | --- | FIXED | Aus Lacroix [:aus] | 1174901 | [MJS] [CI] | No feature-b2g | |
| 1240854 | AssertionError: deleteSession called on client w/o driver! | RESOLVED | --- | INCOMPLETE | Gareth Aye [:gaye] (back from PTO) | [MJS] [CI] | No feature-b2g |
21 Total; 0 Open (0%); 21 Resolved (100%); 0 Verified (0%);
Sandbox Environment Stability
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);
Automated Device Testing with TaskCluster/Treeherder at Bitbar
| ID | Summary | Status | Target milestone | Resolution | Assigned to | Depends on | Blocks | Whiteboard | Feature-b2g |
|---|---|---|---|---|---|---|---|---|---|
| 1224630 | [gatt][q4-q1] Integrate device workers for BitBar device farm into Taskcluster | RESOLVED | --- | WONTFIX | Aus Lacroix [:aus] | 1230275 | 1161727 | [TC] [MSJ] [CI] | No feature-b2g |
| 1230275 | [gatt] [device-ci] Investigate current state of BitBar Device Worker for TaskCluster | RESOLVED | --- | WONTFIX | 1224630 | [TC] [MJS] [CI] | No feature-b2g |
2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);
Dependency Tracking
Detailed Program Plan
| Action Item | Engineering Owner | QA Owner | UX Owner | Bugzilla ID | Planned Done | Actual Done |
|---|---|---|---|---|---|---|
Program Stakeholders
| Role | Name | IRC |
|---|---|---|
| Sponsor | David Scravaglieri | scravag |
| EPM | Julie McCracken | Julie |
| EM | Dylan Oliver | doliver |
| TL | Gareth Aye | gaye |
| TL | Ghislain (Aus) Lacroix | aus |
| QA | Pallavi Yaramada | pallavi |
| QA | Johan Lorenzo | jlorenzo |
- EPM = Engineering Program Manager
- EM = Engineering Manager
- PM = Product Manager
- TL = Tech Lead
- QA = Quality Assurance
