Confirmed users
606
edits
Sheriffing/How To/Mobile Trees: Difference between revisions
Jump to navigation
Jump to search
add how to classify
(added repositories with description, tree and repository links and escalation channels) |
(add how to classify) |
||
| Line 12: | Line 12: | ||
** Repository: [https://github.com/mozilla-mobile/reference-browser reference-browser] on GitHub | ** Repository: [https://github.com/mozilla-mobile/reference-browser reference-browser] on GitHub | ||
** Escalation channel: #android-components on Slack | ** Escalation channel: #android-components on Slack | ||
===Differences to autoland and mozilla-central=== | |||
* If a task shall run once more, this must be done from "Custom Actions" > "rerun" | |||
* Failures are tracked in multiple tools | |||
** Raptor failures in Bugzilla's [https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=Raptor Testing :: Raptor] component | |||
** Issues with the release pipeline like signing issues into Bugzilla's [https://bugzilla.mozilla.org/enter_bug.cgi?product=Release%20Engineering Release Engineering] | |||
** Non-Raptor test failures shall be reported into the Github repository mentioned above. | |||
** Test results are often stored externally and not in the log. | |||
===Accessing the test results=== | |||
Many test tasks stored the test results in an external database and the log will only provide the url to the results in that database. | |||
<ol> | |||
<li>Select the failed task.</li> | |||
<li>Open the unformatted log.</li> | |||
<li>Go the end of the log and look for a failure line like this: | |||
<code>[task 2020-06-17T10:41:25.807Z] Error: Matrix failed: matrix-147257ou2tpig FINISHED failure https://console.firebase.google.com/project/moz-reference-browser-230023/testlab/histories/bh.b4e77beaed81bc1c/matrices/6129539140917351455</code></li> | |||
<li>Copy the url.</li> | |||
<li>Open the url in a new tab.</li> | |||
<li>The main pane lists the devices on which the tests are run. Repeat the following steps for each device with a failure.</li> | |||
<li>Click on the device with the failure. A list of 'shards' will reopen which are similar to 'chunks' on autoland and mozilla-central: The tests are distributed across the shards.</li> | |||
<li>Open each failed shard in a new tab (click link with mouse wheel).</li> | |||
<li>Depending on the task, the view opened will provide details on the failure. The tab '''Test cases''' shows which test failed.</li> | |||
<li>If don't know if there is already a Github issue tracking the issue, go to the related Github project mentioned above.</li> | |||
<li>Click onto the tab 'Issues'.</li> | |||
<li>Insert the test name from the clipboard and search for it.</li> | |||
<li>If there is no open Github issue, create one. | |||
<ul> | |||
<li>Set the issue title as 'Intermittent testName errorSummary'.</li> | |||
<li>Provide a link to the log.</li> | |||
<li>Copy and paste the stacktrace (if present).</li> | |||
<li>For further questions about test failures, the [https://chat.mozilla.org/#/room/#mobile-test-eng:mozilla.org mobile test engineering channel] is the primary contact.</li> | |||
</ul> | |||
</li> | |||
<li>Copy the url of the issue.</li> | |||
<li>Back in Treeherder, select the task (e.g. Ctrl + click and paste the url into the comment field.</li> | |||
<li>Save the classification.</li> | |||
</ol> | |||