New test environments: Difference between revisions

New test environments (view source)

653 bytes added , 20 August 2019

75

edits

@@ Line 39: / Line 39: @@
 The first step of any new test environment is to enable the test platform on Tryserver.
-At each step, ensure the taskgraph is sound. This can be verified using <code>./mach taskgraph full -v</code>
+At the bare minimum, ensure the taskgraph is sound with each step. This can be verified using <code>./mach taskgraph full -v</code>
-If the taskgraph can be generated, a push to Tryserver would at least pass the Gecko Decision Task.
 == Enable build ==
@@ Line 206: / Line 204: @@
 * does <code>./mach taskgraph full</code> succeed with test sets and test platforms defined?
 * do the tests show up in the try fuzzy selector?
+== Obtain baseline ==
+Once build and tests are enabled on Tryserver, it is time to run a baseline push to take inventory of suites that pass and fail.
+Using <code>./mach try fuzzy --no-artifact</code>, push tests belonging to the new platform to the Tryserver. Do not use artifact builds for the baseline push as some test results (outside of the compiled tests such as <code>cppunit</code>) are affected by the artifact build.
+=== Example ===
+[https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=119fc9a78c7c62c7bace963a4bb88673ee9cab51 Debian 10]
+=== Checklist ===
+* does the build succeed?
+* are the tests scheduled?
+* does each test job run to completion?
+* which tests pass?
+* which tests fail?