Confirmed users
955
edits
Buildbot/Talos/Data: Difference between revisions
→Terminology: another clarification
(→Suite: update) |
(→Terminology: another clarification) |
||
| Line 9: | Line 9: | ||
When a build is completed we run a series of test (unittest and performance "talos") jobs. Each job reserves a machine for itself, then runs the script which sets up, installs, executes the test, generates the results, and cleans up after itself. In general we try to ensure the jobs complete in 30 minutes or less. | When a build is completed we run a series of test (unittest and performance "talos") jobs. Each job reserves a machine for itself, then runs the script which sets up, installs, executes the test, generates the results, and cleans up after itself. In general we try to ensure the jobs complete in 30 minutes or less. | ||
For Talos we have a series of jobs and each job runs 1 or more | For Talos we have a series of jobs and each job runs 1 or more test suites (with the exception of tp and xperf, jobs run 2-4 suites at a time). A suite would be something like 'ts_paint', 'Canvasmark', or 'tp5'. Each Suite will run its respective subtests, and provide a summary number which is representative of a meaningful aggregation of the individual subtest results. When all the suites in a job have completed, the results will be output (uploaded in some cases) and we will be able to look for regressions, view data in a graph, and query for the summarized data. | ||
=== Suite === | === Suite === | ||