Confirmed users
1,927
edits
Auto-tools/Meetings/2015-02-23: Difference between revisions
Auto-tools/Meetings/2015-02-23 (view source)
Revision as of 18:32, 23 February 2015
, 23 February 2015→Holidays and Trips
| (21 intermediate revisions by 16 users not shown) | |||
| Line 1: | Line 1: | ||
= Notices, Highlights, Roundtable = | = Notices, Highlights, Roundtable = | ||
* Significant Contributions | * Significant Contributions | ||
** Vikas Mishra has being doing awesome work on the perfherder ui frontend: {{bug|1132506}}, {{bug|1131183}} | |||
** Julien Parkouss continues doing great work on mozregression, helping mentor various new contributors (e.g. {{bug|1128601}}) in addition to fixing and improving the software | |||
** adusca and vaibhavmagarwal for contributions mozilla_ci_tools | |||
* Introductions: Maja Frydrychowicz and Syd Polk | |||
= Newsgroup and Blog Posts = | = Newsgroup and Blog Posts = | ||
* https://gbrownmozilla.wordpress.com/2015/02/13/more-memory-for-the-android-emulator/ | * https://gbrownmozilla.wordpress.com/2015/02/13/more-memory-for-the-android-emulator/ | ||
* https://mrcote.info/blog/2015/02/16/pulse-update/ | * https://mrcote.info/blog/2015/02/16/pulse-update/ | ||
* http://wrla.ch/blog/2015/02/measuring-e10s-vs-non-e10s-performance-with-perfherder/ | |||
= Goal Updates = | = Goal Updates = | ||
| Line 20: | Line 25: | ||
** {{bug|906712}} fixed | ** {{bug|906712}} fixed | ||
** {{bug|1096488}}, support for non-remote <-> remote pages in e10s in progress (blocking e10s in firefox-ui-tests) | ** {{bug|1096488}}, support for non-remote <-> remote pages in e10s in progress (blocking e10s in firefox-ui-tests) | ||
** {{bug|1107336}} | |||
*** WPT Runner changes added | |||
*** Green try run, just fixing patch for review (reviewboard and my local copy aren't best friends) | |||
=== Support the conversion of targeted P1 mozmill tests to Marionette and get them running in CI [hskupin] === | === Support the conversion of targeted P1 mozmill tests to Marionette and get them running in CI [hskupin] === | ||
* details: Perform the CI work necessary to get mozmill tests converted to Marionette running for update | * details: Perform the CI work necessary to get mozmill tests converted to Marionette running for update tests, and security tests. This entails writing new Marionette test libraries for these features, and getting converted tests running on the existing mozmill CI systems. | ||
* stretch goal: get test results reported to Treeherder | * stretch goal: get test results reported to Treeherder | ||
* stretch goal: support the conversion of Search tests | * stretch goal: support the conversion of Search tests | ||
* '''progress since last update''': | * '''progress since last update''': | ||
** Given by the feedback of QE we are now targeting security and update tests as P1 | |||
** Lots of activities here with about 7 security tests in the work and close to be landed - for others the deps will also land soon | |||
** Bob Silverberg started on the software update module and will continue with the about window soon; Henrik will start this week with the implementation of the necessary harness features | |||
** Thanks to a couple of our contributors who are working on patches! | |||
=== Resolve P1 bugs blocking the release of Marionette 1.0 [AutomatedTester, ato, jgraham] === | === Resolve P1 bugs blocking the release of Marionette 1.0 [AutomatedTester, ato, jgraham] === | ||
| Line 31: | Line 43: | ||
* stretch goal: release Marionette 1.0 | * stretch goal: release Marionette 1.0 | ||
* '''progress since last update''': | * '''progress since last update''': | ||
** [ato] {{Bug|1107706}}: Blocking tests on B2G have been resolved, now working on adapting emulator callbacks to the new dispatching mechanism. No more known issues, expect to start rebasing this week. | |||
** {{bug|1118298}} done by contributor :gioyik | |||
** [jgraham] - Taught wires to launch Firefox (via very basic Rust ports of mozrunner and mozprofile); started working on a better testsuite to cover the webdriver protocol. | |||
===''Supporting Tasks''=== | ===''Supporting Tasks''=== | ||
| Line 41: | Line 56: | ||
* bugs: {{bug|1109218}}, {{bug|1121616}} | * bugs: {{bug|1109218}}, {{bug|1121616}} | ||
* '''progress since last update''': | * '''progress since last update''': | ||
** | ** work has continued on the MozReview UI - some portions reviewed and landed | ||
** autoland configured to use https | ** autoland configured to use https | ||
| Line 58: | Line 73: | ||
* details: We want to use Treeherder to store and display Talos performance data, because Datazilla is being deprecated and Graphserver doesn't support the kind of performance analysis we'd like to perform on the data. | * details: We want to use Treeherder to store and display Talos performance data, because Datazilla is being deprecated and Graphserver doesn't support the kind of performance analysis we'd like to perform on the data. | ||
* '''progress since last update''': | * '''progress since last update''': | ||
** UI improvements from wlach and mishrasvikras | |||
** Starting on support for summary series ({{bug|1108832}}) | |||
== Treeherder == | == Treeherder == | ||
| Line 70: | Line 87: | ||
* bug: {{bug|1113873}} | * bug: {{bug|1113873}} | ||
* '''progress since last update''': | * '''progress since last update''': | ||
** Viewing logs based on error lines is working. | |||
** Infinite scrolling of logs is partially working. | |||
** Created a Celery task to generate the summary from the full "raw" structured log. *almost* working... | |||
** ON TRACK - Expect to be done by end of February | |||
=== {{defer|}} Develop a minimal UI that sheriffs can use to file new intermittents [edmorley] === | === {{defer|}} Develop a minimal UI that sheriffs can use to file new intermittents [edmorley] === | ||
| Line 92: | Line 113: | ||
* '''progress since last update''': | * '''progress since last update''': | ||
** slow but steady progress | ** slow but steady progress | ||
** all fields and comments implemented, working through minor outstanding issues and testing | |||
** [https://dl.dropboxusercontent.com/u/16292140/1096798-wip-3.png screenshot] [https://dl.dropboxusercontent.com/u/16292140/1096798-wip-4.png screenshot] | |||
=== Implement versioning framework for REST API [dkl] === | === Implement versioning framework for REST API [dkl] === | ||
| Line 97: | Line 120: | ||
* bug: {{bug|1051056}} | * bug: {{bug|1051056}} | ||
* '''progress since last update''': | * '''progress since last update''': | ||
** Finishing up on final (4th) design iteration. Removal of legacy dependency on library that will be going away once REST is the only API. REST was a wrapper around JSON::RPC (RPC procedure calls over JSON) and had quite a few hacks in the code to remove JSON::RPC specific details before returning REST like data. The new version is written for REST only and will be much simpler. Slow but steady progress. Trying to get up for review late today, early tomorrow. | |||
=== GitHub authentication [dylan] === | === GitHub authentication [dylan] === | ||
* details: Allow users to authenticate with Bugzilla using their GitHub account. This will encourage more contributors and allow us to better integrate GitHub into the Mozilla workflow. | * details: Allow users to authenticate with Bugzilla using their GitHub account. This will encourage more contributors and allow us to better integrate GitHub into the Mozilla workflow. | ||
* bug: {{bug|1118365}} | * bug: {{bug|1118365}} | ||
* '''progress since last update''': | * '''progress since last update''': Steady progress since last update. Several design concessions have had to be made to Bugzilla's authentication stack. First review will be this week. | ||
== DevTools Harness == | == DevTools Harness == | ||
| Line 120: | Line 144: | ||
* bug: {{bug|1121655}} | * bug: {{bug|1121655}} | ||
* '''progress since last update''': | * '''progress since last update''': | ||
** 2015-02-11: Met with dburns, jlal, mcote, mdoglio, ryanvm, sphilp, jgriffin to discuss the meaning of Tier 2 jobs and how they related to this deliverable. Tier 2 jobs where determined to be sheriffable jobs whose sheriffing is limited to filing bugs on failures which can then be ignored by sheriffs and whose failures do not cause the offending changesets to be backed out. These aspects of Tier 2 jobs are [[#Distinguish_between_Tier_1_and_Tier_2_jobs_.5Bmdoglio.5D|Treeherder UI issues]] and not related to the original expectation of this deliverable. | |||
** My understanding, is that this deliverable is now about defining and documenting requirements for non-buildbot jobs. | |||
=== Prototype a retrigger-based bisection tool [armenzg, jmaher] === | === Prototype a retrigger-based bisection tool [armenzg, jmaher] === | ||
| Line 131: | Line 157: | ||
* details: Create a Proof of Concept “big data” project which will store information about every test file we run: test status, error details, test machine and test duration to begin with. We will use this project to develop schemas and queries that work with data this large, and we will use this data to normalize chunk sizes and provide details about which tests never fail. | * details: Create a Proof of Concept “big data” project which will store information about every test file we run: test status, error details, test machine and test duration to begin with. We will use this project to develop schemas and queries that work with data this large, and we will use this data to normalize chunk sizes and provide details about which tests never fail. | ||
* '''progress since last update''': | * '''progress since last update''': | ||
** ActiveData service is up! http://activedata.allizom.org/ | |||
** Use the [http://people.mozilla.org/~klahnakoski/qb/query.html query tool] to send queries in realtime | |||
** Only basic queries are supported by the ActiveData service right now, and not very well. | |||
** Currently has over 1 billion test results | |||
** Three big ($0.50/hour) machines in cluster has reasonable aggregation performance (1x10^9 records in 20seconds) | |||
** Three bigger ETL machines ($1/hour) running 60 python threads each (180 total) can just barely keep up with data volume | |||
** Many opportunities to reduce cost (more efficient algorithms, spot instances, dynamic machine allocation) | |||
* '''next steps''': | |||
** More test cases must be written that will cover the common and simple use cases: Along with making ActiveData pass those tests. | |||
** Add minimal discoverability features: Finding simple things, like names of columns, is currently very difficult or impossible. | |||
** Handle more corner cases to the ETL code as we stumble upon the last of the oddities in the structured logs. | |||
** Answer some real questions with this thing | |||
=== Implement the ability to normalize chunk durations in mochitest [ahal] === | === Implement the ability to normalize chunk durations in mochitest [ahal] === | ||
| Line 150: | Line 189: | ||
** "greening" of tests | ** "greening" of tests | ||
* '''progress since last update''': | * '''progress since last update''': | ||
** egl failures resolved with | ** egl failures resolved with customized kernel, but other frequent failures persist | ||
** will likely setup Android 4.3 emulator tests instead, since that environment seems much more stable | |||
=== Help Releng reduce test load [jmaher] === | === Help Releng reduce test load [jmaher] === | ||
| Line 201: | Line 241: | ||
= Holidays and Trips = | = Holidays and Trips = | ||
* mcote, jmaher, and jgriffin in Toronto Tuesday-Friday this week. | |||
= Misc = | = Misc = | ||