MozillaWiki

ReleaseEngineering/BuildFaster: Difference between revisions

Page Discussion

ReleaseEngineering/BuildFaster (view source)

Revision as of 21:36, 8 November 2012

1,635 bytes removed ,  8 November 2012
→‎Current activities
 
(47 intermediate revisions by 8 users not shown)
Line 19: Line 19:


= Current activities =
= Current activities =
Ensure any bugs that are filed for this effort have [buildfaster:pN] in the whiteboard.  Where N is some number 1-5.  P1 is highest priority, P5 is the lowest.
[https://bugzilla.mozilla.org/buglist.cgi?list_id=743687&resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=buildfaster:p This is our list of '''current active bugs'''].
If you'd like to file a bug for something you'd like to see included in this effort, you can use [buildfaster:?] in the whiteboard. 
[https://bugzilla.mozilla.org/buglist.cgi?list_id=743751&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=buildfaster%3A%3F This is our list of '''proposed bugs'''].
The table below is for current tasks that are not tracked in bugzilla.
{| style="width: 100%" class="fullwidth-table sortable"
{| style="width: 100%" class="fullwidth-table sortable"
|-
|-
Line 26: Line 36:
| style="background:#DDD" | '''Status'''
| style="background:#DDD" | '''Status'''
|-
|-
| ??
| {{bug|772446}}
| Set up metrics to quantify our current end to end time for tracking
| Move all Linux builds to AWS
| catlee
| jhopkins
| {{StatusHealthy|status=In Design}}
| {{StatusHealthy|status=All Firefox builds are there now, just waiting on Thunderbird to catch-up.}}
|-
| {{bug|658313}}
| Make on-change builds into non PGO, only do PGO on an as available basis
| '''UNOWNED'''
| {{StatusAtRisk|status=Needs owner}}
|-
| ??
| Investigate if we can do 64 bit only 10.6 builds per-checkin (how many 10.5 failures are there?)
| '''UNOWNED'''
| {{StatusAtRisk|status=Needs Owner}}
|-
| {{bug|659328}}
| Determine which suites can be merged to amortize setup cost
| armenzg
| {{StatusHealthy|status=Has Patches}}
|-
| n/a
| Profile Test suites to find individual tests are higher than average time to complete (so devs can fix them)
| jmaher
| {{StatusHealthy|status=First analysis of mochitest-plain complete}}
|-
| {{bug|417044}}
| Perform Universal binary builds as single-pass
| :joey
| {{StatusHealthy|status=In Progress}}
|-
| {{bug|662833}}
| Split config/rules.mk into hierarchy of makefiles
| :joey
| {{StatusHealthy|status=Waiting on Review}}
|-
| {{bug|659222}}
| Branch prioritization broken for test jobs
| catlee
| {{StatusHealthy|status=In Progress}}
|-
| {{bug|661656}}
| Improving network between colos
| zandr
| {{StatusHealthy|status=In Progress}}
|-
| {{bug|586418}}
| Merge some test suites
| '''UNOWNED'''
| {{StatusBlocked|status=Cannot finish until {{bug|594415}} completed}}
|-
| {{bug|594415}}
| Change TBPL to report on individual test suites not buildbot jobs
| Swatinem
| {{StatusAtRisk|status=Patches complete, needs to be unstuck, ctalbert driving}}
|-
| {{bug|661649}}
| Deploy talos pageset to all slaves to avoid download/unpack
| '''UNOWNED'''
| {{StatusAtRisk|status=Needs Owner}}
|-
|-
| {{bug|561235}}
| {{bug|758624}}
| Use server to deal with getting stacks for crashes (no download of symbols onto slave)
| Purchase new machines for win/linux testers that are not tied to aggressive obsolescence schedule
| ted
| coop
| {{StatusHealthy|status=Needs next steps defined}}
| {{StatusBlocked|status=Underway. Various test OSes are being setup and validated.}}
|-
|-
| {{bug|561754}}
| {{bug|784913}}
| Create server for processing minidumps
| migrate linux xpcshell testsuite to AWS
| ted
| {{StatusHealthy|status=Complete?}}
|-
| {{bug|659118}}
| Reuse win7 64bit machines for supporting other OS's
| armenzg
| {{StatusHealthy|status=In Progress}}
|-
| {{bug|660124}}
| Turn off txul and ts in favor of their -paint varieties
| jmaher
| jmaher
| {{StatusBlocked|status=Blocked on reaching consensus, ctalbert helping drive}}
| {{StatusBlocked|status=Blocked on panda work}}
|-
| ??
| Use sdwilsh's notification bot to help devs more easily watch the tree
| sdwilsh?
| {{StatusHealthy|status=Needs next steps, ctalbert to follow up}}
|-
| {{bug|558180}}
| Check mozconfigs into source to avoid buildconfig checkout
| catlee
| {{StatusHealthy|status=In Progress}}
|-
| ??
| Profile mochitest plain
| '''UNOWNED'''
| {{StatusAtRisk|status=Needs Owner}}
|-
| n/a
| Get data on test run time for each of the moth individual tests
| ctalbert
| {{StatusHealthy|status=COMPLETE}}
|-
| {{bug|528231}}
| (loosely related) Get debug symbols for unit test reference platform into breakpad
| nthomas
| {{StatusBlocked|status=Blocked on nthomas's availability}}
|-
|-
| {{bug|571886}}
| {{bug|784913}}
| (loosely related) Create calendar tool to aid Firefox sheriffs
| migrate linux xpcshell testsuite to AWS
| harth and/or webdev
| '''unowned'''
| {{StatusBlocked|status=Blocked on harth's availability}}
| {{StatusBlocked|status=Needs an owner}}
|-
| ??
| Run debug builds on an "as available" basis once we hit capacity limits
| catlee
| {{StatusHealthy|status=Proposed/In design}}
|-
| ??
| Use a packaged, release build for httpd.js when running tests (improve httpd startup cost)
| ctalbert
| {{StatusHealthy|status=Proposed/Investigation}}
|-
| ??
| Purchase new windows and linux machines for test slaves that are not tied to aggressive obsolescence schedule
| zandr
| {{StatusBlocked|status=Underway, but issues getting machines from 3rd party provider}}
|}
|}


== Long Term Ideas ==
== Long Term Ideas ==
* {{bug|657738}} - Automatically determine when oranges happen and auto-star them (no owner/decision on steps yet)
* {{bug|657738}} - Automatically determine when oranges happen and auto-star them (no owner/decision on steps yet)
* {{bug|630534}} - (related) Host tree status outside of tinderbox
* Windows builds on AWS are still too slow, but we can revisit this regularly
** This would help us move away from tinderbox and would free up many of the constraints we currently operate under
* Cross-compile Mac builds on AWS
* Analyze using VMs for "dial-up" capacity
** after legal review, *yes* we think we can do this. Need to find a developer owner to give this a try.
** Would need engineering resources to debug oranges that occur only on vms
* {{bug|772579}} - Can we run unittests on AWS for Linux (and eventually Windows)?
** Would need to run some low numbers of vms on an ongoing basis so that we continue to ensure they are providing results we can trust
** We may need to solve {{bug|617763}} so that these machines can be kept up to date (whether they live in the build VPN or not, i.e. if we use EC2 or something like that...).  Either way we must address the problem that they could be out of date between one dial-in and another dial-in.
* Analyze methods of understanding and optimizing what we run on a per-checkin basis
* Analyze methods of understanding and optimizing what we run on a per-checkin basis
** Test code related to the current patch?
** Test code related to the current patch?
** Run full tests only every x pushes or y time (whichever occurs first).  Between those only run a set of tests related to the patch at hand. (Would need developer override and on-demand testing ability)
** Run full tests only every x pushes or y time (whichever occurs first).  Between those only run a set of tests related to the patch at hand. (Would need developer override and on-demand testing ability)
* less frequent debug builds
* use an optimized xpcshell + httpd.js to run debug tests (depends on <s>{{bug|669953}}</s>)


= Data we need =
= Data we need =
* Profiling on all major harnesses
* Profiling on all major harnesses
** Currently profiling is being done on mochitest harness and tracked [http://brasstacks.mozilla.com/testperf_dashboard/#/mochitest here]. Be sure to select either "opt" or "debug" builds, since choosing "any" is probably not what you want (profiling times differ greatly between the two).
* How many 10.5-only test failures have there been?  Do we need to run 10.5 tests on-checkin?
* How many 10.5-only test failures have there been?  Do we need to run 10.5 tests on-checkin?
* What is the test timing for the suites contained in the moth harness? (ctalbert owns)
* Where are we spending our time during build slave setup/teardown (catlee owns)
* Where are we spending our time during build slave setup/teardown (catlee owns)
* Gather data on specific long-running tests on mochitest & mochitest other
* Gather data on specific long-running tests on mochitest & mochitest other
Line 182: Line 87:
* Joey Armstrong (:joey)
* Joey Armstrong (:joey)
* Clint Talbert (:ctalbert)
* Clint Talbert (:ctalbert)
* Some as-yet undefined set of the A-team (probably Ted, Jmaher and Harth)
* Will Lachance (:wlach)
* Ted Mielczarek (:ted)
* Chris Cooper (:coop)


= Definitions =
= Definitions =
Line 192: Line 99:
= Proposals for consideration =
= Proposals for consideration =
Please add proposals here...
Please add proposals here...
= Meetings =
* [[ReleaseEngineering/BuildFaster/Meetings/2012-10-11|2012-10-11]]
* [[ReleaseEngineering/BuildFaster/Meetings/2012-06-28|2012-06-28]]
* [[ReleaseEngineering/BuildFaster/Meetings/2011-10-26|2011-10-26]]
* [[ReleaseEngineering/BuildFaster/Meetings/2011-09-28|2011-09-28]]
* [[ReleaseEngineering/BuildFaster/Meetings/2011-08-31|2011-08-31]]
* [[ReleaseEngineering/BuildFaster/Meetings/2011-08-17|2011-08-17]]
* [[ReleaseEngineering/BuildFaster/Meetings/2011-08-03|2011-08-03]]
* [[ReleaseEngineering/BuildFaster/Meetings/2011-07-20|2011-07-20]]
* [[ReleaseEngineering/BuildFaster/Meetings/2011-07-07|2011-07-07]]
ChrisCooper
canmove, Confirmed users
2,850

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/319140...486184"