ReleaseEngineering/Maintenance: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(reconfig)
(reconfig)
Line 38: Line 38:
| style="background:#cccccc" | '''When'''
| style="background:#cccccc" | '''When'''
| style="background:#cccccc" | '''Bug #(s)''' - '''Description(s)'''
| style="background:#cccccc" | '''Bug #(s)''' - '''Description(s)'''
|-
| in production
| 2016-12-19 08:26 PT
|
<div style="border: thin grey solid; background-color: lightgrey; float: right; text-align: right; text-size: 80%; padding-left: 5px; padding-right: 5px;">
[https://bugzilla.mozilla.org/buglist.cgi?bug_id=1176784,1324290&query_format=advanced&order=bug_status%2Cbug_id&tweak=1 View list in Bugzilla]
</div>
[https://hg.mozilla.org/build/buildbot-configs buildbot-configs]
* {{bug|1176784}} - reconfigs should not rely on seta server - r=kmoir ([https://hg.mozilla.org/build/buildbot-configs/rev/c61b82cda2ed c61b82cda2ed])
* {{bug|1324290}} - Sign the comm-central win64 builds - r=rail ([https://hg.mozilla.org/build/buildbot-configs/rev/3ba60d3917ff 3ba60d3917ff])
* No bug - Update release config for Fennec-51.0b4-build1 ([https://hg.mozilla.org/build/buildbot-configs/rev/ab541569e70a ab541569e70a])
* No bug - Update release config for Fennec-50.0.2-build1 ([https://hg.mozilla.org/build/buildbot-configs/rev/4cf29de987a0 4cf29de987a0])
* No bug - Update release config for Fennec-51.0b5-build1 ([https://hg.mozilla.org/build/buildbot-configs/rev/1e0a5b9d2fd0 1e0a5b9d2fd0])
* No bug - Update release config for Thunderbird-45.5.1-build1 ([https://hg.mozilla.org/build/buildbot-configs/rev/952bae652661 952bae652661])
* No bug - Update release config for Thunderbird-45.5.1-build2 ([https://hg.mozilla.org/build/buildbot-configs/rev/5c8169fadf9d 5c8169fadf9d])
* No bug - updating seta data for 2016-12-01 ([https://hg.mozilla.org/build/buildbot-configs/rev/71bbb16bb232 71bbb16bb232])
* No bug - Update release config for Thunderbird-51.0b1-build2 ([https://hg.mozilla.org/build/buildbot-configs/rev/9fe6bfabf073 9fe6bfabf073])
* No bug - Update release config for Fennec-50.1.0-build1 ([https://hg.mozilla.org/build/buildbot-configs/rev/f6267b86bcfd f6267b86bcfd])
* No bug - Update release config for Fennec-50.1.0-build2 ([https://hg.mozilla.org/build/buildbot-configs/rev/a26d58ceca67 a26d58ceca67])
* No bug - Update release config for Thunderbird-51.0b1-build3 ([https://hg.mozilla.org/build/buildbot-configs/rev/228acd65b6b9 228acd65b6b9])
* No bug - Update release config for Thunderbird-51.0b1-build4 ([https://hg.mozilla.org/build/buildbot-configs/rev/07a7e88052fb 07a7e88052fb])
* No bug - Update release config for Fennec-51.0b7-build1 ([https://hg.mozilla.org/build/buildbot-configs/rev/f118db65e1d5 f118db65e1d5])
* No bug - Update release config for Thunderbird-45.6.0-build1 ([https://hg.mozilla.org/build/buildbot-configs/rev/e694dd5b252d e694dd5b252d])
|-
|-
| in production
| in production

Revision as of 16:26, 19 December 2016

This page is to track upcoming changes to any part of RelEng infrastructure; buildbot masters, slaves, ESX hosts, etc. This should allow us keep track of what we're doing in a downtime, and also what changes can be rolled out to production without needing a downtime. This should be helpful if we need to track what changes were made when troubleshooting problems.

ReleaseEngineering:BuildbotBestPractices describes how we manage changes to our master buildbots.

Relevant repositories

Non-releng repositories involved in the CI

Treeherder

  • Chief self-deploy recent history (needs VPN; more readable than raw logs, but is cleared if the Chief service is restarted)
  • [treeherderadm.private.scl3.mozilla.com/chief/treeherder.prod/logs/ Chief self-deploy logs archive] (needs VPN; use if the history page is empty)

Mozharness deployments

Merges to the production branch of the mozharness repository are live immediately. Please see that repo for push history.

Puppet deployments

Merges to the production branch of the puppet repository are automatically deployed by Puppet within a 1/2 hour. Please see that repo for push history.

Please note if you are applying a puppet change, unless you have good reason not to do so please merge it to production branch immediately. The reason for this is that we do not want puppet changes to "back up" on default, and then land en masse in production. It can be very difficult to untangle multiple puppet changes that arrive together.

The reason we have a production branch at all, is to allow contributions on default branch from a larger group of people, than those responsible for maintaining the infrastructure controlled by the puppet repo - in other words, a smaller group of people can merge to production, than the group of people that can land on default.

Reconfigs / Deployments

This page is updated by the person who does a reconfig on production systems. Please give accurate times, as we use this page to track down if reconfigs caused debug intermittent problems. (This includes config changes not requiring a "reconfig", such as mozharness changes.)

Outcome should be 'backed out' or 'In production' or some such. Reverse date order pretty please.

Older changes are archived in: 2015 | 2014 | 2013 | 2012 | 2011 | 2010 | 2009

Outcome When Bug #(s) - Description(s)
in production 2016-12-19 08:26 PT

buildbot-configs

  • bug 1176784 - reconfigs should not rely on seta server - r=kmoir (c61b82cda2ed)
  • bug 1324290 - Sign the comm-central win64 builds - r=rail (3ba60d3917ff)
  • No bug - Update release config for Fennec-51.0b4-build1 (ab541569e70a)
  • No bug - Update release config for Fennec-50.0.2-build1 (4cf29de987a0)
  • No bug - Update release config for Fennec-51.0b5-build1 (1e0a5b9d2fd0)
  • No bug - Update release config for Thunderbird-45.5.1-build1 (952bae652661)
  • No bug - Update release config for Thunderbird-45.5.1-build2 (5c8169fadf9d)
  • No bug - updating seta data for 2016-12-01 (71bbb16bb232)
  • No bug - Update release config for Thunderbird-51.0b1-build2 (9fe6bfabf073)
  • No bug - Update release config for Fennec-50.1.0-build1 (f6267b86bcfd)
  • No bug - Update release config for Fennec-50.1.0-build2 (a26d58ceca67)
  • No bug - Update release config for Thunderbird-51.0b1-build3 (228acd65b6b9)
  • No bug - Update release config for Thunderbird-51.0b1-build4 (07a7e88052fb)
  • No bug - Update release config for Fennec-51.0b7-build1 (f118db65e1d5)
  • No bug - Update release config for Thunderbird-45.6.0-build1 (e694dd5b252d)
in production 2016-11-21 09:32 PT

buildbot-configs

in production 2016-11-16 13:34 PT

buildbot-configs

  • No bug - Update release config for Thunderbird-51.0b1-build1 (c28939dcc685)
  • No bug - Update release config for Fennec-51.0b1-build2 (ff67fea64ddf)
  • No bug - Update release config for Fennec-51.0b1-build3 (4a8f906be2ba)

buildbotcustom

in production 2016-11-14 04:42 PT

buildbot-configs

in production 2016-11-07 13:00 PT

buildbotcustom

in production 2016-10-31 12:07 PT

buildbot-configs

in production 2016-10-27 11:38 PT

buildbot-configs

in production 2016-10-25 19:00 PT

buildbot-configs

in production 2016-10-25 09:24 PT

buildbot-configs

in production 2016-10-24 17:58 PT

buildbot-configs

in production 2016-10-21 06:56 PT

buildbot-configs

  • bug 1309894 - Use "poll_all_branches" in configs to explicitly set polling behaviour - r=catlee (f3d8389cb9bf)
  • No bug - Update release config for Fennec-49.0.2-build2 (08de42085241)

buildbotcustom

in production 2016-10-18 08:54 PT

buildbotcustom

in production 2016-10-13 10:14 PT

buildbot-configs

  • bug 1308789 - Set the Win64 builds to be the default try_by_default: True, because we want them to build by default, it's only the Win8 tests that are supposed to be try_by_default: False - r=catlee (a9bbcd872ad9)
  • bug 1309894 - Investigate shipping old release version off of jamun - r=Callek (6f25921852c3)
  • No bug - Update release config for Fennec-50.0b6-build1 (f1f2fc3fa06e)
in production 2016-10-10 07:54 PT

buildbot-configs

  • bug 1292656 - decommission another 10 t-snow-r4 machines - r=coop (02b3c211cc48)
  • bug 1300413 - Use unbuffered python in case we're missing output - r=Callek (1d22c76665b8)
  • No bug - Update release config for Thunderbird-45.4.0-build1 (a4d817817321)
  • No bug - Update release config for Thunderbird-50.0b1-build1 (ab02ce360da9)
  • No bug - Update release config for Fennec-50.0b4-build1 (f854aac09b67)
  • No bug - Update release config for Thunderbird-50.0b1-build2 (acc59373912b)
in production 2016-09-21 10:45 PT

buildbot-configs

  • No bug - Update release config for Fennec-50.0b1-build1 (d17d80076abd)
in production 2016-09-19 22:09 PT

buildbot-configs

in production 2016-09-19 06:37 PT

buildbot-configs

in production 2016-09-14 13:31 PT

buildbot-configs

in production 2016-09-06 10:59 PT

buildbot-configs

in production 2016-09-05 07:59 PT

buildbot-configs

in production 2016-08-26 11:33 PT

buildbot-configs

buildbotcustom

in production 2016-08-25 10:56 PT

buildbot-configs

  • bug 1298013 - Migrate remaining win 7 mochitest bc jobs to AWS - r=catlee (7c1e748101f6)
  • No bug - Update release config for Thunderbird-49.0b1-build4 (26a6a5f90516)
  • No bug - Update release config for Thunderbird-45.3.0-build1 (4b44a217b2ef)

buildbotcustom

in production 2016-08-23 12:23 PT

buildbot-configs

in production 2016-08-23 10:14 PT

buildbot-configs

in production 2016-08-19 05:55 PT

buildbot-configs

buildbotcustom

in production 2016-08-11 13:16 PT

buildbotcustom

in production 2016-08-10 13:50 PT

buildbot-configs

buildbotcustom

in production 2016-08-08 13:15 PT

buildbot-configs

in production 2016-08-04 09:11 PT

buildbot-configs

  • bug 1288043 - migrate win7 gpu jobs to gfx instances - r=rail (199eca90a0a9)
  • bug 1291044 - Use 10 chunks for wpt on win 7 vm debug to prevent timeouts - r=jmaher (c23e2a8603f5)
  • No bug - [PATCH] Bug 1271153 - Adjust automation for the version_display.txt new file on tb - r=rail (06d19bac0eff)
in production 2016-08-01 06:14 PT

buildbot-configs

in production 2016-07-28 04:56 PT

buildbotcustom

in production 2016-07-27 09:54 PT

buildbot-configs

  • No bug - Update release config for Fennec-48.0b11-build1 (9c1804c74761)
  • No bug - Update release config for Fennec-48.0-build1 (49e72329db05)
in production 2016-07-25 07:30 PT

buildbot-configs

in production 2016-07-22 09:39 PT

buildbot-configs

  • bug 1233953 - Intermittent Linux debug command timed out: 7200 seconds elapsed running - r=rail (2f53b99fd268)
  • bug 1282849 - disable fennec debug builds and tests in buildbot - r=kmoir (19f3d53951ae)
  • No bug - Update release config for Fennec-48.0b8-build1 (da62abc85e9d)
  • No bug - Update release config for Fennec-48.0b9-build1 (2b19aa948928)
  • No bug - Update release config for Fennec-48.0b10-build1 (ad5f8c375bcd)
in production 2016-07-12 16:12 PT

buildbot-configs

  • No bug - Update release config for Thunderbird-48.0b1-build2 (205cec6585a1)

buildbotcustom

in production 2016-07-07 11:34 PT

buildbot-configs

buildbotcustom

in production 2016-06-27 18:57 PT

buildbot-configs

in production 2016-06-07 13:55 PT

buildbotcustom

  • bug 1262760 - Use the objdir's virtualenv's python executable. -- fixes typo - r=rail (hopefully) post-landing (6f6a9824ca4d)
in production 2016-06-07 11:56 PT

buildbotcustom

in production 2016-06-02 08:45 PT

buildbot-configs

in production 2016-05-30 06:58 PT

buildbot-configs

in production 2016-05-29 14:55 PT

buildbot-configs

in production 2016-05-27 12:05 PT

buildbot-configs

in production 2016-05-27 09:48 PT

buildbot-configs

in production 2016-05-27 07:55 PT

buildbot-configs

in production 2016-05-27 05:59 PT

buildbotcustom

  • No bug - Backed out changeset 899b41a5e8e5, for xpcshellcommandline bustage (c543cd26f4ad)
in production 2016-05-27 05:51 PT

buildbot-configs

in production 2016-05-26 01:20 PT

buildbot-configs

buildbotcustom

in production 2016-05-24 10:56 PT

buildbot-configs

in production 2016-05-23 09:45 PT

buildbot-configs

in production 2016-05-20 08:37 PT

buildbot-configs

in production 2016-05-19 10:47 PT

buildbot-configs

in production 2016-05-17 09:46 PT

buildbot-configs

in production 2016-05-16 11:22 PT

buildbot-configs

buildbotcustom

in production 2016-05-16 08:15 PT

buildbot-configs

buildbotcustom

in production 2016-05-12 08:43 PT

buildbot-configs

buildbotcustom

in production 2016-05-12 02:03 PT

buildbot-configs

  • bug 1253341 - Run duplicate Talos jobs in AWS for Linux (reverted) - r=Callek (2302dc47cb21)
  • bug 1268542 - Disable Linux64 Debug builds and tests in buildbot - r=kmoir (cfe4c07f8161)
  • bug 1271550 - Add new 10.10.5 machines to releng configs (750ab5df5a06)
  • No bug - Update release config for Thunderbird-45.1.0-build2 (48e0db6d4d22)
  • No bug - Update release config for Thunderbird-45.1.0-build3 (c199b13b84a3)
  • No bug - Backed out changeset c199b13b84a3 to let TB 45.1.0 build3 to restart (a55a7e24bba1)
  • No bug - Update release config for Thunderbird-45.1.0-build3 (3a7c1d3dbace)
  • No bug - Update release config for Fennec-47.0b4-build1 (f7b8a43eed4f)

buildbotcustom

in production 2016-05-06 09:58 PT

buildbot-configs

buildbotcustom

in production 2016-05-02 13:55 PT

buildbotcustom

in production 2016-05-02 13:00 PT

buildbot-configs

buildbotcustom

in production 2016-04-27 10:34 PT

buildbot-configs

in production 2016-04-19 06:35 PT

buildbot-configs

buildbotcustom

in production 2016-04-19 05:32 PT

buildbotcustom

in production 2016-04-13 06:07 PT

buildbot-configs

buildbotcustom

in production 2016-04-04 13:40 PT

buildbot-configs

buildbotcustom

in production 2016-03-29 13:32 PT

buildbot-configs

buildbotcustom

in production 2016-03-21 17:41 PT

buildbotcustom

in production 2016-03-21 14:15 PT

buildbot-configs

buildbotcustom

in production 2016-03-18 12:37 PT

buildbotcustom

in production 2016-03-16 10:35 PT

buildbot-configs

  • bug 1252292 - Don't run Ru-e10s on debug builds - r=me (04c8fe62d3a4)
  • bug 1253341 - Enable talos on linux64 AWS machines on try - r=kmoir (2d6158d681e0)
  • bug 1257230 - Incorrect branding for 45.0.1esr build - r=jlund (b5163b4a7814)
  • No bug - Update release config for Fennec-46.0b2-build1 (216ac6505867)
  • No bug - Update release config for Firefox-38.7.1esr-build1 (2b0b8272dda2)
  • No bug - backouts release change for 38.7.1esr - rev: 2b0b8272dda2 (ea85259cf436)
  • No bug - merges backout 2b0b8272dda2 -> default (7b0276c224b4)
  • No bug - Update release config for Firefox-38.7.1esr-build1 (b735b52c0d5a)
  • No bug - Update release config for Firefox-45.0.1-build1 (b3267390eb7d)
  • No bug - Update release config for Firefox-45.0.1esr-build1 (1f7801999788)
  • No bug - Update release config for Fennec-45.0.1-build1 (e759c66d311c)
  • No bug - Backed out 2d6158d681e0 for suspicion of causing intermittent checkconfig (7c463958098d)

buildbotcustom

  • bug 1253341 - Allow running talos on AWS machines - r=kmoir (42184dbdf8f3)
  • No bug - Backed out changeset 42184dbdf8f3 for suspicion of causing intermittent checkconfig failureis (046c46eb099d)
in production 2016-03-15 07:43 PT

buildbot-configs

in production 2016-03-11 09:50 PT

buildbot-configs

buildbotcustom

in production 2016-03-10 06:17 PT

buildbot-configs

in production 2016-03-09 06:14 PT

buildbot-configs

in production 2016-03-07 19:31 PT

buildbot-configs

in production 2016-03-07 16:12 PT

buildbot-configs

in production 2016-03-07 05:36 PT

buildbot-configs

in production 2016-03-04 13:23 PT

buildbot-configs

in production 2016-03-04 12:26 PT

buildbot-configs

in production 2016-03-04 09:03 PT

buildbot-configs

in production 2016-03-04 07:06 PT

buildbot-configs

in production 2016-03-03 16:54 PT

buildbot-configs

in production 2016-03-02 13:51 PT

buildbot-configs

buildbotcustom

  • No bug - No Bug - shorten travis results notification to be less spammy - r=catlee (4b96f5cb0d0f)
in production 2016-03-02 06:25 PT

buildbot-configs

in production 2016-03-01 07:30 PT

buildbot-configs

in production 2016-02-29 08:10 PT

buildbotcustom

in production 2016-02-25 09:50 PT

buildbot-configs

buildbotcustom

in production 2016-02-24 13:45 PT

buildbot-configs

in production 2016-02-23 13:53 PT

buildbot-configs

in production 2016-02-22 13:17 PT

buildbot-configs

  • No bug - Update release config for Firefox-45.0b8-build1 (5f9a37958eaf)
  • No bug - Update release config for Firefox-45.0b8-build1 (5f9a37958eaf)

buildbotcustom

in production 2016-02-19 13:00 PT

buildbotcustom

in production 2016-02-19 07:03 PT

buildbot-configs

buildbotcustom

in production 2016-02-17 08:46 PT

buildbot-configs

  • bug 1246627 - Schedule e10s jsreftests on win7 patch by aselagea - r=jgriffin (cc37bdb15492)
  • No bug - revert Bug 1246627 - Schedule e10s jsreftests on win7 (71b9ccb8abf1)
  • No bug - Update release config for Fennec-45.0b6-build1 (69f84fb1f35b)
  • No bug - Update release config for Firefox-45.0b6-build1 (df4ed20df968)

buildbotcustom

in production 2016-02-14 08:43 PT

buildbot-configs

buildbotcustom

  • bug 1244510 - Add a try_parser alias for mochitest-e10s-(bc|browser-chrome) patch by aselagea - r=kmoir (c994ec64ba03)
in production 2016-02-12 07:43 PT

buildbot-configs

  • bug 1218589 - Nightly mozilla-central builds of graphene patch by aselagea - r=kmoir (f88d45c3fbe6)
  • bug 1229532 - remove hg bundle builder config - r=bhearsum (f0c9c5b5a7b8)
  • bug 1246627 - Schedule e10s jsreftests on win7 patch by aselagea - r=kmoir (66bc120b896c)
  • bug 787208 - In support of merging comm-central and mozilla-central together, configure cypress as a Thunderbird branch. r=rail - r=Fallen f=jcranmer (b92aaf935173)
  • No bug - Update release config for Firefox-38.6.1esr-build1 (33fe33d65478)
  • No bug - Update release config for Firefox-44.0.2-build3 (09cd8415adb9)
  • No bug - Update release config for Thunderbird-38.6.0-build1 (91cf6aa50268)
  • No bug - Update release config for Firefox-45.0b5-build1 (6376fd9567c4)

buildbotcustom

in production 2016-02-02 14:53 PT

buildbot-configs

  • bug 1242103 - Add browser-chrome-screenshots to m-c and other try - r=armenzg (63614b6564ee)
  • No bug - Update release config for Fennec-45.0b2-build1 (7a121e3abe60)
  • No bug - Update release config for Firefox-45.0b2-build1 (a70a9918a489)
  • No bug - Update release config for Thunderbird-45.0b1-build1 (39642e4c3ba4)
in production 2016-01-26 20:05 PT

buildbotcustom

buildbot-configs

in production 2016-01-26 18:05 PT

buildbotcustom

in production 2016-01-26 16:00 PT

buildbotcustom

buildbot-configs

in production 2016-01-25 20:47 PT

buildbotcustom

in production 2016-01-25 14:54 PT

buildbotcustom

in production 2016-01-25 05:50 PT

buildbot-configs

in production 2016-01-21 12:12 PT

buildbot-configs

in production 2016-01-21 10:56 PT

buildbot-configs

buildbotcustom

in production 2016-01-20 06:16 PT

buildbot-configs

  • No bug - Update release config for Fennec-44.0b11-build1 (370703910927)

buildbotcustom

in production 2016-01-19 07:36 PT

buildbot-configs

buildbotcustom

in production 2016-01-18 06:19 PT

buildbot-configs

mozharness

in production 2016-01-15 08:55 PT

buildbot-configs

  • No bug - Update release config for Fennec-44.0b9-build1 (15dc357fc5bd)
  • No bug - Update release config for Firefox-44.0b9-build1 (df3ee52930d6)

buildbotcustom

in production 2016-01-13 15:03 PT

buildbot-configs

buildbotcustom

in production 2016-01-12 09:04 PT

buildbot-configs

buildbotcustom

mozharness

in production 2016-01-11 08:10 PT

buildbot-configs

buildbotcustom

in production 2016-01-11 07:41 PT

buildbot-configs

buildbotcustom

mozharness

in production 2016-01-07 06:39 PT

buildbot-configs

buildbotcustom