SeaMonkey/StatusMeetings/RelEng

From MozillaWiki
Jump to: navigation, search

SeaMonkey RelEng Status and History Page

This page is to keep track of the status history of the SeaMonkey buildbotmaster and slaves' status.

  • Doubling effort in getting the infra on Azure up.
  • Various Bugs open for building.
    • We have only one macOS builder on the old infrastructure left which seems to fail frequently now.
    • Linux64 Nightly: bug 1275215 is still open but Linux is building most of the time.
    • Linux64 build: bug 1275218 is still open but Linux is building most of the time.
    • Windows building is done on the loaner.
  • [13th May 2018]
    • Tests have been re-enabled and running but have issues.
      • All tests are Red.
      • All trees: macOS builds turned off to preserve the last builder -3 for releases.
    • [comm-central]:
      • All Platforms: Busted and if they work for a day Mozilla will take care of this.
        • Installer and local build fixes reviewed and pushed after the meeting.
        • Windows builds need to switch to mozilla as topsourcedir. Linux and macOS builds might still might work with comm-central as top but deprecated and untested. Buildbot configuration needs to be adapted for this.
      • Do not expect to use the 2.59 release yet other than for checking if it starts. It is fundamentally broken by Mozilla source and api removals.
        • The installer still works...
      • See bug 1452448 for current comm-central breakages.
    • [comm-beta]:
      • All Platforms: See individual platform.
        • Linux*: Orange bug 1415803
        • macOS: builds turned off to preserve -3.
        • Win32: Installer and local builds busted. We do not plan to fix this for 2.58.
      • Do not use the 2.58 release.
      • Windows builds need to switch to mozilla as topsourcedir. Linux and macOS builds might still might work with comm-central as top but now deprecated and untested. Buildbot configuration needs to be adapted for this.
      • See bug 1452448 for current comm-central breakages.
    • [comm-release]:
      • All Platforms: See individual platform.
        • Linux*: Orange bug 1415803
        • macOS: builds turned off to preserve -3.
        • Win32: Dark Red with purple spots.
        • Do not use any 2.57 release from this repo.
        • comm-release branch will not be fixed for 2.57. Instead use the esr60 source.
      • See bug 1379370 for current 2.53 breakages. Tree is at 2.57 already.
    • [comm-esr60]:
      • All Platforms: See individual platform.
        • Linux*: No automation yet.
        • macOS: No automation yet.
        • Win32: No automation yet.
      • Windows builds need to switch to mozilla as topsourcedir. Linux and macOS builds might still might work with comm-central as top but now deprecated and untested.
      • Sidebar, Help and Mail/News are broken because of XUL template removals. See bug 464710.
      • Do not expect to use the 2.57 release yet other than for testing.
      • See bug 1433370 for current 2.57 breakages.
    • [comm-esr52]:
      • All Platforms: See individual platform.
        • Linux*: Orange bug 1415803
        • macOS: builds turned off to preserve -3.
        • Win32: N.N
      • See bug 1313304 for current breakages.
      • Builds from this repo are the only ones fully usable right now.

[8th April 2018]

  • Doubling effort in getting the infra on Azure up.
  • Various Bugs open for building.
    • We have only one OSX64 builder left which seems to fail frequently now.
    • Linux64 Nightly: bug 1275215 is still open but Linux is building most of the time.
    • Linux64 build: bug 1275218 is still open but Linux is building most of the time.
    • Windows building is done on the loaner.
  • [8th April 2018]
    • Tests have been re-enabled and running but have issues.
      • All tests are Red.
    • All trees:
    • [comm-central]:
      • All Platforms: Busted.
        • No GCC 6.4 tooltool packages [TB is using TaskCluster infra so they get the gcc toolchain for free. We still depend on the 6.4 tooltool package. Have yet to ping someone to get it done. None of our systems can build the 6.4 tooltool package.]
        • LLVM_CONFIG not set. bug 1450257
      • Do not expect to use the 2.58 release yet other than for brief testing. It is fundamentally broken by Mozilla source and api removals.
      • Sidebar and Mail/News are broken because of XUL template removals. See bug 464710.
      • Bookmark Manager broken because of places sync api removal. See bug 1378089.
      • Dumps without the fixes for the Bookmark Manager and friends.
      • See bug 1452448 for current comm-central breakages.
    • [comm-beta]:
      • Linux*: Orange bug 1415803
      • OSX64: *backlogged*
      • Win32: Green
      • Sidebar and Mail/News are broken because of XUL template removals. See bug 464710.
      • Bookmark Manager broken because of places sync api removal. See bug 1378089.
      • Dumps without the fixes for the Bookmark Manager and friends.
      • Do not expect to use the 2.57 release yet other than for brief testing.
      • See bug 1433370 for current 2.57 breakages.
    • [comm-release]:
      • Linux*: Orange bug 1415803
      • OSX64: backlogged
      • Win32: Green
      • Do not expect to use the release yet other than for brief testing.
      • See bug 1379370 for current 2.53 breakages. Tree is at 2.56 already.
    • [comm-esr52]:
  • [18th February 2018]
    • Tests have been re-enabled and running but have issues.
      • All tests are Red.
    • All trees:
    • [comm-central]:
      • All Platforms: Green but
        • Windows32: bug 1442554 VS2015 builds affected. Patch got r+. Should be fixed tomorrow.
      • DOMi, cZ and debugQA extensions were temporarily disabled to get basic l10n working in bug 1231349.
      • Do not expect to use the 2.57 release yet other than for brief testing. It is fundamentally broken by Mozilla source and api removals.
      • Sidebar and Mail/News are broken because of XUL template removals. See bug 464710.
      • Bookmark Manager broken because of places sync api removal. See bug 1378089.
      • See bug 1433370 for current 2.57 breakages.
    • [comm-beta]:
      • Linux*: Orange bug 1415803
      • OSX64: *backlogged*
      • Win32: Green
      • Do not expect to use the 2.56 release yet other than for brief testing.
    • [comm-release]:
      • Linux*: Orange bug 1415803
      • OSX64: backlogged
      • Win32: Green
      • Do not expect to use the 2.55 release yet other than for brief testing.
      • See bug 1379370 for current 2.53 breakages. Tree is at 2.55 already.
    • [comm-esr52]:
  • No 2.53 in Bugzilla yet! (fixed)
  • Various Bugs open for building.
    • OSX64: busted due to bug 1260362. Yasm 1.2 needs to be installed. (Ratty: Depends on bug 1257904. How to get traction on this?)
    • Linux64 Nightly: bug 1275215 is still open but Linux is building most of the time.
    • Linux64 build: bug 1275218 is still open but Linux is building most of the time.
    • Windows building is done on the loaner.
    • Various l10n and other bugs bug 1293943 which might also impact the release schedule.
      • CZ langpacks seem to be disabled by default, so that needs to be sorted.
      • DOMi still needs to be sorted.
      • Lightning l10n upload still needs to be sorted.
      • bug 1372167 might help with the l10n issues.
  • Various Bugs open for building.
    • We have only one OSX64 builder left.
    • Linux64 Nightly: bug 1275215 is still open but Linux is building most of the time.
    • Linux64 build: bug 1275218 is still open but Linux is building most of the time.
    • Windows building is done on the loaner.
  • [26th November 2017]
    • Tests have been re-enabled and running but have issues. bug 1295826 tracks the fixes for the Testing Infrastructure.
      • All tests are red.
    • All trees:
      • Problems with Mercurial bug 1415803 make check orange w/ hg unknown command "files".
    • [comm-central]:
      • Linux*: Orange bug 1415803.
      • OSX64: green but the last builder is unable to run tests. Uses SDK 10.11.
      • Win32: green but the loaner can not run the tests.
      • See bug 1420718 for current breakages.
      • DOMi, cZ and debugQA extensions were temporarily disabled to get basic l10n working in bug 1231349.
      • Do not expect to use the 2.56 release yet other than for brief testing.
    • [comm-beta]:
      • Linux*: Orange bug 1415803
      • OSX64: n.n
      • Win32: green
      • See bug 1394144 for current breakages.
      • Do not expect to use the 2.55 release yet other than for brief testing.
    • [comm-release]:
      • Linux*: Orange bug 1415803
      • OSX64: N.N
      • Win32: green
      • See bug 1379370 for current 2.53 breakages. Tree is at 2.54 already.
    • [comm-esr52]:
  • [25th June 2017]
    • Tests have been re-enabled to enable us to find out what's busting them. bug 1295826 tracks the fixes for the Testing Infrastructure.
      • All tests are red.
    • All trees:
      • Problems with Mercurial bug 1368956 make check orange w/ hg unknown command "files".
    • [comm-central]:
    • [comm-beta]:
    • [comm-release]:
    • [comm-esr52]:
  • [18th June 2017]
    • Tests have been re-enabled to enable us to find out what's busting them. bug 1295826 tracks the fixes for the Testing Infrastructure.
      • All tests are red.
    • All trees:
      • Problems with Mercurial bug 1368956 make check orange w/ hg unknown command "files".
    • [comm-central]:
      • Linux*: green
      • OSX64: *busted* Due to mercurial issues. Tracked in bug 1362848.
      • Win32: green
      • See bug 1345770 for current breakages. No new ones for 2.53 yet.
    • [comm-beta]:
      • Linux*: green
      • OSX64: *busted*
      • Win32:green
      • See bug 1345770 for current breakages.
    • [comm-release]:
      • Linux*: green
      • OSX64: *busted*
      • Win32: green
      • See bug 1334779 for current breakages.
    • [comm-esr52]:
  • [7th May 2017]
    • Tests have been re-enabled to enable us to find out what's busting them. bug 1295826 tracks the fixes for the Testing Infrastructure.
      • All tests are red.
    • All trees:
      • bug 1308923 might cause a serious problem with 2.48 and 2.49. Patch in bug 1364977 fixes it and needs to go in all trees when reviewed.
    • [comm-central]:
      • Linux*: green
      • OSX64: *busted* Due to mercurial issues. Tracked in bug 1362848. Might be fixed now after the meeting.
      • Win32: busted. Been working on the puppet issue that I've missed the builds.
      • Builds are unusable but patches are up for all serious issues. See bug 1345770 for current breakages.
      • bug 1363036 for mozilla-central needs to be fixed. Crashes SeaMonkey on tab close.
    • [comm-beta]:
      • Linux*: green
      • OSX64: *busted* Due to mercurial issues. Tracked in bug 1362848. Might be fixed now after the meeting.
      • Win32: busted. Been working on the puppet issue that I've missed the builds.
      • See bug 1334779 for current breakages.
    • [comm-release]:
      • Linux*: green
      • OSX64: *busted* Due to mercurial issues. Tracked in bug 1362848. Might be fixed now after the meeting.
      • Win32: busted. Been working on the puppet issue that I've missed the builds.
      • See bug 1319091 for current breakages.
    • [comm-esr52]:
  • [23rd April 2017]
    • [comm-central]:
      • OSX64: green/orange
      • Win32: busted. Been working on the puppet issue that I've missed the builds.
      • See bug 1345770 for current breakages.
    • [comm-aurora]:
    • [comm-beta]:
      • OSX64: green/orange
      • Win32: busted. Been working on the puppet issue that I've missed the builds.
      • See bug 1334779 for current breakages.
    • [comm-release]:
      • OSX64: green/orange
      • Win32: busted. Been working on the puppet issue that I've missed the builds.
      • See bug 1319091 for current breakages.
    • [comm-esr52]:
      • No automation. [Filed bug 1352820]
      • Was broken by uplift of bug 1299500 for SeaMonkey only. Comment from libertyernie in bug 1353765 indicated that it fixed itself (Might have to do with unified sources).
      • See bug 1313304 for current breakages.
  • [2nd April 2017]
    • Tests have been re-enabled to enable us to find out what's busting them. bug 1295826 tracks the fixes for the Testing Infrastructure.
      • All tests are red.
    • [comm-central]:
      • All platforms: green
      • See bug 1345770 for current breakages.
      • bug 1344411 - 'Lightweight themes are not supported anymore in SeaMonkey' needs a helping hand. If we just port the Firefox build the titlebar will be overwritten by some themes.
    • [comm-aurora]: DISCONTINUED — "Project Dawn" bug 1354818
      • Linux*: Green
      • OSX64: Orange/Green
    • [comm-beta]
      • Pretty much Green/Orange.
      • See bug 1334779 for current breakages.
    • [comm-release]:
      • Pretty much Green/Orange.
      • See bug 1319091 for current breakages.
    • [comm-esr52]:
      • No automation. [Filed bug 1352820]
      • Was broken by uplift of bug 1299500 for SeaMonkey only. Comment from libertyernie in bug 1353765 indicated that it fixed itself (Might have to do with unified sources).
      • See bug 1313304 for current breakages.
  • [5th March 2017]
    • Tests have been re-enabled to enable us to find out what's busting them. bug 1295826 tracks the fixes for the Testing Infrastructure.
      • All tests are red.
    • [comm-central]:
      • All platforms: green
      • See bug 1345770 for current breakages.
      • bug 1344411 - 'Lightweight themes are not supported anymore in SeaMonkey' needs a helping hand. If we just port the Firefox build the titlebar will be overwritten by some themes.
    • [comm-aurora]:
      • Linux*: Green
      • OSX64: Orange/Green
      • See bug 1334779 for current breakages.
    • [comm-beta]
      • Pretty much Green/Orange.
      • See bug 1319091 for current breakages.
    • [comm-release]:
      • Pretty much Green/Orange.
      • See bug 1313304 for current breakages.
    • [comm-esr52]:
      • No automation.
      • Broken by uplift of bug 1299500 which also affects Thunderbird and Firefox.
  • [21st February 2017]
    • Tests have been re-enabled to enable us to find out what's busting them. bug 1295826 tracks the fixes for the Testing Infrastructure.
      • All tests are red.
    • [comm-central]:
    • [comm-aurora]:
      • Linux*: Green
      • OSX64: Orange/Green
      • See bug 1319091 for current breakages.
    • [comm-beta]
      • Pretty much Green/Orange.
      • See bug 1313304 for current breakages.
    • [comm-release]:
      • Green/Orange.
  • [7th February 2017]
    • Tests have been re-enabled to enable us to find out what's busting them. bug 1295826 tracks the fixes for the Testing Infrastructure.
      • All tests are red.
    • [comm-central]:
      • All Platforms: Green (though OSX64 is backlogged, but it should be green).
      • See bug 1334779 for current breakages.
    • [comm-aurora]:
      • Linux*: Green
      • OSX64: Orange/Green
      • See bug 1319091 for current breakages.
    • [comm-beta]
      • Pretty much Green/Orange.
      • See bug 1313304 for current breakages.
    • [comm-release]:
      • Green/Orange.
  • [10th January 2016]
    • Tests have been re-enabled to enable us to find out what's busting them. bug 1295826 tracks the fixes for the Testing Infrastructure.
      • All tests are red.
    • [comm-central]:
      • All Platforms: Green (though OSX64 is backlogged, but it should be green).
      • See bug 1319091 for current breakages.
    • [comm-aurora]:
      • Linux*: Green
      • OSX64: Orange/Green
      • See bug 1313304 for current breakages.
    • [comm-beta]
      • Pretty much Green/Orange.
    • [comm-release]:
      • Green/Orange.
  • [6th December 2016]
    • All nightlies are currently running(bug 1299078 -post-landed-review.)
    • [comm-central]:
      • Linux*: Green
      • OSX64: Orange/Green
      • See bug 1319091 for current breakages.
      • No longer able to drop a url into the browser. bug 1319212
    • [comm-aurora]:
      • Linux*: Green
      • OSX64: Orange/Green
      • See bug 1313304 for current breakages.
    • [comm-beta]
      • Pretty much Green/Orange.
    • [comm-release]:
      • Green/Orange.
  • [22nd November 2016]
    • All nightlies are currently running(bug 1299078 -post-landed-review.)
    • [comm-central]:
      • All Platforms: "Crimson red" due to the XPCOM bug. bug 1318735
      • See bug 1319091 for current breakages.
      • No longer able to drop a url into the browser. bug 1319212
      • Sessionstore problem because jarprefix was removed.bug 1319114
    • [comm-aurora]:
      • Linux*: Green
      • OSX64: Orange/Green
      • See bug 1313304 for current breakages.
    • [comm-beta]
      • Pretty much green/orange.
    • [comm-release]:
      • Green/Orange.
  • [8th November 2016]
    • All nightlies are currently running(bug 1299078 -post-landed-review.)
    • [comm-central]:
      • Linux*: Light red
      • OSX64: Carmine red
      • Windows: Light red dotted on crimson red.
      • See bug 1319091 for current breakages.
      • No longer able to drop a url into the browser (no bug yet).
      • Sessionstore problem because jarprefix was removed (no bug yet).
    • [comm-aurora]:
      • Linux*: Green
      • OSX64: Orange/Green
      • See bug 1313304 for current breakages.
    • [comm-beta]
      • Pretty much green/orange.
    • [comm-release]:
      • Green/Orange.
  • [25th October 2016]
    • All nightlies are currently running(bug 1299078 -post-landed-review.)
    • [comm-central]:
      • All Platforms: Green osx seems to be broken. Windows too.
      • Autocomplete is broken See bug 1300547 and frg status.
      • No longer able to drop a url into the browser (no bug yet).
      • See bug 1313304 for current breakages.
    • [comm-aurora]:
      • Linux*: Green
      • OSX64: Orange/Green
    • [comm-beta]
      • Pretty much green/orange.
    • [comm-release]:
      • Green/Orange.
  • [11th October 2016]
    • All nightlies are currently running(bug 1299078 -post-landed-review.)
    • [comm-central]:
      • All Platforms: Busted. (bug 1309126 and probably others..)
    • [comm-aurora]:
      • Linux*: Green
      • OSX64: Orange/Green
    • [comm-beta]
      • Pretty much green/orange.
    • [comm-release]:
      • Green/Orange.
  • [27th September 2016]
    • All nightlies are currently running (but patch is manually applied to master) (bug 1299078)
    • [comm-central]:
      • Linux*: Green
      • Windows: Busted by bug 1305658.
      • OSX64: Orange.
    • [comm-aurora]:
    • [comm-beta]
      • Linux32 Build: Busted. possibly bug 1293928.. (retriggered... busted on -5... busted on -6, but ok with -10. So this could be a system issue.)
      • Others: Pretty much green/orange.
    • [comm-release]:
      • Green/Orange.
  • [13th September 2016]
    • All nightlies are currently running (but patch is manually applied to master) (bug 1299078)
    • [comm-central]:
      • Linux*/Windows: Green
      • OSX64: Orange.
    • [comm-aurora]:
    • [comm-beta]
      • Pretty much green/orange.
    • [comm-release]:
      • Bustage due to c-b not having yet been merged.
  • [30th August 2016]
    • All nightlies are currently MIA. (bug 1299078 - patch manually applied to master.)
    • [comm-central]:
      • All trunk builds are broken due to mozilla-central bug 956338. No quick fix is possible but Honza Bambas (:mayhemer) is willing to help us. Fixes will be done in comm-central bug 1299116
      • Linux*: Green
      • OSX64: Orange.
    • [comm-aurora]:
    • [comm-beta, comm-release]:
      • pretty much Green/orange.
  • [16th August 2016]
      • [comm-central]:
        • All en-US platforms, even Windows, have been built successfully at least once, and most of them regularly, since the previous meeting. At the moment the most hiccups happen on the Mac.
        • Post-merge nightlies are being published for L32 and L64. Otherwise W32(31-Jul), Mac(01-Aug) just before the merge.
          • OSX64: Builds are backlogged; but currently busted.
      • [comm-aurora]:
        • Only Mac builds are missing. Latest one from 30-Jul.
          • (bug 1295399 but it could be a dupe of bug 1293928, which fixed c-c; but TB's related patch was pushed in April, so we were pretty much sitting on this for two months and since OSX64 slaves have been backlogged for so long.. not surprised this was missed.)
      • [comm-beta]:
        • No 2.46 nightlies yet. 2.46 tinderbox-builds exist for L32, L64 and Mac. Also 2.45 (11-Aug) for W32 (done only on request on the loaner).
      • [comm-release]:
        • 2.45 tinderbox-builds exist for L32 and L64, Mac and for Windows (12-Aug).
    • [3rd August 2016]
      • [comm-central]:
        • All en-US platforms, even Windows, have been built successfully at least once, and most of them regularly, since the previous meeting. At the moment the most hiccups happen on the Mac.
        • Post-merge nightlies are being published for L32 and L64. Otherwise W32(31-Jul), Mac(01-Aug) just before the merge.
      • [comm-aurora]:
        • Only Mac builds are missing. Latest one from 30-Jul.
      • [comm-beta]:
        • No 2.46 nightlies yet. 2.46 tinderbox-builds exist for L32, L64 and Mac, 2.45(29-Jul) for W32 (done only on request on the loaner).
      • [comm-release]:
        • 2.45 tinderbox-builds exist for L32 and L64, 2.44(24-Jul) for Mac, nothing for Windows.
  • [20th June 2016]
    • [comm-central]
      • After merge day, "tinderbox-builds" of 2.47a1 were still being published, but no more so after build ID 20160616123604 for Linux64 and "Linux" (32), and 20160616000806 for "macosx64" (presumably 32/64 "universal" builds). Still nothing on win32 for 2.46a1 and 2.47a1 for other reasons.
    • [comm-aurora]
      • Still no 2.46a2 nightlies. Tinderbox-builds are produced irregularly: for both Linuxes there were none on June 19 but there are on the 20th; Mac has the opposite condition.
  • [7th June 2016]
    • [comm-central & comm-aurora]
      • Yesterday was merge day. The trunk is thus now at SeaMonkey 2.47a1 (Gecko 50.0a1).
      • Currently no nightly builds show up. ewong checks what is wrong.
      • Location bar and Back/Forward buttons broken because of bug 1272401. Ratty is reviewing the patch.
      • bug 1270848 - Back and Forward buttons unavailable may be unrelated to bug 1272401.
    • [comm-beta]
  • [26th April 2016]
    • [*All Trees*]
    • [comm-central]
      • OSX64: busted due to bug 1260362. Yasm 1.2 needs to be installed.
    • [comm-aurora]
      • OSX64: busted due to bug 1260362.
      • Linux x86 and x64 ok.
    • [comm-beta]
    • [comm-release]
  • [12th April 2016]
    • [*All Trees*]
      • Aside for the following specific tree bustages, we are generally busted for all trees due to the fact that our rsa ssh key needs to be changed (currently waiting), so uploading to archive.mo is busted (due to our old key being revoked/removed). Callek is working on this.
    • [comm-central]
      • Linux*: busted because we need to update our GCC to version 4.8.5 this will be fixed in bug 1260099 and bug 1190773.
      • OSX64: busted due to bug 1260362.
    • [comm-aurora, comm-beta]
      • OSX64: busted due to bug 1260362.
  • [29th March 2016]
    • [comm-central]
    • [comm-aurora, comm-beta]
  • [15th March 2016]
    • [comm-central, comm-aurora and comm-beta]
      • Busted due to a host of bugs. [:ewong : currently busy fixing the mess I made with the 2.40 updating to check.]
  • [2nd February 2016]
    • [Tonymec] Building has stopped since 9 January. No tinderbox builds either. [frg] c-c would be clear to me. c-a pulls the wrong Chatzilla and Inspector again. Not sure if this is the cause. See bug 1167346. [IanN] RattyAway has mentioned looking at Jenkins.
  • [19th January 2016]
    • [comm-central]
  • [8th December 2015]
    • No Update. Ewong is hospitalized. Get well soon!
    • Re: Tests. Still broken.
      • Currently bug 1209378 is in progress to fix the perma-brokeness of our testing infrastructure; but it is 'hard'.
  • [24th November 2015]
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see blog for more information).
      • L10n repacks are broken on aurora and central because the patch from bug 1210791 hasn't been reviewed yet.
    • comm-central:
      • Linux*: Green
        • (patches from bug 1187222 were post-land-review pushed.)
      • OSX64: Current status is that it is 'busted'; but this is only because of the backlog. (87 pending build jobs)
        • Possibly busted due to bug 1223633, but will need to get past the busted compile step.
    • comm-aurora:
      • Mostly green.
      • OSX64 is very backlogged; But busted because of bug 1223633. Even if not busted, it'd be orange in some parts but that's due to the firefox.dmg issue bug 958812.
    • comm-beta & comm-release:
  • [10th November 2015]
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see blog for more information).
      • L10n repacks are broken on aurora and central because the patch from bug 1210791 hasn't been reviewed yet.
    • comm-central:
    • comm-aurora:
      • Mostly green. OSX64 orange in some parts but that's due to the firefox.dmg issue bug 958812.
    • comm-beta & comm-release:
  • [27th October 2015]
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see blog for more information).
    • comm-central:
    • comm-aurora:
      • Linux32-debug : busted due to bug 1204162.
        • temporary patch was pushed to c-c so it's green there; but, needs to either be pushed to comm-aurora and comm-beta, or the bug fixed. (depends on bug 1203803).
        • IanN has given approval for the temporary fix to be pushed to comm-aurora and comm-beta.comm-aurora is merging to comm-beta next Monday. If ewong times it right he can push it tomorrow and then we won't need to push it to comm-beta.
      • OSX64: Busted due to bug 1208852 and bug 1218160.
    • comm-beta & comm-release:
      • OSX64: Orange: due to bug 958812 (This has always been orange since a long time ago - unfortunately, not in a galaxy far far away.)
  • [29th September 2015]
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see blog for more information).
    • comm-central to comm-beta:
    • comm-aurora to comm-beta:
      • Linux32-debug : busted due to bug 1204162.
        • temporary patch was pushed to c-c so it's green there; but, needs to either be pushed to c-a and c-b, or the bug fixed. (depends on bug 1203803).
    • comm-release:
      • OSX64: Orange due to bug 958812. (This has always been orange since a long time ago.)
  • [18th August 2015]
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see blog for more information).
    • comm-central & comm-aurora:
  • [1st September 2015]
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see blog for more information).
    • comm-central & comm-aurora:
  • [4th August 2015]
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see blog for more information).
    • comm-central:
  • [21th July 2015]
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see blog for more information).
    • comm-central:
      • All platforms: Busted due to bug 1185583. Patch is awaiting review.
  • [7th July 2015]
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see blog for more information).
    • comm-aurora:
      • All platforms: Busted due to bug 1167346, but fix has been already pushed.
  • [23th June 2015]
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see Release Train section).
  • [9th June 2015]
    • All repos:
      • All repos closed due to Bug 1172750 - db corruption - Fix ETA sometime today.
    • All trees:
      • Windows platform still busted due to needing Win2008R2 installed.
      • We have a loaner which ewong is working on (see Release Train section).
  • [28th April 2015]
    • Windows nightly, aurora and beta builds are broken. Will be fixed by upgrading to Windows Server 2K8 and Visual Studio 2013 (bug 1114876).
    • All Trees
      • OSX64:
        • Build - Orange : bug 958812 - References of firefox.dmg hard-coded in Mozharness tests.
    • [comm-central]:
      • All Nightlies are red due to bug 1155013 as we now require the use of a Socorro token. (Getting the token is tracked by bug 1155011). Currently waiting on what user account will be used for the tokens. rhelmer to advise ewong on this. Also SeaMonkey will need to switch the build process to the new way of symbol uploading.
  • [14th April 2015]
    • Windows nightly, aurora and beta builds are broken. Will be fixed by upgrading to Windows Server 2K8 and Visual Studio 2013 (bug 1114876).
    • All Trees
      • OSX64:
        • Build - Orange : bug 958812 - References of firefox.dmg hard-coded in Mozharness tests.
    • [comm-central]:
      • All Nightlies are red (as of this writing) due to bug 1153187 but they'll turn green/orange when they run next.
    • [comm-aurora]:
      • OSX64 build: red due to a glitch in the network. (could not checkout mozilla-aurora, just retriggered the last build.)
  • [31st March 2015]
    • Windows nightly, aurora and beta builds are broken. Will be fixed by upgrading to Windows Server 2K8 and Visual Studio 2013 (bug 1114876).
    • Linux* slaves are not 'connected' with the master due to 1147835.
    • All Trees
      • OSX64:
        • Build - Orange : bug 958812 - References of firefox.dmg hard-coded in Mozharness tests.
  • [17th March 2015]
    • Windows nightly, aurora and beta builds are broken. Will be fixed by upgrading to Windows Server 2K8 and Visual Studio 2013 (bug 1114876).
    • All Trees
      • OSX64:
        • Build - Orange : bug 958812 - References of firefox.dmg hard-coded in Mozharness tests.
    • [comm-central]
      • Linux*: l10n nightly and l10n dep are running again (patch applied manually to buildbotcustom code)
  • [2nd March 2015]
    • Windows nightly, aurora and beta builds are broken. Will be fixed by upgrading to Windows Server 2K8 and Visual Studio 2013 (bug 1114876).
    • All Trees (Not 100% sure on c-b since it's busted right now)
      • OSX64:
        • Build - Orange : bug 958812 - References of firefox.dmg hard-coded in Mozharness tests.
        • Debug - Red : bug 1137532 - Need to update clang to a newer version.
    • [comm-beta]
      • All Platforms:
        • Patch from bug 1128291 was pushed to DOM-inspector repo, but it only applies (atm) to c-c and c-a. Both c-b and c-r shouldn't use DOM_LATEST_RELEASE tag. The patch from bug 1137110 was pushed to c-b to fix this on c-b. c-r doesn't need fixing as it uses the

right tag.

  • [3rd February 2015]
    • Windows nightly and aurora builds broken. Will be fixed by upgrading to Windows Server 2K8 and Visual Studio 2013 (bug 1114876).
    • [comm-release]
      • Windows builds broken due to infrastructure/network hiccup.
  • [20th January 2015]
  • [25th November 2014]
    • Infrastructure issues
      • linux64 systems are still plagued by bug 1058385. Regular AWOLing. Currently 2 slaves are up.(-5 and -6)
      • NB: Currently the miscellaneous patches to fix the RelEng infra (tooltool et.al) are all applied to the master directly. Once reviewed, the patches will be pushed to their corresponding repositories and the master will be updated properly.
      • NB2: None of the test bustages are listed. Priority is currently set to get the builds running properly.
      • On mozilla-central a win64 change has landed recently and switching to VS2013. SeaMonkey Windows builders are running Vista and VS 2013 has a minimum requirement of Windows 7.
    • [comm-central]
      • WinNT busted : bug 1094097 - fallout from bug 1076020. Have patch. Needs landing on the mozilla/angle github repository.
      • l10n nightly/dep builds busted : bug 1073327 Fix waiting review.
      • blocklist update bustage : bug 1074572 Fix waiting review.
    • [comm-aurora]
  • [11th November 2014]
    • Infrastructure issues
      • linux64 systems are still plagued by bug 1058385. Regular AWOLing. Currently 2 slaves are up.(-5 and -6)
      • NB: Currently the miscellaneous patches to fix the RelEng infra (tooltool et.al) are all applied to the master directly. Once reviewed, the patches will be pushed to their corresponding repositories and the master will be updated properly.
      • NB2: None of the test bustages are listed. Priority is currently set to get the builds running properly.
      • On mozilla-central a win64 change has landed recently and switching to VS2013. SeaMonkey Windows builders are running Vista and VS 2013 has a minimum requirement of Windows 7.
    • [comm-central]
      • WinNT busted : bug 1094097 - fallout from bug 1076020. Have patch. Needs landing on the mozilla/angle github repository.
      • l10n nightly/dep builds busted : bug 1073327 Fix waiting review.
      • blocklist update bustage : bug 1074572 Fix waiting review.
    • [comm-aurora]
  • [28th October 2014]
    • Infrastructure issues
      • linux64 systems are still plagued by bug 1058385. Regular AWOLing. Currently 3 slaves are down.
      • NB: Currently the miscellaneous patches to fix the RelEng infra (tooltool et.al) are all applied to the master directly. Once reviewed, the patches will be pushed to their corresponding repositories and the master will be updated properly.
      • NB2: None of the test bustages are listed. Priority is currently set to get the builds running properly.
      • On mozilla-central a win64 change has landed recently and switching to VS2013. SeaMonkey Windows builders are running Vista and VS 2013 has a minimum requirement of Windows 7.
    • [comm-central]
      • OSX64 busted : bug 1089681 package-manifest.in missing entries.
      • WinNT busted : bug 1053652 ?fixed in mozilla-central upstream?.
      • l10n nightly/dep builds busted : bug 1073327 Fix waiting review.
      • blocklist update bustage : bug 1074572 Fix waiting review.
    • [comm-aurora]
      • blocklist update bustage : bug 1074572 Fix waiting review.
      • OSX64 l10n nightly/dep bustage : related to bug 902876
      • WinNT (build/debug) bustage : RelEng issue. bug 1067567
      • OSX64 bustage: atm, unsure what's busting this. tracked in bug 1090136
  • [12th October 2014]
    • Infrastructure issues
      • linux64 systems are still plagued by plagued by bug 1058385. Regular AWOLing. Currently 7 slaves are down.
      • WinNT systems are ok now. (touch wood)
      • NB: Currently the miscellaneous patches to fix the RelEng infra (tooltool et.al) are all applied to the master directly. Once reviewed, the patches will be pushed to their corresponding repositories and the master will be updated properly.
      • NB2: None of the test bustages are listed. Priority is currently set to get the builds running properly.
      • On mozilla-central a win64 change has landed recently and switching to VS2013. SeaMonkey Windows builders are running Vista and VS 2013 has a minimum requirement of WinXP SP1.
    • [comm-central]
    • [comm-aurora]
      • blocklist update bustage : bug 1074572 Fix waiting review.
      • OSX64 l10n nightly/dep bustage : related to bug 902876
      • WinNT (build/debug) bustage : RelEng issue. bug 1067567
      • OSX64 bustage: changes in OSX code signing paths. bug 1074019
    • [comm-beta]
    • [comm-release]
      • OSX64 bustage : bug 1023644 Patches waiting for review.
  • [30th September 2014]
    • Infrastructure issues
      • linux64 systems are plagued by bug 1058385. Regular AWOLing. Currently 8 slaves are down.
      • WinNT systems were plagued by bug 1069119 and bug 1069117; but they seem to be ok now.
      • NB: Currently the miscellaneous patches to fix the RelEng infra (tooltool et.al) are all applied to the master directly. Once reviewed, the patches will be pushed to their corresponding repositories and the master will be updated properly.
    • [comm-central]
      • All platforms busted : bug 1071497 Patch checked in. Builds now back to "normal".
      • l10n nightly/dep builds busted : bug 1073327 Fix waiting review.
      • blocklist update bustage : bug 1074572 Fix waiting review.
    • [comm-aurora]
      • blocklist update bustage : bug 1074572 Fix waiting review.
      • OSX64 l10n nightly/dep bustage : related to bug 902876
      • WinNT (build/debug) bustage : bug 1053652 Waiting for potential upstream fix.
    • [comm-beta]
    • [comm-release]
      • OSX64 bustage : bug 1023644 Patches waiting for review.
  • [16th September 2014]
    • [rsx11m] The most recent Windows nightly builds were 13-Jun-2014 09:36 for 2.30a1 and 21-Jul-2014 10:17 for 2.29a2. [ewong] Besides infrastructural issues, comm-central has flattened the objdir, which needs further infra changes. [IanN] There has been a lot of build system changes in the last few months.
    • [comm-central]
    • [comm-aurora]
    • [comm-beta]
      • Linux-debug busted due to bug 1053081)
      • OSX debug busted (not yet determined)
  • Historic notes:
    • pymake has been discontinued. We need to install the latest Mozilla-build and tooltool on our windows builders (tracked by bug 1023644 )
    • No langpacks since 29th August 2013 on trunk. Needs investigating.
  • [1st September 2014]
    • All Linux64 machines(except the buildbot master) have decided to go on strike/AWOL/whatever. (Not good, obviously, but not something Callek or ewong can do until DCOps kickstart them, and since Monday is a holiday, it'll be after Tuesday before anything's done.) bug 1058385 Ergo, All trees are closed.
      • w0ts0n from #it took a look at sea-hp-linux64-9 and said it had a 'Disk error'. (Again, not good. In fact, exactly the same situation as sea-hp-linux64-2.)
    • Thankfully, the last slave (-13) went AWOL *after* the 2.29b2 release. (If it was going to go AWOL, after release would be the best time; of course, *not going AWOL* would've been preferential.)
    • Buildbotcustom/config release code is still being tweaked to get release done.
  • [19th August 2014 update]
    • All available linux64 machines are up and running buildbot (CentOS 6.5)
    • Current round of changes have been reviewed and landed.
    • One final round about to be reviewed.
      • The following are 'old' info being kept as a reference:[these entries will be removed with the next Status Meeting]
        • Migrating our Linux builders to CentOS 6 bug 795354. Puppetmaster has been updated. Ewong is able to login to these hosts and buildbot is installed. We still need to get everything setup for parity and do some slight tweaks to the puppet config, but we're pretty close to useable.
        • In Progress: 2.28 beta and final will depend on build system working again for Gecko 31.0
        • bug 983536 libpango version >= 1.22.0 required on Linux.(should be fixed when bug 795354 is fixed).
        • bug 977676 SeaMonkey's puppet broken due to requirement of ffxbld ldap password.
        • bug 943740 is tracking the progress (or lack of one) in building trunk and aurora on Linux & Mac from the downloading user's POV.
        • bug 840426 tracks the buildbot-config and buildbotcustom code changes to support Mock on our linux* slaves.
        • bug 853720 tracks the builders not recognizing mozcrash. (Not exactly sure if it's relevant still, since our builders have been so perma-red for some time; but putting this here just in case.)
    • We are perma-orange on comm-central, comm-aurora and comm-beta due to problems with tests
      • Windows en-US builds are being successfully compiled but tests are not working due to a dependency on python 2.7.3.
    • pymake has been discontinued. We need to install the latest Mozilla-build and tooltool on our windows builders (tracked by bug 1023644 )
    • No langpacks since 29th August 2013 on trunk. Needs investigating.
      • Windows and Mac trunk langpacks directories have been removed on the FTP server, maybe for lack of recent XPIs (later than 2.20a1). Latest Linux langpacks are dated 29 August. Callek told Tonymec on IRC he has a hunch about why but more pressing things to do first.
  • [5th August 2014 update]
    • All available linux64 machines are up and running buildbot (CentOS 6.5)
    • Currently the buildmaster is manually patched (with patches from bug 840427) to run jobs but ewong is continually tweaking the buildbot-config/buildbotcustom code to make the Linux* machines run Mock.
      • The following are 'old' info being kept as a reference:
        • Migrating our Linux builders to CentOS 6 bug 795354. Puppetmaster has been updated. Ewong is able to login to these hosts and buildbot is installed. We still need to get everything setup for parity and do some slight tweaks to the puppet config, but we're pretty close to useable.
        • In Progress: 2.28 beta and final will depend on build system working again for Gecko 31.0
        • bug 983536 libpango version >= 1.22.0 required on Linux.(should be fixed when bug 795354 is fixed).
        • bug 977676 SeaMonkey's puppet broken due to requirement of ffxbld ldap password.
        • bug 943740 is tracking the progress (or lack of one) in building trunk and aurora on Linux & Mac from the downloading user's POV.
        • bug 840426 tracks the buildbot-config and buildbotcustom code changes to support Mock on our linux* slaves.
        • bug 853720 tracks the builders not recognizing mozcrash. (Not exactly sure if it's relevant still, since our builders have been so perma-red for some time; but putting this here just in case.)
    • We are perma-orange on comm-central, comm-aurora and comm-beta due to problems with tests
      • Windows en-US builds are being successfully compiled but tests are not working due to a dependency on python 2.7.3.
  • pymake has been discontinued. We need to install the latest Mozilla-build and tooltool on our windows builders (tracked by bug 1023644 )
  • No langpacks since 29th August 2013 on trunk. Needs investigating.
    • Windows and Mac trunk langpacks directories have been removed on the FTP server, maybe for lack of recent XPIs (later than 2.20a1). Latest Linux langpacks are dated 29 August. Callek told Tonymec on IRC he has a hunch about why but more pressing things to do first.
Retrieved from "https://wiki.mozilla.org/index.php?title=SeaMonkey/StatusMeetings/RelEng&oldid=1195421"