Channel Meeting Details
- Tuesdays and Thursdays at 10am Pacific Time
- irc.mozilla.org #planning for backchannel
Video/Teleconference Details - NEW
- 650-903-0800 or 650-215-1282 x92 Conf# 99951 (US/INTL)
- 1-800-707-2533 (pin 369) Conf# 99951 (US)
- Vidyo Room: Release Coordination
- Vidyo Guest URL
- 1 Attendees
- 2 Previous Actions
- 3 Schedule Update
- 4 Add-ons
- 5 Shield studies
- 6 Stability
- 7 Performance (Thu)
- 8 QE
- 9 Performance
- 10 RelEng
- 11 Marketing/Communications
- 12 User Advocacy
- 13 Roundtable
- 14 Special Topics
- 15 Actions
RyanVM, nicole, jcristau, Ritu, asimonca, philipp, elan, petruta, ,ninu,cornel, Julia, callek,roxana,poiegas, bogdan_maris, jorge, Ioana, selena, Cristian, Alexandru, bogdans, Timea, mboldan,
Current releases dashboard: https://mozilla.github.io/delivery-dashboard
- preparations ongoing for 56.0.1 (win64 migration)
- [jlorenzo] Releng is experiencing trouble to generate special migration partials. We haven't generated such partials in the beta cycle. This is due to the fact that update formats have changed since the last time we manually crafted partials. We hit several bumps along the road, making us start over several times:
1. Our scripts to generate partials have bitrotten. We founds issues along the way. As an example, Taskcluster closed the old Scheduler API, forcing us to find a new way to submit release graphs. We had to find a workaround to submit the right graph. Docker images have bitrotten too. Also, signing format changed from SHA1 to SHA384 2. We also had to tweak these scripts in order to make win64 partials generated on top of win32 original mars. 3. Balrog had to be manually updated to point users to the right partial location.
We're currently blocked by the signing format. "Update verify" tests may also show red because of a platform change.
- Tentative date to be able to push this to release is next week. All stakeholders have been notified.
- 52.4.1 (Mac OS X specific)?
- https://bugzilla.mozilla.org/show_bug.cgi?id=1376163 (crash playing video on macos 10.13)
- Do we have enough ESR users on recent Mac OS X version that justify a dot release?
- Is this covered in our test matrix?
- tests were on 10.12 at the time, we should probably switch to 10.13 now
- [Cornel] Will switch to 10.13 starting with next ESR
- We will do a mac only dot release to address this. Gtb today/tomm and push to ESR on Monday.
- 57.0b5 fennec is out, 100% rollout
- Gtb Desktop 57.0b6 today, push to beta channel tomorrow
- huge number of uplifts, hope to reduce that by next week
- FYI: Shield studies in flight. https://gregglind.github.io/x-shield-inflight-dashboard
- time to unset the early beta flag?
- yes (RyanVM: Done) thanks!
awsy rate: 2.51 (browser 0.83, content 1.68)
- Crash in encoding_rs::Decoder::max_utf16_buffer_length: https://bugzilla.mozilla.org/show_bug.cgi?id=1405571 - 50% of crashes in recent builds
- Nightly crash triage: https://wiki.mozilla.org/NightlyCrashTriage/2017Q4
- New small volume accessibility crash - https://bugzilla.mozilla.org/show_bug.cgi?id=1406054
- Still seeing random crash addresses, mostly on Mac: https://bugzilla.mozilla.org/show_bug.cgi?id=1402087
- 447 crashes/1 install: RtlVirtualUnwind | CrashReporter::JitExceptionHandler
awsy rate 0.93 (browser 0.52, content 0.41)
- pretty good crashing profile, awsy rate is already below what we had at the end of 56.0b cycle; a third of browser crashes are during shutdown
- Startup crash in nsZipArchive::GetItem: https://bugzilla.mozilla.org/show_bug.cgi?id=1404743 - 2.7% of browser crashes
- new signature in 57, needs investigation
- Crash in NewFunctionClone https://bugzilla.mozilla.org/show_bug.cgi?id=1384544 - 2% of browser crashes
- carryover regression from 55; bug stale
- Content crash in IPCError-browser | PBrowserParent::RecvPDocAccessibleConstructor Constructing a top-level PDocAccessible with null COM: https://bugzilla.mozilla.org/show_bug.cgi?id=1383501 - 6.6% of content crashes
- bug has unreviewed patch and & beta uplift in https://bugzilla.mozilla.org/show_bug.cgi?id=1399557 may help too
- Content crash in [@ EMPTY: no crashing thread identified; OK] on google maps: https://bugzilla.mozilla.org/show_bug.cgi?id=1405151 - 37% of content crashes on OS X
awsy rate: 0.84 (browser 0.60, content 0.24)
- Crashes during mediaplayback regressing in 56: https://bugzilla.mozilla.org/show_bug.cgi?id=1404456 & https://bugzilla.mozilla.org/show_bug.cgi?id=1405110 - 7% of content crashes
- affecting intel sandybridge gpus (2011- ...); bugs could need more attention/investigation
- Shutdownhangs in mozilla::dom::workers::RuntimeService::Cleanup: https://bugzilla.mozilla.org/show_bug.cgi?id=1405290 - 5.3% of browser crashes
- regressing in volume during 56; no fix in sight for 56 & 57
awsy rate: release 0.62 - beta 0.96 - nightly 1.69
[philipp] numbers look fairly stable, except on beta
- biggest regression in 57.0b is Crash in ResolvePromise with ublock origin: https://bugzilla.mozilla.org/show_bug.cgi?id=1347984 - 13% of fennec crashes
- a couple of other regressions are on track for receiving a fix: https://bugzil.la/1403787,1403755,1395138
- Nightly rate looks pretty good. New crash, but is it a diagnostic assert added from https://bugzilla.mozilla.org/show_bug.cgi?id=1401737
Aurora / Dev Edition
- 64-bit migration testing on ‘release-cdntest’ moved to Mon, Oct 9
- 56.0.1-build2 signed of as GREEN.
[Brindusa] Started Nightly build validation -few bugs logged (none of them critical or blocker) - still in progress. Tomorrow will send the test results.
- Nightly preliminary feature sign-offs (Mid-Nightly) underway (due Oct 6)
[Julia] DevTools Go Faster - The engineering efforts are paused at least until the end of the year, so the QA team drops the Nightly feature sign off until other implementation updates
Aurora / Dev Edition
- Beta 57.0b5 signed off without any isuses
- 57.0b4 - build 1 Signed off as GREEN last Friday
- 57.0b5 - build 1 Signed off as GREEN yesterday
- 58.0a1 - Nightly Pre-chech Part 1 finished and signed off GREEN on Monday
Aurora/Beta Feature Review
Post-Mortem (Tues 2wks from GA Release)
- 56 post-mortem - https://public.etherpad-mozilla.org/p/56-0-Post-Mortem
- Tentatively planned for Tues 10/10 - A week out from today