QA/StaffMeetings/QA-Staff-02-12-2014

From MozillaWiki
Jump to: navigation, search

Mozilla QA Meeting Feb 12, 2014 (Video Replay)

# Every Weds @ 1:30p Pacific Time - Mozilla HQ, Warp Core 
#
# Vidyo:  https://v.mozilla.com/flex.html?roomdirect.html&key=nL4PESn1IaVh
# Phone:  +1 650 903 0800 or 650-215-1282 x92 Conf# 9245 (US/INTL)
#         +1 800 707 2533 (pin 369) Conf# 9245 (US Toll Free)
#         +1 416 848 3114 x92 Conf# 9245 (Canada)
# IRC:    irc://irc.mozilla.org:6697/#qa

QA Org Items

QA Show and Tell

https://wiki.mozilla.org/QA/Show-and-Tell#Schedule

  • This week: Mike Hoye - Community management and One-n-done
    • Integration ideas to improve our community
  • Next week:
    • Desktop QA

New Hires & Interviews

  • Desktop QA: Kamil Jozwiak
  • Mobile Firefox OS QA: No Jun Park - starting 3/3/2014 (MV for a week, then Toronto)
  • Services: Verbal offer accepted - details to come.

Upcoming Events/Conferences

  • MWC, Monday February 23 - Thursday, February 27
  • Web QA work week: March 17th - 21st, in Mountain View (the new office!)

Notices

  • Monday, February 17th is a U.S. and Ontario Holiday

Travel / PTO

  • [dclarke] Tribes Training (Paris office), April 1- Apr 42
  • [delphine] Working from Paris office, Jan 27- Feb 12
  • [tony] MWC 2/24-2/28, PTO 3/5-3/20
  • [mbrandt] 2/25 - 3/1 Posse Roundup Uncon - focus on CD and quality
  • [marcia] Mozilla Reps Leadership Meetup 2/15-2/16
  • [marcia] MWC February 18 - March 1
  • [geo] 2/28, 3/3 PTO
  • [mwobensmith] Working from London through 2/21
  • [juanb] 2/13 - 2/18 PTO, friend's wedding.

Last Meeting Action Items

  • [Geo] move "Last Meeting" action items to top of template [DONE]
  • [Geo] followup with Clint re: regression range criteria [In Progress, see Roundtable]
  • Karl + others: get items for One and Done [carryover] [In Progress]
  • B2G team owes workweek rollup to rest of team/dev-quality [carryover] [In Progress]
  • Volunteers for Show and Tell [carryover]
    • Resolution: if no volunteer at meeting, will go by rotation sheet and team managers pick.
  • Team: contribute to meetup groups w/ Marcia, email ideas, availability, etc. [carryover]
    • Resolution: not sufficiently actionable, taking off.

Roundtable/Discussion Items

QA Team Updates

Desktop (Juan)

  • Firefox
    • 27.0.1 went to build, will be in QA for potential sign-off by tomorrow afternoon. Target release is Friday.
  • ESR (mwobensmith)
  • Stability (KaiRo)
    • 27.0.1 is due to a stability issue we caught early on but dropped on the floor somewhere on the way. Discussions ongoing on how to improve process to avoid that.
    • We did comparisons between 27 and 28 on Beta early this time and have the main issues covered.
  • Automation ()
    • This is covered by Henrik now.
  • Security (mwobensmith)
    • X-CSP header compatiblity testing, safe browsing tests
  • Community

Mobile (Tony)

  • Firefox for Android (Kevin/Aaron)
  • B2G (Geo/jhammink)
    • [jammink & contractors] Burirun 1.3-3 in progress (~20% as of this morning)
    • [jammink, marcia, others] much to do with hands-on MWC demo testing on Sora, Fireweb, Flame, Open-C, etc.
    • [jsmith] Currently performing surgery to save the trunk build - lots of things busted
    • [jsmith] Lots of exploratory testing on rocketbar
    • [jsmith] Fox Den Demo QA Coordination
    • Stability (nhirata)
    • Metrics/Reporting (geo)
      • Marcia and Geo have put together last week's sample subteam health report, including some basic metrics. Geo fixing and optimizing queries, also want to do this for another week to understand how this looks with week->week velocity info and how much effort it takes to do when not figuring out format/queries/etc on the fly. [where is this located?]
      • Once we're solid on this, will be working on team-wide rollup, including metrics/baselines addressing the Q1 2014 goals.
      • Have also brought up need for metrics w/ performance. Probably will primarily aggregate and reuse acceptance criteria, but examining to see what else we might be able to do.
    • Platform (geo)
      • Team handing off compatibility testing
      • https://bugzilla.mozilla.org/show_bug.cgi?id=902686 -- under review (Change manifest.js to use SpecialPowers.pushPrefEnv)
      • mochitest on-device running against hamachi, roughly same results as inari with a few extra failures. [where are results stored?]
        • [geo]: these are being done unofficially and locally by David Clarke right now, so no results stored. We're figuring out to what extent it works and is stable. Once Treeherder has debuted, we'll look at doing this at dailyish in-house with published results. Until then, platform dev's take was that it would be minimal value (nobody would see it) so we might only do it occasionally ourselves to doublecheck/accept builds and raise flags.
        • Enabling nuwa appears to cause an issue with mochitest on-device, needs more investigation.
      • Currently mapping module owners to test tree in preparation for handing off continuing maintenance of B2G mochitests.
    • Performance (geo)
      • Working with perf team and partner to figure out how to measure and improve scrolling uniformity (consistent amounts of movement per frame of animation). Mason Chang and benwa have been working on scroll-graph to address this: http://benoitgirard.wordpress.com/2013/12/06/introducing-scroll-graph/
      • Focusing on eideticker to deliver tool to calculate sensitivity (i.e. can it accurately distinguish individual frames?) as well as try algorithms to improve sensitivity.
      • Will be trying a private patch against B2G compositor to dump frame timing info rather than fps (rolling average) to see how we can use this data for internally-driven metrics
  • Community Members Onboarding

Services (Edwin)

  • Persona
    • Lots of discussion lately, plan to move moz properties to FxA
  • FxA
    • Test Pass on Desktop/Android
    • Trains are running
    • escalating blockers on monitoring/load testing
    • defining beta criteria and intermittent milestones
  • Sync
    • escalating blockers on monitoring/load testing
    • working with Henrik and Tracy on fixing TPS
  • Community
    • working with 2 on FxA tasks
    • Test day planning in discussion

Web (Stephen)

  • Affiliates
    • No updates
  • B2G/Gaia testing:
  • working to correlate automated tests and manual smoke tests, some new tasks
  • Raymond hooked up the battery harness/ammeter to a Hamachi - now working with Mike Lee to see his/his team's needs for it
  • Bouncer
    • (only internally affecting our tests) - seeing blips in our behind-the-firewall/VLAN-run automation, that's not seen on external-facing tests, which hit the same production target
  • Engagement Projects
    • Small newsletter fixes went out
  • Firefox Health Report
    • Potential release next week - scope of release is currently being determined
  • Firefox Marketplace
    • Weekly release was shipped on 2/11
  • Mozilla.com
    • shipped weekly update on Monday
  • Mozillians
  • Socorro
    • No updates - continous delivery
  • SUMO
  • Community Members Onboarding

Community (Marcia)

https://oneanddone.paas.allizom.org/admin/

    • QA Champions
    • Open Badges:
    • Reps Council Meeting this weekend
      • Will try to evangelize connecting the QA Team with Reps - working to get them looped into our workflow and introduce them to One and Done. Also try to leverage Regional Events as much as possible throughout the year
  • Upcoming Events
  • Testdays (ashughes)
  • Video Work
  • QMO.next (Raymond)
    • Craig is working on features and a new theme for QMO in his spare time. I'll update the team after we push his code to dev/stage.
  • Community Members Onboarding

Firefox Automation (Henrik)

  • General
    • [Henrik] Firefox Automation report – week 1/2 2014: http://www.hskupin.info/2014/02/11/firefox-automation-report-week-12-2014/
    • We are trying to have an automation testday next week on Wednesday
    • We will have Nils and Kamil as duties for Mozmill-CI infrastructure when it comes to releases and merges of Firefox - the time is not manageable by folks in Europe - tranings will start soon
  • Projects
    • Mozmill
      • Bug 965216 - Release Mozmill 2.0.5, mozmill-automation 2.0.5, new environments
      • We were still blocked on a mozprocess bug which let us start Firefox before the old process shutdown on Windows (see bug 970595). Henrik fixed that a couple of minutes ago via bug 947248 - hopefully all mozprocess related bugs blocking mozmill are fixed now
      • Due to the mozmill restart test issues we weren't able to get our Metro tests running in mozmill-ci production. For now we run them with a patched version on Cosmin's machine, which can also set Firefox as default browser through an Autoit script
  • MozTrap (Cameron)
  • Community Members Onboarding

Action Items

  • [marcia] Send out suggestions/requirements for contribution email.
  • [clint] Figure out meeting invite, add recent hires, etc.
  • [raymond] Check with Craig re: presenting proposed new QMO theme

Template available at: https://etherpad.mozilla.org/qa-staff-template

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/StaffMeetings/QA-Staff-02-12-2014&oldid=932783"