Desktop QA Meeting

Chair: Juan

Scribe: Jason

Attendees:

• Marcia Knous
• Tracy Walker
• Jason Smith
• Juan B.
• Anthony Hughes

Previous Actions (add status update here):

Anthony

  * Update Q2 goals based on Bob's feedback 
    * Done -- Bob likes them :)
  * Touch base with metrics on testday dashboard 
    * Done via bugzilla -- no response -- will have to reach out by other means
  * Explain why the MozMill 2.0 goal is being dropped 
    * Done
  *  Start discussion about MozTrap guidelines 
    * Done
  * Pick top 3 areas that need high  priority focus for testing per person and at least one task per area  that a community member could do
    * Not Done -- I think this breaks down to NEED and EASE of contribution (ease probably needs to be ranked higher in our "formula" for priority)

Jason

  *  File a bug to change to auto-response email 
    * Not Started
  *  Create Apps Test Day (May 4th) 
    * Done
  *  Schedule meeting for desktop task list definition 
    * Done
  * Pick top 3 areas that need high  priority  focus for testing per person and at least one task per area  that a  community member could do
    * Not Started
  * Setup a smoke tests review of test cases
    * Done

Juan

  *  Need to talk James about Persona integration with Firefox Desktop Releases 
    * Done: 
      * BrowserID + Sync
      * Target end of Q2
      * Desktop and Mobile (native and B2G)
  *  Find out status on Web API and Web RTC
    * Web RTC:
      * Target Fx17; won't be part of Kilimanjaro
      * Desktop first
    * Web API: Have not gotten status yet.
  * Pick top 3 areas that need high  priority  focus for testing per person and at least one task per area  that a  community member could do
    * Plugins Testing Matrix
      * For community: Look out for Flash/Silverlight/Java/Quicktime incoming bugs and confirm
  * Sync up with Softvision on Task List for Desktop
    * In progress

Marcia

  *  Make a google calendar for QA contribute 
    * Done
  *  Schedule brownbag for crashkill analysis 
    * Will be scheduled when I return to Mt. View next week
  *  MozSpaces - Follow up to see when work weeks are 
    * Ashley Sullivan following up - some still await approval
  *  MozSpaces - Find fix rate manual calculation for top crashers 
    * Sent email to Krupa since QA Dashboard link is down
  * Pick top 3 areas that need high  priority  focus for testing per person and at least one task per area  that a  community member could do
    * In Progress
    * Suggest looking at UNCO crash bugs to see if you can reproduce or ask for additional information [Need]
  * Wireframe for Improved Nightly First Run Page
    * Need to set up meeting with David Boswell but we first need to agree on what we want to emphasize besides the task list
  * How to Document for Advertising for Task List
    * Not started

Tracy

  *  Make a wiki for QA contribute manual process
    * Done - Posted guide here: https://wiki.mozilla.org/QA/Desktop_Firefox/GetInvolved/EmailInquiries
    * Next step - Get people signed up and responding to emails
  * Pick top 3 areas that need high  priority  focus for testing per person and at least one task per area  that a  community member could do
    * Done - Tasks called out at https://wiki.mozilla.org/QA/Desktop_Firefox/GetInvolved
    * Next step - Flesh out guide pages for each task

Agenda:

* Note: Update your status on last meeting + task list meeting above in the notes
* Qualify the Mozilla Web Platform
  * Feature Ownership and Progress
    * Apps in the Cloud Desktop (jsmith)
    * Desktop Web Apps (jsmith)
    * Web API (?)
    * Web RTC (?)
    * Persona Integration (tracy?)
  * k9o
    * Nominating k9o blockers
      * If any bugs maps to a k9o user story, Nominate it.
    * Map out skeletal test plan for k9o usecase testing
      * Example k9o test cases: Search for web apps integration: smoke tests here:   https://moztrap.allizom.org/manage/cases/
    * k9o performance with Taras Glek
      * What are the top partner sites' performance issues
        * Which are the top partners?
      * Who is driving this?
  * One-a-day security fix verification
* Community Retention
  * QA Contribute - Keeping traction on it
    * How are we tracking? Outreach:Response:Contribution ratio?
    * 9 team members have signed up for access to calendar (5 are Desktop)
* Visibility and Accessibility of Desktop QA team
  * Live Test Day - What, When, and How?
* Effectiveness and Efficiency of Release Qualification Process
  * Trial run of nightly beta testing on weekly betas?
    * automation, updates, and smoketests to sign off beta
    * bug fixes and feature/exploratory testing run out-of-band
    * Questions about nightly betas: (juanb)
      * what problem are trying to solve
      * alternatives and contingency plans
  * Buddy System - Other Redundancy Areas
    * Firefox Sync (tracy)
    * Desktop Apps (jsmith)
    * Crashkill (marcia)
  * Desktop Testing Platform Coverage
    * We should be filing bugs like this: https://bugzilla.mozilla.org/show_bug.cgi?id=752493
    * Who consistently looks at QA wanted bugs from the team? Is there a schedule?
  * QA Contact Field bug: Probably need to escalate https://bugzilla.mozilla.org/show_bug.cgi?id=684088 if we want traction on it
    * Let's bring Bob Moss on this bug to get traction
* Misc
  * Upcoming PTO/Working Remote - Preparing for It
    * Marcia
      * May 7th - May 11th - Remote
      * May 14th
      * End of June speaking at conference (~22-23 June)
    * Jason
      * May 30th - 31st - Remote
      * PTO - May 25th, May 29th, June 1st, June 4th
    * Tracy
      * PTO - May 29th - June 1st

Notes:

* AITC - Target Fx15, in progress
* Web Apps into Desktop - In progress
* Web API - Pieces missing, still getting status from John Hammick
* WebRTC - Moved to Fx17 - Drop this from our goals and notify Bob Moss
* Persona Integration - Targeted Fx15, may too be aggressive
* Live Test Day
  * Topic: Plugin Testing (main scenarios, does flash work?)
  * Include presentation and activities
  * When: Mid-June
  * Where: Multi-location
    * Mountain View
    * Vancouver

Action Items:

Anthony

* Pick top 3 areas that need high  priority  focus for testing per person and at least one task per area  that a  community member could do
* Follow-up with Taras on k9o perf
* Add weekly beta test plan security bug verification metric and make sure we hit it
* Sync up with Dave Hunt on Live Test Day and how to do it
* Followup with Win 8 proof of concept feature owner on win 8 metro testing status and issues

Jason

* File a bug to change to auto-response email 
* Pick top 3 areas that need high  priority  focus for testing per person and  at least one task per area  that a  community member could do
* Add desktop web apps smoke tests to MozTrap
* Bring k9o 2-day test strategy sprint to qa-staff meeting (with Stephen Donner)
* Add Anthony to k9o perf discussion with Taras
* Followup with Tony on QA contribute with mobile team

Juan

* Find out status on Web API and Web RTC
* Sync up with Softvision on Task List for Desktop
* Email Bob about WebRTC moving to Fx17, dropping Q2 goal for it
* Followup with Alex on concerns with Rapid Beta approach

Marcia

* Pick top 3 areas that need high  priority   focus for testing per person and at least one task per area  that a   community member could do
* Wireframe for Improved Nightly First Run Page
* MozSpaces - Find fix rate manual calculation for top crashers 
* How to Document for Advertising for Task List
* File a bug on Richard for video support for Live Test Day
* Schedule brownbag for crashkill
* Followup with Bob Moss on qa-contact field bug

Tracy

* Flesh out activities with guide pages for each task
* Make sure folks are signing up for responding to the Contribute mailing list.
* Find out what will be finished for Q2/k9o for sign into browser