QA/StaffMeetings/QA-Staff-01-25-2012: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(6 intermediate revisions by 5 users not shown)
Line 26: Line 26:
* (DavidB) Check out [http://powerball.theautomatedtester.co.uk powerball] and try a simple [http://people.mozilla.org/~jmaher/power-ball-10.xpi addon] to earn points for bugs you file and comment on.
* (DavidB) Check out [http://powerball.theautomatedtester.co.uk powerball] and try a simple [http://people.mozilla.org/~jmaher/power-ball-10.xpi addon] to earn points for bugs you file and comment on.
* Follow up on team discussion that stephend started last evening
* Follow up on team discussion that stephend started last evening
* Virtual Machine maintenance transitioning from Al to Geo. Want to try setting update schedule at every six months. Is this OK?
* Virtual Machine image maintenance transitioning from Al to Geo. Want to try setting update schedule at every six months. Is this OK?
* Has QMO handoff happened?
* Has QMO handoff happened?


Line 84: Line 84:
** Dria (Deb Richardson) is now officially PM
** Dria (Deb Richardson) is now officially PM
** Talked to Browser ID/WebQA team to figure out how to best tackle the issue of testing Browser ID and having infrastructure to have selenium tests for Pancake to run daily.
** Talked to Browser ID/WebQA team to figure out how to best tackle the issue of testing Browser ID and having infrastructure to have selenium tests for Pancake to run daily.
* webapis (jhammink)
** presently main focus with these is now with marionette on B2G (P2), mainly
***SMS
***Telephony
**Insofar as they tie into work with B2G
* b2G (jhammink)
** now building, test driving marionette against b2g on both devices and emulator; have gaia (frontend) working with marionette, python, and JS
****coming soon - blog post coming for all of the above
**Currently manually testing Telephony, SMS, GAIA UI, very hard!
**Test plan for MWC congress demo (last week Feb)
***test plan will cover mainly functionality (use-cases), (but also reliability, performance, security tbd)
**attending B2G workweek in feb
**Two new interns to onboard
**wanna use Vlad Maniak as a resource - and he has expressed interest!


=== WebQA (Stephen)  ===
=== WebQA (Stephen)  ===
Line 125: Line 142:
=== QA Automation Service (Henrik, David)  ===
=== QA Automation Service (Henrik, David)  ===
* General
* General
**
** Selenium 2.17 has been released
*Mozmill
*Mozmill
** Mozmill 1.5.8 release has been targeted for today (including support for running tests in parallel)
** Mozmill 1.5.8 release has been targeted for today (including support for running tests in parallel)
Line 131: Line 148:
====Project updates====
====Project updates====
; CI system for on-demand Firefox builds (Geo)
; CI system for on-demand Firefox builds (Geo)
* Window systems turned out to be challenging re: setup
* Currently finalizing job setup to get prototype fully rolled


; CI system for daily and l10n Firefox Builds (Henrik)
; CI system for daily and l10n Firefox Builds (Henrik)
Line 166: Line 185:


* Mobile (Aaron)  
* Mobile (Aaron)  
* Reviewing sites on native fennec, for possible Android soup from Webkit -> gecko move.  
** Reviewing sites on native fennec, for possible Android soup from Webkit -> gecko move.  


* Dashboard (jsmith)  
* Dashboard (jsmith)  
* In progress of getting a QA Bugzilla Dashboard for Web Apps through Metrics team
** In progress of getting a QA Bugzilla Dashboard for Web Apps through Metrics team
* http://metrics.mozilla.org/pentaho.
** https://metrics.mozilla.com/pentaho/content/pentaho-cdf-dd/Render?solution=metrics&path=bugzilla/solr/QA&file=QA.wcdf


* Goals  
* Goals  
Line 180: Line 199:
=== Today ===
=== Today ===
* Add notes here
* Add notes here
**
Ideas around structure for QA
- no names attach
- general buckets
- general work needing to be done
- not cast in stone
- subject for talks in/out of
- quickly go through it
 
 
high points:
 
Desktop
- Jan 21st.
- 2:30 warpcore: sign off on 10
- chase.com not working ex.  try to catch it all
- general web usage testing
- if there is any bugs, is it blocking?
- plans have been discussed
- mitigating plans
- so far, no wiki for it
- Anthony on 3.6
- in the wiki
 
Mobile
- risk for 1/31
- maybe adjusting the release date for Beta
- Moving parts
- reiterate try to use mobile and get performance in it, responsiveness, pulling up tabs, etc.
- Services side
- new work for back end
- browserid: next 2 trains are big
- b2g
- goal for world congress
- ux demo
- how to flash gecko/gia on a device
 
WebQA
- Affliates
- new contributer
- rajiv is doing onboarding
- Mozillians
- contributer Austin
 
Community
- planning community goals
- Matt approached bmoss for a headset for contributer
- if you want to send a thank you/or tool for a contributer, we will find a way to help
- community giving program
- funding; something small; non specific, just for contributing/in the community; if we get into specifics then it lends to mercenaries.
- studies on compensation.  We don't want mercenaries we want community
 
Automation services
- speed improvements due to support for running in parallel
- memchaser
 
Case Conductor
- 0.8 release
- 1.0 release for the quarter
- ~ 3/1 0.9
- middle of march stabilization
- end of march 1.0
- small projects into case conductor
 
Note:
- rules of snakes
- don't send Bob in a knife to a gunfight
- when you mean a ton of time...
- how much are we going to save ahead of time
- give data for the time, for the bug
- if it doesnt make sense to them, you're not going to persuade them
- give the next level of information
 
Raymond - took over QMO
 
Org:
META groups:
- Community (Marcia)
- triage (bugs) :
- help people log bugs
- pick up the phone; help understand issues to get a solid bug
- QMO
- Automation
- tools : some set that make the right set of tools happen; QA/community can help implement, but no necessarily set it up (permenant)
- tests : need people to actually write the tests in automated tools (nonpermenant/pull to various groups/everyone eventually get there)
 
* Web - Correctness
App Store (UI)
M. O.
Mozillians
Socorro
 
* Server - Performance testing/Load testing/Failover testing/Crypto/Data Store
Sync
Browser ID -> Services
App Store
Socorro
 
* Client (Native; Mac, Win, Android)
Mobile, Desktop
Apps
B2G (might be different)
 
-> we stand behind it because we said it... => shell -> of passing it to waverly.
-> at some point we have to say no.  what willing to stop?
 
--> What's the problem that we're trying to solve?
--> Learn to say "No"
 
- Automation Team versus QA?  Probably no difference.
 
[Clear understanding what the load is / What's going to break?]
 
[Amount of work is limited -> get the automation; maintenance work, = > do it incrementally; maintenance]
 
[Testing/Security testing is in all three buckets]
 
[Security Team]
[Taros : Snappy, Telemetry]
[try to have a standardization - one off ]
 
[strattling buckets?  Community and Web/Server/Client?  ]
 
 
[ Be able to cross train and do what needs to be done; specialty seems broken, expertise should be there though]
 
[ Accountability for the tools? 
- make sure that the group is happy; tool is useful?
- check back with the customer
- break down of communication/management]
 
[ Do it through influence/not authority ]
 
[ Skill sets are different for Web, Server, Client, ex selenium for web, but not client; server doesn't necessary have a ui, JSON thingy, ]
Next Step is to figure out details
 
=== Last Meeting ===
=== Last Meeting ===
* Action Items
* Action Items
**
**

Latest revision as of 19:42, 3 February 2012

« previous mtg | QA Staff Home | next mtg »

QA Staff Meeting Notes January 25, 2012

Meeting Details

 # Every other Weds @ 1:30p Pacific Time - Mozilla HQ, Warp Core 
 #
 # Vidyo:  https://v.mozilla.com/flex.html?roomdirect.html&key=UK1zyrd7Vhym
 # Phone:  +1 650 903 0800 or 650-215-1282 x92 Conf# 95312 (US/INTL)
 #         +1 800 707 2533 (pin 369) Conf# 95312 (US Toll Free)
 #         +1 416 848 3114 x92 Conf# 95312 (Canada)
 # IRC:    irc://irc.mozilla.org:6697/#qa

QA Org Items

  • New Hires & Interviews
    • item
  • Upcoming Events
    • item
  • PTOs

Discussion Items

  • Usefulness of qa-contribute mailing list -- too much spam recently; what is it used for again? (Aaron)
  • Test Day 1/20 Reflection (Jason)
  • (DavidB) Check out powerball and try a simple addon to earn points for bugs you file and comment on.
  • Follow up on team discussion that stephend started last evening
  • Virtual Machine image maintenance transitioning from Al to Geo. Want to try setting update schedule at every six months. Is this OK?
  • Has QMO handoff happened?

QA Team Updates

Desktop Firefox (Juan)

  • Fx.Next
    • Firefox 10 sign off meeting is today at 2:30 PM in WarpCore
      • If you know of any blocking issues please raise them to Juan, Marcia, or Anthony
  • Silent Update
    • Default Compatible Add-ons:
      • Huge thank you to every who helped with Default Compatible Add-ons testing; shipping for add-ons back to Firefox 4
    • Add-on Hotfix
      • Hotfix add-on is live on AMO for beta users, will enable us to simplify the chemspill process for *some* chemspills
  • Crash Investigation
  • Maintenance and Security (ashughes)
    • Firefox 3.6.26
      • Testing is underway, targeting push to Beta tomorrow, release on Tuesday

Browser Technologies (Tony)

  • Issues
    • Tracy taking Services QA Community stewardship.
    • Fennec Native is at risk for 1/31 beta date
  • Fennec (kbrosnan/aaronmt)
  • Sync Client (tracy)
    • Android Native Sync code drop .3 is nearly ready. It should go to m-c today or tomorrow.
    • Add-on sync is bubbling up the channels (Fx 11) gps has added a series of test to TPS for this feature.
    • Switching from Mozmill to Sikuli for UI automation of sync
  • Test pilot (tracy)
    • nothing to report
  • Sync Server (jbonacci, jrgm)
  • BrowserID (jbonacci, jrgm)
    • Trains continue to roll through
      • Train 17: Bug 713961 - QA and deploy BrowserID train-2011.12.28 to production
      • Train 18: Bug 715728 - QA and deploy BrowserID train-2012.01.05 to production
      • Train 19: Bug 719243 - QA and deploy BrowserID train-2012.01.18 to production
    • Big additions in the current and next train:
      • Support for primaries
      • Support for localization
      • UI changes
    • BrowserID team is actively pursuing leads to act as primaries (email domains, etc).
    • QA work continues towards client-side automation
    • QA is also working with Dev on front-end and back-end unit tests
    • Other upcoming activities include build-out and deployment of PHX colo
  • Jetpack(?)
  • Pancake (nhirata)
    • Infrastructure changes, UX changes still occurring with nothing to test on the front end.
    • Dria (Deb Richardson) is now officially PM
    • Talked to Browser ID/WebQA team to figure out how to best tackle the issue of testing Browser ID and having infrastructure to have selenium tests for Pancake to run daily.
  • webapis (jhammink)
    • presently main focus with these is now with marionette on B2G (P2), mainly
      • SMS
      • Telephony
    • Insofar as they tie into work with B2G
  • b2G (jhammink)
    • now building, test driving marionette against b2g on both devices and emulator; have gaia (frontend) working with marionette, python, and JS
        • coming soon - blog post coming for all of the above
    • Currently manually testing Telephony, SMS, GAIA UI, very hard!
    • Test plan for MWC congress demo (last week Feb)
      • test plan will cover mainly functionality (use-cases), (but also reliability, performance, security tbd)
    • attending B2G workweek in feb
    • Two new interns to onboard
    • wanna use Vlad Maniak as a resource - and he has expressed interest!

WebQA (Stephen)

Community (Marcia)

'

QA Automation Service (Henrik, David)

  • General
    • Selenium 2.17 has been released
  • Mozmill
    • Mozmill 1.5.8 release has been targeted for today (including support for running tests in parallel)

Project updates

CI system for on-demand Firefox builds (Geo)
  • Window systems turned out to be challenging re: setup
  • Currently finalizing job setup to get prototype fully rolled
CI system for daily and l10n Firefox Builds (Henrik)
  • System runs stable within the last week - no fall-outs from Pulse
  • Due to lack of documentation for Pulse some issues with queues had to be fixed
Automated testing for add-ons default to compatible
  • Currently a WIP patch is available for testing
  • Waiting for the final list of add-ons to test including platforms and the MPT machines
MemChaser extension
  • Need to finish up the logging feature before we can go public on AMO
  • Working with the JS team on getting an API for garbage collection implemented
  • Collecting ideas for a zombie compartment detector
  • Feedback from various groups is absolutely positive
Case Conductor
  • Doing preliminary testing on 0.8 (pure-Django) drop
  • Feature list for 0.8 up-to-date in Tracker
    • Everything above the ".8 Feature Complete" bar is expected to be in 0.8 release
  • On-track for 0.8 complete on 2/3
  • Q1/2012 Goals created.

Apps (David)

  • Test Day on 1/20 Results (All)
    • We built a well documented software project, with github pages, pydocs ..etc.
    • ~7 contributors, ran into issues getting code up and running
    • Gained insight on future direction of apps extension automation framework
    • Began shift to java + selenium + sikuli + jbehave.
  • Mochi Tests
    • Met with Clint, got details on mochi tests, how to write / where to check them in ..etc.
    • Began experimentation process
  • Mobile (Aaron)
    • Reviewing sites on native fennec, for possible Android soup from Webkit -> gecko move.
  • Goals
    • Seed new test framework
    • Prepare for MWC.

Notes, Takeaways and Action Items

Today

  • Add notes here

Ideas around structure for QA - no names attach - general buckets - general work needing to be done - not cast in stone - subject for talks in/out of - quickly go through it


high points:

Desktop - Jan 21st. - 2:30 warpcore: sign off on 10 - chase.com not working ex. try to catch it all - general web usage testing - if there is any bugs, is it blocking? - plans have been discussed - mitigating plans - so far, no wiki for it - Anthony on 3.6 - in the wiki

Mobile - risk for 1/31 - maybe adjusting the release date for Beta - Moving parts - reiterate try to use mobile and get performance in it, responsiveness, pulling up tabs, etc. - Services side - new work for back end - browserid: next 2 trains are big - b2g - goal for world congress - ux demo - how to flash gecko/gia on a device

WebQA - Affliates - new contributer - rajiv is doing onboarding - Mozillians - contributer Austin

Community - planning community goals - Matt approached bmoss for a headset for contributer - if you want to send a thank you/or tool for a contributer, we will find a way to help - community giving program - funding; something small; non specific, just for contributing/in the community; if we get into specifics then it lends to mercenaries. - studies on compensation. We don't want mercenaries we want community

Automation services - speed improvements due to support for running in parallel - memchaser

Case Conductor - 0.8 release - 1.0 release for the quarter - ~ 3/1 0.9 - middle of march stabilization - end of march 1.0 - small projects into case conductor

Note: - rules of snakes - don't send Bob in a knife to a gunfight - when you mean a ton of time... - how much are we going to save ahead of time - give data for the time, for the bug - if it doesnt make sense to them, you're not going to persuade them - give the next level of information

Raymond - took over QMO

Org: META groups: - Community (Marcia) - triage (bugs) : - help people log bugs - pick up the phone; help understand issues to get a solid bug - QMO

- Automation - tools : some set that make the right set of tools happen; QA/community can help implement, but no necessarily set it up (permenant) - tests : need people to actually write the tests in automated tools (nonpermenant/pull to various groups/everyone eventually get there)

  • Web - Correctness

App Store (UI) M. O. Mozillians Socorro

  • Server - Performance testing/Load testing/Failover testing/Crypto/Data Store

Sync Browser ID -> Services App Store Socorro

  • Client (Native; Mac, Win, Android)

Mobile, Desktop Apps B2G (might be different)

-> we stand behind it because we said it... => shell -> of passing it to waverly. -> at some point we have to say no. what willing to stop?

--> What's the problem that we're trying to solve? --> Learn to say "No"

- Automation Team versus QA? Probably no difference.

[Clear understanding what the load is / What's going to break?]

[Amount of work is limited -> get the automation; maintenance work, = > do it incrementally; maintenance]

[Testing/Security testing is in all three buckets]

[Security Team] [Taros : Snappy, Telemetry] [try to have a standardization - one off ]

[strattling buckets? Community and Web/Server/Client? ]


[ Be able to cross train and do what needs to be done; specialty seems broken, expertise should be there though]

[ Accountability for the tools? - make sure that the group is happy; tool is useful? - check back with the customer - break down of communication/management]

[ Do it through influence/not authority ]

[ Skill sets are different for Web, Server, Client, ex selenium for web, but not client; server doesn't necessary have a ui, JSON thingy, ]


Next Step is to figure out details

Last Meeting

  • Action Items
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/StaffMeetings/QA-Staff-01-25-2012&oldid=394012"