QA SoftVision Team/Desktop

From MozillaWiki
Jump to: navigation, search

Contents

MANAGEMENT AND COMMUNICATION

Project Updates

Hot Projects

Meetings

    • Wednesdays, 8:30am Pacific Time
    • Telephone:
    • 650-903-0800 or 650-215-1282 x92 Conf# 99779 (US/INTL)
    • 1-800-707-2533 (pin 369) Conf# 99779 (US)
    • Vidyo Room: QA-Desktop
    • Vidyo Guest URL
    • Agenda and notes: etherpad

IRC

Mailing Lists

  • firefox-qe@mozilla.com
  • dev-quality@mozilla.com
  • dev-platform@mozilla.com

Bug watchers

  • the list of watches from Bugzilla can be found here

Blogs

TESTING ENVIRONMENTS

Hardware and Operating Systems Coverage

  • Matrix can be found here

REGULAR CYCLE QA DESKTOP TEAM SCHEDULING

Day/Week Monday Tuesday Wednesday Thursday Friday
Week 1 Merge day
RC/ESR testing if needed
General tasks 		Beta 1 testing Beta 1 testing - sign-off
Feature testing - new in Aurora 		General tasks
Feature testing - new in Aurora		 General tasks
Preparing docs for Beta 3 testday
Aurora new features sign-offs
Week 2 General tasks Beta 2 testing - sign-off General tasks General tasks
Final tweaks for Beta 3 testday 		Beta 3 testing - sign-off
Beta 3 testday
Week 3 General tasks
Beta 3 testday follow up 		Beta 4 testing - sign-off General tasks General tasks Beta 5 testing - sign-off
Preparing docs for Beta 7 testday
Week 4 General tasks Beta 6 testing - sign-off General tasks General tasks
Final tweaks for Beta 7 testday 		Beta 7 testing - sign-off
Beta 7 testday
Preparing docs for Aurora testday
Week 5 General tasks
Beta 7 testday follow up		 Beta 8 testing - sign-off General tasks General tasks
Final tweaks for Aurora testday 		Beta 9 testing - sign-off
Aurora testday
Beta features sign-off
Week 6 General tasks
Aurora testday follow up		 RC testing - sign-off ESR testing - sign-off
Feature testing - Aurora feats 		ESR testing - sign-off
Feature testing - Aurora feats 		General tasks
Pre-Beta features sign-offs

BETA TESTING COVERAGE

= Test areas distribution in the Beta Cycle

FIREFOX 40 - RESULTS AND COVERAGE OVER BETA CYCLE

FIREFOX 39 - RESULTS AND COVERAGE OVER BETA CYCLE

General Compatibility Testing - SMOKE AND EXPLORATORY TESTING usually done at least once in the Beta Cycle:

General Compatibility Testing Area Details
WEB COMPATIBILITY Testing involves the following websites:
VIDEO COMPATIBILITY Testing involves opening and playback video content:
  • HTML5 video content
  • *.webm video files
  • *.ogv video files
  • *.mp4 video files
  • video and audio content that makes use of the getUserMedia API
  • VP8 and VP9 video content
PDF COMPATIBILITY
ADD-ON COMPATIBILITY Testing involves installing popular add-ons from AMO

(top 10 Add-ons from Most Popular Extensions page https://addons.mozilla.org/ro/firefox/extensions/?sort=users)

AUDIO COMPATIBILITY Testing involves opening and playback audio files:
  • *.mp3 audio files
  • *.ogg audio files
  • audio files that make use of the Opus codec
  • audio files that make use of the Web Audio API
PLUG-IN COMPATIBILITY Testing involves browsing websites that use the following plugins:
  • Flash Player plug-in
  • Silverlight plug-in
  • Unity Web Player plug-in
  • Java plug-in

Other areas - SMOKE AND EXPLORATORY TESTING - usually done once in the Beta Cycle:

    • [GRAPHICS]
    • [SCROLL & ZOOM]
    • [SEARCH]
    • [SYNC]
    • [BREAKPAD END-TO-END FUNCTIONALITY]
    • [VERIFY UPDATE APPLIES WITH ROOT OWNED FILES]
    • [SECURE BROWSING]
    • [OS X SIGNING]
    • [OS X 32-BIT MODE]
    • [PRINTING]

Feature tests - SMOKE AND EXPLORATORY TESTING - specific features going out in the release - usually done at the beginning of the Beta Cycle (BETA 1 BUILD) and throughout the beta cycle depending on the fixes that are landed in Beta cycle. See examples below:

    • [HELLO] - was tested in Fx 39 and Fx 40
    • [HELLO - CONTEXT] - was tested in Fx 40
    • [POCKET]- was tested in Fx 38.0.5 and Fx 39
    • [EME]- was tested in Fx 38.0.5, Fx 39, Fx 40

BETA TESTING DETAILS

1. Beta 1 testing (week 1)

    • Feature Tracking
    • Select test cases for the features merged from Aurora
    • Create MozTrap Beta 1 Test Run based on testcases above
    • Determine additional areas that need exploratory testing based on analysis of the Beta 1 pushlog
    • Result: Test Plan

2. Determine areas to cover through regression testing (week 2 - before Beta 2)

3. Beta 2 -> RC testing (weeks 2 -> 5)

4. Triage bugs for verification in Beta (weeks 1 -> 3)

5. Beta features sign-off (week 5)

    • Feature Tracking
    • Feature owners monitor new fixes + perform additional testing (weeks 1 -> 5)
    • Feature owners analyze remaining issues and do a final assessment on quality of the Beta features
    • Feature owners send final sign-off email for Beta features to dev leads & release drivers (Friday week 5 or Monday week 6 latest)

GENERAL TASKS DETAILS

1. Bug Verification

2. Bug Investigations

    • Queries & Tracking
    • Task is highly time consuming
    • Few contributors are helping out: Aleksej, Alice

3. Bug Triage

What does this mean

    • The validation or invalidation of unconfirmed bugs logged by community testers
    • Bugs are to be commented and their resolution set accordingly to one of the resolutions: RESOLVED – WFM, INCOMPLETE, DUPLICATE, INVALID, WONTFIX or to NEW
    • Each bug that is set to NEW has to have a regression window - Guide, Tool

Process and documents

    • Queries & Tracking
    • One team member is responsible for regularly checking incoming bugs
    • Striving to unload more work towards the community, and lower time spent by us on this task

4. Feature testing

What does this mean

    • QA Owners are assigned to each feature as early as possible
    • QA Owners are responsible for testing and signing off on their features

Process and documents

    • Tracking by Version
    • Nightly - Draft documentation is created and Exploratory testing is performed to shake out most bugs
    • Aurora - Test Plan is created, Test Cases are created and Run
    • Beta - Bug Verification for additional fixes, and final Exploratory/Regression testing
    • Some features are not signed-off - they are monitored and owners help with Triage and Verification (e.g. MSE)

5. Component Watch

What does this mean

    • Owners will cover: triage of new Unconfirmed bugs, investigation of bugs, and verification of fixes in certain area

Process and documents

    • Queries
    • Major components split among team members
    • Considered a lower priority area with not much time spent on it

6. e10s Regression testing

    • Main QA Page
    • SV Tracking
    • Working to cover all areas at least once
    • Coordinating with Juan and Tracy
    • Generally assigned for weeks 1->4

TESTDAY DETAILS

1. Beta 3 testday (Friday week 2)

    • Generally focused on new features in Beta
    • Results

2. Beta 7 testday (Friday week 4)

    • Generally focused on higher risk areas towards the end of Beta

3. Aurora testday (Friday week 5)

    • Generally focused on Aurora features set to enter Beta + areas most affected by fixes in Aurora

4. Weekly Bug Triage day (every Monday week 1 -> 6)

5. Weekly Bug Verification day (every Wednesday week 1 -> 6)

Retrieved from "https://wiki.mozilla.org/index.php?title=QA_SoftVision_Team/Desktop&oldid=1088151"