MozillaWiki

QA SoftVision Team/Desktop: Difference between revisions

< QA SoftVision Team
 
(48 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= Management and Communication =
= MANAGEMENT AND COMMUNICATION =
=== Project Updates ===
== Project Updates ==
=== Hot Projects ===


=== Meetings ===
* Firefox 40 Beta testing - Test Plan: https://wiki.mozilla.org/Releases/Firefox_40/Test_Plan
* [https://wiki.mozilla.org/QA/Fennec/Waverley#Weekly_Meetings Weekly Waverly] meeting
* Projects/features https://wiki.mozilla.org/QA/Desktop_Firefox/Features


Tuesdays, 8am PT / 4pm GMT
== Meetings ==
Dial in: 800-707-2533
* [https://wiki.mozilla.org/QA/Firefox/TeamMeetings Weekly QA Meeting]
password 369, conference #245


=== IRC ===
** Wednesdays, 8:30am Pacific Time
** Telephone: 
** 650-903-0800 or 650-215-1282 x92 Conf# '''99779''' (US/INTL)
** 1-800-707-2533 (pin 369) Conf# '''99779''' (US)
** Vidyo Room: QA-Desktop
** Vidyo [https://v.mozilla.com/flex.html?roomdirect.html&key=RU7dVysDFakY Guest URL]
** Agenda and notes: [https://etherpad.mozilla.org/firefox-qa-standup-2015-q3 etherpad]
 
== IRC ==
* [http://wbe02.mibbit.com/?settings=1b10107157e79b08f2bf99a11f521973&server=irc.mozilla.org&channel=%23qa #qa] on irc.mozilla.org
* [http://wbe02.mibbit.com/?settings=1b10107157e79b08f2bf99a11f521973&server=irc.mozilla.org&channel=%23qa #qa] on irc.mozilla.org
* [http://wbe02.mibbit.com/?settings=1b10107157e79b08f2bf99a11f521973&server=irc.mozilla.org&channel=%23firefox #firefox] on irc.mozilla.org
* [http://wbe02.mibbit.com/?settings=1b10107157e79b08f2bf99a11f521973&server=irc.mozilla.org&channel=%23firefox #firefox] on irc.mozilla.org
* [http://wbe02.mibbit.com/?settings=1b10107157e79b08f2bf99a11f521973&server=irc.mozilla.org&channel=%23tabcandy #tabcandy] on irc.mozilla.org
* [http://wbe02.mibbit.com/?settings=1b10107157e79b08f2bf99a11f521973&server=irc.mozilla.org&channel=%23devtools #devtools] on irc.mozilla.org
=== Mailing Lists ===
* Use internal Waverly mailing list to communicate active work to managers


=== Bug watchers ===
== Mailing Lists ==
* the list of watches from Bugzilla can be found [http://bit.ly/kHPNtk here]
* firefox-qe@mozilla.com
* dev-quality@mozilla.com
* dev-platform@mozilla.com


=== Spreadsheets ===
== Bug watchers ==
* Unconfirmed bug triage - [http://bit.ly/lwFNN6 link]
* the list of watches from Bugzilla can be found [https://docs.google.com/spreadsheets/d/1rrgKoNFVN7f4rokKK6bAzWkAdzzO0N7JDNIYMzd--1I/edit?hl=en_US#gid=7 here]
* Exploratory work (testcases) - [http://bit.ly/lBKz5S link]
* Fixed bugs verification - [http://bit.ly/jt53Lb link]
* Updated testcases - [http://bit.ly/m7rOv0 link]


===Blogs===
== Blogs ==  
*most contributers have blogs and a lot of information can be found out by watching these blogs
*most contributers have blogs and a lot of information can be found out by watching these blogs
*list of most important blogs:
*list of most important blogs:
Line 35: Line 37:
**Planet Firefox - http://planet.firefox.com/
**Planet Firefox - http://planet.firefox.com/


= Ramp-up and testing environment =
= TESTING ENVIRONMENTS =
=== First Steps ===
* Ramp-up document - [http://bit.ly/iA59QB link]


=== Hardware and Operating Systems ===
=== Hardware and Operating Systems Coverage ===
* Hardware
* Matrix can be found [https://docs.google.com/spreadsheets/d/1w9tUnMCxdKN4gmu-i35J0bJpIT7c9dDCt7C8W5ypw1o/edit#gid=0 here]
** CPU: P4 3.0GHz
** RAM: 2GB
** Video card: MSI Nvidia Geforce N210
* Operating systems
** Windows XP
** Windows 7 x86, x64
** Windows Vista
** Mac OS X 10.6
** Ubuntu 10.10 x86, x64


= Tasks =
= REGULAR CYCLE QA DESKTOP TEAM SCHEDULING =
== Weekly==
{| class="wikitable"
{| class="fullwidth-table sortable" style="width: 500px;"
|-
! Day/Week !! Monday !! Tuesday !! Wednesday !! Thursday !! Friday
|-
|-
| style="font-weight: bold; background: #DDD; width:35%" |
| Week 1 || Merge day <br> RC/ESR testing if needed <br> General tasks || Beta 1 testing || style="background:#9D9; text-align:center;" | Beta 1 testing - sign-off <br> Feature testing - new in Aurora || General tasks <br> Feature testing - new in Aurora|| style="background:#9D9; text-align:center;"| General tasks <br> Preparing docs for Beta 3 testday <br> Aurora new features sign-offs
| style="font-weight: bold; background: #DDD; width:35%" | Channel
| style="font-weight: bold; background: #DDD; width:35%" | Links
|-
|-
| Bug triage
| Week 2 || General tasks || style="background:#9D9; text-align:center;"| Beta 2 testing - sign-off || General tasks || General tasks <br> Final tweaks for Beta 3 testday || style="background:#9D9; text-align:center;"|Beta 3 testing - sign-off <br> Beta 3 testday
| Nighly
| [http://bit.ly/lwFNN6 Spreadsheet link]
|-
|-
| Fixed bugs verification
| Week 3 ||General tasks <br> Beta 3 testday follow up ||style="background:#9D9; text-align:center;" | Beta 4 testing - sign-off || General tasks || General tasks || style="background:#9D9; text-align:center;" | Beta 5 testing - sign-off <br> Preparing docs for Beta 7 testday
| Nightly/Aurora
| [http://bit.ly/jt53Lb Spreadsheet link]
|-
|-
| Exploratory testing
| Week 4 ||General tasks  || style="background:#9D9; text-align:center;"| Beta 6 testing - sign-off || General tasks || General tasks <br> Final tweaks for Beta 7 testday || style="background:#9D9; text-align:center;"| Beta 7 testing - sign-off <br> Beta 7 testday <br> Preparing docs for Aurora testday
| Aurora
| [http://bit.ly/lBKz5S Spreadsheet link]
|-
|-
| Update testcases
| Week 5 ||General tasks <br> Beta 7 testday follow up|| style="background:#9D9; text-align:center;" | Beta 8 testing - sign-off || General tasks || General tasks <br> Final tweaks for Aurora testday || style="background:#9D9; text-align:center;" | Beta 9 testing - sign-off <br> Aurora testday <br> Beta features sign-off
| NApp
| [http://bit.ly/m7rOv0 Spreadsheet link]
|-
|-
| Week 6 ||General tasks <br> Aurora testday follow up|| style="background:#9D9; text-align:center;" | RC testing - sign-off || style="background:#9D9; text-align:center;" | ESR testing - sign-off <br> Feature testing - Aurora feats || style="background:#9D9; text-align:center;" | ESR testing - sign-off <br> Feature testing - Aurora feats || style="background:#9D9; text-align:center;" | General tasks <br> Pre-Beta features sign-offs
|}
|}
===Bug triage===
* the validation or invalidation of unconfirmed bugs logged by community testers (the link with the bugzilla query can be found in the spreadsheet, in the date header)
* bugs  are to be commented and their resolution set accordingly to one of the resolutions: RESOLVED – WFM, INCOMPLETE, DUPLICATE, INVALID, WONTFIX or to NEW
* each bug that is set to NEW has to have a regression window - http://quality.mozilla.org/docs/bugzilla/guide-to-triaging-bugs-for-firefox/finding-a-regression-window/


===Fixed bugs verifications===
= BETA TESTING COVERAGE =
* the verification of fixed bugs -issues should be commented and resolution set to either RESOLVED FIXED or REOPENED
== Test areas distribution in the Beta Cycle =
* verification is performed across all platforms against which the bug was logged
* Please note that the plan is subject to change, depending on the bug fixes that are landing in every beta build
** See High level plan for Firefox Beta starting with Fx 39 Beta Cycle https://docs.google.com/spreadsheets/d/1NMSL8RLbDoQHVKGwxsl64B2aA0YseOrpuAobVrfhmsc/edit#gid=240008071
 
=== FIREFOX 40 - RESULTS AND COVERAGE OVER BETA CYCLE ===
* See a more detailed plan for Firefox 40 Beta Cycle:
** https://docs.google.com/spreadsheets/d/1ZGOcdnEVy8RC1OPf2YPzKNPY3Df9lLmrdKarwjPcbFA/edit#gid=1291138301
 
=== FIREFOX 39 - RESULTS AND COVERAGE OVER BETA CYCLE ===
* See a more detailed plan for Firefox 39 Beta Cycle:
** https://docs.google.com/spreadsheets/d/1TjZTco65z79UmaQfARPYlrmlCb_QykEaMy2F3-M_gkA/edit#gid=1291138301
 
== General Compatibility Testing - SMOKE AND EXPLORATORY TESTING usually done at least once in the Beta Cycle:==
 
{| class="wikitable"
|-
! General Compatibility Testing Area !! Details
|-
| WEB COMPATIBILITY || Testing involves the following websites:<br />
* '''http://www.alexa.com/topsites''' (minimum top 10 websites) <br />
* '''https://www.google.com/maps/about/explore/?status=invite'''<br />
* '''http://www.youtube.com/html5'''
|-
| VIDEO COMPATIBILITY || Testing involves opening and playback video content:<br />
* '''HTML5 video content'''<br />
* '''*.webm video files'''<br />
* '''*.ogv video files'''<br />
* '''*.mp4 video files'''<br />
* '''video and audio content that makes use of the getUserMedia API'''<br />
* '''VP8 and VP9 video content'''
|-
| PDF COMPATIBILITY ||
|-
| ADD-ON COMPATIBILITY || Testing involves installing popular add-ons from AMO <br />
(top 10 Add-ons from Most Popular Extensions page https://addons.mozilla.org/ro/firefox/extensions/?sort=users)
|-
| AUDIO COMPATIBILITY || Testing involves opening and playback audio files:<br />
* '''*.mp3 audio files'''<br />
* '''*.ogg audio files'''<br />
* '''audio files that make use of the Opus codec'''<br />
* '''audio files that make use of the Web Audio API'''
|-
| PLUG-IN COMPATIBILITY|| Testing involves browsing websites that use the following plugins:<br />
* '''Flash Player plug-in''' <br />
* '''Silverlight plug-in'''<br />
* '''Unity Web Player plug-in'''<br />
* '''Java plug-in'''
 
|}
 
== Other areas - SMOKE AND EXPLORATORY TESTING - usually done once in the Beta Cycle:==
** [GRAPHICS]
** [SCROLL & ZOOM]
** [SEARCH]
** [SYNC]
** [BREAKPAD END-TO-END FUNCTIONALITY]
** [VERIFY UPDATE APPLIES WITH ROOT OWNED FILES]
** [SECURE BROWSING]
** [OS X SIGNING]
** [OS X 32-BIT MODE]
** [PRINTING]
 
== Feature tests - SMOKE AND EXPLORATORY TESTING - specific features going out in the release - usually done at the beginning of the Beta Cycle (BETA 1 BUILD) and throughout the beta cycle depending on the fixes that are landed in Beta cycle. See examples below:==
** [HELLO] - was tested in Fx 39 and Fx 40
** [HELLO - CONTEXT] - was tested in Fx 40
** [POCKET]- was tested in Fx 38.0.5 and Fx 39
** [EME]- was tested in Fx 38.0.5, Fx 39, Fx 40
 
= BETA TESTING DETAILS =
=== 1. Beta 1 testing (week 1) ===
** [https://docs.google.com/spreadsheets/d/1Rn-F3Kg_1_VznIxxXkAGGL8mVMSAdamZZI4f1O2r8HA/edit?pli=1#gid=1917376294  Feature Tracking]
** Select test cases for the features merged from Aurora
** Create MozTrap Beta 1 Test Run based on testcases above
** Determine additional areas that need exploratory testing based on analysis of the Beta 1 pushlog
** [https://wiki.mozilla.org/Releases/Firefox_39/Test_Plan/Beta/1  Result: Test Plan]
 
=== 2. Determine areas to cover through regression testing (week 2 - before Beta 2) ===
** [https://goo.gl/mquisO Query for all fixes in the new Beta (not in-testsuite & not already assigned for verification)]
** Based on the query determine areas most affected by fixes, and that pose a higher risk
** Determine the areas that will be covered by Regression tests
** Create MozTrap Beta Regression Test Run - will be run throughout the entire cycle
** [https://docs.google.com/spreadsheets/d/1jm5hlGDoTNp_Hp4VvIkcS7c5LENkpjwJT3n2iQP2Zjk/edit#gid=814109842 Result: Beta Triage]
 
=== 3. Beta 2 -> RC testing (weeks 2 -> 5) ===
** [https://goo.gl/VmtxOn Triage fixes that landed since the previous Beta => bugs to verify + areas to test (Monday & Thursday before GTB)]
** [https://goo.gl/95Bn9t Final triage for fixes in current Beta => bugs to verify + areas to test (Tuesday & Friday after GTB)]
** [https://goo.gl/3VGvP7 Individual Beta Triage Result]
** Determine regression tests to run + exploratory testing needed + environments
** [https://wiki.mozilla.org/Releases/Firefox_39/Test_Plan/Beta/2 Result: Test Plan]
 
=== 4. Triage bugs for verification in Beta (weeks 1 -> 3) ===
** [https://goo.gl/XnZ3h5 Query for all fixes in the new Beta (not in-testsuite & not already assigned for verification)]
** Based on the query triage bugs into: qe-verify+, qe-verify-, [good first verify], ignored
** [https://goo.gl/MRwViB Result: Bugzilla Query]
 
=== 5. Beta features sign-off (week 5) ===
** [https://docs.google.com/spreadsheets/d/1Rn-F3Kg_1_VznIxxXkAGGL8mVMSAdamZZI4f1O2r8HA/edit?pli=1#gid=1917376294 Feature Tracking]
** Feature owners monitor new fixes + perform additional testing (weeks 1 -> 5)
** Feature owners analyze remaining issues and do a final assessment on quality of the Beta features
** Feature owners send final sign-off email for Beta features to dev leads & release drivers (Friday week 5 or Monday week 6 latest)
 
= GENERAL TASKS DETAILS =
=== 1. Bug Verification ===
** [https://goo.gl/mt72iw Beta Bug verification = top priority]
** Beta fixes to verify are mostly triaged by us
** [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=21 General Bug Verification tracking Document]
** [https://docs.google.com/spreadsheets/d/1UXW6X1HjPeoMSm4Q8DTOki5aBno9BfbBRoX8RCYvS8U/edit#gid=0 Sprints Bug verification = secondary priority]
** Sprints fixes to verify are mostly triaged by the Dev Teams / Marco Mucci
** Striving to unload more work towards the community
 
=== 2. Bug Investigations ===
** [https://docs.google.com/spreadsheets/d/1ZHyVmkQWAzIzk4n7afML-BcGZS4gyjjrouafgxFExB0/edit#gid=16 Queries & Tracking]
** Task is highly time consuming
** Few contributors are helping out: Aleksej, Alice
 
=== 3. Bug Triage ===
'''What does this mean'''
** The validation or invalidation of unconfirmed bugs logged by community testers
** Bugs are to be commented and their resolution set accordingly to one of the resolutions: RESOLVED – WFM, INCOMPLETE, DUPLICATE, INVALID, WONTFIX or to NEW
** Each bug that is set to NEW has to have a regression window - [http://quality.mozilla.org/docs/bugzilla/guide-to-triaging-bugs-for-firefox/finding-a-regression-window/ Guide], [http://harthur.github.com/mozregression/ Tool]
 
'''Process and documents'''
** [https://docs.google.com/spreadsheets/d/1KM_OFcOO4PV_zkZCwUwI6zYqnGuSNaOZGpsc83mxoH8/edit#gid=15 Queries & Tracking]
** One team member is responsible for regularly checking incoming bugs
** Striving to unload more work towards the community, and lower time spent by us on this task


===Exploratory testing===
=== 4. Feature testing ===
* ad-hoc testing on specific/error proned areas of the browser
'''What does this mean'''
* testcases are to be created during this kind of testing and written down in the specific spreadsheet
** QA Owners are assigned to each feature as early as possible
* Litmus FFTs/BFTs can be used as a starting point for the exploratory
** QA Owners are responsible for testing and signing off on their features
* each NEW bug has to have a regression window - http://quality.mozilla.org/docs/bugzilla/guide-to-triaging-bugs-for-firefox/finding-a-regression-window/
* Logged bugs should respect the best practices presented here:
** http://quality.mozilla.org/docs/bugzilla/starter-kit/how-to-write-a-proper-bug/
** http://quality.mozilla.org/docs/bugzilla/starter-kit/how-to-write-a-proper-bug-part-2/


===Update testcases in Litmus===
'''Process and documents'''
* based on the Litmus Unclear, Broken / Fail reports, testcases should be kept up-to-date
** [https://docs.google.com/spreadsheets/d/1Rn-F3Kg_1_VznIxxXkAGGL8mVMSAdamZZI4f1O2r8HA/edit?pli=1#gid=12 Tracking by Version]
* all modifications are to be registered in the created spreadsheet
** Nightly - Draft documentation is created and Exploratory testing is performed to shake out most bugs
** Aurora - Test Plan is created, Test Cases are created and Run
** Beta - Bug Verification for additional fixes, and final Exploratory/Regression testing
** Some features are not signed-off - they are monitored and owners help with Triage and Verification (e.g. MSE)


==Periodically==
=== 5. Component Watch ===
'''What does this mean'''
** Owners will cover: triage of new Unconfirmed bugs, investigation of bugs, and verification of fixes in certain area


Beside the weekly activities there are also specific tasks that are performed before the channels are merged and a new release is launched (Aurora, Beta, Final). These are as follow:
'''Process and documents'''
** [https://docs.google.com/spreadsheets/d/1rrgKoNFVN7f4rokKK6bAzWkAdzzO0N7JDNIYMzd--1I/edit#gid=7 Queries]
** Major components split among team members
** Considered a lower priority area with not much time spent on it


Pre Aurora, Pre Beta, Pre Release specific tests:
=== 6. e10s Regression testing ===
*Smoketests
** [https://wiki.mozilla.org/QA/Electrolysis Main QA Page]
*BFTs (Mozmill and Manual)
** [https://docs.google.com/spreadsheets/d/1e8u8QmGxRnl3M-kUz4nCwuxUhdnn27RiMZl2dI7Gatc/edit#gid=0 SV Tracking]
*Features owners sign off
** Working to cover all areas at least once
*Ad-hoc testing
** Coordinating with Juan and Tracy
** Generally assigned for weeks 1->4


= Ownerships  =
= TESTDAY DETAILS =
=== 1. Beta 3 testday (Friday week 2) ===
** Generally focused on new features in Beta
** [https://etherpad.mozilla.org/testday-20150123 Results]


* an ownership guideline can be found [http://quality.mozilla.org/docs/misc/feature-ownership-guidelines/ here]
=== 2. Beta 7 testday (Friday week 4) ===
** Generally focused on higher risk areas towards the end of Beta


==Firefox features==
=== 3. Aurora testday (Friday week 5) ===
* the list of owners can be consulted [http://bit.ly/jVeQAf here]
** Generally focused on Aurora features set to enter Beta + areas most affected by fixes in Aurora


==Litmus==
=== 4. Weekly Bug Triage day (every Monday week 1 -> 6) ===
* the list of owners and subgroups can be found [https://wiki.mozilla.org/QA/Execution/Litmus/Ownership here]
** Focused on involving the community to triage Unconfirmed bugs
** [https://wiki.mozilla.org/Bugdays/Bug-Triage Bug Triage Results]


= Additional Information =
=== 5. Weekly Bug Verification day (every Wednesday week 1 -> 6) ===
* please add info
** Focused on involving the community to verify [good first verify] bug fixes
** [https://wiki.mozilla.org/Bugdays/Bug-verification Bug Verification Results]

Latest revision as of 10:57, 4 August 2015

MANAGEMENT AND COMMUNICATION

Project Updates

Hot Projects

Meetings

    • Wednesdays, 8:30am Pacific Time
    • Telephone:
    • 650-903-0800 or 650-215-1282 x92 Conf# 99779 (US/INTL)
    • 1-800-707-2533 (pin 369) Conf# 99779 (US)
    • Vidyo Room: QA-Desktop
    • Vidyo Guest URL
    • Agenda and notes: etherpad

IRC

Mailing Lists

  • firefox-qe@mozilla.com
  • dev-quality@mozilla.com
  • dev-platform@mozilla.com

Bug watchers

  • the list of watches from Bugzilla can be found here

Blogs

TESTING ENVIRONMENTS

Hardware and Operating Systems Coverage

  • Matrix can be found here

REGULAR CYCLE QA DESKTOP TEAM SCHEDULING

Day/Week Monday Tuesday Wednesday Thursday Friday
Week 1 Merge day
RC/ESR testing if needed
General tasks		 Beta 1 testing Beta 1 testing - sign-off
Feature testing - new in Aurora		 General tasks
Feature testing - new in Aurora		 General tasks
Preparing docs for Beta 3 testday
Aurora new features sign-offs
Week 2 General tasks Beta 2 testing - sign-off General tasks General tasks
Final tweaks for Beta 3 testday		 Beta 3 testing - sign-off
Beta 3 testday
Week 3 General tasks
Beta 3 testday follow up		 Beta 4 testing - sign-off General tasks General tasks Beta 5 testing - sign-off
Preparing docs for Beta 7 testday
Week 4 General tasks Beta 6 testing - sign-off General tasks General tasks
Final tweaks for Beta 7 testday		 Beta 7 testing - sign-off
Beta 7 testday
Preparing docs for Aurora testday
Week 5 General tasks
Beta 7 testday follow up		 Beta 8 testing - sign-off General tasks General tasks
Final tweaks for Aurora testday		 Beta 9 testing - sign-off
Aurora testday
Beta features sign-off
Week 6 General tasks
Aurora testday follow up		 RC testing - sign-off ESR testing - sign-off
Feature testing - Aurora feats		 ESR testing - sign-off
Feature testing - Aurora feats		 General tasks
Pre-Beta features sign-offs

BETA TESTING COVERAGE

= Test areas distribution in the Beta Cycle

FIREFOX 40 - RESULTS AND COVERAGE OVER BETA CYCLE

FIREFOX 39 - RESULTS AND COVERAGE OVER BETA CYCLE

General Compatibility Testing - SMOKE AND EXPLORATORY TESTING usually done at least once in the Beta Cycle:

General Compatibility Testing Area Details
WEB COMPATIBILITY Testing involves the following websites:
VIDEO COMPATIBILITY Testing involves opening and playback video content:
  • HTML5 video content
  • *.webm video files
  • *.ogv video files
  • *.mp4 video files
  • video and audio content that makes use of the getUserMedia API
  • VP8 and VP9 video content
PDF COMPATIBILITY
ADD-ON COMPATIBILITY Testing involves installing popular add-ons from AMO

(top 10 Add-ons from Most Popular Extensions page https://addons.mozilla.org/ro/firefox/extensions/?sort=users)

AUDIO COMPATIBILITY Testing involves opening and playback audio files:
  • *.mp3 audio files
  • *.ogg audio files
  • audio files that make use of the Opus codec
  • audio files that make use of the Web Audio API
PLUG-IN COMPATIBILITY Testing involves browsing websites that use the following plugins:
  • Flash Player plug-in
  • Silverlight plug-in
  • Unity Web Player plug-in
  • Java plug-in

Other areas - SMOKE AND EXPLORATORY TESTING - usually done once in the Beta Cycle:

    • [GRAPHICS]
    • [SCROLL & ZOOM]
    • [SEARCH]
    • [SYNC]
    • [BREAKPAD END-TO-END FUNCTIONALITY]
    • [VERIFY UPDATE APPLIES WITH ROOT OWNED FILES]
    • [SECURE BROWSING]
    • [OS X SIGNING]
    • [OS X 32-BIT MODE]
    • [PRINTING]

Feature tests - SMOKE AND EXPLORATORY TESTING - specific features going out in the release - usually done at the beginning of the Beta Cycle (BETA 1 BUILD) and throughout the beta cycle depending on the fixes that are landed in Beta cycle. See examples below:

    • [HELLO] - was tested in Fx 39 and Fx 40
    • [HELLO - CONTEXT] - was tested in Fx 40
    • [POCKET]- was tested in Fx 38.0.5 and Fx 39
    • [EME]- was tested in Fx 38.0.5, Fx 39, Fx 40

BETA TESTING DETAILS

1. Beta 1 testing (week 1)

    • Feature Tracking
    • Select test cases for the features merged from Aurora
    • Create MozTrap Beta 1 Test Run based on testcases above
    • Determine additional areas that need exploratory testing based on analysis of the Beta 1 pushlog
    • Result: Test Plan

2. Determine areas to cover through regression testing (week 2 - before Beta 2)

3. Beta 2 -> RC testing (weeks 2 -> 5)

4. Triage bugs for verification in Beta (weeks 1 -> 3)

5. Beta features sign-off (week 5)

    • Feature Tracking
    • Feature owners monitor new fixes + perform additional testing (weeks 1 -> 5)
    • Feature owners analyze remaining issues and do a final assessment on quality of the Beta features
    • Feature owners send final sign-off email for Beta features to dev leads & release drivers (Friday week 5 or Monday week 6 latest)

GENERAL TASKS DETAILS

1. Bug Verification

2. Bug Investigations

    • Queries & Tracking
    • Task is highly time consuming
    • Few contributors are helping out: Aleksej, Alice

3. Bug Triage

What does this mean

    • The validation or invalidation of unconfirmed bugs logged by community testers
    • Bugs are to be commented and their resolution set accordingly to one of the resolutions: RESOLVED – WFM, INCOMPLETE, DUPLICATE, INVALID, WONTFIX or to NEW
    • Each bug that is set to NEW has to have a regression window - Guide, Tool

Process and documents

    • Queries & Tracking
    • One team member is responsible for regularly checking incoming bugs
    • Striving to unload more work towards the community, and lower time spent by us on this task

4. Feature testing

What does this mean

    • QA Owners are assigned to each feature as early as possible
    • QA Owners are responsible for testing and signing off on their features

Process and documents

    • Tracking by Version
    • Nightly - Draft documentation is created and Exploratory testing is performed to shake out most bugs
    • Aurora - Test Plan is created, Test Cases are created and Run
    • Beta - Bug Verification for additional fixes, and final Exploratory/Regression testing
    • Some features are not signed-off - they are monitored and owners help with Triage and Verification (e.g. MSE)

5. Component Watch

What does this mean

    • Owners will cover: triage of new Unconfirmed bugs, investigation of bugs, and verification of fixes in certain area

Process and documents

    • Queries
    • Major components split among team members
    • Considered a lower priority area with not much time spent on it

6. e10s Regression testing

    • Main QA Page
    • SV Tracking
    • Working to cover all areas at least once
    • Coordinating with Juan and Tracy
    • Generally assigned for weeks 1->4

TESTDAY DETAILS

1. Beta 3 testday (Friday week 2)

    • Generally focused on new features in Beta
    • Results

2. Beta 7 testday (Friday week 4)

    • Generally focused on higher risk areas towards the end of Beta

3. Aurora testday (Friday week 5)

    • Generally focused on Aurora features set to enter Beta + areas most affected by fixes in Aurora

4. Weekly Bug Triage day (every Monday week 1 -> 6)

5. Weekly Bug Verification day (every Wednesday week 1 -> 6)

Retrieved from "https://wiki.mozilla.org/index.php?title=QA_SoftVision_Team/Desktop&oldid=1088151"