QA/Large-Allocation header: Difference between revisions

Jump to navigation Jump to search

QA/Large-Allocation header (view source)

Revision as of 08:47, 13 March 2017

6,766 bytes added ,  13 March 2017
ter
(cvb)
(ter)
Line 46: Line 46:


=== Out of Scope ===
=== Out of Scope ===
Only the Large-Allocation feature behavior will be included in the testing efforts. Any others , are not going to be evaluated in order to find whether it satisfies the specified requirements or not.
Only the Large-Allocation feature behavior will be included in the testing efforts. Any others technologies, are not going to be evaluated in order to find whether it satisfies the specified requirements or not.


= Requirements for testing =
= Requirements for testing =
Line 69: Line 69:
|-
|-
| 1 || The custom Firefox 53 build works as expected when the LargeAllocation header is present on content loads ||  
| 1 || The custom Firefox 53 build works as expected when the LargeAllocation header is present on content loads ||  
* the browser which have a custom pref `dom.largeAllocation.testing.allLoads` does not behave strangely when loading content specific to this feature. <br>
* the browser which have a custom pref `dom.largeAllocation.testing.allLoads` does not behave strangely when loading content specific to this feature <br>
  || all scenarios provided must have the right expected result || Manual || Desktop Team
  || all scenarios provided must have the right expected result || Manual || Desktop Team
|-
|-
|}


== Testing approach ==
== Testing approach ==
* set up a smoke/acceptance suite that checks the core functionality associated to this feature, along with a few major UI components
* set up a smoke/acceptance suite that checks the core functionality associated to this feature, along with a few major UI components
* set up an exploratory section within this suite that checks all the UI components
* set up an exploratory section within this suite that checks all the UI components
== Builds ==
This section should contain links for builds with the feature:
The feature is available on Firefox 53 starting with 2017-02-02.
* Links for Nightly builds - available since 02-02-2017 [https://archive.mozilla.org/pub/firefox/nightly/2017/02/2017-02-02-11-01-08-mozilla-central/ link]
  Note that the special flag which turns it on for every website is only in the Firefox 53 custom build. [https://archive.mozilla.org/pub/firefox/try-builds/michael@thelayzells.com-e7f71fef67aa64227f32c6926099e834750c4ca6/try-win32/]
== Test Execution Schedule ==
The following table identifies the anticipated testing period available for test execution.
{| class="wikitable" style="width:100%"
|-
! Project phase !! Start Date !! End Date
|-
| Start project
|style="text-align:center;" | 2017-01-12
|style="text-align:center;" | tbd
|-
| Study documentation/specs received from developers
|style="text-align:center;" | 2017-02-28
|style="text-align:center;" | 2017-03-10
|-
| QA - Test plan creation
|style="text-align:center;" | 2017-03-13
|style="text-align:center;" | 2016-03-13
|-
| QA - Test cases/Env preparation
|style="text-align:center;" | 2017-03-13
|style="text-align:center;" | 2017-03-13
|-
| QA - Nightly Testing
|style="text-align:center;" | n/a
|style="text-align:center;" | n/a
|-
| QA - Aurora Testing
|style="text-align:center;" | n/a
|style="text-align:center;" | n/a
|-
| QA - Beta Testing
|style="text-align:center;" | tbd
|style="text-align:center;" | tbd
|-
| Release Date
|style="text-align:center;" | tbd
|style="text-align:center;" | tbd
|}
== Testing Tools ==
Detail the tools to be used for testing, for example see the following table:
{| class="wikitable" style="width:50%"
|-
! Process !! Tool
|-
| Test plan creation || Mozilla wiki
|-
| Test case creation || Google docs/Test Rail
|-
| Test case execution || TestRail
|-
| Bugs management || Bugzilla
|}
== Overview ==
First released on Nightly in 02-02-2017 starting with [https://archive.mozilla.org/pub/firefox/nightly/2017/02/2017-02-02-11-01-08-mozilla-central/ link]
Track the dates and build number where feature was merged to Aurora
Track the dates and build number where feature was merged to Release/Beta
== Risk analysis ==
* Identify the high-risk assumptions.
* Identify existing bugs on the feature with high risk.
* Identify if other areas are affected by the fix.
= References =
* List and links for specs
  - see the MDN article for Large-Allocation [https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Large-Allocation]
* Meta bug: Bug [https://bugzilla.mozilla.org/show_bug.cgi?id=1331083]
Test cases can be found also on TestRail at: [https://testrail.stage.mozaws.net/index.php?/suites/view/494&group_by=cases:section_id&group_order=asc]
{| class="wikitable" style="width:80%"
|-
! Test Areas !! Covered !! Details
|-
| Private Window
|style="text-align:center;" | Yes ||
|-
| Multi-Process Enabled
|style="text-align:center;" | Yes||
|-
| Multi-process Disabled
|style="text-align:center;" | Yes ||
|-
| '''UI'''
||  ||
|-
| Mouse-only operation 
|style="text-align:center;" | Yes ||
|-
| Keyboard-only operation 
|style="text-align:center;" | Yes ||
|-
| Display (HiDPI)
|style="text-align:center;" | Yes ||
|-
| Interraction (scroll, zoom)
|style="text-align:center;" | Yes ||
|-
| Usability and/or discoverability testing 
|style="text-align:center;" | Yes  || Is this feature user friendly
|-
| Does this affect partner builds? Partner build testing 
|style="text-align:center;" | Yes || yes/no options, add comment with details about who will lead testing
|-
|-
| ''' Web Compatibility '''
||  || depends on the feature
|-
| Testing against target sites 
|style="text-align:center;" | Yes ||
|-
| Survey of many sites for compatibility 
|style="text-align:center;" | Yes ||
|-
| ''' Interoperability  '''
||  || depends on the feature
|-
| Common protocol/data format with other software: specification available. Interop testing with other common clients or servers. 
|style="text-align:center;" | Yes  ||
|-
| Coordinated testing/interop across the Firefox products: Desktop, Android, iOS 
|style="text-align:center;" |  ||
|-
| Interaction of this feature with other browser features 
|style="text-align:center;" | Yes ||
|}
== Test suite ==
Large-Allocation header - Sign off (Fx53) testsuite in [tba] (need Testrail account)<br />
= Bug Work =
Tracking bug – meta tracker [https://bugzilla.mozilla.org/show_bug.cgi?id=1331083 bug 1331083]
====== Bug fix verification ======
Bugs associated to [https://bugzilla.mozilla.org/show_bug.cgi?id=1331083 Large-Allocation header]
<bugzilla>
{
"id":"1304140, 1329331, 1331087, 1331525, 1332343, 1333936, 1334309, 1334586, 1338241",
"include_fields":"id,summary,status,resolution,priority"
}
</bugzilla>
====== Logged bugs ======
<bugzilla>
{
"id":"",
"include_fields":"id,summary,status,resolution, priority"
}
</bugzilla>
== Criteria ==
Check list
* All Criteria under each section of Quality Assurance Strategy should be green.
* All test cases should be executed
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
== Results ==
'''Mid-Aurora, Pre-Beta and Pre-Release Sign-off'''
List of OSes that will be covered by testing<br />
*Link for the tests run
** Full Test suite
== Checklist ==
{| class="wikitable" style="width:60%"
|-
! Exit Criteria !! Status !! Notes/Details
|-
|  Testing Prerequisites (specs, use cases)
| style="text-align:center;" |  {{mdone|}}
| style="text-align:center;" |
|-
|  Testing Infrastructure setup
|style="text-align:center;" |  {{mdone|}} ||
|-
|  Test Plan Creation
| style="text-align:center;" | {{mdone|}}  ||
|-
|  Test Cases Creation
|style="text-align:center;" | tbd ||
|-
|  Full Functional Tests Execution
|style="text-align:center;" |  tbd ||
|-
|  Automation Coverage
|style="text-align:center;" | N/A ||
|-
|  Performance Testing
|style="text-align:center;" | N/A ||
|-
|  All Defects Logged
|style="text-align:center;" | {{mdone|}} ||
|-
|  Critical/Blockers Fixed and Verified
|style="text-align:center;" |  {{mdone|}}  ||
|-
|  Metrics/Telemetry
|style="text-align:center;" | N/A ||
|-
|  QA Signoff  - Nightly Release
|style="text-align:center;" |  N/A ||
|-
|  QA Aurora - Full Testing
|style="text-align:center;" |  N/A
|style="text-align:center;" |
|-
|  QA Signoff  - Aurora Release 
|style="text-align:center;" | 
|style="text-align:center;"| 
|-
|-
|  QA Beta - Full Testing
|style="text-align:center;" | 
|style="text-align:center;"|
|-
|  QA Signoff  - Beta Release
|style="text-align:center;" | 
|style="text-align:center;" |
|-
Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1165429"

Navigation menu