MozillaWiki

QA/Netmonitor Phase II (Launchpad): Difference between revisions

Page Discussion

QA/Netmonitor Phase II (Launchpad) (view source)

Revision as of 07:55, 14 September 2017

2,229 bytes added ,  14 September 2017
→‎Checklist: vbnV
(→‎Scope: nbv)
 
(31 intermediate revisions by the same user not shown)
Line 23: Line 23:
|-
|-
| 14/06/2017 || 1.0 || Ciprian Georgiu || Created first draft
| 14/06/2017 || 1.0 || Ciprian Georgiu || Created first draft
|-
|-
| 28/08/2017 || 1.1 || Ciprian Georgiu || Updates
|-
|-
|}
|}
Line 29: Line 32:


== Purpose ==
== Purpose ==
Detail the purpose of this document. For example:
This document describes the following topics:
* The test scope, focus areas and objectives
* the test scope, focus areas and objectives
* The test responsibilities
* the test responsibilities
* The test strategy for the levels and types of test for this release
* the test strategy for the levels and types of test for this release
* The entry and exit criteria
* the entry and exit criteria
* The basis of the test estimates
* the basis of the test estimates
* Any risks, issues, assumptions and test dependencies
* any risks, issues, assumptions and test dependencies
* The test schedule and major milestones
* the test schedule and major milestones
* The test deliverables
* the test deliverables


== Scope ==
== Scope ==
Line 59: Line 62:
== Scope of Testing ==
== Scope of Testing ==
=== In Scope ===
=== In Scope ===
The Network Monitor (netmonitor) shows you all the network requests Firefox makes (for example, when a page is loaded or when an XMLHttpRequest is performed) , how long each request takes, and details of each request. You can edit the method, query, header and resend the request as well. Read this MDN (https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor) article to learn how to use the tool.
The Network Monitor (netmonitor) shows you all the network requests Firefox makes (for example, when a page is loaded or when an XMLHttpRequest is performed), how long each request takes, and details of each request. You can edit the method, query, header and resend the request as well. Read this MDN (https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor) article to learn how to use the tool.


This feature purpose is to run Net panel on the top of Launchpad. Testing efforts will be focused on confirming that Net Monitor Phase II (Launchpad) is working as expected across platforms.
The testing effort for the Netmonitor Phase II (Launchpad) will be invested on the following areas:
The testing effort for the Network Monitor Launchpad will be invested on the following areas:
* Netmonitor runs as intended in the browser tab (Launchpad) and the Launchpad interface is displayed when opening localhost:8000
* functionality: basic and advanced functionality to be verified according to the existing requirements;
* Devtools Launchpad communicates properly with Firefox (the remote debugging server) and list all opened tabs from Firefox
* usability: this feature must be easy to use and straightforward;
* the Toolbar is displayed well at the top of the main window and it provides all the functionality according to the existing requirements
* the feature lists all the network requests made in the course of loading the page and each request is displayed in its own row
* Netmonitor is cleared by default each time Firefox is navigating to a new page or the current page is releaded
* Netmonitor request columns has the intended behavior when toggling different columns
* Netmonitor request details open as intended a new pane and has the following pages: Headers, Cookies, Params, Response, Timings, Security (only for secure pages) Preview.
* Context menu and Submenu works as expected and the functionality is intact
* Hide/show request details panel button work well when Firefox is resized
* Statistics panel shows the correct information and it can be open from Performance analysis icon
* Netmonior works as intended when other themes are applied


=== Out of Scope ===
=== Out of Scope ===
Only the Net Monitor Phase II (Launchpad) feature functionality  will be included in the testing efforts. Net Monitor MVP [https://bugzilla.mozilla.org/show_bug.cgi?id=1307743] or any others functionalities, that are not related to this feature, are not going to be evaluated in order to find whether it satisfies the specified requirements or not.
 
Testing will not check:
* how this feature works on a Hidpi configuration
* if this feature works with RTL locales
* whether the feature works well when Launchpad is loaded in Chrome
* Netmonitor opened in Devtools


== Environments ==
== Environments ==
* Windows 10 x64
* '''Operating systems:'''
* Windows 7 x86
** Windows 10 x64 and Windows 7 x86
* Mac OS X 10.11.6  
** Mac OS X 10.11.6
* Ubuntu 16.04 x64 LTS
** Ubuntu 16.04 x64 LTS
Other configuration/environmental setup needed
 
* documentation on how to load '''Net Monitor in Launchpad''': http://searchfox.org/mozilla-central/source/devtools/client/netmonitor/README.md
* '''Devices: n/a'''
* '''Other configuration/environmental setup needed:'''
** documentation on how to load '''Net Monitor in Launchpad:''' http://searchfox.org/mozilla-central/source/devtools/client/netmonitor/README.md


== Channel dependent settings (configs) and environment setups ==
== Channel dependent settings (configs) and environment setups ==
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
In order to run the Network Monitor in the browser tab, the following packages are required:
 
* node >= 6.9.x
=== Nightly ===
* npm >= 3.x
<div class="mw-collapsible-content">
* yarn >= 0.21.x
only those mentioned in the Environments section
* Firefox either the released version or build from the source code.
</div>
 
=== Beta ===
<div class="mw-collapsible-content">
only those mentioned in the Environments section
</div>
 
=== Post Beta / Release ===
<div class="mw-collapsible-content">
only those mentioned in the Environments section
</div>
</div>


== Risk Assessment and Coverage ==
== Risk Assessment and Coverage ==
Line 102: Line 108:
! ID !! Description / Threat Description !! Covered by Test Objective !!  Magnitude !! Probability !! Priority !! Impact Score  
! ID !! Description / Threat Description !! Covered by Test Objective !!  Magnitude !! Probability !! Priority !! Impact Score  
|-
|-
| RAC-1 || Netmonitor (Launchpad) Details panel (Header, Cookies, Params, Response etc.) || Test Cases || 3-High || 3-Almost Certain || 3-High || 27
| RAC-1 || Netmonitor (Launchpad) Details panel (Header, Cookies, Params, Response etc.) are not showing the correct information for the selected request || TO-1, TO-2, TO-3, TO-4, TO-5, TO-6 || 3-High || 3-Almost Certain || 3-High || 27
|-
| RAC-2 || Filtering requests (by content type, by whether they are XMLHttpRequests or WebSocket requests, by URL, or by request properties) are not working correctly || TO-7 || 2-Moderate || 3-Posibble || 2-Medium || 12
|-
|-
| RAC-2 || Toolbar view buttons || Test Cases || 2-Moderate || 3-Posibble || 2-Medium || 12
| RAC-3 || Context menu is not displayed when right clicking the list requests || TO-9 || 2-Moderate || 2-Possible || 3-High || 12
|-
|-
| RAC-3 || Context menu || Test Cases|| 2-Moderate || 2-Possible || 3-High || 12
| RAC-4 || Context menu and Submenu not saving the correct data for a selected request || TO-10, TO-9 || 2-Moderate || 2-Possible || 3-High || 12
|-
|-
| RAC-4 || Submenu for Context menu || Test Cases || 2-Moderate || 2-Possible || 3-High || 12
| RAC-5 || Netmonitor (sub)checkboxes cannot be accessed with mouse || TO-10|| 2-Moderate || 1-Unlikely|| 3-High || 6
|-
|-
| RAC-5 || Netmonitor checkbox || Test Cases|| 2-Moderate || 1-Unlikely|| 3-High || 6
| RAC-6 || Netmonitor may not run in Launchpad and the interface is not displayed || || 2-Moderate || 1-Unlikely|| 3-High || 6
|}
|}


Line 132: Line 140:
{| class="wikitable"
{| class="wikitable"
|-
|-
! Ref !! Function !! Test Objective !! Evaluation Criteria !! RAC !! Test Type !! Owners  
! Ref !! Function !! Evaluation Criteria !! Test Objective !! RAC !! Test Type !! Owners  


|-
|-
| 1 || Netmonitor (Launchpad) Details panel (Header) is working properly ||  
| TO-1 || Netmonitor (Launchpad) Details panel (Header) || The tab lists basic information about the request: request URL, request method, remote IP address and port, status code with link to MDN doc (if available), the HTTP request and response headers that were sent and a button to edit and resend the request
The Headers tab is properly displayed and lists the basic info about the request:
  || Verifying that Detail panel (Header) displays the correct information for each request and if the request's method, URL, parameters, headers are resented.  || RAC-1 ||Manual || Desktop Team
* request URL
* request method
* remote IP address and port
* status code
* the HTTP request and response headers that were sent
* a button to edit and resend the request <br>
  || all scenarios provided must have the right expected result || RAC-1 ||Manual || Desktop Team
|-
|-
| 2 || Netmonitor (Launchpad) Details panel (Cookies) ||  
| TO-2 || Netmonitor (Launchpad) Details panel (Cookies) ||  
The Cookies tab correctly lists full details of any cookies sent with the request or response.<br>
The tab lists full details of any cookies sent with the request or response. As with headers, Cookies gives the option to filter the cookies.
  || all scenarios provided must have the right expected result || RAC-1 ||Manual || Desktop Team  
  || Verifying that Detail panel (Cookies) displays the correct cookies for the selected request and check if filtering works as intended. || RAC-1 ||Manual || Desktop Team  
|-
|-


| 3 || Netmonitor (Launchpad) Details panel (Params) ||  
| TO-3 || Netmonitor (Launchpad) Details panel (Params) ||  
The Params tab is properly displays the GET parameters and POST data of a request. <br>
The tab displays the GET parameters and POST data of a request. <br>
  || all scenarios provided must have the right expected result || RAC-1 || Manual || Desktop Team  
  || Verifying that the GET parameters and POST data of a request are displayed in the Params's panel.  || RAC-1 || Manual || Desktop Team  
|-
|-


| 4 || Netmonitor (Launchpad) Details panel (Response) ||  
| TO-4 || Netmonitor (Launchpad) Details panel (Response) ||  
The tab shows the corect response for the selected request type. (e.g.: HTML, JS, or CSS)
The Response tab is properly displayed and if the response is HTML, JS, or CSS, it will be shown as text.
|| Verifying that response is shown as text for HTML, JS, or CSS, as an inspectable object for a JSON and a tab preview for a image. || RAC-1 ||Manual || Desktop Team  
 
If the response is JSON, it will be shown as an inspectable object.
If the response is an image, the tab displays a preview. <br>
|| all scenarios provided must have the right expected result || RAC-1 ||Manual || Desktop Team  
|-
|-


| 5 || Netmonitor (Launchpad) Details panel (Timings) ||  
| TO-5 || Netmonitor (Launchpad) Details panel (Timings) ||  
The tab breaks a network request down into the following subset of the stages defined in the HTTP Archive specification.
The Timings tab is displayed properly with the following subset of the stages defined in the HTTP Archive specification:
|| Verifying that the following specification: Blocked, DNS resolution, Connecting, Sending, Waiting, Receiving changes when a different request is clicked. || RAC-1 || Manual || Desktop Team  
* Blocked
* DNS resolution
* Connecting
* Sending
* Waiting
* Receiving. <br>
|| all scenarios provided must have the right expected result || RAC-1 || Manual || Desktop Team  
|-
|-


| 6 || Netmonitor (Launchpad) Details panel (Security)  ||  
| TO-6 || Netmonitor (Launchpad) Details panel (Security)  ||  
The Security tab is properly displayed with details about the secure connection used including the protocol, the cipher suite, and certificate details.<br>
The tab is displayed if the site is being served over HTTPS.
|| all scenarios provided must have the right expected result || RAC-1 || Manual || Desktop Team  
|| Verifying that the tab contains details about the secure connection used including the protocol, the cipher suite, and certificate details. || RAC-1 || Manual || Desktop Team  
|-
|-


| TO-7 || Toolbar view buttons ||


| 7 || Toolbar view buttons ||
The buttons filter the requests for each case:  All, HTML, CSS, JS, XHR, Fonts, Images, Media, Flash, WS, Other and Filter Urls.
 
  || Verifying that all buttons works and requests are filtered for every scenarios. || RAC-2 || Manual || Desktop Team  
Toolbar view button All, HTML, CSS, JS, XHR, Fonts, Images, Media, Flash, WS, Other and Filter Urls work as intended.<br>
  || all filter buttons works as expected showing the correct filter for each case || RAC-2 || Manual || Desktop Team  
|-
|-


| 8 || Clear button ||  
| TO-8 || Clear button ||  


Clear button work as intended.<br>
The button gives an option to clear all the requests at once.
  || all filter buttons works as expected showing the correct filter for each case || RAC-2 || Manual || Desktop Team  
  || Verifying that no requests are shown in Netmonitor after the button is pressed. || RAC-2 || Manual || Desktop Team  
|-
|-


| TO-9 || Context menu ||


| 9 || Context menu ||  
Context-clicking on a row in the list displays the context menu and containing all options according to requests.
 
|| Verifying that functionality is intact and all the following options are displayed on context menu: Copy URL, Copy URL Parameters, Copy POST Data (only for POST requests), Copy as cURL, Copy Request Headers, Copy Response Headers, Copy Response, Copy Image as Data URI (only for images), Copy All As HAR, Save All As HAR, Save Image As (only for images), Edit and Resend, Open in New Tab, Start Performance Analysis for the page. || RAC-3 || Manual || Desktop Team  
Context menu's options (eg: Save all As har, Edit and Resend, Open in New Tab and Start Performance Analysis) works as intended.<br>
|| all scenarios provided must have the right expected result || RAC-3 || Manual || Desktop Team  
|-
|-


| 10 || Submenu for Context menu ||  
| TO-10 || Submenu for Context menu ||  
Submenu's options (eg: Copy URL, Copy URL Parameters, Copy POST Data, Copy as cURL etc.) works as intended.<br>
Submenu's options is opened form Context menu while hovering over Copy string.
|| all scenarios provided must have the right expected result || RAC-4 || Manual || Desktop Team  
|| Verifying that Submenu options works as intended as well as functionally. Copy URL, Copy URL Parameters, Copy POST Data, Copy as cURL etc, saves the correct data for each request.   || RAC-4 || Manual || Desktop Team  
|-
|-


 
| TO-11 || Netmonitor checkbox  ||  
| 11 || Netmonitor checkbox  ||  
Netmonitor checkboxes gives the option to uncheck the columns as well as adding a new one.
Netmonitor's columns works as intended. <br>
  || Verifying that columns are added in netmonitor as intended and can be unchecked or reset to the default state. || RAC-5 || Manual || Desktop Team  
  || all scenarios provided must have the right expected result || RAC-5 || Manual || Desktop Team  
|-
|-


Line 228: Line 215:
| Start project  
| Start project  
|style="text-align:center;" | 6/12/2017  
|style="text-align:center;" | 6/12/2017  
|style="text-align:center;" | tbd
|style="text-align:center;" | 29/09/2017
|-
|-
| Study documentation/specs received from developers
| Study documentation/specs received from developers
Line 248: Line 235:
|-
|-
| QA - Beta Testing  
| QA - Beta Testing  
|style="text-align:center;" | tbd
|style="text-align:center;" | 08/22/2017
|style="text-align:center;" | tbd
|style="text-align:center;" | 09/26/2017
|-
|-
| Release Date  
| Release Date  
|style="text-align:center;" | tbd
|style="text-align:center;" colspan="2" | 09/26/2017
|style="text-align:center;" | tbd
|}
|}


Line 272: Line 258:


== Overview ==
== Overview ==
Track the dates and build number where feature was released to Nightly
* Feature was released to Nightly: [https://archive.mozilla.org/pub/firefox/nightly/2017/04/2017-04-10-03-02-21-mozilla-central/ 55.0a1 (2017-04-10)]
  Track the dates and build number where feature was merged to Release/Beta
* Feature was merged to Beta: [https://archive.mozilla.org/pub/firefox/candidates/56.0b1-candidates/build1/ 56.0b1 (2017-08-07)]
* Feature was merged to Release: TBD


= References =
= References =
* List and links for specs
* List and links for specs
  List and links for available specs - http://searchfox.org/mozilla-central/source/devtools/client/netmonitor/README.md
** [http://searchfox.org/mozilla-central/source/devtools/client/netmonitor/README.md run Netmonitor in Launchpad]
** [https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor MDN article]
 
* Meta bug
* Meta bug
(netmonitor-phaseII) [meta] Load Net Panel In Launchpad) [https://bugzilla.mozilla.org/show_bug.cgi?id=1348737]
** {{bug|1348737}} - (netmonitor-phaseII) [meta] Load Net Panel In Launchpad


= Testcases =  
= Testcases =  
Line 288: Line 277:
|-
|-
| Private Window  
| Private Window  
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Multi-Process Enabled  
| Multi-Process Enabled  
|style="text-align:center;" | ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Multi-process Disabled  
| Multi-process Disabled  
|style="text-align:center;" |  ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Theme (high contrast)  
| Theme (high contrast)  
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| '''UI'''  
| '''UI'''  
Line 303: Line 292:
|-
|-
| Mouse-only operation   
| Mouse-only operation   
|style="text-align:center;" |  ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Keyboard-only operation   
| Keyboard-only operation   
|style="text-align:center;" | ||
|style="text-align:center;" | No ||
|-
|-
| Display (HiDPI)  
| Display (HiDPI)  
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Interaction (scroll, zoom)  
| Interaction (scroll, zoom)  
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Usable with a screen reader   
| Usable with a screen reader   
|style="text-align:center;" | || e.g. with NVDA
|style="text-align:center;" | No || e.g. with NVDA
|-
|-
| Usability and/or discoverability testing   
| Usability and/or discoverability testing   
|style="text-align:center;" | || Is this feature user friendly
|style="text-align:center;" | Yes || Is this feature user friendly
|-
|-
| RTL build testing   
| RTL build testing   
|style="text-align:center;" | ||
|style="text-align:center;" | No ||
|-
|-
| '''Help/Support'''  
| '''Help/Support'''  
Line 327: Line 316:
|-
|-
| Help/support interface required   
| Help/support interface required   
|style="text-align:center;" |  || Make sure link to support/help page exist and is easy reachable.
|style="text-align:center;" | Yes || Make sure link to support/help page exist and is easy reachable.
|-
|-
| Support documents planned(written)   
| Support documents planned(written)   
|style="text-align:center;" |  || Make sure support documents are written and are correct.
|style="text-align:center;" | Yes || Make sure support documents are written and are correct.


|-
|-
Line 337: Line 326:
|-
|-
| Feature upgrades/downgrades data as expected   
| Feature upgrades/downgrades data as expected   
|style="text-align:center;" | ||
|style="text-align:center;" | No ||
|-
|-
| Does sync work across upgrades   
| Does sync work across upgrades   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Requires install testing     
| Requires install testing     
|style="text-align:center;" | || separate feature/application installation needed (not only Firefox)
|style="text-align:center;" | No || separate feature/application installation needed (not only Firefox)
|-
|-
| Affects first-run or onboarding     
| Affects first-run or onboarding     
|style="text-align:center;" | || Florin/Lawrence are investigating if there is a dedicated QA for this, or we should test? Should be an yes/no and if is yes should add in detail column the team/person assigned.
|style="text-align:center;" | No || Florin/Lawrence are investigating if there is a dedicated QA for this, or we should test? Should be an yes/no and if is yes should add in detail column the team/person assigned.
|-
|-
| Does this affect partner builds? Partner build testing   
| Does this affect partner builds? Partner build testing   
|style="text-align:center;" | || yes/no options, add comment with details about who will lead testing
|style="text-align:center;" | No || yes/no options, add comment with details about who will lead testing


|-
|-
Line 356: Line 345:
|-
|-
| Enterprise administration   
| Enterprise administration   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Network proxies/autoconfig   
| Network proxies/autoconfig   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| ESR behavior changes   
| ESR behavior changes   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Locked preferences   
| Locked preferences   
|style="text-align:center;" |  ||
|style="text-align:center;" | No ||


|-
|-
Line 372: Line 361:
|-
|-
| Temporary or permanent telemetry monitoring   
| Temporary or permanent telemetry monitoring   
|style="text-align:center;" | || List of error conditions to monitor
|style="text-align:center;" | No || List of error conditions to monitor
|-
|-
| Telemetry correctness testing   
| Telemetry correctness testing   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Server integration testing   
| Server integration testing   
|style="text-align:center;" |  ||  
|style="text-align:center;" | No ||  
|-
|-
| Offline and server failure testing   
| Offline and server failure testing   
|style="text-align:center;" |  ||
|style="text-align:center;" | No ||
|-
|-
| Load testing   
| Load testing   
|style="text-align:center;" |  ||
|style="text-align:center;" | No ||


|-
|-
Line 391: Line 380:
|-
|-
| Addon API required?   
| Addon API required?   
|style="text-align:center;" |  ||  
|style="text-align:center;" |  No ||  
|-
|-
| Comprehensive API testing   
| Comprehensive API testing   
|style="text-align:center;" |  ||  
|style="text-align:center;" | No ||  
|-
|-
| Permissions   
| Permissions   
|style="text-align:center;" |  ||  
|style="text-align:center;" | No ||  
|-
|-
| Testing with existing/popular addons
| Testing with existing/popular addons
|style="text-align:center;" |  ||  
|style="text-align:center;" | No ||  


|-
|-
Line 407: Line 396:
|-
|-
| 3rd-party security review   
| 3rd-party security review   
|style="text-align:center;" |  ||  
|style="text-align:center;" | No ||  
|-
|-
| Privilege escalation testing
| Privilege escalation testing
|style="text-align:center;" |  ||  
|style="text-align:center;" | No ||  
|-
|-
| Fuzzing   
| Fuzzing   
|style="text-align:center;" |  ||  
|style="text-align:center;" | No ||  


|-
|-
Line 420: Line 409:
|-
|-
| Testing against target sites   
| Testing against target sites   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Survey of many sites for compatibility   
| Survey of many sites for compatibility   
|style="text-align:center;" |  ||  
|style="text-align:center;" | No ||  


|-
|-
Line 430: Line 419:
|-
|-
| Common protocol/data format with other software: specification available. Interop testing with other common clients or servers.   
| Common protocol/data format with other software: specification available. Interop testing with other common clients or servers.   
|style="text-align:center;" |  ||  
|style="text-align:center;" | Yes ||  
|-
|-
| Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS   
| Coordinated testing/interop across the Firefoxes: Desktop, Android, iOS   
|style="text-align:center;" | ||  
|style="text-align:center;" | No ||  
|-
|-
| Interaction of this feature with other browser features   
| Interaction of this feature with other browser features   
|style="text-align:center;" |  ||  
|style="text-align:center;" | No ||  
|}
|}


== Test suite ==
== Test suite ==
Full Test suite - Link to test rail - tascases should be added under Firefox Desktop project [https://testrail.stage.mozaws.net/index.php?/suites/view/1108&group_by=cases:section_id&group_order=asc link]
* Full Test suite - [https://testrail.stage.mozaws.net/index.php?/suites/view/1108&group_by=cases:section_id&group_order=asc TestRail link]
Smoke Test suite - Link with the tests - if available/needed.
* Smoke Test suite - [https://testrail.stage.mozaws.net/index.php?/suites/view/1108&group_by=cases:section_id&group_order=asc TestRail link]
Regression Test suite - Link with the tests - if available/needed.
* Regression Test suite - n/a.


= Bug Work =
= Bug Work =
Line 484: Line 473:
'''Nightly testing'''<br />
'''Nightly testing'''<br />


List of OSes that will be covered by testing<br />
* List of OSes that will be covered by testing:<br />
*Link for the tests run
** Windows 10 x64
** Full Test suite, link to TestRail - Tests Runs and Results [https://testrail.stage.mozaws.net/index.php?/runs/overview/17 link]
** Windows 7 x86
** Daily Smoke, if needed/available
** Ubuntu 16.04 x64
** Regression Test suite, if needed/available
** Mac OS X 10.11.6
<br />
* Link for the tests run - [https://testrail.stage.mozaws.net/index.php?/reports/view/411 link]


'''Merge to Beta Sign-off'''<br />
'''Merge to Beta Sign-Off'''
List of OSes that will be covered by testing<br />
* List of OSes that will be covered by testing<br />
*Link for the tests run
** Windows 10 x64
** Full Test suite
** Windows 7 x86
** Mac OS X 10.11.6
** Ubuntu 16.04 x64
* Link for the tests run - [https://testrail.stage.mozaws.net/index.php?/reports/view/473 link]


= Checklist =
== Checklist ==
{| class="wikitable" style="width:60%"
{| class="wikitable"
|-
|-
! Exit Criteria !! Status !! Notes/Details
! Exit Criteria !! Status !! Notes/Details
|-
|-
| Testing Prerequisites (specs, use cases)  
| Testing Prerequisites (specs, use cases)  
| style="text-align:center;" |  Done
| style="text-align:center;" |  {{mdone|}}
| style="text-align:center;" |  
| style="text-align:center;" |  
|-
|-
| Testing Infrastructure setup  
| Testing Infrastructure setup  
|style="text-align:center;" |  Done ||  
|style="text-align:center;" |  {{mdone|}} ||  
|-
|-
| Test Plan Creation  
| Test Plan Creation  
| style="text-align:center;" | Done ||  
| style="text-align:center;" |  {{mdone|}} ||  
|-
|-
| Test Cases Creation  
| Test Cases Creation  
|style="text-align:center;" |  Done ||  
|style="text-align:center;" |  {{mdone|}} ||  
|-
|-
|  Automation Coverage ||  
| Full Functional Tests Execution
|style="text-align:center;" |  {{mdone|}}
|style="text-align:center;" |   
|style="text-align:center;" |   
|-
|-
| Performance Testing
| Automation Coverage
|style="text-align:center;" | ||  
|style="text-align:center;" | N/A ||  
 
|-
|-
| All Defects Logged || ||  
| Performance Testing
|style="text-align:center;" | <font style="font-weight:bold;font-size:80%;">[NOT APPLICABLE]</font> ||  
|-
|-
| Critical/Blockers Fixed and Verified || ||
| All Defects Logged
|-
|style="text-align:center;" | {{mdone|}} ||  
|  Metrics/Telemetry||
|style="text-align:center;" |  
|-
| Basic/Core functionality Nightly testing
|style="text-align:center;" | 
|style="text-align:center;" |
|-
|-
| QA mid-Nightly Signoff||
| Critical/Blockers Fixed and Verified
|style="text-align:center;" | Done (YELLOW) [https://testrail.stage.mozaws.net/index.php?/reports/view/411 see testrail report]
|style="text-align:center;" | {{mdone|}}  ||
|-
|-
| QA Nightly - Full Testing
| Metrics/Telemetry
|style="text-align:center;" | ||  
|style="text-align:center;" | N/A ||
|-
|-
|  QA pre-Beta Signoff||
| QA Signoff - Nightly Release
|style="text-align:center;"| Done (YELLOW) [https://testrail.stage.mozaws.net/index.php?/reports/view/473 see testrail report]
|style="text-align:center;" | {{mdone|}}
|| [https://testrail.stage.mozaws.net/index.php?/reports/view/473 Email sent - YELLOW]
|-
|-
| QA Beta - Full Testing
| QA Beta - Full Testing
|style="text-align:center;" | ||  
|style="text-align:center;" | {{mdone|}} ||  
|-
|-
|  QA pre-Release Signoff ||
| QA Signoff - Beta Release
|style="text-align:center;" | Email to be sent  
| style="text-align:center;" | {{mdone|}} || [https://testrail.stage.mozaws.net/index.php?/reports/view/544 Email sent - GREEN]
|}
|}
Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1179340...1180425"