View source for QA/Client Crash Id

''' QA/Client Crash Id'''<br />


'''Revision History'''


{| class="wikitable" style="width:60%"
|-
! Date !! Version !! Author !! Description 
|-
| 02/27/2017 || 1.0 || Abe Masresha || Created first draft
|-
| 05/03/2017  || 1.0 || Petruta Rasa  ||  Added tests and results
|-
|}

= Overview =
== Purpose ==
The goal of this feature is to align the break pad-generated crash id with a crash report.

== Scope ==
* Testing will focus on: 

::: Stability of the browser with this work.
::: Functionality and stability of Crash Reporter Client.
::: Comparing contents of pings from ping sender and restart.
::: Comparing values of minidumpSha256Hash of ping sender ping and restart ping
::: Comparing SHA256 hash of the minidump file and value of minidumpSha256Hash from one of the crash pings.
::: Comparing SHA256 hash of the minidump file and value of minidumpSha256Hash from the Content crash ping.

* How testing will be performed:
:::  Manual

== Risk analysis ==
This work includes modifications to both the crash reporter client and the code that handles content process crashes. So the risk will mainly base on those two: 
*First, it may break the crash reporter client in a way that prevents the user from sending crash reports or causes submitted crash reports to be malformed
*Second, it may not send crash pings for the content crash, or send malformed crash pings.

== Ownership ==
 '''Mozilla Development'''<br />
 Gabriele Svelto<br />

'''QA Eng. Softvision Las Vegas'''<br />
[mailto:stefan.georgiev@softvision.com Stefan Georgiev - Team Lead ]<br />
[mailto:amasresha@softvision.com Abe Masresha - Test Lead]<br />
[mailto:jwilliams@softvision.com Justin Williams - Tester]<br />
[mailto:gwimberly@softvision.com Grover Wimberly IV - Tester]<br />
[mailto:kkumari@softvision.com Kanchan Kumari - Tester]<br />
[mailto:petruta.rasa@softvision.com Petruta Rasa - Tester]<br />

= Testing summary = 
== Scope of Testing ==
=== In Scope ===
This feature will be tested manually on Windows, Mac and Linux operating system.

=== Out of Scope ===
Automation will be handled by the developer (if needed)<br />
iOS devices will not be tested

= Requirements for testing =
== Environments: Operating Systems ==
 Windows 10 64bits
 Windows 7 32 bits
 Ubuntu 16.04 32 and 64bits
 MacOSX 10.12
 Android devices

== Firefox Channel and environment setups ==

=== Build ===
Firefox Nightly 55.0a1

= Test Strategy = 
== Test Objectives ==
Verify Firefox is stable with this feature<br>
Verify two crash pings are received. One from the ping sender (first ping) and the other after the browser restarts (second ping).
Verify the first and the second ping contain minidumpSha256Hash field in the payload element.<br />
Verify the first and the second ping have identical value for the minidumpSha256Hash field<br>
Verify SHA256 hash of the minidump file and the value of minidumpSha256Hash in crash ping are identical.<br />
Verify content crash has only one ping<br/>
Verify value of minidumpSha256Hash of the content crash is identical to the hash of the minidump file<br>
Verify Firefox restarts successfully after crash<br>
Verify crash reporter client is responsive and functional after or before the first ping<br>

== Test Execution Schedule ==
{| class="wikitable" style="width:60%"
|-
! Project phase !! Start Date !! End Date
|-
| Start project 
|style="text-align:center;" |02/24/2017  || TBD
|-
| Study documentation/specs received from developers
|style="text-align:center;" |02/24/2017  || 02/16/2017
|-
| QA - Test plan creation 
|style="text-align:center;" | 02/27/2017 || TBD
|-
| QA - Test cases/Env preparation 
|style="text-align:center;" | 02/27/2017  || TBD
|-
| QA - Nightly
|style="text-align:center;" | TBD || TBD
|-
|}

== Testing Tools ==
Detail the tools to be used for testing, for example see the following table:
{| class="wikitable" style="width:50%"
|-
! Process !! Tool
|-
| Test plan creation || Mozilla wiki
|-
| Test case creation || [https://docs.google.com/a/mozilla.com/spreadsheets/d/1O9cf70gxWkmmHTW3FIY_u8C-tc8LuAwDU0ai2TSdm64/edit?usp=sharing Test Cases GDoc]
|-
| Test case execution || [https://testrail.stage.mozaws.net/index.php?/suites/view/925&group_by=cases:section_id&group_order=asc]
|-
| Bugs management || Bugzilla

|}

= Status = 
== Overview ==
= References =
Test cases: [https://testrail.stage.mozaws.net/index.php?/suites/view/925&group_by=cases:section_id&group_order=asc Test Cases TestRail] or [https://docs.google.com/a/mozilla.com/spreadsheets/d/1O9cf70gxWkmmHTW3FIY_u8C-tc8LuAwDU0ai2TSdm64/edit?usp=sharing Test Cases GDoc]
          
Bugs:  [https://bugzilla.mozilla.org/show_bug.cgi?id=1322611   Bug 1322611]<br />

= Testcases = 
== Overview ==
 Summary of testing scenarios

= Test Areas =
{| class="wikitable" style="width:80%"
|-
! Test Areas !! Covered !! Details
|-
| Private Window 
|style="text-align:center;" | Yes || Few tests will be made on private window as well
|-
| Non e10s Window 
|style="text-align:center;" | Yes || Few tests will be made with non e10s window as well
|-
|| '''Install/Upgrade''' 
||  || 
|-
| Feature upgrades/downgrades data as expected   
|style="text-align:center;" | Yes  || Feature will be tested against browser updates
|-
| ''' Data Monitoring ''' 
||  || 
|-
| Temporary or permanent telemetry monitoring   
|style="text-align:center;" | Yes || Data from Local telemetry server will be monitored
|-
| Telemetry correctness testing   
|style="text-align:center;" | Yes || Data correctness will be tested.
|-
| Server integration testing   
|style="text-align:center;" | Yes || Feature will send data to a telemetry server and this functionality will be tested
|-
| Offline and server failure testing   
|style="text-align:center;" | Yes || Server offline option will be included in testing
|-
| ''' Add-ons ''' ||
|-
| Testing with existing/popular add-ons
|style="text-align:center;" | Yes || feature functionality will be tested with popular addons
|-
|-
| ''' Web Compatibility ''' 
||  ||  
|-
| Survey of popular sites for compatibility   
|style="text-align:center;" | Yes || Firefox will be tested for stability

|-
| ''' Interoperability  ''' 
||  ||  
|-
| Common protocol/data format with other software: specification available. Interop testing with other common clients or servers.   
|style="text-align:center;" | Yes ||  Hashed minidump file and restart ping contents will be compared
|-
| Interaction of this feature with other browser features   
|style="text-align:center;" | Yes ||  Firefox will be tested for stability across popular sites.
|}

== Test suite ==
 Full Test suite - [https://testrail.stage.mozaws.net/index.php?/suites/view/925&group_by=cases:section_id&group_order=asc  Test Cases TestRail]/ [https://docs.google.com/a/mozilla.com/spreadsheets/d/1O9cf70gxWkmmHTW3FIY_u8C-tc8LuAwDU0ai2TSdm64/edit?usp=sharing   Test Cases GDoc]<br />

= Bug Work =
Tracking bugs – [https://bugzilla.mozilla.org/show_bug.cgi?id=1322611   bug]<br />

====== Bugs ======
<!--
{| class="wikitable" style="width:50%"
|-
! Bug No !! Summary !! Status !! Firefox Version
|-
||| || || 54
|-
| ||  || || 54
|-

|}
-->

= Sign off =
== Criteria ==
Check list
* All test cases should be executed
* All blocker and critical bugs must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/developer/QA)

== Results ==
'''Feature testing'''<br />

*Links for the tests run
** Full Test suite: [ Test Cases runs TestRail]
** Regression Test suite: [link] if needed/available

== Checklist ==
{| class="wikitable" style="width:60%"
|-
! Exit Criteria !! Status !! Notes/Details
|-
|  Testing Prerequisites (specs, use cases) 
| style="text-align:center;" |  {{mdone|}}
| style="text-align:center;" |  
|-
|  Testing Infrastructure setup 
|style="text-align:center;" | {{mdone|}}
| 
|-
|  Test Plan Creation 
| style="text-align:center;" | {{mdone|}}
| 
|-
|  Test Cases Creation 
|style="text-align:center;" | {{mdone|}}

|style="text-align:center;" | [https://testrail.stage.mozaws.net/index.php?/suites/view/925&group_by=cases:section_id&group_order=asc Test Cases TestRail]/ [https://docs.google.com/a/mozilla.com/spreadsheets/d/1O9cf70gxWkmmHTW3FIY_u8C-tc8LuAwDU0ai2TSdm64/edit?usp=sharing Test Cases GDoc]
|-
|  Full Functional Tests Execution 
|style="text-align:center;" |  
|style="text-align:center;" |  
|-
|  Automation Coverage ||
|style="text-align:center;" | Will be covered by developer if needed
|-
|  Performance Testing ||
|style="text-align:center;" |  Will be covered by developer if needed
|-
|  Load Testing ||
|style="text-align:center;" |  Will be covered by developer if needed
|-
|  All Defects Logged || || 
|-
|  Critical/Blockers Fixed and Verified || || 
|-
|  Metrics/Telemetry|| 
|style="text-align:center;" | 
|-
|  QA Signoff  - mid-Nightly || {{mdone|}}  
|style="text-align:center;" | 
|-
|  QA Signoff  - Nightly Release || {{mdone|}}  
|style="text-align:center;"|
|-
|  QA Beta - Full Testing || {{mdone|}} 
|style="text-align:center;" |   
|-
|  QA Signoff  - Beta Release || {{mdone|}}  
|style="text-align:center;" |  
|}