QA/Platform/Graphics/Features/GPU Process: Difference between revisions

Jump to navigation Jump to search

QA/Platform/Graphics/Features/GPU Process (view source)

Revision as of 22:37, 10 March 2017

4,300 bytes added ,  10 March 2017
→‎Beta
Line 693: Line 693:


== Beta ==
== Beta ==
{| class="wikitable" style="width:60%"
{| class="wikitable" style="width:100%;"
|-
|-
! Criteria !! Status !! Notes/Details
! style="width:30%" | Criteria
! style="width:10%" | Status
! style="width:60%" | Notes
|-
|-
|  Testing Prerequisites (specs, use cases)
! colspan="3" style="text-align:left" | Testing
| style="text-align:center;" | 
| style="text-align:center;" |  
|-
|-
| Testing Infrastructure setup
| style="vertical-align:top" | Manual Test Coverage
|style="text-align:center;" |   ||  
| style="text-align:center; vertical-align:top" |  
|
* Softvision mid-Beta sign-off via Testrail
* Softvision pre-Release sign-off via Testrail
* Beta testday
|-
|-
| Test Plan Creation
| style="vertical-align:top" | Automated Test Coverage
| style="text-align:center;" |   ||  
| style="text-align:center; vertical-align:top" |  
| Coverage limited to process instantiation. Risks to be mitigated via manual testing.
|-
|-
| Test Cases Creation
| Performance Test Coverage
|style="text-align:center;" |   ||  
| style="text-align:center; vertical-align:top" |  
|  
* Passmark benchmark completed by Softvision
* WebGL benchmark completed by Softvision
* HWA Video benchmark completed by Softvision
|-
|-
|  Full Functional Tests Execution
! colspan="3" style="text-align:left" | Bug Reports
|style="text-align:center;" | 
|style="text-align:center;" |
|-
|-
| Automation Coverage ||
| style="vertical-align:top" | [https://mzl.la/2mGuyaD No unresolved blocker bugs] <br /> [https://mzl.la/2mGmR4k No untriaged bugs blocking Fx53]
|style="text-align:center;" |  
| style="text-align:center; vertical-align:top" |
| <u>Bugs pending Firefox 53 status review:</u><br />
<bugzilla display="list">
{
"blocks":"1264543",
"f1":"cf_status_firefox53",
"o1":"anywordssubstr",
"v1":"affected, ?",
"include_fields":"id,summary,status"
}
</bugzilla>
|-
|-
| Performance Testing
| style="vertical-align:top" | [https://mzl.la/2lA94Lt No unverified fixed bugs]
|style="text-align:center;" |  ||
| style="text-align:center; vertical-align:top" |
|<u>Bugs pending Firefox 53 status review:</u><br />
<bugzilla display="list">
{
"blocks":"1264543",
"resolution":"FIXED",
"f1":"cf_status_firefox53",
"o1":"notequals",
"v1":"fixed",
  "f2":"target_milestone",
"o2":"anywordssubstr",
"v2":"---, mozilla53",
"include_fields":"id,summary,status"
}
</bugzilla>
|-
|-
|  All Defects Logged || ||  
! colspan="3" style="text-align:left" | Metrics
|-  
| style="vertical-align:top" | [https://mzl.la/2kU1sq9 GPU_PROCESS_LAUNCH_TIME_MS_2 distribution <= 3s]
| style="vertical-align:top; text-align:center" |
| ''Note: A value larger than 3s likely represents users failing to get GPU Process''
{| class="wikitable"
! Percentile || 2017-03-13 || Net
|-
|-
| Critical/Blockers Fixed and Verified || ||
| 5th
|-
|-
| Metrics/Telemetry||
| 25th
|style="text-align:center;" |
|-
|-
| QA Signoff - Beta 1..n ||  
| 50th
|style="text-align:center;" | Email to be sent
|-
| 75th
|-
| 95th
|-
! 3s Threshold
|-
! 1s Threshold
|}
|-
| style="vertical-align:top" | [https://mzl.la/2kXKjvM GPU_PROCESS_LAUNCH_TIME_MS_2 evolution doesn't regress]
| style="vertical-align:top; text-align:center" |
| ''Note: A value larger than 3s likely represents users failing to get GPU Process''
{| class="wikitable"
! Percentile || 2017-03-13 || Net
|-
| 5th
|-
| 25th
|-
| 50th
|-
| 75th
|-
| 95th
|-
|}
|-
| style="vertical-align:top" | [https://sql.telemetry.mozilla.org/dashboard/gfx-gpu-process-crashes Telemetry graphics crash count/rate largely unchanged]
| style="text-align:center; vertical-align:top" |
|
{| class="wikitable"
! || 2017-03-13 || Net
|-
| Crash Rate Average
|-
| Crash Count Average
|-
| Crash Count Percent
|}
|-
| style="vertical-align:top" | [https://crash-stats.mozilla.com/topcrashers/?product=Firefox&version=53.0a2&days=7&process_type=gpu No unresolved topcrashes correlated to GPU Process]
| style="text-align:center; vertical-align:top" |  
|
<u>GPU crashes rank ?th overall @ ?% with a combined ? reports</u><br />
1st-ranked is #? overall @ ?% (<signature> - <bug>)<br />
2nd-ranked is #? overall @ ?% (<signature> - <bug>)<br />
3rd-ranked is #? overall @ ?% (<signature> - <bug>)<br />
4th-ranked is #? overall @ ?% (<signature> - <bug>)<br />
5th-ranked is #? overall @ ?% (<signature> - <bug>)
|-
| style="vertical-align:top" | Telemetry Experiment reveals no regressions in A/B testing
| style="text-align:center; vertical-align:top" |  
|
* Compare SIMPLE_MEASURES_FIRSTPAINT as a proxy for performance
* Compare crash counts/rates of cohorts in Telemetry
* Compare crash counts/rates of cohorts in Socorro
* Compare crash signatures of cohorts in Socorro
|-
| style="vertical-align:top" | Percentage of users w/GPU Process and hardware profiles
| style="text-align:center; vertical-align:top" |
|
|-
| style="vertical-align:top" | [https://people-mozilla.org/~danderson/moz-gfx-telemetry/www/#view=windows-features Insignificant change in hardware acceleration distribution]
| style="text-align:center; vertical-align:top" |
|
GPU Process Status:<br />
<pre>
Date        Available  Unavailable  Failed  Blocked
2017-03-13           
</pre>
D3D11 Status:<br />
<pre>
Date        Available  Unavailable  Failed  Blocked  Blacklisted
2017-03-13     
</pre>
Media Decoder Status:<br />
<pre>
Date        DXVA D3D9  DXVA D3D11  WMF Software
2017-03-13     
</pre>
|-
| style="vertical-align:top" | [https://mzl.la/2lnuYCz SIMPLE_MEASURES_FIRSTPAINT does not regress more than 5%]
| style="text-align:center; vertical-align:top" |
|
<pre>
Median Improvement
Beta 53
Beta 52
Beta 51
Beta 50
</pre>
''Note: even if it does regress it'll be hard to prove that it's GPU Process related and not some other change(s)''
|-
! colspan="3" style="text-align:left" | QA Signoffs
|-
| Okay to Merge to Release
| style="text-align:center; vertical-align:top" |
|
|-
| Release migration signed-off
| style="text-align:center; vertical-align:top" |
|
|-
| Release notes signed-off
| style="text-align:center; vertical-align:top" |
|
|-
|-
| QA Signoff - Release Migration ||
|style="text-align:center;" | Email to be sent
|}
|}


Ashughes
Confirmed users
14,525

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1165361"

Navigation menu