QA/Fennec/Presentation API: Difference between revisions

Jump to navigation Jump to search

QA/Fennec/Presentation API (view source)

Revision as of 16:08, 14 November 2016

2,529 bytes added ,  14 November 2016
no edit summary
No edit summary
No edit summary
Line 1: Line 1:
Please take notice that this is a very rough draft, with some bullet points for future work
= Overview =
== Purpose ==
The purpose of this wiki is to serve as a general test plan for verifying that Presentation API is correctly implemented.


Bugzilla:
== Ownership ==
https://bugzilla.mozilla.org/show_bug.cgi?id=1184036
Developer contact: [mailto:schien@mozilla.com Shih-Chiang Chien (:schien)]


QA: [mailto:mihai.g.pop@softvision.ro Mihai Pop (:MihaiP)]


Testplan:
== Overall Status ==
*Run w3c test cases (include all tests from http://w3c-test.org/presentation-api/)
*Run a smoke test on Shih-Chiang Chien’s build (verify that the new bugs regressed from presentation API = check vs latest nightly)
*Monitor crash-rate (check with schien to have crash reporter enabled on his build)
*Until then, check adb logcat
*Monitor power consumption (schien’s build vs latest nightly)
*https://developer.android.com/studio/profile/battery-historian.html


Test run for http://w3c-test.org/presentation-api/controlling-ua/ :
<small>'''<font color="orange">[AT RISK]</font>'''</small> Nightly 53: it's going to be prefed on, on '''15-11-2016'''
*https://docs.google.com/spreadsheets/d/1uub8yabL0-_7TUaBPZ7sbhqYNcn936IYQSU6vNhMULY/edit#gid=2140443213
* At risk due to {{Bug|1313368}} - [Presentation API] Fennec stalls when trying to load a page (This is reproducing only on particular devices and on Presentation API pages, so we don't know exactly how many users this issue is affecting)


Concerns:
= Testing Summary =
*Bug 1313368 - [Presentation API] Fennec stalls when trying to load a page
== Scope of testing ==
*Bug 1313404 - [Presentation API] Fennec crashes after running a test from http://w3c-test.org
The testing will be focusing on:
* Verify that the manual tests made by W3C for presentation API pass
* Monitor crash rate, stability, and power consumption
* Check that no important issues are regressing from the feature work
 
== Environments ==
Testing will be performed on following devices:  
* Nexus Player
* Chromecast
* Phones and Tablets
** Android 4.4+
 
== Specifications ==
* https://www.w3.org/TR/presentation-api/#api
 
== User stories ==
* Use cases and requirements for Presentation API
** https://github.com/w3c/presentation-api/blob/gh-pages/uc-req.md
 
== Acceptance Criteria==
*The feature does not introduce new regressions that affect performance and stability
*All w3c tests pass
 
==KPIs / Measurements Used:
*Qualitative:
** No negative feedback from users
 
= Testing details =
* Environment:
- Receiver (Presentation Display): "Nexus Player + TV" or "Chromecast +TV"
- Sender: Android device with Android 4.4 or higher
- Fennec Build: https://people-mozilla.org/~schien/presentation-api/fennec-52.0a1.en-US.android-arm.apk
* Test cases(Latest test result: 14.11.2016):
- https://docs.google.com/spreadsheets/d/1uub8yabL0-_7TUaBPZ7sbhqYNcn936IYQSU6vNhMULY/edit#gid=1378915478
  - All tests passed except a known w3c issue, and 2 tests are Not Applicable
- https://docs.google.com/spreadsheets/d/1xfbFJw2nfSAL3Bd4LIXMkdUseI1Cq3aZW8ZrDmU5Hc4/edit#gid=284504573
  - 11/12 tests PASSED
 
=Bug Work=
==Tracking bugs==
* Bugs to track the Presentation API feature
* [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1184036&hide_resolved=0 Dependencies]
* meta bug: {{Bug|1184036}} - (1-UA_Presentation_API) [meta] 1-UA support for Presentation API
ex:
<onlyinclude>
<bugzilla>
{
"id": "1197690, 1268758, 1069230, 1304976, 1304969, 1215426, 1288297, 1306536, 1254488, 1282003, 1304971, 1208417, 1297475, 1297018, 1316592, 1313601, 1252788, 1262779, 1274749, 1204709, 1299705, 1291971, 1224445, 1205233, 1267923, 1314229, 1161440, 1305352, 1299061, 1316584, 1228526, 1289974, 1235124, 1232105, 1236777, 1278205, 1306210, 1273628, 1267965, 1153063, 1214179, 1234492, 1295087, 1151002, 1259349, 1205237, 1235123, 1292057, 1121566, 1268810, 1285870, 1258600, 1120308, 1258978, 1288600, 1301259, 1313404, 1214176, 1254489, 1212702, 1205222, 1272197, 1254484, 1258602, 1276380, 1276116, 1263107, 1192101, 1314213, 1080474, 1152522, 1268795, 1194049, 1173263, 1195221, 1205219, 1234128, 1313368, 1275157, 1304978, 1254477, 1210340, 1311375, 1305351"
}
</bugzilla>
</onlyinclude>
 
=Signoff=
== Criteria ==
* All test cases should be executed
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed
 
== Results ==
'''Nightly testing'''
*Not landed yet
 
'''Aurora testing'''
*Not landed yet
 
'''Beta testing'''
*Not landed yet
MihaiP
Confirmed users
1,235

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1154565"

Navigation menu