QA/Fennec/Presentation API: Difference between revisions

QA/Fennec/Presentation API (view source)

Revision as of 16:08, 14 November 2016

2,529 bytes added , 14 November 2016

no edit summary

MihaiP

Confirmed users

1,235

edits

@@ Line 1: / Line 1: @@
-Please take notice that this is a very rough draft, with some bullet points for future work
+= Overview =
+== Purpose ==
+The purpose of this wiki is to serve as a general test plan for verifying that Presentation API is correctly implemented.
-Bugzilla:
+== Ownership ==
-https://bugzilla.mozilla.org/show_bug.cgi?id=1184036
+Developer contact: [mailto:schien@mozilla.com Shih-Chiang Chien (:schien)]
+QA: [mailto:mihai.g.pop@softvision.ro Mihai Pop (:MihaiP)]
-Testplan:
+== Overall Status ==
-*Run w3c test cases (include all tests from http://w3c-test.org/presentation-api/)
-*Run a smoke test on Shih-Chiang Chien’s build (verify that the new bugs regressed from presentation API = check vs latest nightly)
-*Monitor crash-rate (check with schien to have crash reporter enabled on his build)
-*Until then, check adb logcat
-*Monitor power consumption (schien’s build vs latest nightly)
-*https://developer.android.com/studio/profile/battery-historian.html
-Test run for http://w3c-test.org/presentation-api/controlling-ua/ :
+<small>'''<font color="orange">[AT RISK]</font>'''</small> Nightly 53: it's going to be prefed on, on '''15-11-2016'''
-*https://docs.google.com/spreadsheets/d/1uub8yabL0-_7TUaBPZ7sbhqYNcn936IYQSU6vNhMULY/edit#gid=2140443213
+* At risk due to {{Bug|1313368}} - [Presentation API] Fennec stalls when trying to load a page (This is reproducing only on particular devices and on Presentation API pages, so we don't know exactly how many users this issue is affecting)
-Concerns:
+= Testing Summary =
-*Bug 1313368 - [Presentation API] Fennec stalls when trying to load a page
+== Scope of testing ==
-*Bug 1313404 - [Presentation API] Fennec crashes after running a test from http://w3c-test.org
+The testing will be focusing on:
+* Verify that the manual tests made by W3C for presentation API pass
+* Monitor crash rate, stability, and power consumption
+* Check that no important issues are regressing from the feature work
+== Environments ==
+Testing will be performed on following devices:
+* Nexus Player
+* Chromecast
+* Phones and Tablets
+** Android 4.4+
+== Specifications ==
+* https://www.w3.org/TR/presentation-api/#api
+== User stories ==
+* Use cases and requirements for Presentation API
+** https://github.com/w3c/presentation-api/blob/gh-pages/uc-req.md
+== Acceptance Criteria==
+*The feature does not introduce new regressions that affect performance and stability
+*All w3c tests pass
+==KPIs / Measurements Used:
+*Qualitative:
+** No negative feedback from users
+= Testing details =
+* Environment:
+ - Receiver (Presentation Display): "Nexus Player + TV" or "Chromecast +TV"
+ - Sender: Android device with Android 4.4 or higher
+ - Fennec Build: https://people-mozilla.org/~schien/presentation-api/fennec-52.0a1.en-US.android-arm.apk
+* Test cases(Latest test result: 14.11.2016):
+ - https://docs.google.com/spreadsheets/d/1uub8yabL0-_7TUaBPZ7sbhqYNcn936IYQSU6vNhMULY/edit#gid=1378915478
+   - All tests passed except a known w3c issue, and 2 tests are Not Applicable
+ - https://docs.google.com/spreadsheets/d/1xfbFJw2nfSAL3Bd4LIXMkdUseI1Cq3aZW8ZrDmU5Hc4/edit#gid=284504573
+   - 11/12 tests PASSED
+=Bug Work=
+==Tracking bugs==
+* Bugs to track the Presentation API feature
+* [https://bugzilla.mozilla.org/showdependencytree.cgi?id=1184036&hide_resolved=0 Dependencies]
+* meta bug: {{Bug|1184036}} - (1-UA_Presentation_API) [meta] 1-UA support for Presentation API
+ex:
+<onlyinclude>
+<bugzilla>
+{
+ "id": "1197690, 1268758, 1069230, 1304976, 1304969, 1215426, 1288297, 1306536, 1254488, 1282003, 1304971, 1208417, 1297475, 1297018, 1316592, 1313601, 1252788, 1262779, 1274749, 1204709, 1299705, 1291971, 1224445, 1205233, 1267923, 1314229, 1161440, 1305352, 1299061, 1316584, 1228526, 1289974, 1235124, 1232105, 1236777, 1278205, 1306210, 1273628, 1267965, 1153063, 1214179, 1234492, 1295087, 1151002, 1259349, 1205237, 1235123, 1292057, 1121566, 1268810, 1285870, 1258600, 1120308, 1258978, 1288600, 1301259, 1313404, 1214176, 1254489, 1212702, 1205222, 1272197, 1254484, 1258602, 1276380, 1276116, 1263107, 1192101, 1314213, 1080474, 1152522, 1268795, 1194049, 1173263, 1195221, 1205219, 1234128, 1313368, 1275157, 1304978, 1254477, 1210340, 1311375, 1305351"
+}
+</bugzilla>
+</onlyinclude>
+=Signoff=
+== Criteria ==
+* All test cases should be executed
+* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed
+== Results ==
+'''Nightly testing'''
+*Not landed yet
+'''Aurora testing'''
+*Not landed yet
+'''Beta testing'''
+*Not landed yet