MozillaWiki

QA/Fennec/draft mihai 2

< QA‎ | Fennec
Revision as of 16:00, 14 November 2016 by MihaiP (talk | contribs) (→‎Overall Status)

Overview

Purpose

The purpose of this wiki is to serve as a general test plan for verifying that Presentation API is correctly implemented.

Ownership

Developer contact: Shih-Chiang Chien (:schien)

QA: Mihai Pop (:MihaiP)

Overall Status

<medium>[AT RISK]</medium> Nightly 53: it's going to be prefed on, on 15-11-2016

Testing Summary

Scope of testing

The testing will be focusing on:

  • Verify that the manual tests made by W3C for presentation API pass
  • Monitor crash rate, stability, and power consumption
  • Check that no important issues are regressing from the feature work

Environments

Testing will be performed on following devices:

  • Nexus Player
  • Chromecast
  • Phones and Tablets
    • Android 4.4+

Specifications

User stories

Acceptance Criteria

  • The feature does not introduce new regressions that affect performance and stability
  • All w3c tests pass

==KPIs / Measurements Used:

  • Qualitative:
    • No negative feedback from users

Testing details

  • Environment:
- Receiver (Presentation Display): "Nexus Player + TV" or "Chromecast +TV"
- Sender: Android device with Android 4.4 or higher
- Fennec Build: https://people-mozilla.org/~schien/presentation-api/fennec-52.0a1.en-US.android-arm.apk
  • Test cases(Latest test result: 14.11.2016):
- https://docs.google.com/spreadsheets/d/1uub8yabL0-_7TUaBPZ7sbhqYNcn936IYQSU6vNhMULY/edit#gid=1378915478
  - All tests passed except a known w3c issue, and 2 tests are Not Applicable
- https://docs.google.com/spreadsheets/d/1xfbFJw2nfSAL3Bd4LIXMkdUseI1Cq3aZW8ZrDmU5Hc4/edit#gid=284504573
  - 11/12 tests PASSED

Bug Work

Tracking bugs

  • Bugs to track the Presentation API feature
  • Dependencies
  • meta bug: bug 1184036 - (1-UA_Presentation_API) [meta] 1-UA support for Presentation API

ex:

Full Query
ID Summary Priority Status
1069230 [Stingray] Presentation API webidl implementation P1 RESOLVED
1080474 [Presentation WebAPI] Device discovery mechanism -- RESOLVED
1120308 [Presentation WebAPI] control protocol establishment and offer-answer exchange -- RESOLVED
1121566 TEST-UNEXPECTED-FAIL | dom/presentation/tests/xpcshell/test_presentation_device_manager.js | xpcshell return code: 0 -- RESOLVED
1151002 Port Bug 1120308 to comm-central - [Presentation WebAPI] control protocol establishment and offer-answer exchange -- RESOLVED
1152522 Intermittent test_tcp_control_channel.js | xpcshell return code: 0 P3 RESOLVED
1153063 trun on logging for debug in test_tcp_control_channel.js -- RESOLVED
1161440 [Presentation WebAPI] trusted UI for device selection on Firefox OS -- RESOLVED
1173263 Integrate with the devices related interface of Presentation API -- RESOLVED
1192101 [Presentation WebAPI] support PresentationRequest / PresentationAvailability / getSession(s) P1 RESOLVED
1194049 NsdManager increases power consumption -- RESOLVED
1195221 Implement LocalPresentationSessionTransport for 1-UA devices -- RESOLVED
1197690 [Presentation WebAPI] Support resuming a disconnected presentation session -- RESOLVED
1204709 Intermittent test_presentation_receiver_start_session_error.html | The control channel is closed abnormally. - got +0, expected 2147500036 -- RESOLVED
1205219 [Presentation WebAPI] Support terminate semantics P1 RESOLVED
1205222 [Presentation WebAPI] Add PresentationReceiver and navigator.presentation.receiver P1 RESOLVED
1205233 [Presentation WebAPI] Make Presentation API available for browser chrome code P1 RESOLVED
1205237 [Presentation WebAPI] get self IP address on Fennec for initiating presentation session P1 RESOLVED
1208417 [Presentation WebAPI] Integrate multi-screen over HDMI into Presentation API -- RESOLVED
1210340 [Presentation WebAPI] Support close semantics -- RESOLVED
1212702 [Presentation WebAPI] Rename to PresentationConnection P1 RESOLVED
1214176 [Presentation WebAPI] fix typo in TCPPresentationServer.js P1 RESOLVED
1214179 [Presentation WebAPI] regression after changing device Id from service name to host name P1 RESOLVED
1215426 [Presentation WebAPI] Grant access to browser receiving pages -- RESOLVED
1224445 Add NS_NATIVE_OPENGL_CONTEXT handling to nsWindow::GetNativeData() on gonk -- RESOLVED
1228526 [Presentation WebAPI] support device filtering in presentation device management -- RESOLVED
1232105 [Presentation WebAPI] provide a trusted UI for device selection on Fennec -- RESOLVED
1234128 [Presentation WebAPI] navigator.presentation.receiver is null in 1-UA use case. -- RESOLVED
1234492 [Presentation WebAPI] Add role in PresentationService -- RESOLVED
1235123 [Presentation WebAPI] Receive PresentationRequestUIGlue events from shell-remote -- RESOLVED
1235124 [Presentation WebAPI] Start multi-screen by Presentation API -- RESOLVED
1236777 Implement FrozenArray in webidl P3 NEW
1252788 Integrate ChromeCast into Presentation API P5 RESOLVED
1254477 [Presentation WebAPI] receiver should always return null for the defaultRequest -- RESOLVED
1254484 [Presentation WebAPI] resolve the relative URL to absolute URL -- RESOLVED
1254488 [Presentation WebAPI] check mixed security context policy while starting a PresentationRequest -- RESOLVED
1254489 [Presentation WebAPI] change initial state of PresentationConnection to 'connecting' -- RESOLVED
1258600 [Presentation WebAPI] implement PresentationConnectionClosedEvent -- RESOLVED
1258602 [Presentation WebAPI] implement PresentationConnectionList -- RESOLVED
1258978 [Presentation WebAPI] would warning if |NotifyTransportClosed| is async called after |transport.close| P5 RESOLVED
1259349 [Presentation WebAPI] support device filtering via request URL while doing device selection -- RESOLVED
1262779 Support device-selection for Presentation API on Fennec -- RESOLVED
1263107 [Presentation WebAPI] need to implement the rejection of offer/answer handshake P5 RESOLVED
1267923 Move the CSS rules to ContextualIdentityService P1 RESOLVED
1267965 [Presentation WebAPI] PresentationService should call NotifySessionConnect for new coming session -- RESOLVED
1268758 [Presentation WebAPI] implement allow-presentation sandboxing flag -- RESOLVED
1268795 [Presentation WebAPI] create a sandbox for loading presented content -- RESOLVED
1268810 [Presentation WebAPI] set sandboxed auxiliary navigation browsing context flag on receiving browsing context -- RESOLVED
1272197 [Presentation WebAPI] implement connect / disconnect / launch command -- RESOLVED
1273628 2.09 - 5.37% sessionrestore / tpaint (windows7-32, windows8-64, windowsxp) regression on push 9ffc0498ad7e (Tue May 17 2016) -- RESOLVED
1274749 Useless one free space between dropmarker and separator in address bar -- VERIFIED
1275157 [Presentation WebAPI] Lifetime management of PresentationConnection -- RESOLVED
1276116 Useless one free space between dropmarker and separator in address bar - round 2, fight -- VERIFIED
1276380 [Presentation WebAPI] support terminating a presentation in receiving presentation context -- RESOLVED
1278205 [Presentation WebAPI] enable Presentation API via preference instead of permission -- RESOLVED
1282003 Make nsScreenManagerAndroid support multiple screens. P1 RESOLVED
1285870 Show custom presentation on Chromecast. -- RESOLVED
1288297 [Presentation WebAPI] Support constructing PresentationRequest with multiple URLs -- RESOLVED
1288600 [Presentation WebAPI] No available device, return NotFoundError exception -- VERIFIED
1289974 [Presentation WebAPI] provide a trusted UI for device selection on Firefox P5 RESOLVED
1291971 [Presentation WebAPI] enable Presentation-API-related web platform test on treeherder -- RESOLVED
1292057 [Presentation WebAPI] add complete NSPR log for core service and webidl implementation -- RESOLVED
1295087 [Presentation WebAPI] Implement PresentationDeviceProvider for Chromecast devices on Fennec. P2 RESOLVED
1297018 Download notification is not displayed in the android notification bar P1 VERIFIED
1297475 Provide a way for system add-ons to add handlers for nsIContentPermissionRequests -- VERIFIED
1299061 [Presentation WebAPI] Expose the browser that the request was originated in -- RESOLVED
1299705 [Presentation WebAPI][UI] Need icons for device selection -- RESOLVED
1301259 [Presentation WebAPI] close the existing transport first before reconnect while in connected state. -- RESOLVED
1304969 [Presentation API] TV screen doesn’t return to home screen after terminating presentation from controller -- RESOLVED
1304971 [Presentation API] Failed to launch same presentation from fennec to another TV display -- RESOLVED
1304976 [Presentation API] Controller doesn't show an error message when the presentation url with a bogus url P3 RESOLVED
1304978 [Presentation API] Failed to re-launch presentation -- RESOLVED
1305351 Add ChromeCastDisplay for Presentation API. -- RESOLVED
1305352 [Presentation API] Implement PresentationRequestUIGlue for Fennec. -- RESOLVED
1306210 [Presentation WebAPI] Expose the principal of the device request -- RESOLVED
1306536 Do not display origin when displayURI is set to false in PermissionUI.jsm -- RESOLVED
1311375 [Presentation API] Fail to get expect result status when running "startNewPresentation_unsettledpromise-manual" -- RESOLVED
1313368 [Presentation API] Fennec stalls when trying to load a page P3 RESOLVED
1313404 [Presentation API] [debug build only] Fennec crashes after running a test from http://w3c-test.org P3 RESOLVED
1313601 Disable presentation mode when taps disconnect from Chromecast on Android Notification Bar. -- RESOLVED
1314213 hideNotNow doesn't work in PermissionUI.jsm -- RESOLVED
1314229 [Presentation API] presentation.receiver should be null if not in corresponding context -- RESOLVED
1316584 [Presentation WebAPI] No way to terminate the presentation from Android notifications on Android KitKat P3 RESOLVED
1316592 [Presentation WebAPI] Nearby Devices dialog still displayed after TV display is closed P3 RESOLVED

84 Total; 1 Open (1.19%); 78 Resolved (92.86%); 5 Verified (5.95%);


Signoff

Criteria

  • All test cases should be executed
  • All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed

Results

Nightly testing

  • Not landed yet

Aurora testing

  • Not landed yet

Beta testing

  • Not landed yet
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec/draft_mihai_2&oldid=1154560"