QA/Fennec/DevTools Remote Debugging

From MozillaWiki
< QA‎ | Fennec
Jump to: navigation, search

Overview

Purpose

The purpose of this wiki is to serve as a general test plan for verifying that DevTools Remote Debugging is supported in Fennec.

Ownership

Developer contact:

Julian Descottes

QA contacts:

Andrei Bodea

Hani Yacoub

Overall Status

[In Progress] : Nightly 67 in 3/07/2019

Testing Summary

Scope of testing

The testing will be focus on the results of DevTools Remote Debugging with various devices. USB connection: making sure we can connect to devices, that the basic debugging experience is there and the workflow works. Connecting to network devices: be able to connect to Firefox 66, 67, 68 (there’ll be backward compatibility messages for other versions) In theory you can try to connect to anything, we don’t plan to prevent it but it might not work

Environments

Testing will be performed on following devices:

  • Phones
    • [ Sony Xperia Z3 Android 5.1.1]
  • Tablet

Specifications

Test page:

We should test to check the results of DevTools Remote Debugging with various devices.

Acceptance Criteria

All Test cases have passed and all P1 bugs are fixed.

Testing details

Test Cases

  • [link [Fx 67] DevTools Remote Debugging

Testing days

Date: 02-02-2019

  • Created Test Plan
  • Started working on TC's

Date: 08-03-2019

  • First testing roud
  • Functional and exploratory testing on Nightly 67.0a1 (2019-03-08)
  • Devices:

Date: 11-03-2019

  • First test report is sent
  • Please note that this email stands as a test report from “DevTools Remote Debugging” feature on Nightly 67. It is not a formal sign off, as the feature isn't going to ride the Fx67 train and the plan is to ship in Firefox 68. The implementation of this feature is not completed and currently does not require a pre-Beta sign-off.

Date: 17-04-2019

  • Functional and exploratory testing on Nightly 68.0a1 (date)

Date: 19-04-2019

  • Functional and exploratory testing on Nightly 68.0a1 (2019-04-17)
  • Devices:
  • The DevTools Remote Debugging NG - Test Results email is sent

Date: 27-06-2019

  • Based on latest update this is moved on Fx69 and to sign-off will be sent together with Desktop side, please note that the Mobile Android part is very small requires to connect the device to the desktop, and the rest is Desktop side, confirmed with Julian Descottes.

Bug Work

Full Query
ID Priority Summary Status Resolution Assigned to Whiteboard
1046897 P3 WiFi discovery confused when primary interface changes NEW
1304378 P3 Support DevTools Addons on developer toolboxes connected to remote tabs NEW triaged
1462211 -- [meta] Remote debugging NG Milestone 1 RESOLVED FIXED
1463604 -- [meta] Remote debugging NG Milestone 0 RESOLVED FIXED
1469089 P3 [meta] Connect to wifi devices from about:debugging NEW
1469093 P3 Implement connected toolbox NEW
1470352 P3 Implement PIN based WiFi authentication NEW
1481256 P5 Expose option to open/close listening port from about:debugging (i.e. provide a UI for the --start-debugger-server option) NEW
1488505 P3 [remote-dbg-next] allow to enable / disable wifi debugging NEW
1488513 P2 [remote-dbg-next] List content processes in new about:debugging ASSIGNED Julian Descottes [:jdescottes]
1492738 P3 [remote-dbg-next] Screencasting device when remote debugging NEW
1494541 -- [meta] Remote debugging NG Milestone 2 NEW
1494542 -- [meta] Remote debugging NG Milestone 3 RESOLVED INVALID
1496659 -- [meta] Remote debugging NG Milestone 3+ RESOLVED FIXED
1497780 P3 [remote-dbg-next] Add screenshot feature to new about:debugging RESOLVED WONTFIX
1498008 P3 [remote-dbg-next] Create proper icon for WiFi debugging NEW
1498009 P3 Add telemetry to record possible causes of discovery/connection failure (WiFi) NEW
1498874 P3 There is no indication that clicking "reload" in 'about:debugging#addons' on a temporary extension has done anything. NEW
1501934 -- Disable or hide "Refresh devices" button if usb devices are not enabled RESOLVED DUPLICATE
1505133 P3 [remote-dbg-next] UX-implementation: create a Toggle switch component to use for toggling WiFi/USB debugging NEW
1505795 -- [remote-dbg-next] Update the range of supported versions to accommodate for Fennec rollout RESOLVED DUPLICATE
1506661 -- [remote-dbg-next] Support Manifest URL link for addon targets with Remote Runtimes RESOLVED DUPLICATE
1509313 P3 [remote-dbg-next] Show ADB process status in about:debugging NEW
1509348 -- Fix animations in collapsible sections RESOLVED DUPLICATE
1510947 -- [remote-dbg-next] [meta] Accessibility for new about:debugging NEW
1511779 P3 [remote-dbg-next] Check if ADB command listDevices can be removed NEW
1512308 -- [remote-dbg-next] Runtime page header: add device information RESOLVED DUPLICATE
1512441 -- [remote-dbg-next] Unregister Service worker from new about:debugging RESOLVED DUPLICATE
1516344 P5 [meta] unify all the different button styles for in-content pages of Firefox NEW
1517676 -- [remote-dbg-next] Even if close the Firefox connected via network location, status of the item on sidebar does not update RESOLVED DUPLICATE
1518122 P3 [remote-dbg-next] Review using fronts directly in our data structure instead of doing lookups NEW
1519036 P3 Use in-content/common.css in about:debugging NEW
1519338 P3 [dt-docs] Add "how to do localization" section to our README file NEW dt-docs
1525553 P3 Sources pane is empty when debugging extensions RESOLVED INVALID
1527240 -- [remote-dbg-next] Polish runtime header actions RESOLVED DUPLICATE
1536733 -- [remote-dbg-next] [meta] Backward compatibility issues between Firefox 68 and 67 RESOLVED WORKSFORME
1539451 P3 [remote-dbg-next] Disable WebIDE and the Connect... page by default RESOLVED FIXED Julian Descottes [:jdescottes]
1539461 P3 [remote-dbg-next] Remove codebase for the old aboutdebugging RESOLVED FIXED Julian Descottes [:jdescottes]
1539462 P3 [remote-dbg-next] Remove the codebase for WebIDE and the Connect page RESOLVED FIXED Julian Descottes [:jdescottes]
1540179 P3 Provide an option to pause and debug the remote browser on startup RESOLVED DUPLICATE
1540610 P3 [remote-dbg-next] Move connection state of runtime to ui state NEW remote-debugging-technical-debt
1540633 -- [remote-dbg-next] Allow to set breakpoints during Fenix startup RESOLVED DUPLICATE
1545407 -- [remote-dbg-next] Improve worker debugging workflow RESOLVED DUPLICATE
1546268 -- Vertical scrollbar is partly offscreen RESOLVED DUPLICATE
1547658 P1 [remote-dbg-next] Second line in sidebar is cut off RESOLVED FIXED Ola Gasidlo [:ogasidlo] [:ola] OOO - please ping :ladybenko:
1548461 -- [remote-dbg-next] Connection timeout warning looks too big in the sidebar RESOLVED FIXED Julian Descottes [:jdescottes]
1548762 -- [remote-dbg-next] Message "No devices discovered" should have color set to grey 50 RESOLVED FIXED Ola Gasidlo [:ogasidlo] [:ola] OOO - please ping :ladybenko:
1550651 P3 Don't register AboutDebuggingNewRegistration.jsm on GeckoView RESOLVED FIXED Julian Descottes [:jdescottes]
1552502 P3 Recording profile via the "Profile performance" button crashes remote Firefox (Mobile Beta) RESOLVED INACTIVE
1552806 P3 [remote-dbg-next] Setup page intro text should not mention "tabs" category for "This Firefox" RESOLVED FIXED Francis Houle
1552956 P2 [remote-dbg-next] Allow to inspect targets in a window instead of a tab NEW
1553027 -- [meta] Update new remote debugging to ship with 69 instead of 68 RESOLVED FIXED
1553042 P1 [remote-dbg-next] Enable about:debugging in Beta and Release for Firefox 69 RESOLVED FIXED Julian Descottes [:jdescottes]
1553090 P3 [remote-dbg-next] Add header messages in aboutdebugging errors and warnings NEW
1553092 P2 [remote-dbg-next] Add padding to error / warning details NEW
1553214 P1 Console from about:debugging not working with Firefox for Android RESOLVED FIXED Julian Descottes [:jdescottes] [qa-triaged]
1553879 P3 [remote-dbg-next] Addon debugging can block Firefox main process if extensions.webextensions.remote is false NEW
1554404 P2 [remote-dbg-next] Inspect buttons for service workers are disabled without explanation or alternative RESOLVED FIXED Julian Descottes [:jdescottes]
1555622 P3 [remote-dbg-next] Allow to edit preferences on remote browser from about:debugging NEW
1556648 P3 Investigate serviceworker registration issue in multie10s aboutdebugging mochitest NEW
1556649 P3 Cleanup leftover tabs, workers, addons after mochitest timeout NEW
1557170 P2 Make Service Workers debugging compatible with remote debugging RESOLVED FIXED Julian Descottes [:jdescottes]
1557250 P3 Remove extra getDeviceDescription call in new about:debugging sw push NEW
1560236 P3 Align the empty section messages ("Nothing yet") with the section title's text NEW
1560551 P3 Refactor RuntimePage component(s) NEW
1561284 P2 [remote-dbg-next] Fenix/Firefox Preview stuck in "waiting for browser" unless a tab is opened NEW
1565182 P3 [remote-dbg-next] Add mochitest to check that favicons are not blank NEW
1566443 P3 Rename aboutdebugging-new to aboutdebugging RESOLVED FIXED Julian Descottes [:jdescottes]
1566763 P3 Review DEVTOOLS_ABOUTDEBUGGING_OPENED_COUNT after removing old aboutdebugging RESOLVED FIXED Julian Descottes [:jdescottes]
1568304 P3 Add integration test for local process debugging via about:debugging RESOLVED FIXED Julian Descottes [:jdescottes]
1569410 P2 Impossible to debug extension popups with new about:debugging RESOLVED FIXED Julian Descottes [:jdescottes]
1571240 P3 Missing RTL support for the new about:debugging VERIFIED FIXED Itiel
1571241 P3 about:debugging sidebars should not have dotted outlines when clicking them NEW
1571703 P3 DebuggerClient created for local target inspection is never destroyed RESOLVED FIXED Alexandre Poirot [:ochameau]
1571801 P3 Unregister service worker takes a long time with sw-refactor enabled NEW
1577564 P3 Instrument when a Firefox client gets remotely debugged NEW
1583313 P3 Remove code related to WebIDE WiFi debugging NEW
1585505 P3 Update GECKOVIEW_REFERENCE_BROWSER_BUILD.md document NEW
1589087 P3 Latest Firefox Preview Nightly doesn't have a $version in Remote Debugging RESOLVED FIXED Julian Descottes [:jdescottes]
1591098 P3 Inspect Main Process opens about:devtools-toolbox using the actorID as target id RESOLVED WORKSFORME
1593715 P3 [10.15] adb extension causes macOS Catalina permission prompt when about:debugging opens NEW
1596867 P3 [remote-dbg-next] Temporary extensions should be listed on remote devices RESOLVED FIXED Luca Greco [:rpl] [:luca] [:lgreco]
1608039 P3 Remote toolbox should not redirects to the connection error page when the connection is lost NEW
1617237 P3 See/change remote URL from about:devtools-toolbox RESOLVED FIXED Micah [:mtigley] (she/her)
1617240 P3 Reload remote tab from about:devtools-toolbox RESOLVED FIXED David Walsh :davidwalsh
1618788 P2 Inspecting the same tab twice on a remote runtime leads to a blank toolbox RESOLVED DUPLICATE
1630005 P3 Support debug target navigation when fission is enabled RESOLVED DUPLICATE dt-fission-future
1634471 P3 Handle navigation failures when changing URLs NEW
1635536 P3 Simplify CSS for rendering the target's URL input and title NEW
1638889 P3 While remote profiling/debugging, prevent device's screen from going to sleep NEW

90 Total; 44 Open (48.89%); 45 Resolved (50%); 1 Verified (1.11%);


Signoff

Criteria

  • All test cases should be executed
  • All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed

Results

Nightly testing

  • There were no blockers found while testing this feature.
  • All bugs are fixed and verified

Beta testing NA