Highlights in recent work on Remote Protocol and WebDriver BiDi.
We are gradually publishing blog posts about how our work in Firefox is helping the web testing ecosystem.
Aside from that, we also landed a number of small improvements to our Remote Protocol implementation. Here are some highlights:
f6fd4b85734e Henrik Skupin Bug 1641839 - [remote] Only emit Page.frameAttached and Page.frameAttached for frames. r=remote-protocol-reviewers,maja_zf 21693836e4ec Henrik Skupin Bug 1639467 - [remote] Enable page events after loading initial page in browser_frameAttached.js. r=remote-protocol-reviewers,maja_zf 5b83370e0913 Henrik Skupin Bug 1636373 - [remote] Only emit Runtime.executionContextsCleared event for top-level execution contexts. r=remote-protocol-reviewers,maja_zf 0704e56577ad Henrik Skupin Bug 1644695 - [remote] Return the frameId of the frame when DOM.describeNode is called with a frame element. r=remote-protocol-reviewers,maja_zf 59cd08b79678 Henrik Skupin Bug 1637640 - [remote] Enable page load events for frames. r=remote-protocol-reviewers,maja_zf df561c571cfd Maja Frydrychowicz Bug 1646923 - [remote] Add metadata checks for skipped and extra tests r=remote-protocol-reviewers,whimboo 06435d32c9fe Gabriel Luong Bug 1595710 - Add support for "platform" to Emulation.setUserAgentOverride r=remote-protocol-reviewers,whimboo c52d1bf844e8 Maja Frydrychowicz Bug 1632722 - IO.read hits 'too many arguments' r=remote-protocol-reviewers,jgraham 5198f71d9969 Henrik Skupin Bug 1662549 - [puppeteer] Don't set "toolkit.telemetry.server" to prevent side-effects. r=remote-protocol-reviewers,maja_zf adc5e45e75aa Henrik Skupin Bug 1661390 - [remote] Use a UUID as session id for targets. r=remote-protocol-reviewers,jgraham 3803f3a513e4 James Graham Bug 1660893 - Don't crash when the remote agent can't be initialized, r=remote-protocol-reviewers,whimboo 6b6e8d122736 Maja Frydrychowicz Bug 1665283 - [puppeteer] Remove datareporting.healthreport.about.reportUrl from Launcher.ts r=remote-protocol-reviewers,whimboo 1616add3a380 Henrik Skupin Bug 1669230 - [remote] Remove --remote-debugger command line argument. r=remote-protocol-reviewers,jgraham dd12a0bec46f Henrik Skupin Bug 1675266 - [remote] Add debuggerAddress as attribute on the Remote Agent interface. r=remote-protocol-reviewers,maja_zf 15713ccc419b Henrik Skupin Bug 1675266 - [remote] Report correct browser name/version, and user agent string for "/json/version" httpd handler. r=remote-protocol-reviewers,maja_zf e12b84262d8b Henrik Skupin Bug 1605784 - [remote] Implement Emulation.setTouchEmulationEnabled. r=remote-protocol-reviewers,jdescottes 31021652804c Henrik Skupin Bug 1680101 - [remote] Temporarily force disabling Fission until Remote Agent is compatible. r=remote-protocol-reviewers,maja_zf 80442bbcf255 Maja Frydrychowicz Bug 1672461 - [remote] Pin package dependencies when installing Puppeteer in Mozilla CI r=remote-protocol-reviewers,whimboo ff54087a0e8e Henrik Skupin Bug 1680740 - [remote] Implement Target.getTargets. r=remote-protocol-reviewers,maja_zf b1e57d60a768 Henrik Skupin Bug 1680962 - [remote] sendResponse has to always return a result in the response payload. r=remote-protocol-reviewers,jdescottes ef6ad6acbda0 Henrik Skupin Bug 1680814 - [remote] Target events have to emit a complete TargetInfo dictionary. r=remote-protocol-reviewers,jdescottes 8afb25365681 Henrik Skupin Bug 1553854 - [remote] Implement Runtime.consoleAPICalled. r=remote-protocol-reviewers,maja_zf bd8ae5a00b8e Henrik Skupin Bug 1669218 - [geckodriver] Add "moz:debuggerAddress" capability to offer an opt-in for the Remote Protocol. r=webdriver-reviewers,jgraham b2e729917288 Henrik Skupin Bug 1671205 - [marionette] Return moz:debuggerAddress in capabilities when Remote Agent is active. r=marionette-reviewers,jdescottes,maja_zf
- When navigating to web pages with iframes included, all relevant page navigation events are sent out for each and every frame now. To get this finished various other small fixes were necessary too. The formerly added preference remote.frames.enabled is no longer necessary because frame handling is enabled by default now.
- For the next while, the team will be focused on making Marionette Fission-compatible. In the meantime, we’ll be gathering feedback from Puppeteer users and anyone else experimenting with Remote Protocol before we resume our work in that area.
- 🎉 We completed the Beta MVP milestone on May 20th. That included a lot of major CDP end-points and events needed to use Puppeteer for automation. With that milestone achieved Firefox now passes
- Our work will continue on another milestone after some outreach and the prioritization of the remaining highly requested CDP end-points and events. In the meantime we will focus on Fission compatibility.
- Etienne provided another patch that implements Network.getAllCookies.
- Partial support for iframes has been added, and as such the Page.frameAttached and Page.frameDetached events are emitted now. It also means that the frameId property of the Network.requestWillBeSent and Network.responseReceived events can correctly reference the related frame. Right now this feature is pref’ed off behind the remote.frames.enabled preference, but we hope to enable it by default relatively soon.
- Fixed bugs between May 20th and June 2nd
- Fixed bugs between May 6th and May 19th
- 🌟 Contributor Etienne Bruines implemented offline mode for Network.emulateNetworkConditions to allow testing frameworks to verify the correct offline behavior for web applications. Thanks, Etienne!
- The Runtime.executionContextCreated and Runtime.executionContextDestroyed events are now correctly emitted when subframes are created and destroyed. To not break users of Puppeteer while further work for frames needs to be landed, both events are temporarily put behind the `remote.frames.enabled` preference, and not sent out by default.
- The Network.requestWillBeSent event has been updated to include request headers and document URL.
- The Network.responseReceived event is available now. Some data about the response, such as security state, will be implemented later.
- loaderId of document request is now included in Page.lifecycleEvent
- Page.getFrameTree now returns all the frames of the current tab target and is now also Fission compatible.
- Page.navigate now waits until the response of the initial document request has been received and is therefore able to return errorText when navigation fails.
- See all the fixed bugs from April 7th to May 5th.
- Fixed bugs between Feb 25th and April 6th
- Maja addedFirefox Nightly support to Google Puppeteer’s browser-fetching feature. This means that as of v3, Puppeteer users will have an equally convenient install+launch+connect flow for both Firefox and Chrome.
- Henrik landed several fixes and enhancements for both the Runtime.evaluate and Runtime.callFunctionOn API’s.
- Henrik added basic implementations for DOM.describeNode and DOM.resolveNode.
- We can now run Wordpress Gutenberg end-to-end tests against Firefox Nightly and we pass 52% of all the tests out of the box. To reach that status a fair amount of work has been done over the last couple of months.
- Fixed bugs between Jan 15th and Feb 24th
- Henrik finished off the patch from David Burns for Page.setCookie(s).
- Henrik added support for the "clip" argument in Page.captureScreenshot
- The Puppeteer project is now running their CI jobs against Firefox Nightly in addition to Chrome
- Fixed bugs between Dec 18th and Jan 14th
- 🌟 mohitsingh1930 fixed some missing default parameters in the remote agent
- 🌟 Kstheking0 refactored the remote browser-chrome tests and made the task function arguments to use an object
- Henrik implemented Network.deleteCookies and parts of Network.getCookies to allow consumers of Puppeteer to interact with site cookies.
- Henrik implemented Network.setCacheDisabled to allow ignoring the cache for each network request.
- Henrik fixed Emulation.setUserAgentOverride to only affect the current tab target.
- Henrik started the work on making the Remote Agent Fission compatible.
- You can see a demo of our Puppeteer support during the Berlin All-Hands.
- Fixed bugs between Dec 4th and Dec 17th
- Henrik implemented parts of Page.getLayoutMetrics to retrieve the layout (fixed) viewport and the full content size. Both are needed for better screenshot capabilities like capturing the full page.
- Henrik added support for Page.printToPDF to allow the current web page to be printed to a PDF file. This also includes the implementation for handling streams via IO.read and IO.close.
2019-11-05 and 2019-12-03
Introduction: The Web Predictability team (#interop) is implementing a subset of CDP in Firefox; we’ve had a working prototype for some time, and our next milestone is to provide initial cross-browser support in Google's Puppeteer (a popular Node.js library for headless browser automation).
- Fixed bugs between Nov 5th and Dec 3rd
- 🌟 Jacob Stenson removed the unnecessarily returned frameId property from the Page.loadEventFired event
- We started shipping the CDP-based remote debugging server on the Firefox Nightly release channel(see --remote-debugger flag)
- Maja added Firefox support to Puppeteer’s launcher API; users will be able to run Puppeteer with Firefox in addition to Chrome as of the next Puppeteer release.
- Andreas added a new section for Remote Debugging in about:support to expose the status if Firefox is under remote control and a visual cue in the Firefox UI when the browser is remotely controlled
- We’re steadily implementing more CDP methods on track toward our MVP
- Andreas made it possible to use Page.reload ignoring the browser’s cache
- Thomas Kosmas removed the unnecessarily returned timeout property from various page navigation events
- Henrik added support for Target.activateTarget, which switches to the specified target (tab) in any of the open browser windows. That also fixed a bug in Page.bringToFront, which caused the command to not wait for the window’s “activate” and “focus” events.
- Henrik made it possible for Page.captureScreenshot to save the captured screenshots in different formats and quality
- Andreas added support for Security.setIgnoreCertificateErrors to allow bypassing security exceptions when loading documents with invalid or otherwise bad TLS certificates
- Henrik added support for Emulation.setUserAgentOverride which allows clients to replace the user agent with a custom value.