Auto-tools/New Contributor/Quarter of Contribution/Web Driver Infrastructure

From MozillaWiki
Jump to: navigation, search
Web Platform Tests – WebDriver Infrastructure
Andreas Tolfsen (ato on IRC, GitHub)


The Web Platform Tests are part of the W3C-coordinated initiative Test the Web Forward to build a cross-browser testsuite for open web platform technologies.

The project is an attempt to provide tests that can be run in all browsers, giving browser vendors confidence that they are shipping software that is compatible with other browsers. This gives web authors assurance that they can rely on the web platform to deliver on the promise of working across browsers and devices, without needing extra layers of abstraction to cover the gaps left by specification editors and implementors.

Project Outline

A yet unsolved problem is that not all test cases can be automated using content JavaScript; we’re lacking a way to control- and test features that require user input. The goal of this project is to allow wptrunner, the Web Platform Tests runner harness, to automatically run tests that would ordinarily require user interaction, using WebDriver.

In more detail, the goal of this project is to implement a “meta-circular” WebDriver client. This will have an API implemented in JavaScript that runs in the browser. The API will use an HTTP- or WebSockets backend to call a different WebDriver client implemented in Python in the wptserve server. This Python code will in turn call back into the browser using the WebDriver API, enabling emulation of human-like interactions to happen from tests written purely in cross-browser JavaScript.

wptrunner already uses WebDriver to drive the tests in a browser. It loads the test documents, listens for test results from testharness.js (the harness that wpt tests are written using), and then reports them back. By giving the in-browser content JS the ability to call out to the WebDriver session controlling its own browser session, the tests will be able to issue trusted OS events and emulate user interaction in a way that hasn’t previously been possible.

This project will allow the web platform to ensure cross-browser interoperability in privileged and permissions-guarded areas that content JS is not able to reach.

The project is outlined further in Github issue #2161.

Learn More

To learn more about this project we recommend:

  • Join IRC! It’s the best place to ask questions about the project and get help if you are stuck. #testing on and #ateam on are great channels to be in. Note that many people are most active in European timezones.
  • Checkout web-platform-tests from GitHub. Make sure that you are able to start the server and load tests in your browser; see the README file for instructions on how to do this.
  • If you’d like, try fixing one of the difficulty:easy bugs listed on the web-platform-tests issue tracker. These are generally not strictly related to this project, but they will help you understand the workings of web-platform-tests.
  • Run the manifest script in your web-platform-tests checkout to generate a test manifest.
  • Checkout wptrunner and try running a test against your local browser. See the README for more instructions, but something like:
wptrunner --binary=/path/to/firefox/binary --tests /path/to/web-platform-tests --metadata /path/to/web-platform-tests --ssl-type none --log-mach - dom/historical.html
  • Alternatively, if you have a Firefox source tree and build, you can run web-platform-tests using mach: mach web-platform-tests dom/historical.html.
  • Read about manual test cases in WPT.
  • Learn about WebDriver.

Ideal Candidate

Things we are looking for in an ideal candidate:

  • Ability to write code and design good APIs in both JavaScript and Python
  • Willing to learn how to write WebDriver adaptors
  • Creative ideas on how to proxy data between JavaScript and Python
  • Eagerness to learn about network and socket programming, protocols, and serialisation
  • Interest in test automation and UI/functional automation
  • Willing to learn, ask questions, and try new things out