The Problem

The paradigm of our entire automation solution works on a very simple concept. Here's the concept: 1. We create a profile of our making 2. We start the browser in that profile with a URL.

This concept will not work in a browser-less world like b2g where we are running services on "bare metal" HTML rendering engines. Also, b2g gives us a unique opportunity to plug in automation at a lower level than we have before so that we can better wire in automation and testability into our products without bolting this ability onto it as an afterthought as we've don in the past.

There are a few options for doing this.

Approach 1: "Bazooka" aka JsBridge

JsBridge is a tool from the mozmill tool chain that creates a listening socket inside of the product. By communicating to this socket, you can send in raw javascript and have it executed in a browser context. It's essentially RPC for javascript.

Upside: It's relatively easy to do Downside: The API isn't that good, and isn't really going to be an amenable solution to platform developers. It's not something I think we can actually upstream into the Gecko platform.

Approach 2: "Four tons" aka webdriver

Webdriver works through a HTTP REST interface. They have a well-defined API for interacting with a UI system (content only right now, no reason it can't be amended to work with chrome as well) and for running random scripts in the browser context. The current webdriver idea is to embed our old friend httpd.js into the browser to provide the HTTP interface for the protocol. This takes too much memory on fennec and doesn't work.

Upside: gives developers of our web apps and web front end a common understandable test framework that they should be used to (i.e. selenium). Also, it has a very well-defined interface, which is moving to be a web-testing W3C standard and we should support it in that right anyway in the browser.

Downside: It's heavy. The API requires keep-alives, proper HTTP Responses, needs to accept and parse HTTP POST & Get requests etc. It's not clean, it's not simple, and it's a bit unwieldy to work with. We already know that as-is it won't work in a mobile environment. So, we have to try to cut away pieces of the HTTP server and hope we don't break anything in the process, and that means further forking of an already forked (the webdriver project not only reused httpd.js, they forked it too) httpd.js.

Approach 3: "Surgical Strike" aka The Debugger Approach

The JSD team has a remote javascript debugger interface that opens a socket inside the browser and accepts a JSON protocol over the socket. This is most like what we want, simple, lightweight, versatile. However, the remote debugger socket protocol is not readily extensible for us to use our own protocol over the socket, and it's not clear if we want to be in "debugging" mode to turn on our automation support.

Also, being able to run tests and debug at the same time is important. We don't want to prevent that usecase and it's not clear if we can multiplex JSON over the socket and not confuse the debugger code. However, adding an entirely separate socket system into the platform really seems like egregious overkill. We've got this one wired in now, we should reuse it.

Upside: it's lightweight, already wired in, socket code is finished, we just need to add in our protocol handler.

downside: we don't want to break the jsdebugger stuff.

Another upside: If we write a shim that runs outside of firefox it could handle the http post/get stuff and it could translate the selenium calls into JSON to speak to this socket. Or, alternatively, we could change the selenium bindings for firefox so that it has a translation layer from the REST protocol into the straight JSON protocol that we'd be using. This way we could still gain the benefit of allowing web developers write tests the way they intend to for web apps and web ui.

I think the Debugger Approach is the one to try. It gives us the most versatility with the lightest touch on the product.