Firefox OS/Remote Control

Introduction

Remote control is a new feature for Firefox OS on TV. We hope user can control TV at any device without installing any APP. Consider compatibility, remote control uses HTTP and AJAX as communication protocol between TV and user. User can easily connect to TV via open his/her browser, enter URL on TV and use the virtual touchpad to control TV just in seconds.

Meta Bug: bug 1205939

Prototype Demo Video: http://bit.ly/1Fa2ST1

UX & Visual Spec

UX Spec: http://bit.ly/1jrWqgn

Visual Spec: http://bit.ly/1jrWQU3

Architecture Designs

Remote Control Service: Entry point of remote control feature. Control service related logic and data in runtime.
HTTP Server: Serves connections from user. Two types files can be served:
- Static file: Stored in Remote Control Client App, packaged in app://remote-control-client.gaiamobile.org
- Service script: Handle AJAX request and response for secure connection initialization, PIN code pairing and control event processing. Packaged in resource://gre/res/remotecontrol
Remote Control App: Provides service related user interface on TV. Communicate with Remote Control Service via MozSettings.

User interaction of remote control is divided into three parts:

Establish secure connection
PIN code pairing (optional, by user settings)
Control event processing

Establish secure connection

Remote control is designed to protect data transmitted between client and server, for example: user input string may contains privacy data. Client is forced to exchange symmetric key (AES-GCM) with server in the first time. After key exchange, server uses UUID in cookie to identify client and key to decrypt messages. Following describes how keys are exchanged:

User opens the URL
Client requests page without valid UUID.
Server response secure.html as requested page for key exchange.
Client acquire RSA public key from secure.sjs.
Secure.sjs replies RSA public key SPKI in base64.
Client import RSA public key and wrap symmetric key.
Client sends wrapped symmetric key to secure.sjs in base64.
Secure.sjs replies with a ticket number.
Secure.sjs unwraps symmetric key with RSA private key.
Secure.sjs saves symmetric key.
Remote Control Service generates a new UUID, encrypted with symmetric key and binds encrypted UUID with ticket number.
Client polls encrypted UUID from secure.sjs.
Client decrypt received UUID to double confirm server receives correct symmetric key.
Client reloads the URL with UUID.

PIN code pairing

Control event processing

Ajax Protocol

Touch Events

 {
   type: 'touchstart',
   detail: {
     width: <touch panel width, integer, in pixels>,
     height: <touch panel height, integer, in pixels>,
     identifier: <A simple unique identifier for server to distinguish touch events between different clients, string>
   }
 }

 {
   type: 'touchmove',
   detail: {
     dx: <dx between current point and starting point, integer, in pixels>,
     dy: <dy between current point and starting point, integer, in pixels>,
     identifier: <should be identical with "touchstart">,
     duration: <duration from "touchstart", integer, in milliseconds>
   }
 }

 {
   type: 'touchend',
   detail: {
     dx: <same as "touchmove">,
     dy: <same as "touchmove">,
     identifier: <should be identical with "touchstart">,
     duration: <same as "touchmove">,
     swipe: <the gesture within this touch, string, should be "left", "right", "up" and "down">
   }
 }

Scroll Events

There are "scrollstart", "scrollmove" and "scrollend" events which details are all identical with corresponding touch events above.

Key Events

 {
   type: 'keypress',
   detail: <KeyEvent constant, string, sush as "DOM_VK_RETURN">
 }

Input Events

 {
   type: 'input',
   detail: {
     clear: <whether to clear the entire string in the current focused input field, boolean>,
     string: <new string to append, string>,
     keycode: <a specified key to be pressed after the string inputted, integer>
   }
 }

Custom Events

 {
   type: 'custom',
   action: <custom action name, string>,
   ...
 }

Pairing

Meta Bug: bug 1207996

Flowchart

First Time Connection

Enter PIN Code

Resume Connection

Dismiss Pairing

Protocol

from client to server via AJAX

 {
   pincode: <pincode>
 }

response when success

 {
   verified: true,
   uuid: <uuid>
 }

reponse when error

 {
   verified: false,
   reason: 'expired' / 'invalid'
 }

internal events at server side

 {
   type: 'mozChromeRemoteControlEvent',
   detail: {
     action: 'pin-created',
     pincode: <pincode>
   }
 }

 {
   type: 'mozChromeRemoteControlEvent',
   detail: {
     action: 'pin-destroyed'
   }
 }

 {
   type: 'mozContentEvent',
   detail: {
     type: 'remote-control-pin-dismissed',
     detail: {
       reason: 'timeout' / 'manually'
     }
   }
 }

Secure Connection

To protect private data between user and TV, we provide a secure connection which refers to SSL and jCryption. Here is the concept:

Client requests RSA public key from TV.
Client sends symmetric key and PIN code (if any) to TV, encrypted with public key.
TV decrypts symmetric key with private key
TV generates an UUID, encrypts with symmetric key then sends 2 UUIDs to client, one is encrypted the other is not.
Client decrypts UUID with symmetric key, confirm UUID
Begin remote control with symmetric key.

Goal of Remote Control is to create an easy to use way for every device, we hide public key exchange at the background. User doesn't need to input a long URL with public key nor scan QRCode. Everything is done automatically without interrupting user experience.