Firefox OS/Remote Control: Difference between revisions

Revision as of 06:26, 18 April 2016

Introduction

Remote control is a feature for Firefox OS on TV. After Send tab to TV, user can easily extend browsing experience on TV via Firefox on Android, using virtual touchpad to control TV.

Meta Bug: bug 1205939

Prototype Demo Video: http://bit.ly/1Fa2ST1

UX & Visual Spec

UX Spec: http://bit.ly/1jrWqgn

Visual Spec: http://bit.ly/1jrWQU3

Architecture Designs

Remote Control Service: Entry point of remote control feature. Control service related logic and data in runtime.
TLS server: Serves connections from user.

User interaction of remote control is divided into two parts:

Peer authentication
Control event processing

Peer authentication

Remote control is designed to protect data transmitted between client and server, for example: user input string may contains privacy data. The connection is based on self-signed TLS server. To prevent man in the middle(MITM) attack, we adopt J-PAKE to exchange symmetric key for peer authentication. Following describes how it works:

The user sends the tab to TV on fennec.
Client sends request to handshake without ID.
Server replies with the handshake type is 1st time handshake.
Client computes J-PAKE round 1 and sends the result to server.
Server generates PIN code and show on screen, computes J-PAKE round 1 and sends the result to client.
Server computes J-PAKE round 2 with PIN code attached first 12 characters as weak secret.
User enter PIN code, client computs J-PAKE round 2 with PIN code attached first 12 characters as weak secret.
Client sends round 2 result to server.
Server dismisses PIN code notification on screen, sends round 2 result to client.
Both client and server compute J-PAKE final round, gets AES and HMAC key.
Server use two keys to get a signature of H(H(AES key)), then send the signature to client.
Client verifies the signature matches it has.
Client use two keys to get a signature of H(AES key), then send the signature to server.
Server verifies the signature matches it has.
Server replies with the handshake finish with client's ID for connection in the future.

Re-authentication

User is only required to input PIN code when first time connect to TV. In the second time, client and server leverage previous AES key value and derive new AES and HMAC key. This can reduce number of user input and make the procedure from send tab to TV to remote control more smoothly.

The user sends the tab to TV on fennec.
Client sends request to handshake with ID.
Server looks up ID/AES key mapping and replies with the handshake type is 2nd time handshake.
Client computes J-PAKE round 1 and sends the result to server.
Server computes J-PAKE round 1 and sends the result to client.
Client and server computes J-PAKE round 2 with previous AES key value as weak secret.
Client sends round 2 result to server.
Server sends round 2 result to client.
Both client and server compute J-PAKE final round, gets new AES and HMAC key.
Server use two keys to get a signature of H(H(AES key)), then send the signature to client.
Client verifies the signature matches it has.
Client use two keys to get a signature of H(AES key), then send the signature to server.
Server verifies the signature matches it has.
Server replies with the handshake finish.

After authentication, client can sends control event to server.

Control event processing

Control page provides: 1) a virtual touchpad, 2) a virtual vertical scrollbar, 3) 3 function keys. Every touch, swipe or click will be generated to an event in JSON format then send to TV. Following describes how control event is sent and processed:

User operates control page.
Client generates and sends the event in JSON format.
Server parses the event to JSON object, dispatch to client.sjs.
Client.sjs parses event and dispatch to Gecko or Gaia system app.

Data encryption/decryption

There are three kinds of data are encrypted while tranmission:

Client polls UUID in establish secure connection stage
Client sends PIN code in pin code pairing
Client sends event in control event processing

As data are encrypted by symmetric key, AES-GCM, there are two things need noticed:

AES-GCM need initialization vector(IV) to encrypt/decrypt. Currently, IV is random value for each data, length is 12 bytes. Encrypted data is appended after IV. Receiver needs to slice first 12 bytes as IV to decrypt.
Each message type is string, use TextEncoder.encode() to encode as an UInt8Array, then encrypt to ArrayBuffer. Vice versa.
Data encryption/decryption is asynchronous but HTTP request need response immediately. For UUID and PIN code case, remote control uses a ticket number to get status after decryption. But for event, we reduce polling result but use latest event's result instead.

Ajax Protocol

RSA public key exchange

Request

 {
   action: 'require-public-key'
 }

Response

 {
   publicKey: <RSA-OEAP SPKI in base64>
 }
 {
   error: <reason>
 }

Send symmetric key

Request

 {
   action: 'send-symmetric-key',
   wrappedSymmetricKey: <AES-GCM wrapped by RSA-OEAP in base64>
 }

Response

 {
   ticket: <ticket>
 }
 {
   error: <reason>
 }

Poll UUID

Request

 {
   action: 'poll-uuid'
   ticket: <ticket>
 }

Response

 {
   done: true,
   encryptedUUID: <IV + UUID encrypted by AES-GCM, in base64>,
 }
 {
   done: true,
   error: <reason>
 }
 {
   done: false // Symmetic key unwrapping or encrypting UUID
 }

Pair PIN code

Request

 {
   action: 'pair-pincode'
   encryptedPIN: <IV + PIN encrypted by AES-GCM in base64>
 }

Response

 {
   ticket: <ticket>
 }

Poll pair result

Request

 {
   action: 'poll-pair-result'
   ticket: <ticket>
 }

Response

 {
   done: true
   verified: <boolean>
   reason: <reason>
 }
 {
   done: false
 }

Encrypted control events

Client sends encrypted event in query string as:

 <IP_Address>:<Port>?message=<IV + encrypted event by AES-GCM in base64>

Response

 {
   verified: <boolean>
 }

After decrypted, following are control events:

Touch Events

 {
   type: 'touchstart',
   detail: {
     width: <touch panel width, integer, in pixels>,
     height: <touch panel height, integer, in pixels>,
     identifier: <A simple unique identifier for server to distinguish touch events between different clients, string>
   }
 }

 {
   type: 'touchmove',
   detail: {
     dx: <dx between current point and starting point, integer, in pixels>,
     dy: <dy between current point and starting point, integer, in pixels>,
     identifier: <should be identical with "touchstart">,
     duration: <duration from "touchstart", integer, in milliseconds>
   }
 }

 {
   type: 'touchend',
   detail: {
     dx: <same as "touchmove">,
     dy: <same as "touchmove">,
     identifier: <should be identical with "touchstart">,
     duration: <same as "touchmove">,
     swipe: <the gesture within this touch, string, should be "left", "right", "up" and "down">
   }
 }

Scroll Events

There are "scrollstart", "scrollmove" and "scrollend" events which details are all identical with corresponding touch events above.

Key Events

 {
   type: 'keypress',
   detail: <KeyEvent constant, string, sush as "DOM_VK_RETURN">
 }

Input Events

 {
   type: 'textinput',
   detail: {
     clear: <whether to clear the entire string in the current focused input field, boolean>,
     string: <new string to append, string>,
     keycode: <a specified key to be pressed after the string inputted, integer>
   }
 }

Custom Events

 {
   type: 'custom',
   detail: {
     action: <custom action name, string>,
     ...
   }
 }

Pairing (Full version, without secure connection)

Meta Bug: bug 1207996

Flowchart

First Time Connection (deprecated in secure connection)

Enter PIN Code (deprecated in secure connection)

Resume Connection (deprecated in secure connection)

Dismiss Pairing

Protocol

from client to server via AJAX (deprecated in secure connection)

 {
   pincode: <pincode>
 }

response when success (deprecated in secure connection)

 {
   verified: true,
   uuid: <uuid>
 }

reponse when error

 {
   verified: false,
   reason: 'expired' / 'invalid'
 }

internal events at server side

 {
   type: 'mozChromeRemoteControlEvent',
   detail: {
     action: 'pin-created',
     pincode: <pincode>
   }
 }

 {
   type: 'mozChromeRemoteControlEvent',
   detail: {
     action: 'pin-destroyed'
   }
 }

 {
   type: 'mozContentEvent',
   detail: {
     type: 'remote-control-pin-dismissed',
     detail: {
       reason: 'timeout' / 'manually'
     }
   }
 }

Secure Connection (deprecated)

To protect private data between user and TV, we provide a secure connection which refers to SSL and jCryption. Here is the concept:

Client requests RSA public key from TV.
Client sends symmetric key and PIN code (if any) to TV, encrypted with public key.
TV decrypts symmetric key with private key
TV generates an UUID, encrypts with symmetric key then sends 2 UUIDs to client, one is encrypted the other is not.
Client decrypts UUID with symmetric key, confirm UUID
Begin remote control with symmetric key.

Goal of Remote Control is to create an easy to use way for every device, we hide public key exchange at the background. User doesn't need to input a long URL with public key nor scan QRCode. Everything is done automatically without interrupting user experience.