Phishing Protection: Server Spec

Safe Browsing Server Specification

Niels Provos

This documents contains the relevant information necessary to implement a server that talks the Google Safe Browsing protocol. The Google Safe Browsing protocol does not use binary data but instead uses human-readable text only. An implementation is relatively straight forward.

The protocol consists of four different HTTP requests:

  * */getkey*: This request requires SSL support.  It provides the client with a private key for confidential communication with the server.
  * */lookup*: Asks the server to render a verdict on the provided target URL.  This request needs to be encrypted with the private key from above.
  * */update*: The client provides a list of tables that it wants the server to update.  The server either provides the full content of the current tables or incremental updates to bring the client's tables up to the current version.
  * */report*: Used to report when users visit phishing pages and if they decided to heed the warning or ignore it.

Data Encoding Standards

Encrypted data or key material needs to be encoded so that it can be transmitted as part of a _HTTP GET_ request. We use Websafe Base64 encoding with the following table:

/**
* Our websafe alphabet. Value 64 (=) is special; it means "nothing."
*/ 
ENCODED_VALS_WEBSAFE = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" +
                      "abcdefghijklmnopqrstuvwxyz" 
                      "0123456789-_=";

The encoding works just like regular Base64 encoding. Each 6-bit of data gets mapped into a websafe encoded character and vice versa. The *=* character is used for padding.

Key Value Pairs

All server responses are in the form of key value pairs. They need to follow this format:

key:<value length>:value

If a key is unknown to the client, it must ignore it. Most clients should simply ignore the length -- it's a historical artifact and not necessary in most circumstances. A client should skip lines of names it does not understand, and should be tolerant of blank lines. However, a server should still generate a correct length field even if most clients are just going to ignore it.

GetKey Request

The GetKey request is used when the browser starts up to create a shared secret key between the client and the server. The secret key is used to encrypt lookup requests and also for authenticating table updates. The protocol does not enforce the use of a secret key but it's strongly encouraged. To be secure, the client makes the a GetKey request over SSL.

A server needs to respond with the following data:

clientkey:24:pOAblTUiZFkLSv3xRiXKKQ==
wrappedkey:24:MTqdJvrixHRGAyfebvaQWYda

The client key is a 16-byte long random nonce generated by the server when receiving the GetKey request. The wrapped key is the random nonce encrypted by a server key. The wrappedkey is opaque to the client and a server may implement any encryption algorithm it sees fit. The wrappedkey allows the server to reconstruct the client key without requiring per-client state. It is up to the server to include verification information into the wrapped key that might allow it to determine if decrypting it was successful.

If the server key changes, the server can prepend

pleaserekey:1:1

to responses for lookup or update requests to tell the client to request a new client key.

Lookup Requests

Sample format of an encrypted lookup request:

GET /safebrowsing/lookup?sourceid=firefox-antiphish&features=TrustRank&client=navclient-auto-tbff&encver=1&nonce=-151363793
  &wrkey=MTpuN9U8dJcjmXboTmcm8edD&encparams=8R_ixk0eg40SLt0sN0rPcx4ahPttnQUkFjrn_cq53I3d&

Sample format of an unencrypted lookup request:

GET /safebrowsing/lookup?sourceid=firefox-antiphish&features=TrustRank&client=navclient-auto-tbff&q=http%3A//www.honeyd.org

The output from the server is either empty or contains:

phishy:1:1

If the request was unencrypted or contained a bad wrapped key, a server may also output

pleaserekey:1:1

to instruct the client to issue a new GetKey? request.

To decrypt an encrypted a