View source for CloudServices/SoftRelease

= Overview =

Firefox Mobile and Firefox Desktop both follow a specific release cycle that makes it hard to 
ship experimental features, try out small changes on a subset of users or ramp up a new feature to avoid huge peaks on our infrastructure.

The ''SoftRelease'' service offers a way to ramp up or bucket-test a new feature shipped in Firefox Mobile or Desktop.

Use cases examples:

* ramping up Firefox Hello for our user base by making it accessible to 10% of the user base and growing it to 100% once we are confident that the server infrastructure works well.
* activating a new feature for specific regions in Firefox. 
* making small UI variations like what they're doing at the Foundation for their "End Of the Year" campaign, see https://fundraising.mozilla.org/testing-testing-and-more-testing/

= General Principle =

When Firefox Mobile or Dekstop starts, it sends a request to the SoftRelease service to ask if
a feature has to be activated or not.

The proposed API is a single HTTP endpoint that contains the name of the feature:

<pre>
GET https://features.services.mozilla.com/<feature_name>

{'enabled': true}
</pre>

The server analyzes the client IP and specific headers like the User Agent, and returns a JSON 
mapping containing the answer.

When the '''enabled''' key is sent back, the client gets a YES/NO answer and acts upon it.
For example, for Firefox Hello, the decision to display the Hello button or not could be done
by this call:

<pre>
GET https://features.services.mozilla.com/hello
</pre>

In some other cases, the feature is activated but we want to make different versions.
it's preferrable to let the client decide what to do, given a list of options values sent back by the server.

For example, in a campaign page, an UI has a button with two options that may vary amongst users: 
its color and its text. The client can call the server to get back the values to use:

<pre>
GET https://features.services.mozilla.com/campaign-2015

{'color': '#ff0000',
 'value': 'Click Here'}
</pre>

When the client wants to get several features at once, it can batch its requests 
by calling the root endpoint:

<pre>
GET https://features.services.mozilla.com?features=campaign-2015,hello

{
 'hello': {'enabled': true}, 
 'campaign-2015': {'color': '#ff0000', 'value': 'Click Here'}
}
</pre>

= Dashboard =

To manage the responses, the service provides a dashboard where admins can:

* add or remove a feature name
* list the different possible responses that can be returned to a user
* configure a policy for the server to decide which response to return

We provide 3 policies

* '''Weighted''': defines a percentage for each possible response.
* '''Geolocation''': associates regions to possible responses.
* '''User-Agent''': associates User-Agents to possible responses.

Policies can be combined.

== Examples ==


=== Example 1 ===

We want to try out a new donation campaign UI for French users. In their case, we want to set the color
of the button Green. For other users, we want a Blue color.

steps:

#. We add a new "campaign-button-color" feature via the dashboard
#. we add the two possible responses: 

  <pre>
  Green => {'color': 'green'}
  </pre>

  <pre>
  Blue => {'color': 'blue'}
  </pre>

#. we associate each response to a Geolocation Policy

  <pre>
  France: Green
  </pre>

  <pre>
  default: Blue
  </pre>

= Metrics =

Collecting metrics during A/B testing is important to follow & understand what's the impact
of the different versions of a feature.

The proposed service does not provide any server-side metrics, but returns a unique id 
for each combination returned for a given feature.

e.g. :

<pre>
GET https://features.services.mozilla.com/<feature_name>

{'enabled': true, 'id': '4fa1d44e-2f9d-4cd3-a660-85e892c0ace9'}
</pre>

The id is guaranteed to stay unique and consistent and can be used by the application
to track the different combinations.

= Related Works =

There are two related works at Mozilla:

* The Campaign Manager - https://wiki.mozilla.org/CloudServices/Roadmaps/Campaign-Manager
* Abatar - https://github.com/dannycoates/abatar

The campaign manager focuses on targeted messaging for Android users, but its principle is quite similar to
the current proposal: the client asks the server for messages to display - and that message may vary from
one client to the other.

Abatar is a client-side only A/B testing framework: all the possible combinations are stored in the
client and the decision is made with the execution context.

Abatar plans to have a server side at some point, that will serve some javascript files
that contains the combinations, but the decision will still be made by the client.

For evident practical reasons, a project using Abatar will want to own the releasing and publishing process
of all its javascript files, so deporting this to a service like the one described in
this document would add too much complexity.