Telemetry/Testing

Verifying Pings

Testing methodology

1. Set up your Firefox browser to allow for inspecting browser initiated ssl requests
2. Set the desired pref’s for a test case
3. Restart your browser to ensure an accurate starting point for the test
4. Follow test steps
5. Trigger pings of type (see triggering pings) to validate functionality

Browser Prefs

FHR Prefs

• datareporting.healthreport.service.enabled
• datareporting.healthreport.uploadEnabled

UT Prefs

• toolkit.telemetry.enabled
• toolkit.telemetry.unified - enables UT instead of just the old telemetry

Triggering Pings

Trigger Telemetry pings

1. Disable a plugin

• This will change your saved browser configuration and trigger a ping
• Note that this is limited to 1 ping every 5 minutes, which may or may not be frequently enough for testing purposes

1. Use the browser console

• In your browser console import TelemetryController: Components.utils.import("resource://gre/modules/TelemetryController.jsm");
• Trigger a simple ping with: TelemetryController.submitExternalPing("foobar", {});

Trigger FHR pings

1. Update the datareporting.healthreport.servicenextDataSubmissionTime pref to be approx. 60 seconds from ‘now'

• I used http://currentmillis.com/ to grab the current UTC millis and added 60000 to it
• When the millis elapses a ping will be sent to FHR within a minute or so

Client Testing

Notes

There are a handful of prefs required to effectively test telemetry pings and behavior. See the validating pings section for more info and set up instructions.

The two endpoints were data is sent are https://fhr.data.mozilla.com/1.0/submit/ and https://incoming.telemetry.mozilla.org/submit/

For Old Telemetry and Unified Telemetry, all data is sent to the incoming.telemetry URL with the parameter ?v=4. OT sends "saved-session" pings, UT sends "main" pings.

Test Cases

1. Everything disabled

Initial State: Clean start up with everything disabled, upload enabled

Steps:

• Trigger all pings

Expected result: No data should be sent to either fhr.data or incoming.telemetry

2. Everything enabled

Initial State: Clean start up with everything enabled, upload enabled

Steps:

• Trigger all pings

Expected result: Data is sent to fhr.data, main and saved-session pings are sent to incoming.telemetry

3. FHR and OT enabled, UT disabled

Initial State: clean start up with FHR and OT enabled, upload enabled, UT disabled

Steps:

• Trigger all pings

Expected result: Data is sent to fhr.data and saved-session pings are sent to incoming.telemetry

4. FHR enabled, but upload disabled

Initial State: clean start up with FHR enabled, upload disabled

Steps:

• Trigger all pings

Expected result: no data sent to either fhr.data or incoming.telemetry

5. Everything enabled, but upload disabled

Initial State: clean start up with everything enabled, upload disabled

Steps:

• Trigger all pings

Expected result: no data sent to fhr.data or incoming.telemetry

6. Everything enabled, confirm UT is sending, then disable UT

Initial State: Active instance with everything enabled, upload enabled

Steps:

• Trigger all pings
• Confirm incoming.telemetry is receiving main pings
• Disable UT

Expected result: incoming.telemetry stops receiving main pings, saved-session pings continue to send

7. Everything enabled, confirm UT is sending, then disable upload

Initial State: Active instance with everything enabled, upload enabled

Steps:

• Trigger all pings
• Confirm incoming.telemetry and fhr.data is receiving main pings
• Disable upload

Expected result: fhr.data no longer receives data

8. FHR and OT enabled, confirm OT is sending, then enable UT

Initial State: Active instance with FHR and OT enabled, UT disabled, upload enabled

Steps:

• Trigger all pings
• Confirm incoming.telemetry is receiving saved-sessoin pings but not main pings
• Enable UT

Expected result: Data is sent to fhr.data and incoming.telemetry receives both saved-session and main pings

9. Disabling FHR upload results in a delete request being sent

Initial State: FHR and Upload prefs are enabled

Steps:

• Trigger all pings
• Confirm fhr.data and incoming.telemetry are receiving pings
• Disable upload

Expected result: Delete request is sent to https://fhr.data.mozilla.com/1.0/submit/

Server Testing

The data pipeline testing involves validating the accuracy and amount of data processed to ensure the incoming pings are counted correctly so that the graphs we produce and make decisions from are reliable.

Testing the servers involves 3 parts:

1. Pipeline Execution
2. Data validation
3. Load

Pipeline Execution

• Sample Data Generator
• Script/tool to submit sample data for processing, possibly based on test/load profiles
• Verify the pipeline successfully processes the data and no errors are raised

Data validation

• Sample Data Generator
• Script/tool to submit sample data for processing, possibly based on test/load profiles
• Analyze and compare actual output to expected output (based on script/sample data input)

Load Testing

• Load is expected to be about 1000rps in beta spiking to 2000 during peak times, with each request being approx. 100kb
• Load is expected to be about 3000rps in production spiking to 8000 during peak times, with each request being approx. 100kb

Inspecting Browser Initiated SSL Requests

pro-tip: if starting fx from the command line you can add the -jsdebugger switch to auto-open the toolbox

1. Open the inspector (option-command-i on mac)
2. Click the gear/settings icon
3. Scroll all the way down to advanced settings (should be on the bottom right)
4. Enable both remote and chrome debugging
5. In about:config make sure devtools.debugger.prompt-connection is set to false
6. Restart firefox
7. In the customize menu there is a developer menu (the wrench, move it to your toolbar for quick access if you want)
8. In the developer menu open the Browser Toolbox, alternatively you can use Tools -> Web Developer -> Browser Toolbox
9. View and use the network tab as you normally would in the regular inspector, except now you can capture events from browser level requests (pocket, sync, telemetry, etc.), including SSL