Confirmed users
95
edits
Telemetry/Testing: Difference between revisions
→Browser Prefs: add link to preferences documentation
StuartPhilp (talk | contribs) (Testing Notes and Cases for the Telemetry Pipeline) |
Gfritzsche (talk | contribs) (→Browser Prefs: add link to preferences documentation) |
||
| (2 intermediate revisions by 2 users not shown) | |||
| Line 7: | Line 7: | ||
:# Restart your browser to ensure an accurate starting point for the test | :# Restart your browser to ensure an accurate starting point for the test | ||
:# Follow test steps | :# Follow test steps | ||
:# Trigger pings of | :# Trigger pings of various types (see triggering pings) to validate functionality | ||
Once you have tested against the live servers, you can validate against a local server to get a deeper look at the pings. | |||
:# Set up a local server to accept pings for inspection | |||
:# Set up your Firefox browser to send pings to your local server | |||
:# Trigger pings of various types to validate functionality | |||
==Browser Prefs== | ==Browser Prefs== | ||
[https://gecko.readthedocs.org/en/latest/toolkit/components/telemetry/telemetry/preferences.html Preferences documentation] | |||
====FHR Prefs==== | ====FHR Prefs==== | ||
| Line 18: | Line 27: | ||
:* toolkit.telemetry.enabled | :* toolkit.telemetry.enabled | ||
:* toolkit.telemetry.unified - enables UT instead of just the old telemetry | :* toolkit.telemetry.unified - enables UT instead of just the old telemetry | ||
:* toolkit.telemetry.unifiedIsOptIn - makes Telemetry opt-in even with UT enabled | |||
:* toolkit.telemetry.optoutSample - makes Telemetry opt-out for a 5% sample of users if UT is on and opt-in | |||
==Triggering Pings== | ==Triggering Pings== | ||
| Line 27: | Line 38: | ||
:::* In your browser console import TelemetryController: Components.utils.import("resource://gre/modules/TelemetryController.jsm"); | :::* In your browser console import TelemetryController: Components.utils.import("resource://gre/modules/TelemetryController.jsm"); | ||
:::* Trigger a simple ping with: TelemetryController.submitExternalPing("foobar", {}); | :::* Trigger a simple ping with: TelemetryController.submitExternalPing("foobar", {}); | ||
:# Restart your browser | |||
:::* When the browser is shutdown and then started again, a new telemetry session should begin. You should see saved-session | |||
====Trigger FHR pings==== | ====Trigger FHR pings==== | ||
| Line 34: | Line 47: | ||
=Client Testing= | =Client Testing= | ||
====Pref Control Test Suite==== | |||
====Notes==== | ====Notes==== | ||
This suite tests that the behavior of pings (recorded and uploaded) is controlled correctly by the selected browser preferences. This suite should be run against the live servers to make sure the correct endpoints are hit, and that data is or is not sent as expected based on the prefs that are set. | |||
There are a handful of prefs required to effectively test telemetry pings and behavior. See the validating pings section for more info and set up instructions. | There are a handful of prefs required to effectively test telemetry pings and behavior. See the validating pings section for more info and set up instructions. | ||
The two endpoints | The two endpoints that pings should be sent to are https://fhr.data.mozilla.com/1.0/submit/ and https://incoming.telemetry.mozilla.org/submit/ | ||
For Old Telemetry and Unified Telemetry, all data is sent to the incoming.telemetry URL with the parameter ?v=4. OT sends "saved-session" pings, UT sends "main" pings. | For Old Telemetry and Unified Telemetry, all data is sent to the incoming.telemetry URL with the parameter ?v=4. OT sends "saved-session" pings, UT sends "main" pings. The ping types can be inferred from the ping request URL, e.g. https://incoming.telemetry.mozilla.org/submit/telemetry/14f9e835-d100-b44d-b833-8c027d220453/main/Firefox/42.0a1/nightly/20150721030212?v=4 is a /main/ ping as identified after the ID. A saved-session would look something like https://incoming.telemetry.mozilla.org/submit/telemetry/78eff6fd-40ca-6742-8630-5dbbff090e89/saved-session/Firefox/42.0a1/nightly/20150721030212?v=4. | ||
====Test Cases==== | ====Test Cases==== | ||
| Line 118: | Line 135: | ||
::* Disable upload | ::* Disable upload | ||
: '''Expected result''': Delete request is sent to https://fhr.data.mozilla.com/1.0/submit/ | : '''Expected result''': Delete request is sent to https://fhr.data.mozilla.com/1.0/submit/ | ||
====Ping Type Test Suite==== | |||
====Notes==== | |||
There are several types of pings to look at: | |||
https://gecko.readthedocs.org/en/latest/toolkit/components/telemetry/telemetry/pings.html#ping-types | |||
Additionally under the main ping type, there are several reasons: | |||
https://gecko.readthedocs.org/en/latest/toolkit/components/telemetry/telemetry/main-ping.html | |||
Each of these ping types has specific conditions that trigger them, as outlined in the docs. These cases have been covered in this test suite. | |||
The two main types of pings are the old saved-session ping, and the new main ping. The old saved-session ping will soon be removed, and will be handled by UT's main ping with a reason of saved-session. | |||
main ping reasons: | |||
aborted-session | |||
environment-change | |||
shutdown | |||
daily | |||
saved-session | |||
====Session and Subsession Validation Test Suite==== | |||
=Server Testing= | =Server Testing= | ||