Stub Attribution Notes/Test Plan (scratchpad)
Dependencies (tools/systems):
Questions:
- All (supported) Windows platforms, or?
- Which Firefox editions?
- How to test (all five?) codes/fields?
- Source
- Medium
- Campaign
- Content
- "Referrer" alone?
- Do Not Track support
- How/does JavaScript-disabled interplay with DNT?
- Which specific browsers (vendors + versions) properly support it/if have it enabled, we'll honor the setting?
- How/does JavaScript-disabled interplay with DNT?
- Which locale(s)?
- Entry-points on Mozilla.com. Which page(s)? Or all download pages?
- https://www.mozilla.org/en-US/firefox/all/
- https://www.mozilla.org/en-US/firefox/new/?scene=2 (and do the ?scene=[] parameters matter?)
- JavaScript enabled *and* disabled cases?
- Should the attribution code(s) be present on Download buttons when JavaScript is disabled?
- How/who checks the Telemetry ping from the client? Via Dashboard?
- Do we need to cover the upgrade-path scenario? i.e.:
- user downloads and installs using the special stub installer w/tracking code
- we verify correct pings, etc.
- user upgrades later to a newer version of Firefox (pave-over install)
- do we still check for this ping?
- How about the successfully installed, then uninstalled case: check to see that client no longer sends ping?
Manual Tests:
- Positive (happy-path) Tests:
- Do Not Track disabled
- Click an ad banner or link which has an attribution (source? medium? campaign? content? referrer?) code
- Verify that the URL which takes you to a Mozilla.org download page contains a stub_attcode with a valid param
- Verify that the same valid stub_attcode param/value gets passed to the Mozilla.org Download Firefox button
- Download and install the stub installer
- Verify (how?) that upon successful installation, the stub installer sends a "success!" ping with the same stub_attcode param/value
- Verify (need specific Windows registry key, here) that the stub_attcode gets written to the registry/on-disk (need to double-check this) with appropriate value(s)
- ...which are?
- Do Not Track disabled
Negative Tests:
- Ensure other installers/binaries don't have/pass on the URL param/stub code
- e.g. Mac, Linux, full Firefox for Windows installers
- Much, much more to flesh out, here
Test Automation Coverage:
- What will the Bedrock unit tests cover?
- Where/how frequently will they run? With each pull request/commit/release?
- What will the Bouncer tests cover?
- Where/how frequently will they run? With each pull request/commit/release? On a cron?