- If you are a Firefox User with questions about the Heartbeat prompt, go to Heartbeat and Shield User support page
- Engineers: see #engineering
- Product managers: see #process
- Release managers: see #launchSchedule
- University Researcher: see the #contact
- 1 SHIELD PROGRAM
- 2 Launch a Shield Study
- 3 Launch Schedule, Backlog, In Progress
- 4 Engineering and Building
- 5 Contact Us
- 6 History
Shield Turns Superstition into Science
Shield is a Firefox user testing platform for proposed, new and existing features and ideas. Broad Applications The Shield system addon targets specific subsets of users, enables features and messaging, and measures the results of those interventions to evaluate the effect of the features/messages on measures (retention, uptake, performance, crashiness) to create insights for the Firefox product and Mozilla as an organization.
Supported Study and Messaging Designs
Shield supports these Study and Outreach Designs
- Preference-flip to control feature
- Preference-flip to control (disabled by default) system addon
- Add-on Based study (opt-in) Shield Studies
- Add-on Based Study (opt-out) Shield Studies
- Recruits users to participate in testing new features and ideas that help us validate product decisions. Our users tell us what to ship.
- Heartbeat Star Rating with action link
- Messaging Study with action link to external site (survey, Brain Games, interface testing, external user task tool)
- Contextual Feature Recommendation
- Customized action after user event
- On-going opt-in cohort for deeper data collection
- Rollout preference study
Choosing A Design for Evaluating Features
Issues in design choice:
- Code deployment: in-tree (either as native code, or system addon) vs. Extension (Legacy Extension or Addon)
- Sensitivity of data / probes collected. In particular, full urls usually require an opt-in design
- Speed vs. fidelity: if possible, use Images, fake prototypes, etc FIRST to do validation. Gather data using **surveys** of Firefox users, Other Browser users or whoever.
- Just an Idea: Do surveys! Minimal wireframe prototypes! Pile-sort! Don't build! Identify needs, user models, and terminology before solving and building!
- if you are already in tree (mozilla-central), including your probes, use "pref-flip"
- Not in tree, use an opt-out (if data is benign).
- Not in tree, need more sensitive data, needs opt-in (at the risk of recruitment bias), choose opt-in (one-off study) or Pioneer.
Alternatives To Shield
Other methods support these designs better:
- early stage ideation: simple hallway / cafe testing and paper prototypes
- You need Chrome / IE / Edge users - we can recruit those using other methods
- You want in-depth narrative data.
Other methods for launching and testing features or messages include
- AMO addon
- Test Pilot
Launch a Shield Study
Overview: File Bug, Send email to Release-Drivers, tell Shield Team to start recruiting.
- How to Launch A Pref Flip Study
- How to Launch an opt-in / opt out study
- Coming soon -* How To Pioneer
Launch Schedule, Backlog, In Progress
- Google Sheet of Proposed and Inflight
- Pref-flip inflight Dashboard
- Experiments Data Viewer
- System Add-on Installs by Version/Channel
Engineering and Building
Shield System Addon - downloads, interprets and runs recipes
The Shield system addon pulls a list potential "recipes" from the Normandy server on a regular (daily) basis. It allows us to quickly control the delivery of recipes outside of the trains. The addon then filters recipes based on client configuration (channel, locale, geo, addons, prefs). The client-side component lets us constrain what actions are available to recipes for security purposes, and also provides a good point for testing the functionality of our fixes without removing our ability to ship fixes fast.
Read details at: Shield client addon concepts
Normandy - User Profile Matching and Recipe Deployment
- Filters for Shield Targeting, included telemetry ping traits, addons list, and other "point in time" profile features.
- Server Ops Wiki
- Normandy Github Repository
- Normandy Issue Tracker
- There's a [https://bugzilla.mozilla.org/
- describecomponents.cgi?product=Shield "Shield" Bugzilla product] if you want to file bugs that depend on other Bugzilla bugs or otherwise need to integrate with Bugzilla.
- Technical Documentation
Shield Studies Addon Utils
- Single File `.jsm`.
- Embed these inside your Legacy Addon or Embedded WebExtension addon.
- Shield-Studies-Addon-Utils on Github
- if you ARE NOT ON VERSION 4+, please stop :)
- No more Jetpack / Addon-sdk!
Experienced Shield Study Authors Include:
- Gregg Lind
- Kamyar Ardekani
- Benton Case
- Luke Crouch
Not Satisfied? For all additional questions or live support
Visit our SLACK Channel: #shield on mozilla.slack.com and say "help".
Shield Core Team
- Project Lead: Gregg Lind [:gregglind]
- Product Owner: Matt Grimes [:Matt_G]
- Data Steward: Rebecca Weiss [:rweiss]
- Development: Michael Cooper [:mythmon], Michael Kelly [:Osmose], Web Engineering * Systems
- Operations: Daniel Thorn [:relud], Benson Wong [:mostlygeek]
- QA: Chris Hartjes [:chartjes], Adrian Florinescu [:aflorinescu], Ovidiu Boca [:Ovidiu]
- Status: Launched / Active Development
- June 2014 * mconner files a tracker bug for enhancing FHR to proactively detect and solve problems for users.
- December 2014 * bsmedberg files bug 1111016 for implementing the smallest-possible set of changes necessary to support Heartbeat (a prompt shown to users asking them to rate Firefox) and Self-Repair (a minimal version of the problem-solving functionality, described in detail in dependent bugs).
- Early 2015
- UITour is enhanced to support Heartbeat prompts, sending user responses to input.mozilla.org.
- Self-Repair is implemented as a set of static files hosted on AWS that Firefox retrieves.
- Late 2015 * Self Repair is renamed to SHIELD since the project covers more than just repairing Firefox. The name covers the server that deploys configuration files (recipes) and the in-product component that executes the recipe.
- Example 1: The Shield server deploys a recipe to a client indicating that the user should be included in the daily Heartbeat sample. The in-product component responds by offering the Heartbeat prompt.
- Example 2: The SHIELD server matches a “preference repair recipe” to the user configuration, and returns it. Firefox runs the recipe steps to fix the broken preference.
- February 2016 * Development begins on the SHIELD backend service, Normandy. Work on the system add-on begins as well.
- April 2016 * Normandy replaces the self-repair server at the domain self-repair.mozilla.org. Development on the system add-on continues.
- December 2016 * SHIELD System Add-on lands in mozilla-central
- 2017 * Shield adds pref-flip studies, v4 of shield utils and many other things.