Services/Shavar/Lists

Overview

To quickly check which lists are being served by shavar:

• prod
• stage

Summary

The purpose of this wiki is to provide information related to the Shavar server lists. These lists will be rolled / phased out from both the (Shavar) server and the (Firefox) client.

Server-side

• What lists are currently being server by the Shavar server in STAGE, in PRODUCTION?
• What lists will be removed / added and when (as of which shavar server deployment)

Client-side

• What lists are currently being consumed by the client (in Nightly, DevEd, Beta, etc.)?
• What lists will be removed / added and when (and in what client version)

Reference

Tesplan:

• see: Services/TrackingProtection/Shavar Server - Testing

#top

Purpose

In this wiki you'll find the following:

• a list-centric matrix of shavar dependencies
  • Can be used to easily view a match-up between lists and (a) when they will be available from the shavar server as well as (b) when they will enabled in a specific version of the Firefox client
• A Basic overview of list "pref sets" used for testing
• A Summary of the various list repositories and what they're used for

Please Note:

• there is currently no whitelist for diary testing (mozfull)
• staging and production list are the same (except moztestpub is only used in staging tests)
• In staging environment, we use the same shavar code, just using different lists which get served from different shavar stacks.
• moztespub lists are generated on stage by committing changes to: https://github.com/mozilla-services/shavar-test-lists

#top

List Types & Repos

1. Disconnect list
   • Canonical list from disconnect
2. "Staging" List
   • Our clone of disconnect list we use for testing changes
3. Editing List
   • Editable black/white lists used by services-QA team for e2e-testing

#top

List Dependency Matrix

List Name	Shavar Server Deploy				Firefox Client Release		Purpose	Comments (Ops/QA/Dev)
	STG	PRD	Add in	Remove in	Add in	Remove in
mozfull-track-digest256	X	X			43		the "strict" Disconnect list (uses the same whitelist)
mozfullstaging-track-digest256	X	X	0.7				testing new entries to the strict blacklist before we submit them to Disconnect	Never shipped w/ FF
mozplugin-block-digest256	X	X	0.7		47
mozplugin2-block-digest256	X	X	0.7		47
mozpub-track-digest256	X	X		0.7	38	42
mozstd-track-digest256	X	X			42
mozstd-trackwhite-digest256	X	X			42
mozstdstaging-track-digest256	X	X	0.7				testing new entries to the default blacklist before we submit them to Disconnect	Never shipped w/ FF
mozstdstaging-trackwhite-digest256	X	X	0.7				testing new entries to the whitelist before we submit them to Disconnect	Never shipped w/ FF
moztestpub-track-digest256	X						testing additions and removals during the end-to-end tests	Never shipped w/ FF
moztestpub-trackwhite-digest256	X						testing additions and removals during the end-to-end tests	Never shipped w/ FF
base-track-digest256	X	X
basew3c-track-digest256	X	X
baseeff-track-digest256	X	X
content-track-digest256	X	X
contentw3c-track-digest256	X	X
contenteff-track-digest256	X	X

#top

List Testing

"Pref Sets"

For testing shavar and tracking protection client behavior (in the browser), we need to set certain custom preferences in about:config.

Below you will find a matrix of test "pref sets" and their usage

Pref Set	Key Feature	Comments
mozstd	determine what list client uses to block things
mozfull	tracking table pref - we switch the blacklist, keep same whitelist
mozstaging	for staging changes to the disconnect list
moztestpub	change white & blacklist because we those are the ones we can edit in github	should be called "editing-list" or "e2e-test". For all other tests, the actual underlying list is the disconnect one (except mozstaging which has another upstream repo)
base	same purpose as mozfull since we set the tracking table to include both "base" (equivalent to mozstd) and "content" (which has the entries in mozfull that aren't in mozstd)

#top