Confirmed users
487
edits
Services/TrackingProtection/Shavar Server - Testing: Difference between revisions
Jump to navigation
Jump to search
m
Services/TrackingProtection/Shavar Server - Testing (view source)
Revision as of 04:11, 16 September 2015
, 16 September 2015Re-organize test cases
Rpappalardo (talk | contribs) m (Move setup to a separate section) |
Rpappalardo (talk | contribs) m (Re-organize test cases) |
||
| Line 95: | Line 95: | ||
'''List Delivery and Creation Test''' | '''List Delivery and Creation Test''' | ||
* | * Follow [[#Setup|Setup]] instructions (create new profile, set prefs!) | ||
* | * Do cache file sanity checks ([[#TEST_.230:_verify_downloaded_.28cached.29_mozpub_list_files|TEST #0]]) with unchanged black/white lists. | ||
'''Additions/Deletions Test''' | '''Additions/Deletions Test''' | ||
* | * Follow [[#Setup|Setup]] instructions (create new profile, set different prefs) | ||
* | * Do cache file sanity checks ([[#TEST_.230:_verify_downloaded_.28cached.29_moztestpub_list_files|TEST #0]]) | ||
* Verify that blacklist and whitelist updates get properly delivered to client | * Verify that blacklist and whitelist updates get properly delivered to client | ||
| Line 107: | Line 107: | ||
* ideally we'd be using the real list | * ideally we'd be using the real list | ||
** we can do that for almost all the | ** we can do that for almost all the end-2-end and validation tests, but we can't do that for the "update" test | ||
* TEST #0 gets executed twice | * TEST #0 gets executed twice | ||
** once w/ [[# | ** once w/ [[#Setup|Setup]] to verify moztestpub, mozstd, mozfull cache files | ||
** once w/ [[# | ** once w/ [[#Setup|Setup]] to verify moztestpub cache files | ||
[[#top]] | [[#top]] | ||
| Line 251: | Line 251: | ||
* !!!!! restart Firefox !!!!!! | * !!!!! restart Firefox !!!!!! | ||
Replace production URL with stage URL for the following prefs: | Replace production URL with stage URL for the following prefs: | ||
| Line 271: | Line 270: | ||
[[#top]] | [[#top]] | ||
== List Delivery & Creation Tests == | == List Delivery & Creation Tests (moztestpub, mozstd, mozfull) == | ||
=== list delivery === | |||
TEST #0 | |||
Verify that all data lists have been delivered and have a reasonable filesize. | |||
Execute the following tests, once for each data list. | |||
* moztestpub | * moztestpub | ||
| Line 280: | Line 283: | ||
* mozfull | * mozfull | ||
=== | === verify list files cache === | ||
'''SETUP''' | |||
Follow setup instructions for all three list files here: [[#Setup|Setup]] | |||
'''SUMMARY''' | '''SUMMARY''' | ||
| Line 348: | Line 351: | ||
</code> | </code> | ||
=== | === verify its-a-tracker page === | ||
* Pull up this page: http://itisatrap.org/firefox/its-a-tracker.html | * Pull up this page: http://itisatrap.org/firefox/its-a-tracker.html | ||
| Line 356: | Line 359: | ||
* If not working, then either about:config issue or client issue - repeat above! | * If not working, then either about:config issue or client issue - repeat above! | ||
=== | === verify CNN page === | ||
* Pull up CNN: http://www.cnn.com | * Pull up CNN: http://www.cnn.com | ||
| Line 362: | Line 365: | ||
* the developer tool console tab should indicate that something was blocked by tracking protection - should say something like URL blocked because of tracking | * the developer tool console tab should indicate that something was blocked by tracking protection - should say something like URL blocked because of tracking | ||
=== | === verify YouTube page === | ||
* Pull up Youtube: https://www.youtube.com | * Pull up Youtube: https://www.youtube.com | ||
| Line 371: | Line 374: | ||
[[#top]] | [[#top]] | ||
== List Additions / Deletions Tests (moztestpub | == List Additions / Deletions Tests (moztestpub) == | ||
=== | === blacklist deletions === | ||
TEST #1 | |||
=== | ==== verify enabled ==== | ||
* Follow setup instructions for moztestpub file here: [[#Setup|Setup]] | |||
* Pull up this test page: https://people.mozilla.org/~fmarier/tracking-test/ | * Pull up this test page: https://people.mozilla.org/~fmarier/tracking-test/ | ||
* ensure that the resource is blocked: | * ensure that the resource is blocked: | ||
| Line 390: | Line 390: | ||
** A message in the: Tools > Web Developer > Browser Console: "The resource at "https://extremetracking.com/" was blocked because tracking protection is enabled." | ** A message in the: Tools > Web Developer > Browser Console: "The resource at "https://extremetracking.com/" was blocked because tracking protection is enabled." | ||
==== delete from blacklist ==== | |||
* Follow setup instructions for moztestpub file here: [[#Setup|Setup]] | |||
* Remove the extremetracking.com json blob from the blacklist on https://github.com/mozilla-services/shavar-test-lists | * Remove the extremetracking.com json blob from the blacklist on https://github.com/mozilla-services/shavar-test-lists | ||
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins) | * This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins) | ||
==== verify disabled ==== | |||
* Start Debuggging | |||
** Restart Firefox from terminal to force list updates on client | |||
** The pref set in STEP #0 (browser.safebrowsing.debug) will output debug messages like so: https://pastebin.mozilla.org/8844272 | |||
** Debug output will start in terminal 10 - 30 secs after you start browser | |||
* update URL has registered correctly (line #8) | * update URL has registered correctly (line #8) | ||
** Registering whitelist- mozpub-trackwhite-digest256 (line #10) | ** Registering whitelist- mozpub-trackwhite-digest256 (line #10) | ||
| Line 413: | Line 412: | ||
[[#top]] | [[#top]] | ||
=== | === blacklist additions === | ||
TEST #2 | |||
==== add to blacklist ==== | |||
* Add the extremetracking.com json blob back to the blacklist on https://github.com/mozilla-services/shavar-test-lists | * Add the extremetracking.com json blob back to the blacklist on https://github.com/mozilla-services/shavar-test-lists | ||
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins) | * This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins) | ||
==== verify enabled ==== | |||
* check that it's blocked again by visiting https://people.mozilla.org/~fmarier/tracking-test/ | * check that it's blocked again by visiting https://people.mozilla.org/~fmarier/tracking-test/ | ||
| Line 428: | Line 428: | ||
[[#top]] | [[#top]] | ||
=== | === whitelist deletions === | ||
TEST #3 | |||
'''SUMMARY''' | '''SUMMARY''' | ||
| Line 436: | Line 438: | ||
* By removing it from whitelist, we should be able to verify that it now gets blocked! | * By removing it from whitelist, we should be able to verify that it now gets blocked! | ||
==== verify enabled ==== | |||
Start Debuggging | Start Debuggging | ||
| Line 443: | Line 445: | ||
* Debug output will start in terminal 10 - 30 secs after you start browser | * Debug output will start in terminal 10 - 30 secs after you start browser | ||
==== delete from whitelist ==== | |||
* Remove googlesyndication.com from the "Google" organization in the resources section of the whitelist - https://github.com/mozilla-services/shavar-test-lists | * Remove googlesyndication.com from the "Google" organization in the resources section of the whitelist - https://github.com/mozilla-services/shavar-test-lists | ||
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins) | * This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins) | ||
==== verify disabled ==== | |||
* update URL has registered correctly (line #8) | * update URL has registered correctly (line #8) | ||
** Registering whitelist- mozstd-trackwhite-digest256 (line #10) - or mozfull | ** Registering whitelist- mozstd-trackwhite-digest256 (line #10) - or mozfull | ||