Confirmed users
1,504
edits
Rpappalardo (talk | contribs) (update loadtesting) |
|||
| (47 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
<span style="color:#FF0000">This page is deprecated!!!!!! Please see: [https://testrail.stage.mozaws.net/index.php?/suites/view/354&group_by=cases:section_id&group_order=asc TestRail testcase repository]</span> | |||
<span style="color:#FF0000">(for TestRail permissions, file an issue here: https://github.com/mozilla-services/shavar-e2e-tests )</span> | |||
[[Category:Test Plans]] | [[Category:Test Plans]] | ||
= Testplan (Overview) = | = Testplan (Overview) = | ||
| Line 33: | Line 40: | ||
'''REFERENCE''' | '''REFERENCE''' | ||
Shavar Lists | |||
* [[Services/Shavar/Lists]] | |||
UI for Tracking protection | UI for Tracking protection | ||
| Line 41: | Line 51: | ||
** client-only test! (shavar-server could be completely broken and we wouldn't know) | ** client-only test! (shavar-server could be completely broken and we wouldn't know) | ||
** good test to run first, because if its broken, no reason to proceed | ** good test to run first, because if its broken, no reason to proceed | ||
* | * https://mozilla.github.io/tracking-test/ | ||
* https://mozilla.github.io/tracking-test/ads.html | |||
* https://mozilla.github.io/tracking-test/analytics.html | |||
* https://mozilla.github.io/tracking-test/social.html | |||
** page checks if list delivery/generation is broken | ** page checks if list delivery/generation is broken | ||
** Uses: http://extremetracking.com/ (essentially within an iframe) | ** Uses: http://extremetracking.com/ (essentially within an iframe) | ||
| Line 61: | Line 74: | ||
PRODUCTION: | PRODUCTION: | ||
* shavar.services.mozilla.com - e2e test URL (no loadtesting on PRODUCTION) | * shavar.services.mozilla.com - e2e test URL (no loadtesting on PRODUCTION) | ||
* tracking-protection.cdn.mozilla.net - CDN for lists | * tracking-protection.cdn.mozilla.net - CDN for lists | ||
| Line 98: | Line 110: | ||
* ideally we'd be using the real list | * ideally we'd be using the real list | ||
** we can do that for almost all the end-2-end and validation tests, but we can't do that for the "update" test | ** we can do that for almost all the end-2-end and validation tests, but we can't do that for the "update" test | ||
[[#top]] | [[#top]] | ||
| Line 113: | Line 118: | ||
'''PURPOSE''' | '''PURPOSE''' | ||
* test that the server is configured properly. | * test that the server is configured properly. | ||
=== STAGE === | === STAGE === | ||
1) Verify the download of all lists: | |||
$ curl -d" " 'https://shavar.stage.mozaws.net/list?client=foo&appver=1&pver=2.2' | $ curl -d" " 'https://shavar.stage.mozaws.net/list?client=foo&appver=1&pver=2.2' | ||
Should return all of these lists: | Should return all of these lists: | ||
ads-track-digest256 | |||
allow-flashallow-digest256 | |||
analytics-track-digest256 | |||
base-track-digest256 | |||
baseeff-track-digest256 | |||
basew3c-track-digest256 | |||
block-flash-digest256 | |||
block-flashsubdoc-digest256 | |||
content-track-digest256 | |||
contenteff-track-digest256 | |||
contentw3c-track-digest256 | |||
except-flash-digest256 | |||
except-flashallow-digest256 | |||
except-flashinfobar-digest256 | |||
except-flashsubdoc-digest256 | |||
except-infobars-digest256 | |||
mozfull-track-digest256 | |||
mozfullstaging-track-digest256 | |||
mozplugin-block-digest256 | |||
mozpub-track-digest256 | |||
mozstd-track-digest256 | |||
mozstd-trackwhite-digest256 | |||
mozstdstaging-track-digest256 | |||
mozstdstaging-trackwhite-digest256 | |||
moztestpub-track-digest256 | |||
moztestpub-trackwhite-digest256 | |||
social-track-digest256 | |||
2) Curl each of the list groups. | |||
Example for MOZSTD: | |||
$ curl -d "mozstd-track-digest256;" 'https://shavar.stage.mozaws.net/downloads?client=foo&appver=1&pver=2.2' | |||
Should return this: | Should return a response similar to this: | ||
n:3600 | n:3600 | ||
i:mozstd-track-digest256 | i:mozstd-track-digest256 | ||
u:tracking-protection.stage.mozaws.net/mozstd-track-digest256/1442275596 | u:tracking-protection.stage.mozaws.net/mozstd-track-digest256/1442275596 | ||
[[#top]] | [[#top]] | ||
=== PRE-PRODUCTION === | === PRE-PRODUCTION === | ||
Stack checks are run on pre-prod to verify that a new app stack returns the same results as the old stack. It verifies the server response. This environment is only up when the tests need to be run, which is only when there is a new stack. | |||
$ curl -k -d" " 'https://shavarprodtest.prod.mozaws.net/list?client=foo&appver=1&pver=2.2' | $ curl -k -d" " 'https://shavarprodtest.prod.mozaws.net/list?client=foo&appver=1&pver=2.2' | ||
Should return all of these lists: | Should return all of these lists: | ||
[* update with next verification] | |||
base-track-digest256 | base-track-digest256 | ||
baseeff-track-digest256 | baseeff-track-digest256 | ||
| Line 197: | Line 191: | ||
mozstdstaging-track-digest256 | mozstdstaging-track-digest256 | ||
mozstdstaging-trackwhite-digest256 | mozstdstaging-trackwhite-digest256 | ||
[[#top]] | [[#top]] | ||
=== PRODUCTION === | === PRODUCTION === | ||
# shavar URL | # shavar URL | ||
$ curl -d" " 'https://shavar.services.mozilla.com/list?client=foo&appver=1&pver=2.2' | $ curl -d" " 'https://shavar.services.mozilla.com/list?client=foo&appver=1&pver=2.2' | ||
# tracking URL (CNAME'd to shavar URL - should produce same output) | # tracking URL (CNAME'd to shavar URL - should produce same output) | ||
$ curl -d" " 'https://tracking.services.mozilla.com/list?client=foo&appver=1&pver=2.2' | $ curl -d" " 'https://tracking.services.mozilla.com/list?client=foo&appver=1&pver=2.2' | ||
Should return all of these lists: | Should return all of these lists: | ||
ads-track-digest256 | |||
allow-flashallow-digest256 | |||
analytics-track-digest256 | |||
base-track-digest256 | base-track-digest256 | ||
baseeff-track-digest256 | baseeff-track-digest256 | ||
basew3c-track-digest256 | basew3c-track-digest256 | ||
block-flash-digest256 | |||
block-flashsubdoc-digest256 | |||
content-track-digest256 | content-track-digest256 | ||
contenteff-track-digest256 | contenteff-track-digest256 | ||
contentw3c-track-digest256 | contentw3c-track-digest256 | ||
except-flash-digest256 | |||
except-flashallow-digest256 | |||
except-flashinfobar-digest256 | |||
except-flashsubdoc-digest256 | |||
mozfull-track-digest256 | mozfull-track-digest256 | ||
mozfullstaging-track-digest256 | mozfullstaging-track-digest256 | ||
mozplugin-block-digest256 | mozplugin-block-digest256 | ||
mozplugin2-block-digest256 | mozplugin2-block-digest256 | ||
mozpub-track-digest256 | |||
mozstd-track-digest256 | mozstd-track-digest256 | ||
mozstd-trackwhite-digest256 | mozstd-trackwhite-digest256 | ||
mozstdstaging-track-digest256 | mozstdstaging-track-digest256 | ||
mozstdstaging-trackwhite-digest256 | mozstdstaging-trackwhite-digest256 | ||
social-track-digest256 | |||
Curl for the mozstd list: | Curl for the mozstd list: | ||
# shavar URL | # shavar URL | ||
$ curl -d"mozstd-track-digest256;" 'https://shavar.services.mozilla.com/downloads?client=foo&appver=1&pver=2.2' | $ curl -d"mozstd-track-digest256;" 'https://shavar.services.mozilla.com/downloads?client=foo&appver=1&pver=2.2' | ||
# tracking URL (CNAME'd to shavar URL - should produce same output) | # tracking URL (CNAME'd to shavar URL - should produce same output) | ||
$ curl -d"mozstd-track-digest256;" 'https://tracking.services.mozilla.com/downloads?client=foo&appver=1&pver=2.2' | $ curl -d"mozstd-track-digest256;" 'https://tracking.services.mozilla.com/downloads?client=foo&appver=1&pver=2.2' | ||
Should return this: | Should return this: | ||
n:3600 | n:3600 | ||
i:mozstd-track-digest256 | i:mozstd-track-digest256 | ||
u:tracking-protection.services.mozilla.com/mozstd-track-digest256/1442275596 | u:tracking-protection.services.mozilla.com/mozstd-track-digest256/1442275596 | ||
[[#top]] | [[#top]] | ||
| Line 389: | Line 364: | ||
* Below are the custom prefs needed for each of the data lists | * Below are the custom prefs needed for each of the data lists | ||
* Not all data lists (and corresponding prefs) are needed for each test | * Not all data lists (and corresponding prefs) are needed for each test | ||
=== SET CUSTOM PREFS === | === SET CUSTOM PREFS === | ||
'''NOTE''' | '''NOTE''' | ||
* Prefs info is permanently located at (https://github.com/mozilla-services/services-test/blob/dev/shavar/e2e-test/prefs.ini) | |||
* Prefs are listed below for stage testing. | * Prefs are listed below for stage testing. | ||
* To test production, replace staging URL (stage.mozaws.net) with production URL (shavar.services.mozilla.com) | * To test production, replace staging URL (stage.mozaws.net) with production URL (shavar.services.mozilla.com) | ||
* all list files are tested in both stage and production (except moztestpub - stage only) | * all list files are tested in both stage and production (except moztestpub - stage only) | ||
'''Prefs for: base list file''' | |||
'''Prefs for: | |||
<source lang="bash"> | <source lang="bash"> | ||
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html "); | user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html"); | ||
user_pref("browser.safebrowsing.debug", true); | user_pref("browser.safebrowsing.debug", true); | ||
user_pref("browser.safebrowsing.enabled", false); | user_pref("browser.safebrowsing.phishing.enabled", false); | ||
user_pref("browser.safebrowsing.malware.enabled", false); | user_pref("browser.safebrowsing.malware.enabled", false); | ||
user_pref("privacy.trackingprotection.enabled", true); | |||
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1); | |||
user_pref("browser.safebrowsing.provider.mozilla.lists", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | ||
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | ||
user_pref("urlclassifier.trackingTable", "test-track-simple,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("urlclassifier.trackingTable", "test-track-simple, | user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256"); | ||
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple"); | </source> | ||
user_pref(" | |||
'''Create a Strict list''' | |||
<source lang="bash"> | |||
1. Create a new Nightly profile. | |||
2. Go into about:preferences#privacy and set Tracking Protection to Always. | |||
3. Click on Change Block List and select strict protection. | |||
4. Make these edits in about:config | |||
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1); | |||
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | |||
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | |||
</source> | |||
'''Create a Cryptomining Fingerprinting list''' | |||
<source lang="bash"> | |||
1. Create a new profile | |||
2. Make these edits in about:config | |||
browser.safebrowsing.provider.mozilla.gethashURL;https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2 | |||
browser.safebrowsing.provider.mozilla.updateURL;https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2 | |||
browser.safebrowsing.provider.mozilla.lists;base-track-digest256,mozstd-trackwhite-digest256,content-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,block-flash-digest256,except-flash-digest256,allow-flashallow-digest256,except-flashallow-digest256,block-flashsubdoc-digest256,except-flashsubdoc-digest256,except-flashinfobar-digest256,ads-track-digest256,social-track-digest256,analytics-track-digest256,fastblock1-track-digest256,fastblock1-trackwhite-digest256,fastblock2-track-digest256,fastblock2-trackwhite-digest256,fastblock3-track-digest256,base-cryptomining-track-digest256,base-fingerprinting-track-digest256 | |||
urlclassifier.trackingTable;test-track-simple,base-track-digest256,base-cryptomining-track-digest256,base-fingerprinting-track-digest256 | |||
browser.safebrowsing.provider.mozilla.nextupdatetime;0 | |||
3. Restart browser | |||
4. Open a Private Browsing window | |||
</source> | </source> | ||
'''Prefs for: moztestpub list file''' | '''Prefs for: moztestpub list file''' | ||
<source> | |||
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html"); | |||
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html "); | |||
user_pref("browser.safebrowsing.debug", true); | user_pref("browser.safebrowsing.debug", true); | ||
user_pref("browser.safebrowsing.enabled", false); | user_pref("browser.safebrowsing.phishing.enabled", false); | ||
user_pref("browser.safebrowsing.malware.enabled", false); | user_pref("browser.safebrowsing.malware.enabled", false); | ||
user_pref("privacy.trackingprotection.enabled", true); | |||
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1); | |||
user_pref("browser.safebrowsing.provider.mozilla.lists", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | ||
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | ||
user_pref("urlclassifier.trackingTable", "test-track-simple,moztestpub-track-digest256"); | |||
user_pref("urlclassifier.trackingTable", "test-track-simple,moztestpub-track-digest256"); | |||
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,moztestpub-trackwhite-digest256"); | user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,moztestpub-trackwhite-digest256"); | ||
</source> | </source> | ||
'''Prefs for: mozstd list file''' | '''Prefs for: mozstd list file''' | ||
<source lang="bash"> | <source lang="bash"> | ||
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html "); | user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html"); | ||
user_pref("browser.safebrowsing.debug", true); | user_pref("browser.safebrowsing.debug", true); | ||
user_pref("browser.safebrowsing.enabled", false); | user_pref("browser.safebrowsing.phishing.enabled", false); | ||
user_pref("browser.safebrowsing.malware.enabled", false); | user_pref("browser.safebrowsing.malware.enabled", false); | ||
user_pref("privacy.trackingprotection.enabled", true); | |||
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1); | |||
user_pref("browser.safebrowsing.provider.mozilla.lists", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | ||
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | ||
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstd-track-digest256"); | |||
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstd-track-digest256"); | |||
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256"); | user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256"); | ||
</source> | </source> | ||
| Line 453: | Line 456: | ||
<source lang="bash"> | <source lang="bash"> | ||
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html "); | user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html"); | ||
user_pref("browser.safebrowsing.debug", true); | user_pref("browser.safebrowsing.debug", true); | ||
user_pref("browser.safebrowsing.enabled", false); | user_pref("browser.safebrowsing.phishing.enabled", false); | ||
user_pref("browser.safebrowsing.malware.enabled", false); | user_pref("browser.safebrowsing.malware.enabled", false); | ||
user_pref("privacy.trackingprotection.enabled", true); | |||
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1); | |||
user_pref("browser.safebrowsing.provider.mozilla.lists", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | ||
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | ||
user_pref("urlclassifier.trackingTable", "test-track-simple,mozfull-track-digest256"); | |||
user_pref("urlclassifier.trackingTable", "test-track-simple,mozfull-track-digest256"); | |||
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256"); | user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256"); | ||
</source> | </source> | ||
| Line 469: | Line 473: | ||
<source lang="bash"> | <source lang="bash"> | ||
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html "); | user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html"); | ||
user_pref("browser.safebrowsing.debug", true); | user_pref("browser.safebrowsing.debug", true); | ||
user_pref("browser.safebrowsing.enabled", false); | user_pref("browser.safebrowsing.phishing.enabled", false); | ||
user_pref("browser.safebrowsing.malware.enabled", false); | user_pref("browser.safebrowsing.malware.enabled", false); | ||
user_pref("privacy.trackingprotection.enabled", true); | |||
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1); | |||
user_pref("browser.safebrowsing.provider.mozilla.lists", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256"); | |||
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | ||
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2"); | ||
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstdstaging-track-digest256,mozfullstaging-track-digest256"); | |||
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstdstaging-track-digest256,mozfullstaging-track-digest256"); | |||
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstdstaging-trackwhite-digest256"); | user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstdstaging-trackwhite-digest256"); | ||
</source> | </source> | ||
[[#top]] | |||
== List Delivery & Creation Tests: moztestpub, mozstd, mozfull, mozstaging, base == | |||
NOTE: | |||
* | * run tests on both STAGE & PROD (w/ the exception of moztestpub which only exists on STAGE) | ||
=== list delivery === | === list delivery === | ||
| Line 508: | Line 499: | ||
Execute the following tests, once for each data list. | Execute the following tests, once for each data list. | ||
* | LIST GROUPS | ||
* moztestpub [STAGE ONLY] | |||
* mozstd | * mozstd | ||
* mozfull | * mozfull | ||
* mozstaging | * mozstaging | ||
* | * base | ||
* strict | |||
* fastblock | |||
==== verify list files cache ==== | ==== verify list files cache ==== | ||
| Line 530: | Line 525: | ||
'''FILESIZES''' | '''FILESIZES''' | ||
whitelists should be < 400K | |||
* mozstd-trackwhite-digest256 | |||
* mozstdstaging-trackwhite-digest256 [NEW] | |||
* moztestpub-trackwhite-digest256 | |||
blacklists should be < 100K | blacklists should be < 100K | ||
* mozfull-track-digest256 | * mozfull-track-digest256 | ||
* mozfullstaging | * mozfullstaging-track-digest256 | ||
* mozstd-track-digest256 | * mozstd-track-digest256 | ||
* mozstdstaging-track-digest256 | * mozstdstaging-track-digest256 | ||
* moztestpub-track-digest256 | * moztestpub-track-digest256 | ||
* base-track-digest256 | |||
content list < 30k | |||
* content-track-digest256 | |||
DNT lists < 1k | |||
* basew3c-track-digest256 | |||
* baseeff-track-digest256 | |||
* contentw3c-track-digest256 | |||
* contenteff-track-digest256 | |||
plugin lists should be > 0 | |||
* mozplugin-block-digest256 | |||
* mozplugin2-block-digest256 | |||
''''CACHE LOCATION''' | ''''CACHE LOCATION''' | ||
The cache has a different location for different operating systems. | |||
Example for moztestpub lists: | |||
<code> | <code> | ||
~/.cache/mozilla/firefox/XXXX/safebrowsing/moztestpub-track{,white}-digest256.* on Linux | ~/.cache/mozilla/firefox/XXXX/safebrowsing/moztestpub-track{,white}-digest256.* on Linux | ||
~/Library/Caches/Firefox/Profiles/XXXX/safebrowsing/moztestpub-track{,white}-digest256.* on Mac | ~/Library/Caches/Firefox/Profiles/XXXX/safebrowsing/moztestpub-track{,white}-digest256.* on Mac | ||
</code> | </code> | ||
'''LIST FILES / SIZE''' | '''LIST FILES / SIZE''' | ||
All files in the safebrowsing folder created for the profile should have full permissions. | |||
Example for moztestpub lists: | |||
for moztestpub lists: | |||
<code> | <code> | ||
-rw-r--r-- 1 johnnyquest staff 52620 Sep 15 10:55 moztestpub-track-digest256.cache | -rw-r--r-- 1 johnnyquest staff 52620 Sep 15 10:55 moztestpub-track-digest256.cache | ||
| Line 596: | Line 580: | ||
</code> | </code> | ||
==== verify list pages ==== | |||
==== verify | |||
repeat test 1 x for each of these list groups: | |||
* STAGE: mozstd, mozfull, moztestpub, base | |||
* PROD: mozstd, mozfull, moztestpub, base | |||
STEPS: | STEPS: | ||
* | * Set up list | ||
* Verify with matching test page: | |||
https://mozilla.github.io/tracking-test/ads.html | |||
https://mozilla.github.io/tracking-test/analytics.html | |||
https://mozilla.github.io/tracking-test/social.html | |||
https://mozilla.github.io/tracking-test/fastblock.html | |||
https://mozilla.github.io/tracking-test/disconnect.html | |||
https://mozilla.github.io/tracking-test/full.html - for the Strict list | |||
==== verify CNN page ==== | ==== verify CNN page ==== | ||
repeat test 1 x for each of: | repeat test 1 x for each of: | ||
* STAGE: moztestpub, mozstd, mozfull, mozstaging | * STAGE: moztestpub, mozstd, mozfull, mozstaging, base | ||
* PROD: | * PROD: mozstd, mozfull, mozstaging, base | ||
STEPS: | STEPS: | ||
| Line 642: | Line 610: | ||
repeat test 1 x for each of: | repeat test 1 x for each of: | ||
* STAGE: mozstd, mozfull, mozstaging | * STAGE: mozstd, mozfull, mozstaging, base | ||
* PROD: mozstd, mozfull, mozstaging | * PROD: mozstd, mozfull, mozstaging, base | ||
Note: skip this test for | Note: skip this test for moztestpub-track-digest256. | ||
STEPS: | STEPS: | ||
| Line 656: | Line 624: | ||
[[#top]] | [[#top]] | ||
== List Additions / Deletions Tests: | == List Additions / Deletions Tests: moztestpub == | ||
NOTE: | |||
* STAGE only | |||
=== blacklist deletions === | === blacklist deletions === | ||
| Line 662: | Line 633: | ||
==== verify enabled ==== | ==== verify enabled ==== | ||
* Follow setup instructions for | * Follow setup instructions for moztestpub file here: [[#Setup|Setup]] | ||
* Pull up this test page: https:// | * Pull up this test page: https://mozilla.github.io/tracking-test/ | ||
* ensure that the resource is blocked: | * ensure that the resource is blocked: | ||
* You should see: | * You should see: | ||
| Line 672: | Line 643: | ||
==== delete from blacklist ==== | ==== delete from blacklist ==== | ||
* Follow setup instructions for | * Follow setup instructions for moztestpub file here: [[#Setup|Setup]] | ||
* Remove the extremetracking.com json blob from the blacklist (track.json) on https://github.com/mozilla-services/shavar-test-lists | * Remove the extremetracking.com json blob from the blacklist (track.json) on https://github.com/mozilla-services/shavar-test-lists | ||
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins) | * This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins) | ||
* Manually trigger an update via the mozilla button on the about:url-classifier page | |||
==== verify disabled ==== | ==== verify disabled ==== | ||
* Restart Firefox from terminal to force list updates on client | * Restart Firefox from terminal to force list updates on client | ||
* Start | * Start Debugging | ||
Verify debugging log (in terminal where you launched Firefox). | |||
Here are some examples of what you might see: | |||
'''update URL has registered correctly''' | '''update URL has registered correctly''' | ||
* i.e: listmanager: 09:44:04 GMT-0700 (PDT): registering | * i.e: listmanager: 09:44:04 GMT-0700 (PDT): registering moztestpub-track-digest256 with https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=43.0a1&pver=2.2 | ||
'''Registering whitelist- mozstd-trackwhite-digest256''' | '''Registering whitelist- mozstd-trackwhite-digest256''' | ||
* listmanager: 09:44:04 GMT-0700 (PDT): registering | * listmanager: 09:44:04 GMT-0700 (PDT): registering moztestpub-trackwhite-digest256 with https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=43.0a1&pver=2.2 | ||
'''Checking for updates on shavar server''' | '''Checking for updates on shavar server''' | ||
* listmanager: 09:44:07 GMT-0700 (PDT): checkForUpdates with https:// | * listmanager: 09:44:07 GMT-0700 (PDT): checkForUpdates with https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=43.0a1&pver=2.2 | ||
'''Request to shavar server begins''' | '''Request to shavar server begins''' | ||
* listmanager: 09:44:07 GMT-0700 (PDT): makeUpdateRequestForEntry_: request | * listmanager: 09:44:07 GMT-0700 (PDT): makeUpdateRequestForEntry_: request moztestpub-track-digest256;a:1440776223 | ||
'''Update success for both lists''' | '''Update success for both lists''' | ||
* listmanager: 09:44:07 GMT-0700 (PDT): update success for mozpub-track-digest256, | * listmanager: 09:44:07 GMT-0700 (PDT): update success for mozpub-track-digest256,moztestpub-trackwhite-digest256 from https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=43.0a1&pver=2.2: 2700 | ||
'''VERIFY''' | '''VERIFY''' | ||
* check that it's no longer blocked by visiting https:// | * check that it's no longer blocked by visiting https://mozilla.github.io/tracking-test/ | ||
* If list with disabled trackingprotection has loaded, you should see: black cat bearing its claws: [https://wiki.mozilla.org/File:Tracking_protection_off.png Cat] | * If list with disabled trackingprotection has loaded, you should see: black cat bearing its claws: [https://wiki.mozilla.org/File:Tracking_protection_off.png Cat] | ||
| Line 713: | Line 688: | ||
==== verify enabled ==== | ==== verify enabled ==== | ||
* check that it's blocked again by visiting https:// | * check that it's blocked again by visiting https://mozilla.github.io/tracking-test/ | ||
* If list with trackingprotection enabled has loaded, you should see: fox with a stop sign: [https://wiki.mozilla.org/File:Tracking_protection_on.png Fox] | * If list with trackingprotection enabled has loaded, you should see: fox with a stop sign: [https://wiki.mozilla.org/File:Tracking_protection_on.png Fox] | ||
| Line 723: | Line 698: | ||
* This test is similar to [[#blacklist_deletions|blacklist deletions]], but now we verify the whitelist update (vs. blacklist update) | * This test is similar to [[#blacklist_deletions|blacklist deletions]], but now we verify the whitelist update (vs. blacklist update) | ||
* Normally googlesyndication | * Normally <tt>googlesyndication.com</tt> and <tt>doubleclick.net</tt> are whitelisted so that they don't get blocked from YouTube | ||
* By removing | * By removing them from whitelist, we should be able to verify that they now get blocked! | ||
==== verify enabled ==== | ==== verify enabled ==== | ||
| Line 735: | Line 710: | ||
==== delete from whitelist ==== | ==== delete from whitelist ==== | ||
* Remove doubleclick.net from the "Google" organization in the resources section of the whitelist (trackwhite.json) - https://github.com/mozilla-services/shavar-test-lists | * Remove <tt>doubleclick.net</tt> and <tt>googlesyndication.com</tt> from the "Google" organization in the resources section of the whitelist (trackwhite.json) - https://github.com/mozilla-services/shavar-test-lists | ||
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins) | * This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins) | ||
* Manually trigger an update via the mozilla button on the about:url-classifier page | |||
==== verify disabled ==== | ==== verify disabled ==== | ||
Verify debugging log: | Verify debugging log (from stdout where you launched Firefox from terminal): | ||
'''update URL has registered correctly''' | '''update URL has registered correctly''' | ||
* i.e: listmanager: 09:44:04 GMT-0700 (PDT): registering | * i.e: listmanager: 09:44:04 GMT-0700 (PDT): registering moztestpub-track-digest256 with https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=43.0a1&pver=2.2 | ||
'''Registering whitelist- | '''Registering whitelist- moztestpub-trackwhite-digest256''' | ||
* listmanager: 09:44:04 GMT-0700 (PDT): registering | * listmanager: 09:44:04 GMT-0700 (PDT): registering moztestpub-trackwhite-digest256 with https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=43.0a1&pver=2.2 | ||
'''Checking for updates on shavar server''' | '''Checking for updates on shavar server''' | ||
* listmanager: 09:44:07 GMT-0700 (PDT): checkForUpdates with https:// | * listmanager: 09:44:07 GMT-0700 (PDT): checkForUpdates with https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=43.0a1&pver=2.2 | ||
'''Request to shavar server begins''' | '''Request to shavar server begins''' | ||
* listmanager: 09:44:07 GMT-0700 (PDT): makeUpdateRequestForEntry_: request mozpub-track-digest256;a:1440776223 | * listmanager: 09:44:07 GMT-0700 (PDT): makeUpdateRequestForEntry_: request mozpub-track-digest256;a:1440776223 | ||
'''Update success for both lists''' | '''Update success for both lists''' | ||
* listmanager: 09:44:07 GMT-0700 (PDT): update success for | * listmanager: 09:44:07 GMT-0700 (PDT): update success for moztestpub-track-digest256,moztestpub-trackwhite-digest256 from https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=43.0a1&pver=2.2: 2700 | ||
'''NOTE''' | '''NOTE''' | ||