MozillaWiki

Services/TrackingProtection/Shavar Server - Testing: Difference between revisions

Page Discussion

Services/TrackingProtection/Shavar Server - Testing (view source)

Revision as of 17:42, 11 January 2019

524 bytes added ,  11 January 2019
→‎SET CUSTOM PREFS
 
(8 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<span style="color:#FF0000">This page is deprecated!!!!!! Please see:  [https://testrail.stage.mozaws.net/index.php?/suites/view/354&group_by=cases:section_id&group_order=asc TestRail testcase repository]</span>
<span style="color:#FF0000">(for TestRail permissions, file an issue here:  https://github.com/mozilla-services/shavar-e2e-tests )</span>
[[Category:Test Plans]]
[[Category:Test Plans]]
= Testplan (Overview) =  
= Testplan (Overview) =  
Line 365: Line 372:
* To test production, replace staging URL (stage.mozaws.net) with production URL (shavar.services.mozilla.com)
* To test production, replace staging URL (stage.mozaws.net) with production URL (shavar.services.mozilla.com)
* all list files are tested in both stage and production (except moztestpub - stage only)
* all list files are tested in both stage and production (except moztestpub - stage only)
*  line 9 of "Prefs for: mozpub list file" removes the whitelist ("-trackwhite-") that's in there by default. Once removed it will not be downloaded anymore.


'''Prefs for: base list file'''


'''Prefs for: moztestpub list file'''
<source lang="bash">
<source>
 
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html");
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html");
user_pref("browser.safebrowsing.debug", true);
user_pref("browser.safebrowsing.debug", true);
Line 381: Line 386:
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("urlclassifier.trackingTable", "test-track-simple,moztestpub-track-digest256");          
user_pref("urlclassifier.trackingTable", "test-track-simple,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,moztestpub-trackwhite-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
</source>
</source>


 
'''Create a Strict list'''
 
'''Prefs for: mozstd list file'''


<source lang="bash">
<source lang="bash">
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html");
1. Create a new Nightly profile.
user_pref("browser.safebrowsing.debug", true);
2. Go into about:preferences#privacy and set Tracking Protection to Always.
user_pref("browser.safebrowsing.phishing.enabled", false);
3. Click on Change Block List and select strict protection.
user_pref("browser.safebrowsing.malware.enabled", false);
4. Make these edits in about:config
user_pref("privacy.trackingprotection.enabled", true);
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1);
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1);
user_pref("browser.safebrowsing.provider.mozilla.lists", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstd-track-digest256");         
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
</source>
</source>


'''Prefs for: mozfull list file'''
'''Create a Cryptomining Fingerprinting list'''


<source lang="bash">
<source lang="bash">
1. Create a new profile
2. Make these edits in about:config
browser.safebrowsing.provider.mozilla.gethashURL;https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
browser.safebrowsing.provider.mozilla.updateURL;https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
browser.safebrowsing.provider.mozilla.lists;base-track-digest256,mozstd-trackwhite-digest256,content-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,block-flash-digest256,except-flash-digest256,allow-flashallow-digest256,except-flashallow-digest256,block-flashsubdoc-digest256,except-flashsubdoc-digest256,except-flashinfobar-digest256,ads-track-digest256,social-track-digest256,analytics-track-digest256,fastblock1-track-digest256,fastblock1-trackwhite-digest256,fastblock2-track-digest256,fastblock2-trackwhite-digest256,fastblock3-track-digest256,base-cryptomining-track-digest256,base-fingerprinting-track-digest256
urlclassifier.trackingTable;test-track-simple,base-track-digest256,base-cryptomining-track-digest256,base-fingerprinting-track-digest256
browser.safebrowsing.provider.mozilla.nextupdatetime;0
3. Restart browser
4. Open a Private Browsing window
</source>
'''Prefs for: moztestpub list file'''
<source>
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html");
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html");
user_pref("browser.safebrowsing.debug", true);
user_pref("browser.safebrowsing.debug", true);
Line 417: Line 430:
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("urlclassifier.trackingTable", "test-track-simple,mozfull-track-digest256");          
user_pref("urlclassifier.trackingTable", "test-track-simple,moztestpub-track-digest256");          
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,moztestpub-trackwhite-digest256");
</source>
</source>


'''Prefs for: mozstaging list file'''
 
 
'''Prefs for: mozstd list file'''


<source lang="bash">
<source lang="bash">
Line 434: Line 449:
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstdstaging-track-digest256,mozfullstaging-track-digest256");                  
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstd-track-digest256");          
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstdstaging-trackwhite-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
</source>
</source>


 
'''Prefs for: mozfull list file'''
'''Prefs for: base list file'''


<source lang="bash">
<source lang="bash">
Line 452: Line 466:
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("urlclassifier.trackingTable", "test-track-simple,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("urlclassifier.trackingTable", "test-track-simple,mozfull-track-digest256");          
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
</source>
</source>


 
'''Prefs for: mozstaging list file'''
'''Prefs for: moztestpub list file (OLD)'''


<source lang="bash">
<source lang="bash">
Line 464: Line 477:
user_pref("browser.safebrowsing.phishing.enabled", false);
user_pref("browser.safebrowsing.phishing.enabled", false);
user_pref("browser.safebrowsing.malware.enabled", false);
user_pref("browser.safebrowsing.malware.enabled", false);
user_pref("privacy.trackingprotection.enabled", true);
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1);
user_pref("browser.safebrowsing.provider.mozilla.lists", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("privacy.trackingprotection.enabled", true);
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstdstaging-track-digest256,mozfullstaging-track-digest256");                  
user_pref("urlclassifier.trackingTable", "test-track-simple,moztestpub-track-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstdstaging-trackwhite-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,moztestpub-trackwhite-digest256");
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,goog-downloadwhite-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256");
user_pref("browser.safebrowsing.provider.mozilla.lists", "moztestpub-track-digest256,moztestpub-trackwhite-digest256");
</source>
</source>
* !!!!!  restart Firefox !!!!!!
Replace production URL with stage URL for the following prefs:
* browser.trackingprotection.updateURL
* browser.trackingprotection.gethashURL
Set pref for debugging:
* browser.safebrowsing.debug = true
** this will give you some will give you some updates on the console
Remove Google malware & phishing protection:
* browser.safebrowsing.phishing.enabled = false
* browser.safebrowsing.malware.enabled = false
* privacy.trackingprotection.enabled = true (no reboot necessary)
'''RESET BROWSER'''
* !!!!!  restart Firefox !!!!!!


[[#top]]
[[#top]]
Line 507: Line 502:
* moztestpub [STAGE ONLY]
* moztestpub [STAGE ONLY]
* mozstd
* mozstd
** mozstd
** mozstdstaging
* mozfull
* mozfull
* mozstaging
* mozstaging
* mozplugin
* base
* base
** base
* strict
** content
* fastblock




Line 588: Line 580:
</code>
</code>


==== verify its-a-tracker page ====  
==== verify list pages ====  


repeat test 1 x for each of these list groups:  
repeat test 1 x for each of these list groups:  
* STAGE: mozstd, mozfull, mozstaging, base
* STAGE: mozstd, mozfull, moztestpub, base
* PROD: mozstd, mozfull, mozstaging, base  
* PROD: mozstd, mozfull, moztestpub, base  
 
Note: skip this test for moztestpub-track-digest256.


STEPS:
STEPS:
* Pull up this page: http://itisatrap.org/firefox/its-a-tracker.html  
* Set up list
* Ensure that it is all green
* Verify with matching test page:
** blacklist and whitelist are working
https://mozilla.github.io/tracking-test/ads.html
* This page doesn't hit shavar server at all
https://mozilla.github.io/tracking-test/analytics.html
* If not working, then either about:config issue or client issue - repeat above!
https://mozilla.github.io/tracking-test/social.html
https://mozilla.github.io/tracking-test/fastblock.html
https://mozilla.github.io/tracking-test/disconnect.html
https://mozilla.github.io/tracking-test/full.html  - for the Strict list


==== verify CNN page ====  
==== verify CNN page ====  
Line 653: Line 646:
* Remove the extremetracking.com json blob from the blacklist (track.json) on https://github.com/mozilla-services/shavar-test-lists
* Remove the extremetracking.com json blob from the blacklist (track.json) on https://github.com/mozilla-services/shavar-test-lists
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins)
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins)
* Manually trigger an update via the mozilla button on the about:url-classifier page


==== verify disabled ====  
==== verify disabled ====  
Line 718: Line 712:
* Remove <tt>doubleclick.net</tt> and <tt>googlesyndication.com</tt> from the "Google" organization in the resources section of the whitelist (trackwhite.json) - https://github.com/mozilla-services/shavar-test-lists  
* Remove <tt>doubleclick.net</tt> and <tt>googlesyndication.com</tt> from the "Google" organization in the resources section of the whitelist (trackwhite.json) - https://github.com/mozilla-services/shavar-test-lists  
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins)
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins)
* Manually trigger an update via the mozilla button on the about:url-classifier page


==== verify disabled ====  
==== verify disabled ====  
Rbillings
Confirmed users
1,504

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1189218...1206080"