MozillaWiki

Services/TrackingProtection/Shavar Server - Testing: Difference between revisions

Page Discussion

Services/TrackingProtection/Shavar Server - Testing (view source)

Revision as of 17:42, 11 January 2019

509 bytes added ,  11 January 2019
→‎SET CUSTOM PREFS
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<span style="color:#FF0000">This page is deprecated!!!!!! Please see:  [https://testrail.stage.mozaws.net/index.php?/suites/view/354&group_by=cases:section_id&group_order=asc TestRail testcase repository]</span>
<span style="color:#FF0000">(for TestRail permissions, file an issue here:  https://github.com/mozilla-services/shavar-e2e-tests )</span>
[[Category:Test Plans]]
[[Category:Test Plans]]
= Testplan (Overview) =  
= Testplan (Overview) =  
Line 162: Line 169:


=== PRE-PRODUCTION ===
=== PRE-PRODUCTION ===
Stack checks are run on pre-prod to verify that a new app stack returns the same results as the old stack. It verifies the server response. This environment is only up when the tests need to be run, which is only when there is a new stack.


  $ curl -k -d" "  'https://shavarprodtest.prod.mozaws.net/list?client=foo&appver=1&pver=2.2'
  $ curl -k -d" "  'https://shavarprodtest.prod.mozaws.net/list?client=foo&appver=1&pver=2.2'
Line 167: Line 175:


Should return all of these lists:
Should return all of these lists:
[* update with next verification]


  base-track-digest256
  base-track-digest256
Line 186: Line 195:


=== PRODUCTION ===
=== PRODUCTION ===
<code>
   # shavar URL
   # shavar URL
   $ curl -d" "  'https://shavar.services.mozilla.com/list?client=foo&appver=1&pver=2.2'
   $ curl -d" "  'https://shavar.services.mozilla.com/list?client=foo&appver=1&pver=2.2'
   # tracking URL (CNAME'd to shavar URL - should produce same output)
   # tracking URL (CNAME'd to shavar URL - should produce same output)
   $ curl -d" "  'https://tracking.services.mozilla.com/list?client=foo&appver=1&pver=2.2'
   $ curl -d" "  'https://tracking.services.mozilla.com/list?client=foo&appver=1&pver=2.2'
</code>


Should return all of these lists:
Should return all of these lists:
<code>
ads-track-digest256
allow-flashallow-digest256
analytics-track-digest256
  base-track-digest256
  base-track-digest256
  baseeff-track-digest256
  baseeff-track-digest256
  basew3c-track-digest256
  basew3c-track-digest256
block-flash-digest256
block-flashsubdoc-digest256
  content-track-digest256
  content-track-digest256
  contenteff-track-digest256
  contenteff-track-digest256
  contentw3c-track-digest256
  contentw3c-track-digest256
except-flash-digest256
except-flashallow-digest256
except-flashinfobar-digest256
except-flashsubdoc-digest256
  mozfull-track-digest256
  mozfull-track-digest256
  mozfullstaging-track-digest256
  mozfullstaging-track-digest256
  mozplugin-block-digest256
  mozplugin-block-digest256
  mozplugin2-block-digest256
  mozplugin2-block-digest256
mozpub-track-digest256
  mozstd-track-digest256
  mozstd-track-digest256
  mozstd-trackwhite-digest256
  mozstd-trackwhite-digest256
  mozstdstaging-track-digest256
  mozstdstaging-track-digest256
  mozstdstaging-trackwhite-digest256
  mozstdstaging-trackwhite-digest256
</code>
social-track-digest256
 


Curl for the mozstd list:
Curl for the mozstd list:
<code>
   # shavar URL  
   # shavar URL  
   $ curl -d"mozstd-track-digest256;" 'https://shavar.services.mozilla.com/downloads?client=foo&appver=1&pver=2.2'
   $ curl -d"mozstd-track-digest256;" 'https://shavar.services.mozilla.com/downloads?client=foo&appver=1&pver=2.2'
   # tracking URL (CNAME'd to shavar URL - should produce same output)
   # tracking URL (CNAME'd to shavar URL - should produce same output)
   $ curl -d"mozstd-track-digest256;" 'https://tracking.services.mozilla.com/downloads?client=foo&appver=1&pver=2.2'
   $ curl -d"mozstd-track-digest256;" 'https://tracking.services.mozilla.com/downloads?client=foo&appver=1&pver=2.2'
</code>


Should return this:
Should return this:
<code>
   n:3600
   n:3600
   i:mozstd-track-digest256
   i:mozstd-track-digest256
   u:tracking-protection.services.mozilla.com/mozstd-track-digest256/1442275596
   u:tracking-protection.services.mozilla.com/mozstd-track-digest256/1442275596
</code>
Curl for the mozfull lists:
<code>
  # shavar URL
  $ curl -d"mozfull-track-digest256;" 'https://shavar.services.mozilla.com/downloads?client=foo&appver=1&pver=2.2'
  # tracking URL (CNAME'd to shavar URL - should produce same output)
  $ curl -d"mozfull-track-digest256;" 'https://tracking.services.mozilla.com/downloads?client=foo&appver=1&pver=2.2'
</code>
Should return this:
<code>
  n:3600
  i:mozfull-track-digest256
  u:tracking-protection.services.mozilla.com/mozfull-track-digest256/1442275596
</code>


[[#top]]
[[#top]]
Line 380: Line 372:
* To test production, replace staging URL (stage.mozaws.net) with production URL (shavar.services.mozilla.com)
* To test production, replace staging URL (stage.mozaws.net) with production URL (shavar.services.mozilla.com)
* all list files are tested in both stage and production (except moztestpub - stage only)
* all list files are tested in both stage and production (except moztestpub - stage only)
*  line 9 of "Prefs for: mozpub list file" removes the whitelist ("-trackwhite-") that's in there by default. Once removed it will not be downloaded anymore.


'''Prefs for: base list file'''


'''Prefs for: moztestpub list file'''
<source lang="bash">
<source>
 
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html");
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html");
user_pref("browser.safebrowsing.debug", true);
user_pref("browser.safebrowsing.debug", true);
Line 396: Line 386:
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("urlclassifier.trackingTable", "test-track-simple,moztestpub-track-digest256");          
user_pref("urlclassifier.trackingTable", "test-track-simple,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,moztestpub-trackwhite-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
</source>
</source>


 
'''Create a Strict list'''
 
'''Prefs for: mozstd list file'''


<source lang="bash">
<source lang="bash">
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html");
1. Create a new Nightly profile.
user_pref("browser.safebrowsing.debug", true);
2. Go into about:preferences#privacy and set Tracking Protection to Always.
user_pref("browser.safebrowsing.phishing.enabled", false);
3. Click on Change Block List and select strict protection.
user_pref("browser.safebrowsing.malware.enabled", false);
4. Make these edits in about:config
user_pref("privacy.trackingprotection.enabled", true);
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1);
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1);
user_pref("browser.safebrowsing.provider.mozilla.lists", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstd-track-digest256");         
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
</source>
</source>


'''Prefs for: mozfull list file'''
'''Create a Cryptomining Fingerprinting list'''


<source lang="bash">
<source lang="bash">
1. Create a new profile
2. Make these edits in about:config
browser.safebrowsing.provider.mozilla.gethashURL;https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
browser.safebrowsing.provider.mozilla.updateURL;https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%MAJOR_VERSION%&pver=2.2
browser.safebrowsing.provider.mozilla.lists;base-track-digest256,mozstd-trackwhite-digest256,content-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,block-flash-digest256,except-flash-digest256,allow-flashallow-digest256,except-flashallow-digest256,block-flashsubdoc-digest256,except-flashsubdoc-digest256,except-flashinfobar-digest256,ads-track-digest256,social-track-digest256,analytics-track-digest256,fastblock1-track-digest256,fastblock1-trackwhite-digest256,fastblock2-track-digest256,fastblock2-trackwhite-digest256,fastblock3-track-digest256,base-cryptomining-track-digest256,base-fingerprinting-track-digest256
urlclassifier.trackingTable;test-track-simple,base-track-digest256,base-cryptomining-track-digest256,base-fingerprinting-track-digest256
browser.safebrowsing.provider.mozilla.nextupdatetime;0
3. Restart browser
4. Open a Private Browsing window
</source>
'''Prefs for: moztestpub list file'''
<source>
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html");
user_pref("browser.startup.homepage", "http://itisatrap.org/firefox/its-a-tracker.html");
user_pref("browser.safebrowsing.debug", true);
user_pref("browser.safebrowsing.debug", true);
Line 432: Line 430:
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("urlclassifier.trackingTable", "test-track-simple,mozfull-track-digest256");          
user_pref("urlclassifier.trackingTable", "test-track-simple,moztestpub-track-digest256");          
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,moztestpub-trackwhite-digest256");
</source>
</source>


'''Prefs for: mozstaging list file'''
 
 
'''Prefs for: mozstd list file'''


<source lang="bash">
<source lang="bash">
Line 449: Line 449:
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstdstaging-track-digest256,mozfullstaging-track-digest256");                  
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstd-track-digest256");          
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstdstaging-trackwhite-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
</source>
</source>


 
'''Prefs for: mozfull list file'''
'''Prefs for: base list file'''


<source lang="bash">
<source lang="bash">
Line 467: Line 466:
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("urlclassifier.trackingTable", "test-track-simple,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("urlclassifier.trackingTable", "test-track-simple,mozfull-track-digest256");          
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstd-trackwhite-digest256");
</source>
</source>


 
'''Prefs for: mozstaging list file'''
'''Prefs for: moztestpub list file (OLD)'''


<source lang="bash">
<source lang="bash">
Line 479: Line 477:
user_pref("browser.safebrowsing.phishing.enabled", false);
user_pref("browser.safebrowsing.phishing.enabled", false);
user_pref("browser.safebrowsing.malware.enabled", false);
user_pref("browser.safebrowsing.malware.enabled", false);
user_pref("privacy.trackingprotection.enabled", true);
user_pref("browser.safebrowsing.provider.mozilla.nextupdatetime", 1);
user_pref("browser.safebrowsing.provider.mozilla.lists", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,test-block-simple,goog-downloadwhite-digest256,mozstd-track-digest256,mozstd-trackwhite-digest256,mozfull-track-digest256,mozplugin-block-digest256,mozplugin2-block-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256,mozstdstaging-track-digest256,mozfullstaging-track-digest256,mozstdstaging-trackwhite-digest256,base-track-digest256,baseeff-track-digest256,basew3c-track-digest256,content-track-digest256,contenteff-track-digest256,contentw3c-track-digest256");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.gethashURL", "https://shavar.stage.mozaws.net/gethash?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("browser.safebrowsing.provider.mozilla.updateURL", "https://shavar.stage.mozaws.net/downloads?client=SAFEBROWSING_ID&appver=%VERSION%&pver=2.2");
user_pref("privacy.trackingprotection.enabled", true);
user_pref("urlclassifier.trackingTable", "test-track-simple,mozstdstaging-track-digest256,mozfullstaging-track-digest256");                  
user_pref("urlclassifier.trackingTable", "test-track-simple,moztestpub-track-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,mozstdstaging-trackwhite-digest256");
user_pref("urlclassifier.trackingWhitelistTable", "test-trackwhite-simple,moztestpub-trackwhite-digest256");
user_pref("urlclassifier.disallow_completions", "test-malware-simple,test-phish-simple,test-unwanted-simple,test-track-simple,test-trackwhite-simple,goog-downloadwhite-digest256,moztestpub-track-digest256,moztestpub-trackwhite-digest256");
user_pref("browser.safebrowsing.provider.mozilla.lists", "moztestpub-track-digest256,moztestpub-trackwhite-digest256");
</source>
</source>
* !!!!!  restart Firefox !!!!!!
Replace production URL with stage URL for the following prefs:
* browser.trackingprotection.updateURL
* browser.trackingprotection.gethashURL
Set pref for debugging:
* browser.safebrowsing.debug = true
** this will give you some will give you some updates on the console
Remove Google malware & phishing protection:
* browser.safebrowsing.phishing.enabled = false
* browser.safebrowsing.malware.enabled = false
* privacy.trackingprotection.enabled = true (no reboot necessary)
'''RESET BROWSER'''
* !!!!!  restart Firefox !!!!!!


[[#top]]
[[#top]]
Line 522: Line 502:
* moztestpub [STAGE ONLY]
* moztestpub [STAGE ONLY]
* mozstd
* mozstd
** mozstd
** mozstdstaging
* mozfull
* mozfull
* mozstaging
* mozstaging
* mozplugin
* base
* base
** base
* strict
** content
* fastblock




Line 603: Line 580:
</code>
</code>


==== verify its-a-tracker page ====  
==== verify list pages ====  


repeat test 1 x for each of these list groups:  
repeat test 1 x for each of these list groups:  
* STAGE: mozstd, mozfull, mozstaging, base
* STAGE: mozstd, mozfull, moztestpub, base
* PROD: mozstd, mozfull, mozstaging, base  
* PROD: mozstd, mozfull, moztestpub, base  
 
Note: skip this test for moztestpub-track-digest256.


STEPS:
STEPS:
* Pull up this page: http://itisatrap.org/firefox/its-a-tracker.html  
* Set up list
* Ensure that it is all green
* Verify with matching test page:
** blacklist and whitelist are working
https://mozilla.github.io/tracking-test/ads.html
* This page doesn't hit shavar server at all
https://mozilla.github.io/tracking-test/analytics.html
* If not working, then either about:config issue or client issue - repeat above!
https://mozilla.github.io/tracking-test/social.html
https://mozilla.github.io/tracking-test/fastblock.html
https://mozilla.github.io/tracking-test/disconnect.html
https://mozilla.github.io/tracking-test/full.html  - for the Strict list


==== verify CNN page ====  
==== verify CNN page ====  
Line 668: Line 646:
* Remove the extremetracking.com json blob from the blacklist (track.json) on https://github.com/mozilla-services/shavar-test-lists
* Remove the extremetracking.com json blob from the blacklist (track.json) on https://github.com/mozilla-services/shavar-test-lists
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins)
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins)
* Manually trigger an update via the mozilla button on the about:url-classifier page


==== verify disabled ====  
==== verify disabled ====  
Line 733: Line 712:
* Remove <tt>doubleclick.net</tt> and <tt>googlesyndication.com</tt> from the "Google" organization in the resources section of the whitelist (trackwhite.json) - https://github.com/mozilla-services/shavar-test-lists  
* Remove <tt>doubleclick.net</tt> and <tt>googlesyndication.com</tt> from the "Google" organization in the resources section of the whitelist (trackwhite.json) - https://github.com/mozilla-services/shavar-test-lists  
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins)
* This will trigger the stage list creation script (Cloud Services QA or Ops team can monitor this in Jenkins)
* Manually trigger an update via the mozilla button on the about:url-classifier page


==== verify disabled ====  
==== verify disabled ====  
Rbillings
Confirmed users
1,504

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1188986...1206080"