QA/Execution/Web Testing/mcomtests

Mozilla.com Automated Test Tracking

• Raymond's Github Repository
• Current test repository SVN
• This needs to be a job in Hudson, ASAP, to augment/replace the current failing tests
  • Run against http://www-fx4.stage.mozilla.com/en-US/firefox/
• Tests
  • Redirect tests , these should cover the redirects in htaccess
    • Both locale detection/redirection+user-agent tests/redirects
  • Video page tests
  • Newsletter page tests
  • Download button tests
    • Note that we don't have to test the mirror links themselves, just that http://www.mozilla.com/products/download.html?product=firefox-3.6.13&os=win&lang=en-US (e.g.) returns a 200 OK, and redirects the page to http://www.mozilla.com/en-US/products/download.html?product=firefox-3.6.13&os=win&lang=en-US, and, if possible, puts up the mirror link (can we test that some event is fired off?)
  • This would be done using PyQuery and urllib (this is needed to switch user agents and verify that the correct button and message appears on each download page)
  • All.html: http://www.mozilla.com/en-US/firefox/all.html
  • All-older.html: http://www.mozilla.com/en-US/firefox/all-older.html
  • Mobile page tests
    • Auto-detection of device?
    • Auto-redirect
  • Customization page tests
  • Features page tests
  • Homepage tests
  • Breadcrumb tests

Dev requirements ideas

• Easy to run locally (when refactoring)
• Every commit in there should be reviewed (to be sure we are not breaking other tests, only adding good new tests)
• Public reporting system (so that we quickly know if any branch is misbehaving)
  • Hudson takes care of it
    • Is there a way to put it on a public URL? I never use the VPN and it would be cool that anyone can access it.

Pascal's QA needs

• images should not be deleted if they are still used by localized pages (happened in the past)
• any CSS change to theme/content.css theme/template.css should mean a test to 3 locales (one rtl, one in cyrillic, one Indian), ideally test that on the front page
• Any new webfont on the site should be automatically tested for compatibility with locales for obvious bugs in glyphs support
• the biggest pain point for me are redirects set up in htaccess which often break locales or fallbacks for locales (our htaccess needs a major cleanup but that's a different issue)
• I would need automated screenshot generation of pages on demand for non-gecko browsers, specifically IE 6/7/8/9. (I have local tools for gecko and webkit, IE is the one difficult to test)