QA/Execution/Web Testing/Project Checklist
This is meant to be a quite-literal checklist for determing "go/no-go" status for your website. Where it doesn't apply, feel free to use it rather as a discussion point, but be sure all points are raised and noted.
- 1 Other Wiki Resources
- 2 Browser compatibility/coding standards
- 3 Video Support
- 4 General
- 5 RSS Feeds
- 6 Logged-in vs. logged-out functionality
- 7 Social media
- 8 Performance/load testing
- 9 Accessibility
- 11 Security
- 12 Project/release-cycle
- 13 Site Config
- 14 Tools
Other Wiki Resources
- Master list of other WebDev guides: https://wiki.mozilla.org/Websites/Guides
Browser compatibility/coding standards
- Get the list of supported browsers somewhere in writing (srsly)
- All supported browsers look "pretty good" and are 100% functional
- If not, exceptions are noted and called out prior to shipping
- Team has gone through https://wiki.mozilla.org/WebDev:FrontendCodeStandards
- What are the supported video formats?
- Appropriate fallback in place and working?
- No Flash
- Firefox 3.6+
- IE 7/8/9
- Opera (latest)
- Chrome (latest)
- Safari (latest)
- Latest Firefox beta
- Appropriate fallback in place and working?
- If this is an outsourced project, a test plan and the test cases that the consultants used to validate their work should be included in the hand-off process.
- In URL paths, / and "" (without the trailing slash) both resolve to the same URI/resource
- HTML validates: http://html5.validator.nu/ or http://validator.w3.org/
- Exceptions can be made for social-network/sharing sites, that may have malformed tags or markup
- Is the robots.txt file configured correctly to prevent bots from spidering to nonsensical pages? (example)
- Validate: http://feedvalidator.org/
- Links all work
- l10n: other locales look fine (special chars, Unicode, etc.)
- Correct timestamps
Logged-in vs. logged-out functionality
- Ask/test what the difference between the logged-in vs. logged-out functionality is
- It's probably quite a bit more than you expected
- Private or public? If private: sandboxed, or test accounts?
- Test with "Secure browsing (https)" (on https://www.facebook.com/editaccount.php?ref=mb&drop) enabled and disabled
- Enable "HTTPS Only | Always use HTTPS" (https://twitter.com/settings/account)
- Length of pre-populated tweets in en-US and other locales (l10n)
- Unescaped vs. escaped characters (string literals)
- Unicode chars
- ShareThis, etc.
- QR Codes
- Try different readers (AT&T Scanner, QuickMark, QR Droid, etc.)
- Doesn't tank on YSlow! (Strive for an A with ruleset v2)
- CDN is hooked up, if needed (static images, CSS, JS, videos) -- see "Site Config" section, too, below
- Load-tested, if needed? Make sure to bring this up
- Is the site generally accessible?
- What's the user messaging?
- Complete the following (taken from WebAppSec/Secure_Coding_QA_Checklist)
- Runs on both HTTP / HTTPS? Mixed-content warnings? Cert set up?
- Should HTTP requests get automatically redirected to HTTPS, by default?
- Check to see that third-party assets use relative paths, where possible
- e.g. Webtrends tags, social media, etc.
- Has a Webdev bug, filed from https://intranet.mozilla.org/webtools/authenticate/login
- Has a test plan (which can augment this, but lists specific bugs, milestones, project owners, etc.)
- Reviewed by team
- Has clear bug-filing/tracking, either in a new Bugzilla component, existing (with whiteboard, etc.), or in Basecamp
- If in Bugzilla, has clear milestones
- Listed on https://wiki.mozilla.org/QA/Execution/Web_Testing#Current_Projects
- Regular status meetings
- IRC channel?
- If needed, a tracking bug with dependencies set for other groups' work (IT, Webdev, Marketing, etc.)
- Make sure to schedule a brief meeting where all parties (Marketing/Webdev/QA) are present and discuss the open issues, and any last-minute (but necessary) changes
- Ask for PRDs, user-flow diagrams, testing notes
- Are the third-party developers cc:d on all relevant bugs? If possible (i.e. not "mozilla-confidential" flagged), have them follow the Bugzilla component (and make sure they have accounts).
- Has a push bug, and a release checklist, even if you don't think they need it (they do)
Optimal Bugzilla Workflow
Each project can be different, but please use this flow if starting a new project:
- A bug is filed in the correct product/component
- Bug is triaged for severity, priority and assigned to a target milestone
- Bug is fixed in work based on next target milestone release
- Dev puts bug# in git comment
- Dev puts commit url in bugzilla comment
- Dev marks bug RESOLVED:FIXED
- Tester verifies bug on stage, bug marked RESOLVED:VERIFIED
- Code is deployed to production
- Tester verifies bug
- Tester verifies deployment/push/server-ops bug
Devs may fix bugs before triage, they should update Target Milestone to the next release.
Whiteboard may be used for various reasons:
- [good first bug] - Communicates a contributor opportunity
- Make sure you get Django-traceback emails, on both prod and trunk/staging
- Any vanity domains? Registered/set up, and working?
- Has a favicon? That works in IE, too?
- reCaptcha APIs on staging?
- Google Maps API key on staging?
- Metrics code is correct/working for the site (typically Webtrends)
- Ask Blake Cutler or Laura Forrest (preferably Laura)
- Any rewrite/redirect rules are in place and working?
- Is the site planned to be hooked up to a CDN? Is it set up for testing on staging, and without any cert-signing problems?
- You're using as many of the tools as appropriate
- XSS Me