QA/Execution/Web Testing/Socorro/Test Plan
< QA | Execution | Web Testing
Jump to navigation
Jump to search
Socorro test plan
Socorro roadmap:
Staging server:
Supported browser(s):
- Firefox 3.6 (Tier 1)
- IE 8 (major functionality/layout only)
- Chrome (major functionality/layout only)
- Safari (major functionality/layout only)
Manual Tests:
- Ensure that the various top-level application views work (Firefox, Thunderbird, etc.)
- As well as their listed versions, beneath
- Check that the versions are accurate/up-to-date/complete (ask!)
- Check they're sorted correctly
- Load each "Report" type:
- Overview
- Crashes per User
- Nightly Builds
- Top Changers
- Top Crashers
- Top Crashers by URL
- Top Crashers by Domain
- Top Crashers by Topsite
- Browse around and ensure there are no JS errors trapped in the Error Console
- Ensure that searching for crashes by ID -and- signature works
- Ensure that the "More Reports" dropdown loads data for each option
- Advanced Search (contains: "flash"):
- http://crash-stats.mozilla.com/query/query?product=Firefox&version=ALL%3AALL&range_value=1&range_unit=weeks&date=09%2F21%2F2010+22%3A10%3A40&query_search=signature&query_type=contains&query=flash&build_id=&process_type=any&hang_type=any&do_query=1
- Since we can't test every combination, ensure that both the "All" filters work, in tandem with random testing of other platforms/versions
- What should be tested in the Advanced Filters section?
- Test JavaScript in form
- Product, Version should be populated based on clicks
- Switch to "Plugins Only," click search - using results (click through to details on a couple)
- Test JavaScript in form
- Test "Search by Branch" returns only crashes for Gecko on that branch
- Do a general search; using results, test each in isolation:
- Test Stack signature startswith, contains, isexactly
- Build Id returns only crashes for that build
- Play with before and within the last fields to change search results
- The "Crashing Thread" section is intact, and the link to the source works
- The "Show/hide other threads" link works
- Do a general search; using results, test each in isolation:
- Topchangers
- Verify switching # of days works
- Download CSV file and ensure it has content
- Ensure RSS feed has data (validate the feed, too)
- Verify switching # of days works
- Load a crash report, and:
- "Details" tab: Ensure that the "Related Bugs" block shows up, and the links work
- "Modules" tab: raw list of DLLs, their versions, debug identifiers, and filenames
- "Raw dump" tab: just what it says, ensure there is data there
- "Extensions" tab:
- If there is an extension listed, click on it; it should go to its AMO details page
- It lists the extension's GUID (email address or hex), version, and "current?" (if available) version
- "Comments" tab:
- Comments show up (crashes don't always have comments; just ensure some do)
- Their timestamps are links to the original report, which should have the same top frame
- "Correlations" tab: click on the "Load" buttons, and ensure that data loads
- Try the "Next" and "Previous" buttons
- Make sure to test out http://crash-stats.stage.mozilla.com/status
- Refresh, ensure that the server time updates
- What else can be tested here, on staging? (Socorro devs, please fill in)
Automated Tests:
- What can be automated from above?
- Values of all <select> / <option> widgets
- For each resulting page, verify the most important elements, and that there is *some* data
- Simple search
- Possible to automate by ID?
- How do we search for stacktraces?
- Values of all <select> / <option> widgets
- Next steps:
- Figure out tests
- Write them
- Get review
- Land in http://viewvc.svn.mozilla.org/vc/projects/socorro_qa/
- Get cron for auto-dumping prod -> staging set up
- Configure Hudson to pull and run