QA/Fennec/iOS/1.1
Contents
Fennec (Firefox for iOS) 1.1 Test Plan
- Release Week: November 2nd, 2015
- QA Contact: Aaron Train; Catalin Suciu;
- IRC: irc://irc.mozilla.org#qa ping atrain; csuciu;
Overview
This is a tracking document that outlines the test strategy that Mozilla QA will follow regarding a thorough test pass of Fennec for iOS releases. It will follow the general plan on steps and process that QA will take to accomplish a "sign off" to the release strategy. Mozilla QA strives to uphold quality software, and a successful pass corresponds to no blocking issues, 'Green'-colored test pass on test cases in both execution and automation, and a thorough bug verification window of all Blocking bugs and Critical bugs.
Areas Covered
Areas not Covered
- Feature Unit Tests
- These are normally covered by feature developers, not QA. We are open to discussion about what we could do in QA to increase Unit Tests other than what automation currently exists. Need more definition here on what QA/Dev should own.
- String Localization Tests
- See more in the L10n Test Deliverable section.
Categories
Feature Tests
- Summary
Any major feature item will have a test plan and a tester who owns that area.
- Planning & Scheduling
The Fennec dev team has a great online AHA! board] to track features by milestone. We will work off this list as a starting point for what we focus on for each release. We will create a basic test plan outlining our major focus areas, how we will test and the test cases. These will live here.
- Test Framework
Automation:
- Results
| Feature | QA Owner | Tracking Bug | Documentation/Test Plan | Feature Health |
|---|---|---|---|---|
| Private Browsing | Cata Suciu | bug 1196227 , bug 1173382 | https://wiki.mozilla.org/QA/Fennec/iOS/pbmode | [ON TRACK] |
| Support Firefox Account 'sign-up' from iOS device | Cata Suciu | bug 1196241 | https://wiki.mozilla.org/QA/Fennec/iOS/fxa_signup | [ON TRACK] |
| Desktop Bookmarks appear in a read-only tree view in my bookmark list | Cata Suciu | bug 1196238 | https://wiki.mozilla.org/QA/Fennec/iOS/sync_bookmarks | [ON TRACK] |
| First-run onboarding tour re-design | Cata Suciu | bug 1203563 | https://wiki.mozilla.org/QA/Fennec/iOS/first_run_tour | [ON TRACK] |
Web Compatibility Tests
- Summary
The Web Compatibility Tests focuses on regression and compatibility with popular websites. For Fennec on iOS we are using WKWebView (WebKit) so comparative testing against Safari will be of value. Any problems discovered in the browser should be reproducible in Safari.
- Test Framework
- Top Sites - coverage of a few popular sites that drives most user traffic
- Results
Test results here
Bug Verifications
- Summary
Verifying fixed bugs in Buzilla gives us additional testing on specific bugs that were fixed.
- Planning & Scheduling
To prioritize and narrow the list of bugs, QA will focus primarily on Fennec bugs that represent highest severity.
- Results
Verified bugs here
L10N Tests
- Summary
Fennec will ship with a number of translations. Currently there are [about 50] languages under development with more coming on board in the future. As of this edit, there are 44 complete.
The l10n community provides the translations and the mobile team does the appropriate packaging and bundling.
- Planning & Scheduling
While traditionally we do not perform translation accuracy tests we should spot check a few languages to ensure there are no major UI failures.
- Test Framework
Execution
- Results
Test Results for executed test cases should be tracked on regular cadence on the wiki
Regression Tests
Collecting Feedback
Our main goal here is to make sure we document and test the feedback channels.
- Reporting tools
- Crash Reporter
- Bugzilla
- GitHub
- IRC: #mobile
- Other
- Apple Store reviews
- Input
