QA/Search refactoring
Overview
Summary
The purpose of this wiki is to serve as a general test plan for verifying Desktop Search engine works as expected after the refactoring.
The testing will be focus on:
- verify that search works correct
- verify that customized user data/profile is not affected after update
Ownership
Developer contacts: Florian Queze
QA:
Brindusa Tot - leading QA efforts
Simona Badau, QA
Ovidiu Boca, QA
Status
Overview
[LANDED] Nightly 45: first landed on - 10.29.2015
Risk analysis
- No Risks found at the moment
References
Meta: Bug 1203167
Requirements for testing
Environments
Testing will be performed on following OSes:
- Mac OS X 10
- Windows 7
- Windows 8
- Windows 10
- Windows XP
- Windows Vista
- Ubuntu 14.04
Test Types
| Type of Testing | Required (Y/N) | Owners | Details/Comments |
|---|---|---|---|
| Manual | Y | Moz QA eng team | |
| Automation | Y | ||
| Performance | N | ||
| Security | N | ||
| Metrics/Telemetry | TBD |
Test cases
Overview
- Perform search from the Search Bar – using different terms
- Perform search from all the available search fields
- Add/Update search engine and validate search results is given by the new engine
- Change search settings (select/deselect engine, remove/restore engine, reorder engines) and validate search results is given by the new engine
- Verify if custom search engine settings (new engines added, reordered engines) are preserved after update
- Verify that after update, search engines settings can be updates
- Install external software's that automatically change the search engine for installed browsers and verify that the default search engine or users custom search engine is not altered
Test suites
Notes:
Manual/Automation: Test will be conducted manually
Browser Window: Test will be conducted in standard browser window and Private browser windows
| Title | Description |
|---|---|
| Perform a search from the Search Bar without using a search term | Launch Firefox.
Click the Search Bar from the toolbar. Don't write anything and press [enter]. |
| Perform a search from the Search Bar using a specific search term | Enter a search term in the Search Bar.
Click the go icon (->) |
| Change the search provider using the Search Bar | Select "Change search settings" when the search drop-down panel is opened.
Set a different provider as default search engine. Perform a search for a random keyword. |
| Perform searches from all the available search fields
|
Perform a search from the Location Bar.
Perform a search from the Search Bar. Perform a search from the about:newtab page. Perform a search from the about:home page. |
| Search toolbar | Launch Firefox.
Click the Search bar text box. Don't write anything and press Enter Click the down arrow next to the magnifying glass icon Select "Change Search Settings" and choose another search engine as default from the Default Search Engine drop-down. Perform searches from search toolbar, URL toolbar, about:newtab and about:home Restart Firefox and check the default search engine |
| Change Search Settings - Add new search engine Youtube | Launch Firefox and open a new tab (Ctrl/Cmd+T).
Open the drop down from the Search box field by clicking on the magnifying glass icon and select "Change Search Settings" Select "Add more search engines..." link Install an add-on from the provided list (for example Youtube) and then check it to be the default search engine, when asked Search using the search field from the new tab page, search bar, URL bar, and about:home page. Restart Firefox and check the search engines installed |
| Change Search Settings - Add new search engine - other | Launch Firefox and open a new tab (Ctrl/Cmd+T).
Navigate to pages that could contain different search engines who can be added to Firefox (such as https://play.google.com/store; https://addons.mozilla.org/en-US/firefox/search/?atype=4). Select the magnifying glass from the search toolbar and add them to the search engines list Add several search engines Set one of the added engine as default. Search using the search field from the new tab page, search bar, URL bar, and about:home page. Restart Firefox and check the search engines installed |
| Change Search Settings - Update default search engine | Launch Firefox and open a new tab (Ctrl/Cmd+T).
Click on the magnifying glass from the Search toolbox and choose "Change Search Settings". Update the default engine, for e.g. set eBay the default engine. Search using the search field from the new tab page, search bar, URL bar, and about:home page. Restart Firefox and check the search engines installed Search |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
| Example | Example |
Full Test Suites: [1]
Bugs Work
Tracking bug: Meta bug 1203167
Bug fix verification
| Bug No | Summary | Status | Firefox Verion |
|---|---|---|---|
| 1203161 | remove the browser.search.cache.enabled pref | Resolved - Fixed | |
Logged bugs
Bug 1224150 - Downgrading to FF 44 and upgrading back to FF 45 breaks the search engine
Sign off
Criteria
- All test cases should be executed
- Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
- All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
Results
Nightly testing
- [In progress] Full Test suite results [2]
- There were no blockers found while testing this feature.
- Manual testing covered Windows 7 x64, Windows 8 x64, Windows 10 x64, Windows XP x32, Windows Vista x32, Ubuntu 14.04 x32 and Mac OS X 10.
Merge to Aurora Sign-off