QA/Search refactoring

Overview

Summary

The purpose of this wiki is to serve as a general test plan for verifying Desktop Search engine works as expected after the refactoring.

The testing will be focus on:

• verify that search works correct
• verify that customized user data/profile is not affected after update

Ownership

Developer contacts: Florian Queze
QA:

Brindusa Tot - leading QA efforts

Simona Badau, QA

Ovidiu Boca, QA

Status

Overview

[LANDED] Nightly 45: first landed on - 10.29.2015

Risk analysis

- No Risks found at the moment

References

Meta: Bug 1203167

Requirements for testing

Environments

Testing will be performed on following OSes:

• Windows 7
• Windows 8
• Windows 10
• Windows XP
• Windows Vista
• Ubuntu 14.04

Test Types

• Manual testing - will be performed by QA Engineering Team
• Automation - ???
• Metrics/Telemetry - ???

Test cases

Overview

• Perform search from the Search Bar – using different terms
• Perform search from all the available search fields
• Add/Update search engine and validate search results is given by the new engine
• Change search settings (select/deselect engine, remove/restore engine, reorder engines) and validate search results is given by the new engine
• Verify if custom search engine settings (new engines added, reordered engines) are preserved after update
• Verify that after update, search engines settings can be updates
• Install external software's that automatically change the search engine for installed browsers and verify that the default search engine or users custom search engine is not altered

Test suites

Full Test Suites: [1]

Bugs Work

Tracking bug: Meta bug Bug 1203167

Bug fix verification

Logged bugs

Bug 1224150 - Downgrading to FF 44 and upgrading back to FF 45 breaks the search engine

Sign off

Criteria

• All test cases should be executed
• Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
• All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)

Results

Full Test suite results [1]
Manual testing covered Windows 7, Windows 10, Ubuntu 14.04 and Mac OS X 10.