67
edits
QA/Partner Search: Difference between revisions
→In Scope
| Line 50: | Line 50: | ||
= Testing summary = | = Testing summary = | ||
== Scope of Testing == | |||
== Scope of Testing == | == Scope of Testing == | ||
=== In Scope === | === In Scope === | ||
The testing effort for the Partner Search improvements | |||
We | Numerous changes are desired in the search area meant to provide better support on locale builds. Testing efforts were done considering the changes included in the following bugs: | ||
We | * [https://bugzilla.mozilla.org/show_bug.cgi?id=1276739 Bug 1276739] replace list.txt with a region-aware JSON file format to allow different search configs within a single locale | ||
The list of top 11 locales | * [https://bugzilla.mozilla.org/show_bug.cgi?id=1309588 Bug 1309588] Update new search JSON to match absearch | ||
* [https://bugzilla.mozilla.org/show_bug.cgi?id=1276740 Bug 1276740] centralize all search plugins shipped in Firefox to mozilla-central | |||
Testing was divided in 2 phases as follows: | |||
* '''The first phase''' included the changes made in Bug 1276739 and Bug 1309588 and was concentrated on the list.txt replacement with the list.json, on the list.json content and on the way search is preserved (same search engines and search codes as before the changes). | |||
After the patch from [https://bugzilla.mozilla.org/show_bug.cgi?id=1276739 Bug 1276739] landed, testing effort for the Partner Search improvements were invested on verifying that search engine list were not affected by this improvement. So we verified that this patch has had no effect on the current search engine list, both the engines that show up for a particular language/locale, as well as their order. | |||
We tested that the top 11 locales have the same engines before the test that they do after the test and that they are in the same order. We also tested that when a user upgrades from a version with list.txt to a version with the list.json that they do not see any changes in their search engines. | |||
We also tested that for the default engine, the URL that is used for a search (including codes) is exactly the same as it was before. | |||
The list of top 11 locales are: | |||
* en-US (en-US will be tested in Canada, the United States and multiple European countries (via a proxy) | * en-US (en-US will be tested in Canada, the United States and multiple European countries (via a proxy) | ||
* de | * de | ||
| Line 65: | Line 75: | ||
* Nl | * Nl | ||
* pt-BR | * pt-BR | ||
After the patch from [https://bugzilla.mozilla.org/show_bug.cgi?id=1309588 Bug 1309588] landed, testing effort were concentrated on the locales with complex json list. We verified that the changes made in the patch are not bundling the "google-nocodes" engine for certain Locales (that is causing Google searches to have firefox-a on the end when they are used for default searches). To be more precised, we verified that Google has no codes when it is used inside the locales territories. | |||
We tested on the following locales with the following proxies set: | |||
BY = Belarus, RU = Russia, UA = Ukraine, KZ = Kazakhstan, TR = Turkey | |||
When Russian Firefox is used within BY, RU, UA, KZ or TR, if you use Foogle, it should NOT have Google codes. (default is Yandex) | |||
When Russian Firefox is used outside of BY, RU, UA, KZ or TR, it should have codes | |||
When Kazakh Firefox is used within BY, RU, UA, KZ or TR, if you use Foogle, it should NOT have Google codes. (default is Yandex) | |||
When Kazakh Firefox is used outside of BY, RU, UA, KZ or TR, it should have codes | |||
When Turkish Firefox is used within BY, RU, UA, KZ or TR, if you use Foogle, it should NOT have Google codes. (default is Yandex) | |||
When Turkish Firefox is used outside of BY, RU, UA, KZ or TR, it should have codes | |||
When Ukrainian Firefox is used within BY, RU, UA, KZ or TR, if you use Foogle, it should NOT have Google codes even though the default is Google. | |||
When Ukrainian Firefox is used outside of BY, RU, UA, KZ or TR, it should have codes | |||
When Taiwanese Firefox is used in Taiwan, the default is Yahoo and when you use Google, it should not have codes. | |||
When Taiwanese Firefox is used in Hong Kong, the default is Yahoo Hong Kong and when you use Google, it should not have codes. | |||
When Taiwanese Firefox is outside of Taiwan or Hong Kong, the default is Google and it should have codes. | |||
*The second phase included the changes made in [https://bugzilla.mozilla.org/show_bug.cgi?id=1276740 Bug 1276740] and the testing efforts were concentrated in verifying that after moving all the search engines into mozilla-central (because they are mostly named differently) they will stay exactly as they were before. In this phase we manually verified that the search engines xmls remained the same for all the locales tested in the previous phase, and for the few cases where the engines were named the same but had different function like on the following locales: de, hsb, dsb, rm, es-AR, DA. | |||
We will also conduct this test with '''partner builds''' as well to make sure that the new search engines do not have a negative effect on partner builds. | We will also conduct this test with '''partner builds''' as well to make sure that the new search engines do not have a negative effect on partner builds. | ||