MozillaWiki

QA/Turning On Search Suggestions: Difference between revisions

Page Discussion

QA/Turning On Search Suggestions (view source)

Revision as of 14:54, 7 June 2017

671 bytes added ,  7 June 2017
→‎Test Objectives
 
(13 intermediate revisions by the same user not shown)
Line 44: Line 44:
Turning On Search Suggestions is a Mozilla feature that aims to enable the search suggestions by default and give the users the possibility to disable them (before Firefox 55, the search suggestions were disabled by default and the users were asked to enable them).  
Turning On Search Suggestions is a Mozilla feature that aims to enable the search suggestions by default and give the users the possibility to disable them (before Firefox 55, the search suggestions were disabled by default and the users were asked to enable them).  


With this feature, a new hint is displayed in the Awesome's bar dropdown informing the users (that had the search suggestions enabled or that ignored the enabling/disabling of them) about the enabling of the search suggestions. The users that had the search suggestions blocked previously, won't see the hint and the search suggestions won't be enabled in their case.
With this feature, a new hint is displayed in the Awesome's bar dropdown informing the users (that ignored the enabling/disabling of the search suggestions or on clean installs) about the enabling of the search suggestions. The users that previously allowed the search suggestions or blocked them, won't see the hint.


=== Out of Scope ===
=== Out of Scope ===
Line 101: Line 101:
| 1 || Search suggestions hint || Search suggestions hint is displayed for the users that do not have the search suggestions blocked || The hint informing users that search suggestions are enabled is displayed in the Awesome bar drop down just under the URL || Manual || RAC-1, RAC-2, RAC-3 || Eng Team
| 1 || Search suggestions hint || Search suggestions hint is displayed for the users that do not have the search suggestions blocked || The hint informing users that search suggestions are enabled is displayed in the Awesome bar drop down just under the URL || Manual || RAC-1, RAC-2, RAC-3 || Eng Team
|-
|-
| 2 || Search suggestions disabling || Search suggestions in the Awesome Bar can be disabled || Clicking on Change Settings in the search suggestions hint takes you to the about:preferences#general from where search suggestions can be disabled || Manual  || || Eng Team
| 2 || Search suggestions disabling || Search suggestions in the Awesome Bar can be disabled || Clicking on Change Settings in the search suggestions hint takes you to the about:preferences#general from where search suggestions can be disabled || Manual  || RAC-1, RAC-2, RAC-3 || Eng Team
|-
|-
| 3 || Accessibility || Search suggestions and the hint announcing them being enabled are visible and usable from accessibility point of view|| Testing for accessibility includes: <br />
| 3 || Accessibility || Search suggestions and the hint announcing them being enabled are visible and usable from accessibility point of view|| Testing for accessibility includes: <br />
Line 107: Line 107:
- Screen Reader validation<br />
- Screen Reader validation<br />
- Keyboard-only   
- Keyboard-only   
|| Manual || || Eng Team
|| Manual || RAC-1, RAC-2, RAC-3 || Eng Team
|-
|-
| 4 || Upgrade-Downgrade || The search suggestions hint behavior at FF Upgrade/Downgrade || One-Off searches Awesomebar buttons are not lost during FF upgrade/downgrade || Manual || || Eng Team
| 4 || Upgrade-Downgrade || The search suggestions hint behavior at FF Upgrade/Downgrade || One-Off searches Awesomebar buttons are not lost during FF upgrade/downgrade || Manual || RAC-1, RAC-2, RAC-3, RAC-4 || Eng Team
|}
|}


Line 115: Line 115:
This section should contain links for builds with the feature  
This section should contain links for builds with the feature  
* Links for Nightly builds:  
* Links for Nightly builds:  
** First Landed on Nightly 55.0a1 - to be updated after the enabling of the search suggestions is done.
** First Landed on Nightly 55.0a1 - on 2017-05-24 - [http://archive.mozilla.org/pub/firefox/nightly/2017/05/2017-05-24-03-02-04-mozilla-central/ link for Windows and Mac OS X], [http://archive.mozilla.org/pub/firefox/nightly/2017/05/2017-05-24-10-02-15-mozilla-central/ link for Ubuntu]


* Links for Beta builds
* Links for Beta builds
Line 138: Line 138:
|-
|-
| QA - Nightly Testing  
| QA - Nightly Testing  
|style="text-align:center;" | ||  
|style="text-align:center;" | 2017.05.25 || 2017.06.05
|-
|-
| QA - Beta Testing  
| QA - Beta Testing  
Line 226: Line 226:
|-
|-
| Support documents planned(written)   
| Support documents planned(written)   
|style="text-align:center;" | ? || Make sure support documents are written and are correct.
|style="text-align:center;" | No || Make sure support documents are written and are correct.


|-
|-
Line 268: Line 268:
|-
|-
| Temporary or permanent telemetry monitoring   
| Temporary or permanent telemetry monitoring   
|style="text-align:center;" | ? || List of error conditions to monitor
|style="text-align:center;" | No || List of error conditions to monitor
|-
|-
| Telemetry correctness testing   
| Telemetry correctness testing   
Line 336: Line 336:


== Test suite ==
== Test suite ==
  Full Test suite - Link to test rail - testcases should be added under Firefox Desktop project [https://testrail.stage.mozaws.net/index.php?/suites/view/813&group_by=cases:section_id&group_order=asc link]
  Full Test suite - [https://testrail.stage.mozaws.net/index.php?/suites/view/813&group_by=cases:section_id&group_order=asc TestRail Link] - [https://wiki.mozilla.org/images/c/c6/Turning_on_Search_Suggestions_-EngQA-_-_TestRail.pdf PDF doc]


= Bug Work =
= Bug Work =
Meta bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=12345 12345 - bug summary]
Meta bug: [https://bugzilla.mozilla.org/show_bug.cgi?id=1344919 1344919- META - search suggestions and one-off buttons in Awesome Bar]


<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
====== Logged bugs ( blocking [https://bugzilla.mozilla.org/show_bug.cgi?id=12345 12345] )======
====== Logged bugs ( blocking [https://bugzilla.mozilla.org/show_bug.cgi?id=1344919 1344919] )======


<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
<bugzilla>
<bugzilla>
     {
     {
         "blocks":[12345],
         "blocks":[1344919],
         "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
         "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
     }
     }
Line 360: Line 360:
<bugzilla>
<bugzilla>
     {
     {
         "blocks":[12345],
         "blocks":[1344919],
         "resolution":"FIXED",
         "resolution":"FIXED",
         "include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"
         "include_fields": "id, priority, component, assigned_to, summary, status, resolution, target_milestone"
Line 369: Line 369:


= Sign off =
= Sign off =
== Criteria ==
== Criteria for Mid Nightly ==
Checklist
Check list
* All the smoke tests (designed to ensure the main functionality is working as expected) are executed.
 
== Criteria for Pre-Beta Release and Pre-Release ==
Check list
* All test cases should be executed
* All test cases should be executed
* Has sufficient automated test coverage (as measured by code coverage tools) - coordinate with RelMan
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)
* All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed (as determined by engineering/RelMan/QA)


== Results ==
== Results ==
'''Nightly testing'''<br />
'''Mid Nightly testing'''<br />


List of OSes that will be covered by testing<br />
- was missed due to the fact the feature landed after the Mid Nightly sign off.
*Link for the tests run
** Full Test suite, link to TestRail - Tests Runs and Results [https://testrail.stage.mozaws.net/index.php?/runs/overview/17 link]
** Daily Smoke, if needed/available
** Regression Test suite, if needed/available
<br />


'''Merge to Beta Sign-off'''<br />
'''Pre Beta testing'''<br />
List of OSes that will be covered by testing<br />
List of OSes that will be covered by testing<br />
*Link for the tests run
*Link for the tests run - [https://testrail.stage.mozaws.net/index.php?/plans/view/4371 TestRail link] [https://wiki.mozilla.org/images/5/56/Manual_Testing_Summary_Turning_on_Search_Suggestions_%28Fx55%29_-_Pre-Beta_feature_sign_off_%282017-06-06%29_-_TestRail.pdf PDF doc]
** Full Test suite


== Checklist ==
== Checklist ==
{| class="wikitable" style="width:60%"
{| class="wikitable" style="width:80%"
|-
|-
! Exit Criteria !! Status !! Notes/Details
! Exit Criteria !! Status !! Notes/Details
|-
|-
|  Testing Prerequisites (specs, use cases)  
|  Testing Prerequisites (specs, use cases)  
| style="text-align:center;" |   
| style="text-align:center;" |  {{mdone|}}
| style="text-align:center;" |  
| style="text-align:center;" |  
|-
|-
|  Testing Infrastructure setup  
|  Testing Infrastructure setup  
|style="text-align:center;" |   ||  
|style="text-align:center;" | {{mdone|}} ||  
|-
|-
|  Test Plan Creation  
|  Test Plan Creation  
| style="text-align:center;" |   ||  
| style="text-align:center;" | {{mdone|}}  ||  
|-
|-
|  Test Cases Creation  
|  Test Cases Creation  
|style="text-align:center;" |   ||  
|style="text-align:center;" | {{mdone|}}  ||  
|-
|-
|  Automation Coverage ||
|  Automation Coverage
|style="text-align:center;" |  
|style="text-align:center;" | N/A
|style="text-align:center;" |
|-
|-
|  Performance Testing  
|  Performance Testing  
|style="text-align:center;" | ||  
|style="text-align:center;" | N/A||  
|-
|-
|  All Defects Logged || ||  
|  All Defects Logged  
|style="text-align:center;" | {{mprog|}}
|style="text-align:center;" | 
|-
|-
|  Critical/Blockers Fixed and Verified || ||  
|  Critical/Blockers Fixed and Verified  
|style="text-align:center;" | {{mprog|}}
|style="text-align:center;" | 
|-
|-
|  Metrics/Telemetry||  
|  Metrics/Telemetry
|style="text-align:center;" | N/A
|style="text-align:center;" |  
|style="text-align:center;" |  
|-
|-
|  Basic/Core functionality Nightly testing
|  QA mid-Nightly Signoff
|style="text-align:center;" | 
|style="text-align:center;" | {{missed|}}
|style="text-align:center;" | 
|style="text-align:center;" |
|-
|  QA mid-Nightly Signoff||
|style="text-align:center;" | Email to be sent
|-
| QA Nightly - Full Testing
|style="text-align:center;" |  ||  
|-
|-
|  QA pre-Beta Signoff||  
|  QA pre-Beta Signoff
|style="text-align:center;"| Email to be sent  
|style="text-align:center;" | {{done|}}
|style="text-align:center;" | [[https://wiki.mozilla.org/images/6/61/-desktop-_-feature-_Turning_on_Search_Suggestions_-_pre-Beta_sign_off_%28YELLOW%29.pdf  Email sent in 6/06/2017]]
|-
|-
|  QA Beta - Full Testing
|  QA Beta - Full Testing
Simona Badau
Confirmed users
1,333

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1170960...1172894"