QA/DateTimeInputTypes: Difference between revisions

-
(-)
 
(26 intermediate revisions by 4 users not shown)
Line 7: Line 7:
! Date !! Version !! Author !! Description  
! Date !! Version !! Author !! Description  
|-
|-
| 11/23/2015 || 1.0 || Brindusa Tot || Created first draft
| 14/09/2016 || 1.0 || Brindusa Tot || Created first draft
|-
| 13/11/2017 || 1.1 || Emil Ghitta || Updates
|-
|-
| || || ||
|}
|}


Line 40: Line 41:
[mailto:rares.bologa@softvisioninc.eu Rares Bologa] (irc: RaresB) - PM for QA team <br />
[mailto:rares.bologa@softvisioninc.eu Rares Bologa] (irc: RaresB) - PM for QA team <br />
[mailto:brindusa.tot@softvisioninc.eu Brindusa Tot] (irc: brindusat) - leading QA efforts <br />
[mailto:brindusa.tot@softvisioninc.eu Brindusa Tot] (irc: brindusat) - leading QA efforts <br />
[mailto:simona.marcu@softvision.ro Simona Badau] (irc: simonab) - QA <br />
[mailto:roxana.leitan@softvision.ro Roxana Leitan] (irc: RoxanaLeitan) - QA
[mailto:liviu.cirdei@softvision.ro Liviu Cardei] (irc: liviucirdei) - QA <br />
[mailto:roxana.leitan@softvision.ro Roxana Leitan] (irc: RoxanaLeintan) - QA


= Testing summary =  
= Testing summary =  
Line 101: Line 100:
- Cursor + Keyboard<br />
- Cursor + Keyboard<br />
- Keyboard-only<br />
- Keyboard-only<br />
- testing using high Contract themes<br />
- testing using High Contrast themes<br />
- Screen reader ???
- Screen reader
|| Manual || Eng Team
|| Manual || Eng Team
|-
|-
Line 109: Line 108:
== Builds ==
== Builds ==
This section should contain links for builds with the feature -  
This section should contain links for builds with the feature -  
* Links for Nightly builds
* Links for Nightly builds - available since 10.26.2016 - [http://archive.mozilla.org/pub/firefox/nightly/2016/10/2016-10-26-03-02-10-mozilla-central/ link]
* Links for Aurora builds
* Links for Beta builds
* Links for Beta builds


Line 125: Line 123:
|-
|-
| Study documentation/specs received from developers
| Study documentation/specs received from developers
|style="text-align:center;" | 2016.09.12 ||  
|style="text-align:center;" | 2016.09.12  
|style="text-align:center;" | 2016.09.15
|-
|-
| QA - Test plan creation  
| QA - Test plan creation  
|style="text-align:center;" | 2016.09.12 ||  
|style="text-align:center;" | 2016.09.12  
|style="text-align:center;" | 2016.09.15 
|-
|-
| QA - Test cases/Env preparation  
| QA - Test cases/Env preparation  
|style="text-align:center;" | 2016.09.12 ||  
|style="text-align:center;" | 2016.09.12  
|style="text-align:center;" | 2016.10.01 
|-
|-
| QA - Nightly Testing  
| QA - Nightly Testing  
|style="text-align:center;" | ||
|style="text-align:center;" | 2016.11.10
|-
|style="text-align:center;" | 2017.09.13
| QA - Aurora Testing
|style="text-align:center;" | || 
|-
|-
| QA - Beta Testing  
| QA - Beta Testing  
|style="text-align:center;" | ||  
|style="text-align:center;" | 2017.09.14
|style="text-align:center;" | 2017.10.27
|-
|-
| Release Date  
| Release Date  
|style="text-align:center;" | ||  
|style="text-align:center;" | 2017.11.14 ||  
|}
|}


Line 163: Line 163:
= Status =  
= Status =  
== Overview ==
== Overview ==
  Track the dates and build number where feature was released to Nightly
===Time===
  Track the dates and build number where feature was merged to Aurora
{| class="wikitable"
Track the dates and build number where feature was merged to Release/Beta
| <b>Basic input field  </b>||<small>'''<font color="green">[LANDED]</font>'''</small> || Nightly 52: first landed, disabled by default
|-
|<b>Pop-up picker ||<small>'''<font color="green">[LANDED]</font>'''</small> </b>||Nightly 52: first landed, de-scoped
|-
|<b>Input field with STEP (displaying warning message when out of range) </b>||<small>'''<font color="green">[LANDED]</font>'''</small> || Nightly 53: first landed,disabled by default
|-
|<b>Input field with Min/Max (displaying warning message when out of range) </b>|| <small>'''<font color="green">[LANDED]</font>'''</small> || Nightly 53: first landed, disabled by default
|-
|<b>Input field with keyboard navigation </b>||  <small>'''<font color="green">[LANDED]</font>'''</small> || Nightly 52: first landed, disabled by default
|-
|}
 
===Date===
{| class="wikitable"
| <b>Basic input field  </b>||<small>'''<font color="green">[LANDED]</font>'''</small> || Nightly 53: first landed, disabled by default
|-
|<b>Pop-up picker ||<small>'''<font color="green">[LANDED]</font>'''</small> </b>||Nightly 52: first landed, disabled by default
|-
|<b>Input field with STEP (displaying warning message when out of range) </b>||<small>'''<font color="green">[LANDED]</font>'''</small> || Nightly 53: first landed,disabled by default
|-
|<b>Input field with Min/Max (displaying warning message when out of range) </b>|| <small>'''<font color="green">[LANDED]</font>'''</small> || Nightly 53: first landed, disabled by default
|-
|<b>Input field with keyboard navigation </b>|| <small>'''<font color="green">[LANDED]</font>'''</small> || Nightly 53: first landed, disabled by default
|-
|}
 
== Risk Assessment and Coverage ==
 
{| class="wikitable"
|-
! ID !! Description / Threat Description !! Magnitude !! Probability !! Priority !! Impact Score
|-
| RAC-1 || User Experience || '''3-High''' || '''3-High''' || '''3-High''' || '''27'''
|-
| RAC-2 || High contrast || '''3-High''' || '''3-High''' || '''3-High''' || '''27'''
|-
| RAC-3 || Localization|| '''3-High''' || ''2-Possible'' || '''3-High'''|| '''18'''
|-
| RAC-4 || Screen reader for picker and input box  || '''3-High''' || ''2-Possible'' || '''3-High''' || '''18'''
|-
| RAC-5 || Input box in multiple frames  || '''3-High''' || ''2-Possible'' || ''2-Medium'' || '''12'''
|-
| RAC-6 || RTL  || 1- Low || ''2-Possible'' ||''2-Medium'' || 4
 
|}
 
'''Values:'''
 
* '''Magnitude:''' 1- Low , ''2-Moderate'', '''3-High'''
 
* '''Probability:''' 1-Unlikely, ''2-Possible'', '''3-Almost Certain'''
 
* '''Priority:''' 1 - Low, ''2-Medium'', '''3-High'''
 
'''Impact Score Breakdown:'''
* An impact value of 1, 2, 3, 4 would describe an area which although should be covered there aren't expected any discoveries of critical issues.
* An impact value of 6, 8, 9, 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
* An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.


== Risk analysis ==
Identify the high-risk assumptions
Identify existing bugs on the feature with high risk
Identify if other areas are affected by the fix
= References =
= References =
* Meta Bug - [https://bugzilla.mozilla.org/show_bug.cgi?id=888320 Bug 888320 (datetime) meta Implement all time and date related input types]
* Meta Bug - [https://bugzilla.mozilla.org/show_bug.cgi?id=888320 Bug 888320 (datetime) meta Implement all time and date related input types]
Line 183: Line 235:


= Testcases =  
= Testcases =  
== Overview ==
Summary of testing scenarios


== Test Areas ==
== Test Areas ==
Line 219: Line 269:
|-
|-
| Usable with a screen reader   
| Usable with a screen reader   
|style="text-align:center;" | ??? || e.g. with NVDA
|style="text-align:center;" | yes || e.g. with NVDA
|-
|-
| Usability and/or discoverability testing   
| Usability and/or discoverability testing   
Line 225: Line 275:
|-
|-
| RTL build testing   
| RTL build testing   
|style="text-align:center;" | ??? ||
|style="text-align:center;" | yes ||
|-
|-
| '''Help/Support'''  
| '''Help/Support'''  
Line 231: Line 281:
|-
|-
| Help/support interface required   
| Help/support interface required   
|style="text-align:center;" | ??? || Make sure link to support/help page exist and is easy reachable.
|style="text-align:center;" | No || Make sure link to support/help page exist and is easy reachable.
|-
|-
| Support documents planned(written)   
| Support documents planned(written)   
Line 253: Line 303:
|-
|-
| Does this affect partner builds? Partner build testing   
| Does this affect partner builds? Partner build testing   
|style="text-align:center;" | ??? || yes/no options, add comment with details about who will lead testing
|style="text-align:center;" | || yes/no options, add comment with details about who will lead testing


|-
|-
Line 276: Line 326:
|-
|-
| Temporary or permanent telemetry monitoring   
| Temporary or permanent telemetry monitoring   
|style="text-align:center;" | ??? || List of error conditions to monitor
|style="text-align:center;" | No || List of error conditions to monitor
|-
|-
| Telemetry correctness testing   
| Telemetry correctness testing   
Line 324: Line 374:
|-
|-
| Testing against target sites   
| Testing against target sites   
|style="text-align:center;" | ??? ||  
|style="text-align:center;" | ||  
|-
|-
| Survey of many sites for compatibility   
| Survey of many sites for compatibility   
Line 344: Line 394:


== Test suite ==
== Test suite ==
  Full Test suite - can be found on TestRails at  [https://testrail.stage.mozaws.net/index.php?/suites/view/169&group_by=cases:section_id&group_order=asc link]
  Full Test suite - can be found on TestRails at  [https://testrail.stage.mozaws.net/index.php?/suites/view/169&group_by=cases:section_id&group_order=asc link] or in the [https://wiki.mozilla.org/File:Date_Time_Input_Types_-EngQA-_-_TestRail.pdf PDF doc]
Regression Test suite - if available/needed.
 


= Bug Work =
= Bug Work =
Meta: [https://bugzilla.mozilla.org/show_bug.cgi?id=888320 Bug 888320 - [meta<nowiki>]</nowiki> (datetime) [meta] implement all time and date related input types]
Meta: [https://bugzilla.mozilla.org/show_bug.cgi?id=888320 Bug 888320 - [meta<nowiki>]</nowiki> (datetime) implement all time and date related input types]


== Logged bugs ( blocking [https://bugzilla.mozilla.org/show_bug.cgi?id=888320 888320 ] ) ==
<bugzilla>
<bugzilla>
     {
     {
         "blocks":[1269956],
         "blocks":[888320],
         "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
         "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
     }
     }
</bugzilla>
</bugzilla>


Meta: [https://bugzilla.mozilla.org/show_bug.cgi?id=1323674 Bug 1323674 -[meta<nowiki>]</nowiki> defects in date time input types]
== Logged bugs ( blocking [https://bugzilla.mozilla.org/show_bug.cgi?id=1323674 1323674 ] ) ==
<bugzilla>
    {
        "blocks":[1323674],
        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
    }
</bugzilla>


<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
== Bug Verification  (blocking [https://bugzilla.mozilla.org/show_bug.cgi?id=888320 888320 ]) ==
 
<bugzilla>
====== Bug fix verification ======
    {
<div class="mw-collapsible-content">
        "blocks":[888320],
{| class="wikitable" style="width:50%"
        "resolution":"FIXED",
|-
        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
! Bug No !! Summary !! Status !! Firefox Verion
    }
|-
</bugzilla>
|  123 || bug summary || NEW || Nighly 45
== Bug Verification  (blocking [https://bugzilla.mozilla.org/show_bug.cgi?id=1323674 1323674 ]) ==
|-
<bugzilla>
| || || ||
    {
|}
        "blocks":[1323674],
</div></div>
        "resolution":"FIXED",
 
        "include_fields": "id, priority, component, assigned_to, summary, status, target_milestone"
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
    }
 
</bugzilla>
====== Logged bugs ======
<div class="mw-collapsible-content">
Bug 111111<br/>
Bug 211111<br/>
</div>
</div>


= Sign off =
= Sign off =
Line 389: Line 442:
== Results ==
== Results ==
'''Nightly testing'''<br />
'''Nightly testing'''<br />
''Time Input Box''


List of OSes that will be covered by testing<br />
List of OSes that will be covered by testing<br />
*Link for the tests run
*[https://testrail.stage.mozaws.net/index.php?/plans/view/1400 Link for the tests run]
 
''Date Input Box and Date Picker''


'''Merge to Aurora Sign-off'''
List of OSes that will be covered by testing<br />
List of OSes that will be covered by testing<br />
*Link for the tests run
*[https://testrail.stage.mozaws.net/index.php?/plans/view/2295 Link for the tests run]
** Full Test suite
 
'''Beta Testing'''
*[https://testrail.stage.mozaws.net/index.php?/reports/view/664 Link for the tests run]


== Checklist ==
== Checklist ==
Line 404: Line 462:
|-
|-
|  Testing Prerequisites (specs, use cases)  
|  Testing Prerequisites (specs, use cases)  
| style="text-align:center;" |  
|style="text-align:center;"| {{mdone|}}
| style="text-align:center;" |  
| style="text-align:center;" |  
|-
|-
|  Testing Infrastructure setup  
|  Testing Infrastructure setup  
|style="text-align:center;" |   ||  
|style="text-align:center;" | N/A ||  
|-
|-
|  Test Plan Creation  
|  Test Plan Creation  
| style="text-align:center;" |   ||  
| style="text-align:center;" | 09.12.2016
| style="text-align:center;" | 09.14.2016
|-
|-
|  Test Cases Creation  
|  Test Cases Creation  
|style="text-align:center;" |   ||  
|style="text-align:center;"| {{mdone|}}
| style="text-align:center;" |  
|-
|-
|  Full Functional Tests Execution  
|  Full Functional Tests Execution  
|style="text-align:center;" |   
|style="text-align:center;"| {{mdone|}}
|style="text-align:center;" |   
| style="text-align:center;" |   
|-
|  Time Input Box Functional Tests Execution
|style="text-align:center;"| {{mdone|}}
| style="text-align:center;" |
|-
Date Input Box/Picker Functional Tests Execution
|style="text-align:center;"| {{mdone|}}
| style="text-align:center;" |
|-
|-
|  Automation Coverage ||
|  Automation Coverage ||
Line 426: Line 494:
|style="text-align:center;" |  ||  
|style="text-align:center;" |  ||  
|-
|-
|  All Defects Logged || ||  
|  All Defects Logged
|style="text-align:center;"| {{mdone|}}
| style="text-align:center;" |  
|-
|-
|  Critical/Blockers Fixed and Verified || ||  
|  Critical/Blockers Fixed and Verified
|style="text-align:center;"| {{mdone|}}
| style="text-align:center;" |  
|-
|-
|  Metrics/Telemetry||  
|  Metrics/Telemetry||  
|style="text-align:center;" |  
|style="text-align:center;" |  
|-
|-
|  QA Signoff  - Nightly Release||  
|  QA Signoff  - Nightly Release
|style="text-align:center;" | Email to be sent  
|style="text-align:center;"| {{mdone|}}
|style="text-align:center;" | Email sent on 2017.09.13
|-
|-
|  QA Aurora - Full Testing  
|  QA Aurora - Full Testing  
|style="text-align:center;" | ||
|style="text-align:center;" | n/a
|-
|style="text-align:center;" | n/a
|  QA Signoff  - Aurora Release ||
|style="text-align:center;"| Email to be sent
|-
|-
|  QA Beta - Full Testing
|  QA Signoff  - Aurora Release
|style="text-align:center;" | ||  
|style="text-align:center;"| n/a
|style="text-align:center;"| n/a
|-
|-
|  QA Signoff  - Beta Release ||  
|  QA Signoff  - Beta Release  
|style="text-align:center;" | Email to be sent  
|style="text-align:center;"| {{mdone|}}
|style="text-align:center;" | Email sent on 2017.10.17
|}
|}
54

edits