QA/Photon Structure: Difference between revisions

Jump to navigation Jump to search

QA/Photon Structure (view source)

Revision as of 08:10, 4 October 2017

2,016 bytes added ,  4 October 2017
-
No edit summary
(-)
Line 19: Line 19:


'''Revision History'''
'''Revision History'''
This section describes the modifications that have been made to this wiki page. A new row has been completed each time the content of this document is updated (small corrections for typographical errors do not need to be recorded). The description of the modification contains the differences from the prior version, in terms of what sections were updated and to what extent.


{| class="wikitable" style="width:60%"
{| class="wikitable" style="width:60%"
Line 33: Line 35:
|-
|-
| 09/12/2017 || 1.4 || Grover Wimberly IV || Includes revisions from Project Managers
| 09/12/2017 || 1.4 || Grover Wimberly IV || Includes revisions from Project Managers
|-
| 10/04/2017 || 1.5 || Emil Ghitta || Updates
|-
|-
|}
|}
Line 39: Line 43:
== Purpose ==
== Purpose ==


The purpose of this test plan is to  
The purpose of this test plan is to:
* Determine the scope of testing Photon Structure related bugs and features
* Determine the scope of testing Photon Structure related bugs and features
* Delegate test responsibilities between various QA personnel
* Delegate test responsibilities between various QA personnel
Line 49: Line 53:


== Scope ==
== Scope ==
The scope of this document is to test and verify new features/bugs in Photon Structure are working as intended.  
This wiki section details the testing that will be performed by the project team for the <b>Photon Structure</b> project. It defines the overall testing requirements and provides an integrated view of the project test activities. Its purpose is to document <u> what will be tested</u> and <u>how testing will be performed</u>.
* Testing is performed on the newest version of Nightly and tested on all three mainstream operating systems of Firefox Nightly.  
* (Windows, Mac OSX, and Linux - Ubuntu Distribution).


== Ownership ==
== Ownership ==


Engineering lead:
*Engineering lead:
  [mailto:gijskruitbosch%2Bbugs%40gmail.com Gijs Kruitbosch]
** [mailto:gijskruitbosch%2Bbugs%40gmail.com Gijs Kruitbosch]
 
*Engineering Team:
Engineering Team:
** [mailto:mdeboer@mozilla.com Mike De Boer]
  [mailto:mdeboer@mozilla.com Mike De Boer]
** [mailto:adw@mozilla.com Drew Willcoxon]
  [mailto:adw@mozilla.com Drew Willcoxon]
*UX Team:  
 
** [mailto:abenson@mozilla.com Aaron Benson]
UX Team:  
** [mailto:bbell@mozilla.com Bryan Bell]
  [mailto:abenson@mozilla.com Aaron Benson]
  [mailto:bbell@mozilla.com Bryan Bell]


QA:  
*QA:
  [mailto:gwimberly@softvision.com Grover Wimberly IV]
** Engineering Team - Nightly Testing
***[mailto:gwimberly@softvision.com Grover Wimberly IV]
** Release Team - Beta/Release Testing
*** [mailto:cornel.ionce@softvisioninc.eu Cornel Ionce] (irc:cornel_ionce) - TL for QA team
*** [mailto:emil.ghitta@softvision.ro Emil Ghitta] (irc:emilghitta) - QA
*** [mailto:ciprian.georgiu@softvision.ro Ciprian Georgiu] (irc:ciprian_georgiu) -QA peer


= Testing summary =  
= Testing summary =  
== Scope of Testing ==
== Scope of Testing ==
=== In Scope ===
=== In Scope ===
Photon Structure - Including menus, buttons, toolbar, overflow panel, customization windows, UI icons and elements, as well as functionality on touch screen on all three major operating systems.
Testing will check if the functionality of the following elements work as intended:
*Menus
*Buttons
*Toolbar
*Overflow panel
*Customization window
*UI icons and elements
 
Testing will also check if the above targeted functionality elements work as expected using touch screen on all three major operating systems.


=== Out of Scope ===
=== Out of Scope ===
Animations, Preferences, Onboarding, Mobile Environments
Testing will <u>not</u> check:
*Animations
*Preferences
*Onboarding


= Requirements for testing =
= Requirements for testing =
== Environments ==
== Environments ==
Windows 10
* '''Operating systems''':
Ubuntu 16.04
**Windows 10
Mac 10.12
**Ubuntu 16.04
**Mac 10.12
* '''Devices''': n/a
* '''Other configuration/environmental setup needed''':
** <code>photon.structure.enabled</code> needs to be set to <code>true</code>, but it should be on by default


== Channel dependent settings (configs) and environment setups ==
== Channel dependent settings (configs) and environment setups ==
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
There are no special settings needed to test this feature, apart from its pref being set to true.
 
=== Nightly ===
<div class="mw-collapsible-content">
photon.structure.enabled = true (Default)
</div>
 
=== Beta ===
<div class="mw-collapsible-content">
photon.structure.enabled = true
</div>
 
=== Post Beta / Release ===
<div class="mw-collapsible-content">
photon.structure.enabled = true
</div>
</div>


= Test Strategy =  
= Test Strategy =  
Line 138: Line 142:
* An impact value of 6, 8, 9, 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
* An impact value of 6, 8, 9, 12 would describe an area in which we expect to find issues but those issues are not expected to be critical.
* An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.
* An impact value of 18 or 27 would describe an area on which it is likely to find issues and those issues to be critical or blockers.
'''RAC:''' Refers to the "Risk Assessment and Coverage" item that is covering.


== Test Objectives ==
== Test Objectives ==
Line 147: Line 153:
! Ref !! Function !! Test Objective !! Evaluation Criteria !! Test Type !! RAC !! Owners  
! Ref !! Function !! Test Objective !! Evaluation Criteria !! Test Type !! RAC !! Owners  
|-
|-
| 1 || Menu Testing || Ensure new menus/panels open correctly || Ensuring the bug is fixed on major desktop operating systems || Manual/Performance || RAC-1 || Eng Team
| 1 || Menu Testing || Verify that the new menus/panels open correctly || Menus and Panels are functional || Manual/Performance || RAC-1 || Eng Team
|-
|-
| 2 || Drag-and-drop functionality/Customize Window || Ensure drag and drop functionality with items from new customize window can seamlessly be swapped to location bar, relevant toolbars or overflow panel || Ensuring the bug is fixed on major desktop operating systems || Manual/Performance || RAC-2, RAC-5, RAC-6 || Eng Team
| 2 || Drag-and-drop functionality/Customize Window || Verify that the drag and drop functionality with items from new customize window can seamlessly be swapped to location bar, relevant toolbars or overflow panel || The functionality of the new customize window allows the user to customize the browser without issues || Manual/Performance || RAC-2, RAC-5, RAC-6 || Eng Team
|-
|-
| 3 || Photon library menu || Ensure the new library menu functions as intended per specification || Ensuring the bug is fixed on major desktop operating systems || Manual/Performance || RAC-3, RAC-4 || Eng Team
| 3 || Photon library menu || Verify that the new library menu is functional as per specifications || The library menu is functional and no issues occur while interacting with it|| Manual/Performance || RAC-3, RAC-4 || Eng Team
|-
|-
| 4 || Photon hamburger window || Ensure the new hamburger menu functions as intended per specification || Ensuring the bug is fixed on major desktop operating systems || Manual/Performance || RAC-4 || Eng Team
| 4 || Photon hamburger window || Verify that the new hamburger menu is functional as per specifications || The new hamburger menu is fully functional and it's different options can be used without issues || Manual/Performance || RAC-4 || Eng Team
|-
|-
| 5 || New page action window || Ensure the new page action menu functions as intended per specification || Ensuring the bug is fixed on major desktop operating systems || Manual/Performance || RAC-4 || Eng Team
| 5 || New page action window || Verify that the new page action menu is functional as per specifications || The new page action menu is functional and no issues occur || Manual/Performance || RAC-4 || Eng Team
|-
|-
| 6 || RTL testing || Ensure the Photon Structure menus operate correctly with RTL || Ensuring the bug is fixed on major desktop operating systems || Manual/Performance || RAC-4 || Eng Team
| 6 || RTL testing || Verify that the Photon Structure menus operate correctly with RTL || The Photon Structure menus are fully functional using different RTL builds || Manual/Performance || RAC-4 || Eng Team
|-
|-
| 7 || Sidebar functionality || Ensure the new functionality with the sidebar works as intended || Ensuring the bug is fixed on major desktop operating systems || Manual/Performance || RAC-7 || Eng Team
| 7 || Sidebar functionality || Verify that the new sidebar functionality works as intended || The new sidebar is fully functional and no issues occur|| Manual/Performance || RAC-7 || Eng Team
|-
|-
| 8 || Exploratory testing with touch screen || Ensure the new structure works with touchscreen devices || Ensuring the bug is fixed on major desktop operating systems || Manual/Performance || RAC-8 || Eng Team
| 8 || Exploratory testing with touch screen || Verify the new structure works with touchscreen devices || The new structure is fully functional using touchscreen devices and no specific issues occur || Manual/Performance || RAC-8 || Eng Team
|-
|-
|}
|}


== Builds ==
== Builds ==
[https://www.mozilla.org/en-US/firefox/channel/desktop/ Up-to-date Nightly build]
This section should contain links for builds with the feature:
*[https://www.mozilla.org/en-US/firefox/channel/desktop/ Up-to-date Nightly build]
*[https://archive.mozilla.org/pub/firefox/candidates/ Beta 57 Builds]


== Test Execution Schedule ==
== Test Execution Schedule ==
Line 175: Line 183:
|-
|-
| Start project  
| Start project  
|style="text-align:center;" | 06/2017 || 07/2017  
|style="text-align:center;" colspan="3" | 06/2017  
|-
|-
| Study documentation/specs received from developers
| Study documentation/specs received from developers
Line 193: Line 201:
|-
|-
| Release Date  
| Release Date  
|style="text-align:center;" | 11/2017 with Firefox 57 || TBD
|style="text-align:center;" | TBD || TBD
|}
|}


Line 204: Line 212:
| Test plan creation || Mozilla wiki
| Test plan creation || Mozilla wiki
|-
|-
| Test case creation || TBD
| Test case creation || [https://testrail.stage.mozaws.net/index.php?/suites/view/1488&group_by=cases:section_id&group_order=asc TestRail]
|-
|-
| Test case execution || TBD
| Test case execution || [https://testrail.stage.mozaws.net/index.php?/suites/view/1488&group_by=cases:section_id&group_order=asc TestRail]
|-
|-
| Bugs management || [https://bugzilla.mozilla.org/buglist.cgi?o5=notsubstring&o1=substring&o2=notsubstring&v5=qe-verify%3F&o4=notsubstring&v1=%5Bphoton-structure%5D&v2=%5Btriage%5D&v4=qe-verify-&f1=status_whiteboard&o3=notsubstring&v3=meta&f4=flagtypes.name&query_format=advanced&f3=keywords&f2=status_whiteboard&f5=flagtypes.name&list_id=13734256 Bugzilla]
| Bugs management || [https://bugzilla.mozilla.org/buglist.cgi?o5=notsubstring&o1=substring&o2=notsubstring&v5=qe-verify%3F&o4=notsubstring&v1=%5Bphoton-structure%5D&v2=%5Btriage%5D&v4=qe-verify-&f1=status_whiteboard&o3=notsubstring&v3=meta&f4=flagtypes.name&query_format=advanced&f3=keywords&f2=status_whiteboard&f5=flagtypes.name&list_id=13734256 Bugzilla]
Line 213: Line 221:
= Status =  
= Status =  
== Overview ==
== Overview ==
Track the dates and build number where feature was released to Nightly
*Track the dates and build number where feature was released to Nightly
Track the dates and build number where feature was merged to Release/Beta
* Feature was merged to Beta - [https://archive.mozilla.org/pub/devedition/candidates/57.0b1-candidates/ 57.0b1 (2017-09-15)]




= References =
= References =
[https://bugzilla.mozilla.org/buglist.cgi?o5=notsubstring&o1=substring&o2=notsubstring&v5=qe-verify%3F&o4=notsubstring&v1=%5Bphoton-structure%5D&v2=%5Btriage%5D&v4=qe-verify-&f1=status_whiteboard&o3=notsubstring&v3=meta&f4=flagtypes.name&query_format=advanced&f3=keywords&f2=status_whiteboard&f5=flagtypes.name&list_id=13721695 Photon Structure Bug List]
*[https://bugzilla.mozilla.org/buglist.cgi?o5=notsubstring&o1=substring&o2=notsubstring&v5=qe-verify%3F&o4=notsubstring&v1=%5Bphoton-structure%5D&v2=%5Btriage%5D&v4=qe-verify-&f1=status_whiteboard&o3=notsubstring&v3=meta&f4=flagtypes.name&query_format=advanced&f3=keywords&f2=status_whiteboard&f5=flagtypes.name&list_id=13721695 Photon Structure Bug List]
*[https://bugzilla.mozilla.org/show_bug.cgi?id=1349210 Meta Bug - Photon - structural changes to menus/toolbars/panels]


= Testcases =  
= Testcases =  
Line 379: Line 388:


== Test suite ==
== Test suite ==
Full Test suite - To be worked on
*Full Test suite - [https://testrail.stage.mozaws.net/index.php?/suites/view/1488&group_by=cases:section_id&group_order=asc TestRail link]


= Bug Work =
= Bug Work =
Line 421: Line 430:
'''Nightly testing'''<br />
'''Nightly testing'''<br />


TBD
* List of OSes that will be covered by testing:<br />
** Windows 10 Surface Pro Touchscreen
** Windows 10 64bit
** Windows 7 64bit
** Ubuntu 16.04 64bit
** MacOS 10.11
* Link for the tests run -[https://testrail.stage.mozaws.net/index.php?/plans/view/6188 link]


'''Merge to Beta Sign-off'''<br />
'''Merge to Beta Sign-Off'''
 
* List of OSes that will be covered by testing<br />
TBD
** TBD
** TBD
** TBD
** TBD
* TBD


== Checklist ==
== Checklist ==
Line 441: Line 460:
|-
|-
|  Test Plan Creation  
|  Test Plan Creation  
| style="text-align:center;" |  {{in progress|}}  
| style="text-align:center;" |  {{mdone|}}  
|style="text-align:center;" |  
|style="text-align:center;" |  
|-
|-
|  Test Cases Creation  
|  Test Cases Creation  
|style="text-align:center;" |  TBA
|style="text-align:center;" |  {{mdone|}}
|style="text-align:center;" |  
|style="text-align:center;" |  
|-
|-
Line 469: Line 488:
|-
|-
|  Basic/Core functionality Nightly testing
|  Basic/Core functionality Nightly testing
|style="text-align:center;" |  {{in progress|}}  
|style="text-align:center;" |  {{mdone|}}  
|style="text-align:center;" |   
|style="text-align:center;" |   
|-
|-
|  QA mid-Nightly Signoff
|  QA mid-Nightly Signoff
|style="text-align:center;"| 8/2017
|style="text-align:center;"| {{mdone|}}
|style="text-align:center;" | Email Sent
|style="text-align:center;" | Email Sent on 2017.08.26
|-
|-
|  QA Nightly - Full Testing  
|  QA Nightly - Full Testing  
|style="text-align:center;" | {{in progress|}}
|style="text-align:center;" | {{in progress|}}
|style="text-align:center;"| Email to be sent
|style="text-align:center;"|  
|-
|-
|  QA pre-Beta Signoff
|  QA pre-Beta Signoff
|style="text-align:center;" | TBA
|style="text-align:center;" | {{mdone|}}
|style="text-align:center;"| Email to be sent  
|style="text-align:center;"| Email sent on 2017.09.15
|-
|-
|  QA Beta - Full Testing
|  QA Beta - Full Testing
|style="text-align:center;"| TBA
|style="text-align:center;"| {{in progress|}}
|style="text-align:center;" | Email to be sent
|style="text-align:center;" |  
|-
|-
|  QA pre-Release Signoff  
|  QA pre-Release Signoff  
|style="text-align:center;" | TBA
|style="text-align:center;" | {{in progress|}}
|style="text-align:center;" | Email to be sent  
|style="text-align:center;" | Email to be sent  
|}
|}
Eghitta
54

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1181777"

Navigation menu