Releases/Firefox 40/Test Plan/Beta/RC: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<small> « [[Releases/Firefox 40/Test Plan/Beta/9 | Firefox 40 Beta 9]] | [[Releases/Firefox 40/Test Plan/Beta/RC | Firefox 40 RC]]  | [[Releases/Firefox 41/Test Plan/Beta/1 | Firefox 41 Beta 1]]  »</small>
<small> « [[Releases/Firefox 40/Test Plan/Beta/9 | Firefox 40 Beta 9]] | [[Releases/Firefox 40/Test Plan/Beta/RC | Firefox 40 RC]]  | [[Releases/Firefox 40/Test Plan/Beta/RC2 | Firefox 40 RC build 4]]  »</small>


= Firefox 40 RC (build 1) Test Plan =
= Firefox 40 RC (build 1 & build 2) Test Plan =
* Release Week: Aug 3rd, 2015. Plan is to ship this build on Tuesday Aug 4th 2015 ([https://www.google.com/calendar/embed?src=bW96aWxsYS5jb21fZGJxODRhbnI5aTh0Y25taGFiYXRzdHY1Y29AZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ Release Calendar])
* Release Week: Aug 3rd, 2015. Plan is to ship this build on Tuesday Aug 4th 2015 ([https://www.google.com/calendar/embed?src=bW96aWxsYS5jb21fZGJxODRhbnI5aTh0Y25taGFiYXRzdHY1Y29AZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ Release Calendar])
* QA Contacts: [mailto:kairo@mozilla.com Robert Kaiser]; [mailto:florin.mezei@softvisioninc.eu Florin Mezei]; [mailto:andrei.vaida@softvision.ro Andrei Vaida]
* QA Contacts: [mailto:kairo@mozilla.com Robert Kaiser]; [mailto:florin.mezei@softvisioninc.eu Florin Mezei]; [mailto:andrei.vaida@softvision.ro Andrei Vaida]
* IRC: irc://irc.mozilla.org#qa ping kairo; FlorinMezei; avaida
* IRC: irc://irc.mozilla.org#qa ping kairo; FlorinMezei; avaida
* Build 1: [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/40.0-candidates/build1/ Firefox 40 RC build 1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/ changeset] | (ID: TBD)
* Build 1: [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/40.0-candidates/build1/ Firefox 40 RC build 1] | [https://hg.mozilla.org/releases/mozilla-release/rev/f6321b14228d changeset] | (ID: 20150803103853)
* Build 2: [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/40.0-candidates/build2/ Firefox 40 RC build 2] | [https://hg.mozilla.org/releases/mozilla-release/rev/6cb7bf295939 changeset] | (ID: 20150804131237)


== QE recommendation regarding the quality of the build ==
== QE recommendation regarding the quality of the build ==
* Build 1
<onlyinclude>
<onlyinclude>
{| class="wikitable" width=50%
{| class="wikitable" width=50%
  ! Recommendation based on results from manual testing
  ! Recommendation based on results from manual testing
  | style="background:#FFA500; text-align:center;" | '''CONDITIONAL PASS'''
  | style="background:#9D9; text-align:center;" | '''PASS'''
|}
 
* Build 2
<onlyinclude>
{| class="wikitable" width=50%
! Recommendation based on results from manual testing
| style="background:#9D9; text-align:center;" | '''PASS'''
|}
|}


Line 17: Line 26:


* Testing is performed around areas most affected by fixes since the previous Beta (in cases when there are no Regression tests to cover these areas, or those tests have already been run as part of the Full Beta Regression set)
* Testing is performed around areas most affected by fixes since the previous Beta (in cases when there are no Regression tests to cover these areas, or those tests have already been run as part of the Full Beta Regression set)
* Test Cases executed and results: [https://docs.google.com/spreadsheets/d/1ZGOcdnEVy8RC1OPf2YPzKNPY3Df9lLmrdKarwjPcbFA/edit#gid=834797701 report]
* Test Cases executed and results: [https://docs.google.com/spreadsheets/d/1ZGOcdnEVy8RC1OPf2YPzKNPY3Df9lLmrdKarwjPcbFA/edit#gid=211402700 report]


=== PLATFORM MATRIX ===
=== PLATFORM MATRIX ===
Line 24: Line 33:
  | class="header" style="width:20%; text-align:center;" | Platform
  | class="header" style="width:20%; text-align:center;" | Platform
  | class="header" style="width:20%; text-align:center;" | Assignee
  | class="header" style="width:20%; text-align:center;" | Assignee
  | class="header" style="width:20%; text-align:center;" | Results: [https://docs.google.com/spreadsheets/d/1ZGOcdnEVy8RC1OPf2YPzKNPY3Df9lLmrdKarwjPcbFA/edit#gid=834797701 report]
  | class="header" style="width:20%; text-align:center;" | Results: [https://docs.google.com/spreadsheets/d/1ZGOcdnEVy8RC1OPf2YPzKNPY3Df9lLmrdKarwjPcbFA/edit#gid=211402700 report]
  |-
  |-
  | (Graphics: )
  | Windows 7 x64 (Graphics: Intel HD 2500)
  |  
  | Camelia Badau
  | style="text-align:center;" | {{|}}
  | style="text-align:center;" | {{mdone|}}
  |-
  |-
  | (Graphics: )
  | Windows 10 x64 (Graphics: nVIDIA GeForce 620)
  |  
  | Cornel Ionce
  | style="text-align:center;" | {{|}}
  | style="text-align:center;" | {{mdone|}}
  |-
  |-
  | (Graphics: )
  | Mac OS X 10.10 (Graphics: Intel Iris Pro 1536 MB)
  |  
  | Alexandra Lucinet
  | style="text-align:center;" | {{|}}
  | style="text-align:center;" | {{mdone|}}
  |-
  |-
  | (Graphics: )
  | Ubuntu 14.04 x86 (Graphics: AMD Radeon HD 6450)
  |  
  | Bogdan Maris
  | style="text-align:center;" | {{|}}
  | style="text-align:center;" | {{mdone|}}
  |}
  |}


Line 50: Line 59:
  | class="header" style="width:20%; text-align:center;" | Results
  | class="header" style="width:20%; text-align:center;" | Results
  |-
  |-
  |  
  | IMAGE SUPPORT
  | Regression Testing
  | Regression Testing
  | style="background:#FFA500; text-align:center;" |  
  | style="background:#9D9; text-align:center;" | PASS
  |-
  |-
  |  
  | WEB COMPATIBILITY
  | Regression Testing
  | Regression Testing
  | style="background:#9D9; text-align:center;" |  
  | style="background:#9D9; text-align:center;" | PASS
  |-
  |-
  |  
  | WEBRTC
  | Regression Testing
  | Smoke Testing
  | style="background:#9D9; text-align:center;" |
  | style="background:#9D9; text-align:center;" | PASS
|-
| BUG [https://bugzilla.mozilla.org/show_bug.cgi?id=1190635 1190635] (build 2)
| Bug Fix Verification
| style="background:#9D9; text-align:center;" | PASS
  |-
  |-
  |  
  | WEB COMPATIBILITY (build 2)
  | Regression Testing
  | Exploratory Testing
  | style="background:#9D9; text-align:center;" |  
  | style="background:#9D9; text-align:center;" | PASS
  |-
  |-
  |}
  |}
Line 85: Line 98:
  | -
  | -
  |-
  |-
  | [https://bugzilla.mozilla.org/show_bug.cgi?id=1150222 First Run for 40 release]
  | [https://bugzilla.mozilla.org/show_bug.cgi?id=1150231 First Run for 40 release]
  | Catalin Varga
  | Catalin Varga
  | {{mok}}
  | {{mrisk}}
  | New feature targeted for Firefox 40 (not yet implemented) - no sign-off (helping with testing)
  | NO sign-off (helping with testing) - issues found during testing
  | -
  | -
  |-
  |-
Line 125: Line 138:
  | {{mdone}}  
  | {{mdone}}  
  | SIGNED OFF
  | SIGNED OFF
  | -
  | [https://wiki.mozilla.org/QA/Hello_context Test Plan]
  |-
  |-
  | [https://bugzilla.mozilla.org/show_bug.cgi?id=1077146 Windows 10 support]
  | [https://bugzilla.mozilla.org/show_bug.cgi?id=1077146 Windows 10 support]
Line 139: Line 152:


== New bugs ==  
== New bugs ==  
<bugzilla>
    NONE   
{
"id":"",
"include_fields":"id,summary,status,resolution"
}
</bugzilla>


== Known Issues - also seen in previous builds ==  
== Known Issues - also seen in previous builds ==  
Line 150: Line 158:
<bugzilla>
<bugzilla>
{
{
  "id":"1151736",
  "id":"1180140",
  "include_fields":"id,summary,status,resolution"
  "include_fields":"id,summary,status,resolution"
}
}
Line 158: Line 166:
<bugzilla>
<bugzilla>
{
{
  "id":"942130,1096357,1113633,1142752,1142945,1142992,1168101,1178271,1184459",
  "id":"1096357,1122858,1142752,1142945,1144722,1147363,1163164,1168101,1176175,1176218,1182606",
  "include_fields":"id,summary,status,resolution"
  "include_fields":"id,summary,status,resolution"
}
}
Line 167: Line 175:
<bugzilla>
<bugzilla>
{
{
  "id":"1173734,1183725",
  "id":"1182424",
  "include_fields":"id,summary,status,resolution"
  "include_fields":"id,summary,status,resolution"
}
}

Latest revision as of 12:19, 7 August 2015

«  Firefox 40 Beta 9 | Firefox 40 RC | Firefox 40 RC build 4  »

Firefox 40 RC (build 1 & build 2) Test Plan

QE recommendation regarding the quality of the build

  • Build 1
Recommendation based on results from manual testing PASS
  • Build 2
Recommendation based on results from manual testing PASS

Manual Testing summary - Platform Coverage and Areas Tested

  • Testing is performed around areas most affected by fixes since the previous Beta (in cases when there are no Regression tests to cover these areas, or those tests have already been run as part of the Full Beta Regression set)
  • Test Cases executed and results: report

PLATFORM MATRIX

Platform Assignee Results: report
Windows 7 x64 (Graphics: Intel HD 2500) Camelia Badau [DONE]
Windows 10 x64 (Graphics: nVIDIA GeForce 620) Cornel Ionce [DONE]
Mac OS X 10.10 (Graphics: Intel Iris Pro 1536 MB) Alexandra Lucinet [DONE]
Ubuntu 14.04 x86 (Graphics: AMD Radeon HD 6450) Bogdan Maris [DONE]

AREAS TESTED

Tested Area Testing Type Results
IMAGE SUPPORT Regression Testing PASS
WEB COMPATIBILITY Regression Testing PASS
WEBRTC Smoke Testing PASS
BUG 1190635 (build 2) Bug Fix Verification PASS
WEB COMPATIBILITY (build 2) Exploratory Testing PASS

Features

See the Feature Release Tracking gdoc for a general overview


Feature Owner Status Details Test Plan
Async plugin init Catalin Varga [DONE] SIGNED OFF -
First Run for 40 release Catalin Varga [AT RISK] NO sign-off (helping with testing) - issues found during testing -
Suggested tiles: Sponsored Cornel Ionce [DONE] SIGNED OFF Test Plan
Suggested tiles: Affiliate Cornel Ionce [DONE] SIGNED OFF Test Plan
New Add-on Signing Vasilica Mihasca DROPPED FROM 40 Feature dropped from Firefox 39 & 40 (not enough add-ons signed) -
Unified telemetry/FHR Stuart Philp DROPPED FROM 40 Testing performed is tracked in bug 1169037 -
OMTC by default on Linux Alexandra Lucinet [DONE] SIGNED OFF -
Hello - Context Bogdan Maris [DONE] SIGNED OFF Test Plan
Windows 10 support Andrei Vaida / Cornel Ionce [AT RISK] New feature targeted for Firefox 40 - weekly status reports (NO final sign-off) -

Results

New bugs

    NONE

Known Issues - also seen in previous builds

Major issues also seen in previous builds

Full Query
ID Summary Status Resolution
1180140 The user is unable to join back the conversation after a crash RESOLVED FIXED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);


Low impact issues also seen in previous builds

Full Query
ID Summary Status Resolution
1096357 "Unable to run script because scripts are blocked internally" JS error appear in browser console while browsing facebook and youtube on MacOS RESOLVED INCOMPLETE
1122858 The Firefox Hello window opens to the top left edge of the screen if the Hello icon is placed in the hamburger menu RESOLVED INCOMPLETE
1142752 (NS_NOINTERFACE) [nsIWebProgress.DOMWindow] exception in_docShellsToWindows when opening customize mode RESOLVED FIXED
1142945 Always deny fullscreen should not be a possible input RESOLVED FIXED
1144722 Hello button cannot be positioned at the top of Menu panel RESOLVED INCOMPLETE
1147363 [Youtube][HTML5] Settings set after having clicked "Next" are lost when exiting fullscreen RESOLVED WORKSFORME
1163164 DevTools dumps out errors on DebuggerProgressListener.prototype._getWindowsInDocShell RESOLVED DUPLICATE
1168101 Reader View icon is shown on some Youtube pages RESOLVED WORKSFORME
1176175 Page Info is not using ellipsis everywhere to depict overflowed content NEW
1176218 [Youtube][HTML5] 8K videos do not work at all, or Audio only VERIFIED FIXED
1182606 Context menu for video elements should check .mozHasAudio property NEW

11 Total; 2 Open (18.18%); 8 Resolved (72.73%); 1 Verified (9.09%);


Branch Bug Fix Verifications

Full Query
ID Summary Status Resolution
1182424 Always leave unified Telemetry behavior on for people who opted into Telemetry VERIFIED FIXED

1 Total; 0 Open (0%); 0 Resolved (0%); 1 Verified (100%);


Status tags

  • [NOT STARTED] - {{mnotstarted}} - little or no work has started on this task
  • [ON TRACK] - {{mok}} - task is actively being worked on and those assigned feel comfortable and confident in the quality of the build.
  • [DONE] - {{mdone}} - task is completed and those assigned feel comfortable and confident in the quality of the build.
  • [AT RISK] - {{mrisk}}- blocking bug / high number of bugs / other confidence-shakers makes us feel not comfortable with the quality of the build.
  • [IN PROGRESS] - {{mprog}} - actively being tested on
  • [INVESTIGATION NEEDED] - {{minvest}} - more investigation needed
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_40/Test_Plan/Beta/RC&oldid=1088857"