QA/Pocket integration: Difference between revisions

Jump to navigation Jump to search

QA/Pocket integration (view source)

Revision as of 12:36, 18 April 2016

13,675 bytes added ,  18 April 2016
no edit summary
(the "hostname" pref no longer exists, and doesn't need to be set for testing)
No edit summary
 
(24 intermediate revisions by 2 users not shown)
Line 51: Line 51:
== Test cases ==
== Test cases ==
===Overview===
===Overview===
Targeted scenarios:
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
<small>see the list</small>
Targeted scenarios
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
* The Pocket button is displayed by default between the Bookmarks and the Downloads buttons from the toolbar, '''IF''' Pocket is enabled by default.
* The Pocket button is displayed by default between the Bookmarks and the Downloads buttons from the toolbar, '''IF''' Pocket is enabled by default.
Line 102: Line 101:
<font color="gray">'''[QE-VERIFY-]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1158289 Bug 1158289] - Optimize Reader View's line length to have between 45 and 75 characters<br/>
<font color="gray">'''[QE-VERIFY-]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1158289 Bug 1158289] - Optimize Reader View's line length to have between 45 and 75 characters<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1158281 Bug 1158281] - Match Pocket's Reader View Sepia Theme<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1158281 Bug 1158281] - Match Pocket's Reader View Sepia Theme<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1161514 Bug 1161514] - The sign-up panel shown by the Pocket button displays top and bottom borders on Mac and Windows<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1160678 Bug 1160678] - Pocket door hangers arent automatically closed<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1161881 Bug 1161881] - Enable Pocket by default (in supported locales)<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1162147 Bug 1162147] - "View Pocket List" link in bookmarks menu doesn't match UI spec
:↳ <small>'''2015-05-07'''</small>: set <code>qe-verify-</code> instead since the bug was marked as <code>wontfix</code>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1162193 Bug 1162193] - Pocket initial panel load displays incorrectly<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1161810 Bug 1161810] - Allow opening the Pocket panel<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1162713 Bug 1162713] - Implement "Save Link to Pocket" context menu item<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1162014 Bug 1162014] - Long tags cut off after trying to save them, overlayed/hidden by white area<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1162033 Bug 1162033] - Pocket panel closes itself while hovering over pocket toolbar button<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1162056 Bug 1162056] - While signed into Firefox Accounts for Sync: 'Sign In with Firefox' button links to sign up page, should be sign in page<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1162697 Bug 1162697] - spinner when saving to pocket feels like a responsiveness problem<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1162816 Bug 1162816] - After signing up for Pocket using Firefox, the onboarding experience shows Chrome screenshots<br/>
<font color="gray">'''[QE-VERIFY-]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1162253 Bug 1162253] - Update the Pocket Menu Icon with the correct aspect ratios<br/>
<font color="gray">'''[QE-VERIFY-]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1162660 Bug 1162660] - Pocket Save Panel should use the system font<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1155521 Bug 1155521] - Figure out what to do with users to have the Pocket add-on / SocialAPI button installed<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1163360 Bug 1163360] - Update Pocket code to latest version (May 9th code drop)<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1157017 Bug 1157017] - Pocket offline experience<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1161504 Bug 1161504] - The "Save" button from the Pocket panel appears to have a visited styling applied<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1163319 Bug 1163319] - Pocket button in hamburger menu breaks layout<br/>
<font color="green">'''[QE-VERIFY+]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1163349 Bug 1163349] - "View Pocket List" menuitem not working<br/>
</div>
</div>
</div>
</div>
Line 113: Line 133:
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1156878 Bug 1156878] - Send a request to the server when clicking the Pocket toolbar button
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1156878 Bug 1156878] - Send a request to the server when clicking the Pocket toolbar button
:↳ <small>'''2015-04-30'''</small>: confirmed fixed on 40.0a1, across platforms
:↳ <small>'''2015-04-30'''</small>: confirmed fixed on 40.0a1, across platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1160578 Bug 1160578] - Odd empty door hanger panel when you're on an internal Firefox page
:↳ <small>'''2015-05-07'''</small>: confirmed fixed on 40.0a1, across platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1155518 Bug 1155518] - Implement "Save to Pocket" context menu item
:↳ <small>'''2015-05-07'''</small>: confirmed fixed on 40.0a1, across platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1161510 Bug 1161510] - main.js:481:12 throws an error in the Browser Console when "Sign Up with Firefox" is clicked
:↳ <small>'''2015-05-07'''</small>: confirmed fixed on 40.0a1, across platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1155519 Bug 1155519] - Add "View Pocket Items" menuitem to the bookmarks menu
:↳ <small>'''2015-05-08'''</small>: confirmed fixed on 40.0a1, across platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1160577 Bug 1160577] - Error message is no longer shown when a page isn't readable
:↳ <small>'''2015-05-08'''</small>: confirmed fixed on 40.0a1, across platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1160678 Bug 1160678] - Pocket door hangers arent automatically closed
:↳ <small>'''2015-05-08'''</small>: confirmed fixed on 40.0a1, across platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1161504 Bug 1161504] - The "Save" button from the Pocket panel appears to have a visited styling applied
:↳ <small>'''2015-05-11'''</small>: confirmed fixed across branches and platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1163319 Bug 1163319] - Pocket button in hamburger menu breaks layout
:↳ <small>'''2015-05-11'''</small>: confirmed fixed across branches and platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1163349 Bug 1163349] - "View Pocket List" menuitem not working
:↳ <small>'''2015-05-11'''</small>: confirmed fixed on 39.0a2 and 38.0.5b1 (build1), across platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1157682 Bug 1157682] - Should ignore query string / hashes for checking for "home page"
:↳ <small>'''2015-05-11'''</small>: confirmed fixed across branches and platforms
<font color="green">'''[VERI]'''</font> [https://bugzilla.mozilla.org/show_bug.cgi?id=1157197 Bug 1157197] - Please enable reader view for 38.0.5
:↳ <small>'''2015-05-11'''</small>: confirmed fixed on 38.0.5b1 (build1) and platforms
</div>
</div>
</div>
</div>
Line 134: Line 176:
[https://bugzilla.mozilla.org/show_bug.cgi?id=1161522 Bug 1161522] - The user is able to save Pocket's "My List" to itself, twice
[https://bugzilla.mozilla.org/show_bug.cgi?id=1161522 Bug 1161522] - The user is able to save Pocket's "My List" to itself, twice
:↳ <small>'''2015-05-05'''</small>: found while performing functional testing on 40.0a1
:↳ <small>'''2015-05-05'''</small>: found while performing functional testing on 40.0a1
[https://bugzilla.mozilla.org/show_bug.cgi?id=1162445 Bug 1162445] - The Pocket toolbar button's functionality is not properly adapting to private browsing, where the user is no longer signed in
:↳ <small>'''2015-05-07'''</small>: found while performing functional testing on 40.0a1
[https://bugzilla.mozilla.org/show_bug.cgi?id=1163576 Bug 1163576] - Unable to remove pages from the Pocket list
:↳ <small>'''2015-05-11'''</small>: found while validating Firefox 38.0.5b1 (build1)
[https://bugzilla.mozilla.org/show_bug.cgi?id=1163613 Bug 1163613] - The panel shown when attempting to Save to Pocket in offline mode is dismissed too soon
:↳ <small>'''2015-05-11'''</small>: found while validating Firefox 38.0.5b1 (build1)
[https://bugzilla.mozilla.org/show_bug.cgi?id=1163581 Bug 1163581] - Unexpected error message displayed when navigating from Pocket list
:↳ <small>'''2015-05-11'''</small>: found while validating Firefox 38.0.5b1 (build1)
:↳ <small>'''2015-05-11'''</small>: closed as <code>invalid</code> by Gavin Sharp, as the issue will be tracked Pocket-side
[https://bugzilla.mozilla.org/show_bug.cgi?id=1163577 Bug 1163577] - Enter Reader View button available in Pocket List page
:↳ <small>'''2015-05-11'''</small>: found while validating Firefox 38.0.5b1 (build1)
[https://bugzilla.mozilla.org/show_bug.cgi?id=1163917 Bug 1163917] - The built-in Pocket is NOT completely disabled if Pocket is already installed and used as an add-on
:↳ <small>'''2015-05-12'''</small>: found while validating Firefox 38.0.5b1 (build2)
</div>
</div>
</div>
</div>


==Milestones==  
==Milestones==
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto">
===Firefox 38.0.5 Beta 1===
===Firefox 38.0.5 Beta 1===
<div class="mw-collapsible-content">
{| class="wikitable"
|-
| || '''Build 1''' || '''Build 2'''
|-
| '''Builds''' || [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/38.0.5b1-candidates/build1/ FTP], [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/38.0.5b1-candidates/build1/ HTTP] || [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/38.0.5b1-candidates/build2/ FTP], [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/38.0.5b1-candidates/build2/ HTTP]
|-
| '''ID''' || 20150510205200 || 20150511143336
|-
| '''Changeset''' || [https://hg.mozilla.org/releases/mozilla-release/rev/32b69592b334 32b69592b334] ([https://hg.mozilla.org/releases/mozilla-release/pushloghtml pushlog]) || [https://hg.mozilla.org/releases/mozilla-release/rev/fff143cacb66 fff143cacb66] ([https://hg.mozilla.org/releases/mozilla-release/pushloghtml pushlog])
|}
<big>'''Manual testing summary'''</big>
* '''Test coverage'''
** Smoke testing — targeting the primary functionalities of the feature
** [https://etherpad.mozilla.org/Fx38-0-5b1 Exploratory testing] — targeting various other scenarios surrounding the main functionalities available, including negative and edge cases
** Bug fix verification — [http://mzl.la/1zWYZhq Bugzilla query]
* '''Platform coverage'''
** Windows 7 (x64)
** Windows 8.1 (x86)
** Mac OS X 10.9.5
** Ubuntu 14.04 (x64)
* '''Reported bugs for this build'''
** [normal] [https://bugzil.la/1163576 Bug 1163576] - Unable to remove pages from the Pocket list
*** <font style="background-color:#ECECEE; border-radius:3px;border-style:solid; border-color:#ECECEE; border-width:3px;font-size:90%;">UPDATE 2015-05-12</font> marked as '''Resolved Fixed''' and pending uplift
** [normal] [https://bugzil.la/1163577 Bug 1163577] - Enter Reader View button available in Pocket List page
** [normal] [https://bugzil.la/1163613 Bug 1163613] - The panel shown when attempting to Save to Pocket in offline mode is dismissed too soon
** [normal] [https://bugzil.la/1163581 Bug 1163581] - Unexpected error message displayed when navigating from Pocket list
*** <font style="background-color:#ECECEE; border-radius:3px;border-style:solid; border-color:#ECECEE; border-width:3px;font-size:90%;">UPDATE 2015-05-12</font> closed as '''Invalid''' by Gavin Sharp, as the issue will be tracked on Pocket's end
** [normal] [https://bugzil.la/1163651 Bug 1163651] - [Windows]View Pocket List icon from Bookmarks menu is missing.
** [normal] [https://bugzil.la/1163655 Bug 1163655] - View Pocket List menuitem is not at top if Bookmarks button is moved to menu panel.
** [normal] [https://bugzil.la/1163917 Bug 1163917] - The built-in Pocket is NOT completely disabled if Pocket is already installed and used as an add-on
* '''Test results'''
** Smoke test results:
<blockquote>
<blockquote>
{| class="wikitable"
{| class="wikitable"
|-
|-
| '''Builds''' || [url FTP], [url HTTP]
| '''Platform''' || '''Results''' ([https://moztrap.mozilla.org/results/cases/?filter-run=7156 report])
|-
| Windows 7 (x64) || <center><b><font style="font-size:80%;color:red;">[FAILED]</font></b></center>
|-
| Windows 8.1 (x86) || <center><b><font style="font-size:80%;color:red;">[FAILED]</font></b></center>
|-
| Mac OS X 10.9.5 || <center><b><font style="font-size:80%;color:red;">[FAILED]</font></b></center>
|-
| Ubuntu 14.04 (x64) || <center><b><font style="font-size:80%;color:red;">[FAILED]</font></b></center>
|}
</blockquote>
</blockquote>
 
:* Overall feature status after testing: {{mrisk|}}
:* Recommendation from QA: <font style="font-size:80%;color:orange;">'''[CONDITIONAL SIGN OFF]'''</font> — pending the review of newly introduced bugs
</div>
</div>
 
===Firefox 38.0.5 Beta 2===
{| class="wikitable"
| '''Builds''' || [ftp://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/38.0.5b2-candidates/build1/ FTP], [https://ftp.mozilla.org/pub/mozilla.org/firefox/candidates/38.0.5b2-candidates/build1/ HTTP]
|-
|-
| '''ID''' || ---
| '''ID''' || 20150514163436
|-
|-
| '''Changeset''' || [url rev] ([url pushlog])
| '''Changeset''' || [https://hg.mozilla.org/releases/mozilla-release/rev/195e873a8ab1 195e873a8ab1] ([https://hg.mozilla.org/releases/mozilla-release/pushloghtml pushlog])
|}
|}


<big>'''Manual testing summary'''</big>
<big>'''Manual testing summary'''</big>
* '''Test coverage'''
* '''Test coverage'''
** [url Smoke testing] — targeting the primary functionalities of the feature
** Smoke testing — targeting the primary functionalities of the feature
** Exploratory testing — targeting various other scenarios surrounding the main functionalities available, including negative and edge cases
** [https://etherpad.mozilla.org/Fx38-0-5b2 Exploratory testing] — targeting various other scenarios surrounding the main functionalities available, including negative and edge cases
** Bug fix verification — Bugzilla query
* '''Platform coverage'''
* '''Platform coverage'''
** Windows 1 (x86/64)
** Windows XP (x86)
** Windows 2 (x86/64)
** Windows 8 (x64)
** Mac OS X 10.#
** Mac OS X 10.8.5
** Ubuntu # (x86/64)
** Ubuntu (x86)
* '''Reported bugs for this build'''
* '''Reported bugs for this build'''
** [severity] Bug # - Bug_summary
** [normal] [https://bugzil.la/1165291 Bug 1165291] - Pocket Social Api is not correctly disabled
* '''Test results'''
* '''Test results'''
** Smoke test results:
** Smoke test results:
<blockquote>  
<blockquote>  
<blockquote>
{| class="wikitable"
{| class="wikitable"
|-
|-
| '''Platform''' || '''Results''' ([url report])
| '''Platform''' || '''Results''' ([https://moztrap.mozilla.org/results/cases/?filter-run=7215 report])
|-
|-
| Windows 1 (x86/64) || <center>{{mok|}}</center>
| Windows XP (x86) || <center><b><font style="font-size:80%;color:red;">[FAILED]</font></b></center>
|-
|-
| Windows 2 (x86/64) || <center>{{mok|}}</center>
| Windows 8 (x64) || <center><b><font style="font-size:80%;color:red;">[FAILED]</font></b></center>
|-
|-
| Mac OS X 10.# || <center>{{mok|}}</center>
| Mac OS X 10.8.5 || <center><b><font style="font-size:80%;color:red;">[FAILED]</font></b></center>
|-
|-
| Ubuntu # (x86/64) || <center>{{mok|}}</center>
| Ubuntu (x86) || <center><b><font style="font-size:80%;color:red;">[FAILED]</font></b></center>
|}
|}
</blockquote>
</blockquote>
</blockquote>


:* Overall feature status after testing: ---
:* Overall feature status after testing: <b><font style="color:green;font-size:80%;">[GOOD]</font></b>
:* Recommendation from QA: ---
:* Recommendation from QA: <b><font style="color:green;font-size:80%;">[SIGN OFF]</font></b>


==Sign off ==
==Sign off ==
Line 186: Line 302:
===Results===
===Results===
{{mok|}}''' Beta 38.0.5''' ''(yyyy-mm-dd)''
{{mok|}}''' Beta 38.0.5''' ''(yyyy-mm-dd)''
* <font style="font-size:80%;color:orange;">'''[CONDITIONALLY SIGNED OFF]'''</font> 38.0.5b1-build2 ''(2015-05-12)''
* <b><font style="color:green;font-size:80%;">[SIGNED OFF]</font></b> 38.0.5b2-build1 ''(2015-05-15)''
= Pocket System Add-on =
== Ownership ==
Developer contacts: Shane Caraveo
QA Lead: [mailto:paul.silaghi@softvision.ro Paul Silaghi], :pauly on IRC
== Summary ==
Pocket moved to a system add-on in [https://bugzil.la/1215694 Bug 1215694], which landed on Firefox 46.
== Test Cases ==
Draft test suites:
* {{mdone|}} [https://docs.google.com/spreadsheets/d/1wl9QDPj6-zQfUJd328KOXI75pJX1EOsP5nD9wkRNUO8/edit#gid=2009706707 Smoke and Regression Test Suites]
== Important bugs ==
<font color="red"><small>'''[META]'''</small></font> [https://bugzil.la/1215694 Bug 1215694] - Move Pocket to a built-in add-on
'''[NEW]''' [https://bugzil.la/1263599 Bug 1263599] - "View Pocket List" option from the bookmarks menu gone after restart
==Sign off ==
* <b><font style="color:green;font-size:80%;">[SIGNED OFF]</font></b> 46.0b11 ''(2016-04-18)''
Paul.silaghi
Confirmed users
891

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/1073149...1128095"

Navigation menu