QA/Reading list: Difference between revisions
Andrei.vaida (talk | contribs) m (→Overview: update related to MozTrap test suites) |
Andrei.vaida (talk | contribs) m (→Results: updates related to the upcoming pre-beta feature sign-off) |
||
Line 259: | Line 259: | ||
** Manual testing covered Windows 7 (x64), Windows 8.1 (x86), Ubuntu 14.04 (x64) and Mac OS X 10.9.5. | ** Manual testing covered Windows 7 (x64), Windows 8.1 (x86), Ubuntu 14.04 (x64) and Mac OS X 10.9.5. | ||
'''Merge to Beta Sign-off' | '''Merge to Beta Sign-off''' | ||
* {{ok|}} Beta 38 ''( | * {{ok|}} Beta 38 ''(2015-03-30)''. | ||
** MozTrap Test Run: [https://moztrap.mozilla.org/results/cases/?filter-run=6748 results] | |||
'''Merge to Release Sign-off' | '''Merge to Release Sign-off''' | ||
* {{ok|}} RC 38.0 ''(date)''. | * {{ok|}} RC 38.0 ''(date)''. |
Revision as of 15:01, 24 March 2015
Desktop Reading List
Ownership
Developer contacts: Justin Dolske, Blair McBride, Drew Willcoxon
QA Lead: Andrei Vaida, :avaida on IRC
Status
Overview
[LANDED] Nightly 38: first landed, disabled by default
[LANDED] Aurora 38: landed, disabled by default
[ON TRACK] Beta 38: tbd
[ON TRACK] Release 38: tbd
Risk analysis
Event | Probability | Impact | Current situation | Mitigation steps |
---|---|---|---|---|
Adding a website to Reading List fails to work. | tbd | high | tbd | Periodical regression testing covers this area. |
Opening a website from the Reading List fails to work. | tbd | high | tbd | Periodical regression testing covers this area. |
Removing a website from Reading List fails to work. | tbd | medium | tbd | Periodical regression testing covers this area. |
The Reading List sidebar toggle button fails to work. | improbable | high | tbd | Periodical regression testing covers this area. |
Local reading lists are not saved properly and some/all added items are missing. | tbd | high | tbd | Periodical regression testing covers this area. |
Reading List items are not synced up with the Firefox Account. | tbd | medium | tbd | Periodical regression testing covers this area. |
Reading List items are not synced down from the Firefox Account. | tbd | high | tbd | Periodical regression testing covers this area. |
Summary
Context
Reading List for Firefox Desktop is based on a cross-product set of features, scheduled for release with Firefox 38.
The features planned for Reading List v1 are:
- a synced reading list for a user signed into a FxA
- a local only reading list for a user not signed into a FxA
- a Reader Mode that integrates with the Reading List for reading and saving
What will not be available in Reading List v1:
- offline reading support, currently available only on mobile
- web access independent of Firefox (cloud based access)
Enabling the feature
Reading List is available starting with Firefox 38 and it can be enabled by setting the following pref to true:
browser.readinglist.enabled
UPDATE 2015-03-22: the pref is true by default on 39.0a1.
|
References
- The UX planned for Reading List is available here.
Test cases
Overview
- [DONE] Draft test suite
- [DONE] MozTrap test suites
Targeted scenarios
Testing focused on basic functionality
- Enable Reading List
- Disable reading List
- Check the pre-populated Reading List using a new profile
- Add a web page to local Reading List using the Location Bar
- Enable the Reading List sidebar
- Remove a web page from local Reading List using the sidebar
- Enable the Reading List sidebar from the Bookmarks Menu entry
- Sync down the Reading List from a Firefox Account
- Sync up local Reading List to a Firefox Account
Testing focused on interoperability
- Add web page to Reading List from Reader View's sidebar (non-pb & pb)
- Add web page to Reading List from Location Bar and restart Firefox (non-pb & pb)
- Add web page to Reading List from Library
- note: since the websites from Reading List are in some way similar to bookmarks, it might be possible to add a new website to the Reading List from the Library; same goes for removing a website from the list
- Open a website from History and add it to Reading List (non-pb & pb)
- Restore a closed tab from History and add it to Reading List (non-pb & pb)
- Open a website, bookmark it and add it to Reading List (non-pb & pb)
- Open a bookmarked website and add it to Reading List (non-pb & pb)
- Bookmark a website from the Reading List (non-pb & pb)
- Import bookmarks and add one to Reading List
- Add a website to Reading List from context menu (non-pb & pb)
- Merge local Reading List with a second Reading List associated to a Firefox Account
- Merge two different Reading Lists associated to two different Firefox Accounts
- Disable Reading List syncing from about:preferences#sync
- Initiate manual sync after adding a website to Reading List
- note: as in the case of Bookmarks, adding a website to Reading List might be designed to initiate up-syncing automatically
- Remove pre-populated Reading List websites (non-pb & pb)
- Open website from Reading List and display it in Reader View (non-pb & pb)
- Add several websites to Reading List subsequently (at least 15)
- Open several websites from the Reading List sidebar subsequently, in separate tabs (at least 15)
- Pin a tab where Reading List sidebar is enabled
Testing based on recent bugs and regressions
- Enable the Reading List sidebar on Firefox Developer Edition
- Add web pages with lengthy titles to Reading List
- Check how the Reading List and Reader View buttons are displayed in the Location Bar
- Check how the Reading List button from the Location Bar changes according to its status
- Attempt to add an in-content page to Reading List
- Add a web page to Reading List from Reader View's controls
- Check the Reading List button styling on Windows OS
- Check the "Add page to Reading List" button styling on Firefox Developer Edition
- Check the "Add page to Reading List" button styling on high-contrast themes
- Add a web page to Reading List from a private window
Testing Reading List with e10s
- Reading List should work just as well with e10S enabled (default on in 39.0a1)
- Is this really still true? --> Reading List is currently NOT compatible with e10s. It seems to work fine in e10s.
Bug work
Tracking bug
meta: Bug 1123529 - (desktop-readinglist) [meta] Reading List on desktop
- blocks: bugzilla query
- depends on: bugzilla query
Bug triage +/- for verification
[QE-VERIFY+] Bug 1137231 - Double Separator In Bookmarks Menu
- ↳ 2015-02-26: most likely caused by Bug 112440
[QE-VERIFY+] Bug 1124647 - [ReadingList] Implement reading list controls in the reader mode control bar
- ↳ 2015-03-06: qe-verify+ granted by Florin Mezei
[QE-VERIFY+] Bug 1142217 - Make the Add/Remove to Reading List toggle button of the reader view actually work
[QE-VERIFY+] Bug 1134497 - Implement infopanels for in-product FTU Reading List/View tour
[QE-VERIFY+] Bug 1134507 - Implement infopanels to promote Reader View / Reading List when first seen
[QE-VERIFY+] Bug 1143388 - Clicking reading list item from bookmarks menu should open it in reader view
[QE-VERIFY+] Bug 1141353 - Fix ReadingList bookmarks nativemenu on OSX
[QE-VERIFY+] Bug 1144380 - selected/active state on reading list sidebar persists after switching tabs
[QE-VERIFY+] Bug 1144377 - hover state on reading list sidebar items remains after the mouse leaves the list
[QE-VERIFY+] Bug 1144774 - Add to reading list button is blurry
[QE-VERIFY+] Bug 1144781 - Add to reading list button sometimes doesn't hide when mouseing out from the urlbar
[QE-VERIFY+] Bug 1144823 - reading list sidebar should be sorted by most recently added
[QE-VERIFY+] Bug 1144382 - title text in reading list sidebar item wraps poorly, leaving partial text visible
[QE-VERIFY+] Bug 1145372 - The add to reading list url bar button is larger than the other url bar buttons
[QE-VERIFY+] Bug 1145409 - Show the site favicon for each reading list item
Bug fix verification
[VERI] Bug 1124400 - [ReadingList] Add section to bookmarks popup for reading list items and actions
- ↳ 2015-02-25: verified fixed on 39.0a1, across platforms; follow ups to be made according to Comment 12
- ↳ 2015-03-02: verified fixed on 38.0a1, across platforms
[NEW] Bug 1133611 - [ReadingList] Add promo blurb to ReadingList bookmarks menu
[NEW] Bug 1133610 - [ReadingList] Add promo blurb to ReadingList sidebar
[VERI] Bug 1133485 - Hook up "Add to ReadingList" button in desktop ReaderMode
- ↳ 2015-03-23: verified fixed on 38.0a1, across platforms
[NEW] Bug 1131461 - [ReadingList] Show status text in the URLBar when interacting with the ReadingList button
[NEW] Bug 1131460 - [ReadingList] Add-to-readinglist button in the URLbar should have an animation similar to the bookmarking animation
[NEW] Bug 1131457 - [ReadingList] Add a button to the URLBar that allows adding the current page to the Reading List
[VERI] Bug 1123517 - [ReadingList] Implement basic sidebar that lists unread ReadingList items
- ↳ 2015-02-25: verified fixed on 39.0a1 and 38.0a2, across platforms; potential issues reported
[VERI] Bug 1133662 - [ReadingList] Show something useful in the sidebar when the Reading List is empty
- ↳ 2015-03-20: verified fixed on 39.0a1, across platforms
[NEW] Bug 1124153 - [ReadingList] Make the list of items in the Reading List sidebar scale well
[VERI] Bug 1123525 - [ReadingList] Allow deleting items via the Reading List sidebar
- ↳ 2015-03-20: verified fixed on 39.0a1, across platforms
[NEW] Bug 1123519 - [ReadingList] Allow adding items to the Reading List via the sidebar
[VERI] Bug 1123518 - [ReadingList] Add thumbnails/favicons to items in the Reading List sidebar
- ↳ 2015-03-23: verified fixed on 39.0a1, across platforms; Bug 1146348 was filed as a follow-up
[VERI] Bug 1137231 - Double Separator In Bookmarks Menu when ReadingList is disabled
- ↳ 2015-03-02: verified fixed on 39.0a1 and 38.0a2, across platforms
[VERI] Bug 1124647 - [ReadingList] Implement reading list controls in the reader mode control bar
- ↳ 2015-03-09: verified fixed on 39.0a1 and 38.0a2, across platforms
[VERI] Bug 1133489 - Hook up "Open ReadingList" button in desktop ReaderMode
- ↳ 2015-03-09: verified fixed on 39.0a1 and 38.0a2, across platforms
[VERI] Bug 1142217 - Make the Add/Remove to Reading List toggle button of the reader view actually work
- ↳ 2015-03-16: verified fixed on 39.0a1, across platforms
[VERI] Bug 1144382 - title text in reading list sidebar item wraps poorly, leaving partial text visible
- ↳ 2015-03-23: verified fixed on 39.0a1, across platforms
[VERI] Bug 1144380 - active state on reading list sidebar persists after switching tabs
- ↳ 2015-03-23: verified fixed on 39.0a1, across platforms
[VERI] Bug 1144823 - reading list sidebar should be sorted by most recently added
- ↳ 2015-03-23: verified fixed on 39.0a1, across platforms
[VERI] Bug 1145372 - The add to reading list url bar button is larger than the other url bar buttons
- ↳ 2015-03-23: verified fixed on 39.0a1, across platforms
[VERI] Bug 1144680 - The Reading List button from the Location Bar is available for in-content pages
- ↳ 2015-03-23: verified fixed on 39.0a1, across platforms
[VERI] Bug 1144675 - The Reading List button from the Location Bar should have a distinct icon for pages currently in the list
- ↳ 2015-03-23: verified fixed on 39.0a1, across platforms
[VERI] Bug 1144377 - hover state on reading list sidebar items remains after the mouse leaves the list
- ↳ 2015-03-23: verified fixed on 39.0a1, across platforms
[VERI] Bug 1144781 - Add to reading list button sometimes doesn't hide when mouseing out from the urlbar
- ↳ 2015-03-23: verified fixed on 39.0a1, across platforms, via Bug 1144675
[VERI] Bug 1145911 - Enable reading list by default
- ↳ 2015-03-24: verified fixed on 39.0a1, across platforms
Logged bugs
Bug 1137089 - The Reading List sidebar header is white on the developer edition theme
- ↳ 2015-02-26: found while verifying the fix pushed for Bug 1123517
- ↳ 2015-02-27: this is actually an issue affecting all sidebars on the Developer Edition theme, so it's not strictly related to the Reading List feature
Bug 1143689 - Longer item titles from the Reading List are displayed as cutoff in the sidebar
- ↳ 2015-03-16: found while verifying the fix pushed for Bug 1142217
Bug 1144666 - There's no separator displayed in the Location Bar between the Reading List and Reader View buttons
- ↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
Bug 1144668 - Clicking the Add to Reading List button from the Location Bar does not trigger the Show Bookmarks icon animation
- ↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
- ↳ 2015-03-19: marked as DUPE of Bug 1131460 by Tim Nguyen
Bug 1144675 - The Reading List button from the Location Bar should have a distinct icon for pages currently in the list
- ↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
Bug 1144680 - The Reading List button from the Location Bar is available for in-content pages
- ↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
Bug 1144684 - The status of Reading List's button from the Location Bar is not syncing with the one from Reader View's controls
- ↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
Bug 1144691 - Windows displays a blue glow hover effect for the Reading List button in the Location Bar
- ↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
Bug 1144702 - The Reading List button from the Location Bar is not adapting to high contrast and Developer Edition themes
- ↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
- ↳ 2015-03-23: closed as DUPE of Bug 1134658, by Gavin Sharp
Bug 1144716 - Toggling the Reading List button from the Location Bar of private windows throws errors in the Browser Console
- ↳ 2015-03-18: found while verifying the fix pushed for Bug 1131457
- ↳ 2015-03-23: closed as WFM, as the error is no longer reproducible on 39.0a1
Bug 1145565 - The "Show Reading List Sidebar" button from the bookmarks menu should be visible at all times
- ↳ 2015-03-20: found while running Reading List test cases
Bug 1145609 - The Reading List item removal button should have the same tooltip as the one from the Location Bar
- ↳ 2015-03-20: found while verifying the fix pushed for Bug 1123525
Sign off
Criteria
- All the test cases were executed.
- All the blocker, critical, major bugs have been fixed.
Results
Merge to Aurora Sign-off
- [DONE] Aurora 38 (2015-02-27).
- There were no blockers found while testing this feature.
- Manual testing covered Windows 7 (x64), Windows 8.1 (x86), Ubuntu 14.04 (x64) and Mac OS X 10.9.5.
Merge to Beta Sign-off
- [ON TRACK] Beta 38 (2015-03-30).
- MozTrap Test Run: results
Merge to Release Sign-off
- [ON TRACK] RC 38.0 (date).