Releases/Firefox 28/Test Plan: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 280: Line 280:
  |-
  |-
  | [https://bugzilla.mozilla.org/show_bug.cgi?id=936317 936317]
  | [https://bugzilla.mozilla.org/show_bug.cgi?id=936317 936317]
  | <span style="color:grey">'''VERIFIED'''</span>
  | <span style="color:green">'''VERIFIED'''</span>
  |-
  |-
  | [https://bugzilla.mozilla.org/show_bug.cgi?id=930588 930588]
  | [https://bugzilla.mozilla.org/show_bug.cgi?id=930588 930588]

Revision as of 09:15, 13 December 2013

« Firefox 27 | Firefox 28 | Firefox 29 »

Firefox 28 Test Plan

QA Lead: Anthony Hughes

Schedule

  • 2013.10.28: Nightly
  • 2013.12.09: Aurora
  • 2014.01.17: Pre-Beta feature sign-offs
  • 2014.01.21: Beta 1
  • 2014.01.28: Beta 2
  • 2014.01.31: Beta 3
  • 2014.02.04: Beta 4
  • 2014.02.07: Beta 5
  • 2014.02.11: Beta 6
  • 2014.02.14: Beta 7
  • 2014.02.18: Beta 8
  • 2014.02.21: Beta 9
  • 2014.02.25: Beta 10
  • 2014.03.02: Release Candidate
  • 2014.03.03: Final

Nightly

Features

Feature Tracking

Tentative

Sign-off Criteria
  • All user stories covered by manual or automated tests
  • Manual testing complete with no blocking regressions
  • Automated testing complete with no blocking regressions
  • 0 untriaged unconfirmed bugs
  • 0 untriaged unverified fixes
Status
Feature Owner Documentation Tracked Bugs Aurora Merge Sign-off Pre-beta Sign-off
Australis Paul Silaghi, Cornel Ionce Wiki page Disabled NA
Shumway Paul Silaghi, Bogdan Maris Wiki page
Thumbnail Service Alexandra Lucinet Wiki page fixed bugs
Input type=color Manuela Muntean, Paul Silaghi Wiki page 1 bug
Session Restore Cornel Ionce fixed bugs
Disallow enumeration of navigator.plugins Catalin Varga 1 bug
MP4 Support No need for QA owner 1 bug NA
Enable WEBGL_draw_buffers No need for QA owner 1 bug NA NA
Port DOM Promises to Workers No need for QA owner - covered automatically 1 bug NA NA
Notes
  • Sign-off work should be done on Holly builds (nightly without Australis).

How to Help

Fixed Bug Verification

  • Download and install the latest Firefox Nightly
  • Load this list of bug reports
  • Read each bug report until you find one you understand and want to test
  • Download and install the version of Firefox used to report the bug
  • Using the older build, test the claim made in the bug report to see if you can reproduce it
    • If you cannot reproduce the bug move on to another bug or ask the reporter to verify the fix
  • Using the latest Nightly, test to see if you can reproduce the bug
    • If you cannot reproduce the bug set the STATUS to VERIFIED FIXED and add a comment describing your testing
    • If you can still reproduce the bug set the STATUS to REOPENED and add a comment describing your testing
  • If you need help, send email to Anthony Hughes with the bug summary in the Subject field

Reported Bugs

  • Download and install the latest Firefox Nightly
  • Load this list of bug reports
  • Read each bug report until you find one you understand and want to test
  • Test the claim made in the bug report and see if you can reproduce it
  • If you can reproduce the bug, set the STATUS to NEW, add a comment describing your testing, and try to find a regression window
  • If you need help, send email to Anthony Hughes with the bug summary in the Subject field

Stability Bugs

  • Review the Firefox 28 Explosiveness Report
    • Any signatures rising significantly (>= 2.0 explosiveness rating) without a bug number should be reported to Bugzilla and nominated for tracking
    • If there is already a bug report, update the report with the latest numbers
    • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
    • Ensure the bug is listed below using the bug number template
  • Review the Firefox 28 Topcrash Report
    • Any signature in the top 10 without a bug number should be reported to Bugzilla and nominated for tracking
    • If there is already a bug report, update the report with the latest ranking information
    • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
    • Ensure the bug is listed below using the bug number template
  • Review the Firefox 28 tracked bugs that aren't tracked in 27 or 26.
    • Each bug listed needs to have its rank updated once per week
    • Use the topcrash reports from crash-stats to get the rank for each branch
    • Add this information to the bug report
    • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
    • Conversely, remove the topcrash keyword if the crash is no longer in the top-10
  • For each of these tasks, ensure the etherpad is updated with your name and the date checked

QA Wanted

  • Download and install the latest Firefox Nightly
  • Review this list of bug reports to find a bug you both understand and want to test (each should contain a comment detailing what's needed from QA)
    • If the bug needs steps to reproduce, try to find a reproducible set of steps
    • If the bug needs a regression window, try to find a regression window
    • If the bug is fixed and needs verification, test to confirm you can reproduce the bug on an older build but not the latest build
  • Any tested bugs should be added to the list below using the bug number template
  • Remove the qawanted keyword from any fulfilled request
  • If you need help, send email to Anthony Hughes with the bug summary in the Subject field

Testdays

Aurora

Branch Migration

Tasks

  • [DONE] Updates verified throttled on aurora channel (QA)
  • [DONE] Set mozmill-aurora update tests to auroratest channel in MM-CI
  • Automation daily testruns reviewed for failures
  • Features qualified for uplift to Aurora by feature owners
  • All flagged bug fixes verified
  • All unconfirmed bugs triaged
  • Topcrash reports reviewed for stability concerns
  • Sign-off and verify updates on the aurora channel

Automated Testing

Review the following reports and file bugs if any failures are found:

Feature Sign-offs

Confirm that all enabled features pass smoketests for primary user stories and that disabled features are disabled. Update the status of the feature below as follows:

  • SIGNED-OFF if the feature is enabled and passes all tests
  • DISABLED if the feature is disabled and verified disabled
  • FAILED if serious issues were encountered related to the feature

Additionally, sign-off any features not listed below tracked in this list

Feature Owner Status
Australis Cornel Ionce DISABLED
Shumway Bogdan Maris DISABLED
Thumbnail Service Alexandra Lucinet NA
<input type=color> Paul Silaghi SIGNED-OFF
Disallow enumeration of navigator.plugins Catalin Varga SIGNED-OFF
Session Restore Improvements Cornel Ionce SIGNED-OFF
Notification center support for web notifications Bogdan Maris
Manifest Editor Petruta Rasa
VP9 video decoder Andrei Vaida
Gamepad API Alexandra Lucinet
Notes
  • Thumbnail Service was eventually re-enabled in Firefox 27 and it was signed off there. There are no major (nor many) changes for this feature for Firefox 28.

Bug Fix Verifications

  • Triage this list of bugs
  • Attempt to reproduce the bug as described in a known broken build and the latest Aurora build
  • Update the bug with the results of your testing
  • List the bug in the table below with one of the following statuses:
    • VERIFIED if you verified the fix
    • NEED-INFO if you need follow-up on the bug
    • DROPPED if fix is unverifiable
    • REOPENED if you reopened the bug
Bug Status
937789 VERIFIED
917380 VERIFIED
925521 VERIFIED
946484 VERIFIED
936317 VERIFIED
930588 VERIFIED
919076 VERIFIED
930141 VERIFIED
941397 VERIFIED
934155 VERIFIED
934261 VERIFIED
941099 VERIFIED
932176 VERIFIED
941027 VERIFIED
925470 VERIFIED
932361 VERIFIED

Unconfirmed Bug Triage

  • Triage this list of bugs
  • Attempt to reproduce the bug as described
  • Update the bug with the results of your testing
  • List the bug in the table below with one of the following statuses:
    • CONFIRMED if you set the bug to NEW
    • DROPPED if you set the bug to RESOLVED
    • UNCONFIRMED if you left the bug UNCONFIRMED
Bug Status
948205 UNCONFIRMED
935697 DROPPED
945379 UNCONFIRMED
939291 DROPPED
941303 UNCONFIRMED
945175 UNCONFIRMED
947913 CONFIRMED
945086 UNCONFIRMED
945497 UNCONFIRMED
948090 UNCONFIRMED
945500 UNCONFIRMED

Post Sign-off Tasks

  • Updates verified unthrottled on aurora channel (QA)

Beta

Bugs

Potential Blockers

UNTRIAGED: 0 bugs

TRACKED: 1 bugs

  • bug 942427 Australis: No highlights indicated on Windows 7 High Contrast visual style

FIXED: 2 bugs

  • bug 947916 "Couldn't load XPCOM" & missing pgort100.dll after updating Nightly (2013-12-09) [needs verification]
  • bug 944774 Holly is not applying nightly updates on windows [needs verification]

DROPPED: 3 bugs

  • bug 947930 Latest Nightly Firefox Build does not starts on windows platforms [dupe of bug 947916]
  • bug 944565 Browser would not restart after check update from About Holly [dupe of bug 944774]
  • bug 940364 Australis: Nightly not displaying web pages after update [WORKSFORME]

Issues Found in Testing

UNTRIAGED: 0 bugs

TRACKED: 0 bugs

FIXED: 0 bugs

AT RISK: 0 bugs

DROPPED: 0 bugs

Top Crash Bugs

UNTRIAGED: 0 bugs

TRACKED: 12 bugs

  • bug 947240 crash in mozilla::layers::CompositorD3D11::DrawQuad(mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::EffectChain const&, float, mozilla::gfx::Matrix4x4 const&)
  • bug 947213 crash in mozilla::layers::DeprecatedTextureHostBasic::UpdateImpl(mozilla::layers::SurfaceDescriptor const&, nsIntRegion*, nsIntPoint*)
  • bug 941869 `anonymous namespace::ReportErrorRunnable::ReportError(JSContext*, mozilla::dom::workers::WorkerPrivate*, bool, mozilla::dom::workers::WorkerPrivate*, nsString const&, nsString const&, nsString const&, unsigned int, unsigned int, unsigned int, unsign...
  • bug 937220 mozalloc_abort(char const* const) | NS_DebugBreak | js::AtomizeChars(js::ExclusiveContext*, wchar_t const*, unsigned __int64, js::InternBehavior)
  • bug 934533 Nightly Nov-04, crash in gfxContext::gfxContext(mozilla::gfx::DrawTarget*) if gfx.content.azure.enabled = false
  • bug 931794 startup crash in mozilla::dom::WindowBinding::get_content
  • bug 931911 crash in mozilla::a11y::SelectionManager::ProcessSelectionChanged(nsISelection*)
  • bug 883059 crash in mozilla::a11y::EventQueue::PushEvent
  • bug 880140 crash in nsDeviceContextSpecX::EndPage @ decode_data when printing [verifyme]
  • bug 869027 crash in JS_HasPropertyById - fixed, needs verification
  • bug 812683 [Win7] startup crash in _d2d_clear_surface @ CContext::RestorePipelineStateImpl w/ASUS GamerOSD
  • bug 805406 crash in gfxContext::PushClipsToDT with Direct2D 1.1 (d3d11.dll 6.2 or 6.3)
  • bug 767343 crash in nsSupportsStringImpl::SetData with abort message: "###!!! ABORT: OOM: file e:\builds\moz2_slave\m-cen-w32-ntly\build\xpcom\string\src\nsTSubstring.cpp, line 348" [VERIFYME]

FIXED: 2 bugs

  • bug 945308 crash in mozilla::a11y::HyperTextAccessible::GetBoundsInFrame(nsIFrame*, unsigned int, unsigned int)
  • bug 942819 crash in mozilla::layers::DeviceManagerD3D9::Init()


AT RISK: 0 bugs

DROPPED: 0 bugs

QA Wanted

UNTRIAGED: 0 bugs

TRACKED: 0 bugs

FIXED: 0 bugs

AT RISK: 0 bugs

DROPPED: 0 bugs

Verifications

How to Verify a Fix
  • Pick a bug from this list
  • Download a build that is known to reproduce the bug and test to make sure you can see the bug
  • Download a build that is said to be fixed and test to make sure you cannot see the bug
  • If the bug reproduces in the fixed build:
    • set the STATUS field to REOPENED and provide a comment detailing your testing
  • If the bug does not reproduce in the fixed build:
    • set the status flag to verified for the version you tested
    • set the STATUS field to VERIFIED FIXED if the version tested is the same as the TARGET MILESTONE
    • any bugs marked verified will automatically appear in the Verified Fixes section below

UNTRIAGED: 0 bugs

TRACKED: 0 bugs

FIXED: 0 bugs

AT RISK: 0 bugs

DROPPED: 0 bugs

Unconfirmed

Pick a bug from this list and try to reproduce the issue using the information provided. If you reproduce the issue, provide a description of the circumstances, set the status to NEW, and move the bug into a component which matches the issue (ie. a bookmarks issue would go in the Bookmarks component). If you can't reproduce the issue ask the reporter for more information by setting the need-info? flag to their email address.

UNTRIAGED: 0 bugs

TRACKED: 0 bugs

FIXED: 0 bugs

AT RISK: 0 bugs

DROPPED: 0 bugs

Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_28/Test_Plan&oldid=789235"