Releases/Firefox 5b1/Test Plan:Updates

From MozillaWiki
Jump to: navigation, search

Fx5 Channel switcher (not-auto)

Qualifying 5.0b1

Note: MUs will not be offered to FF3.7a1,a2 because of a change in size of billboard. MUs will be offered to FF3.7a3/4/5 and to FF4.0b* users --> 5.0b1.

Relevant Bugs

https://bugzilla.mozilla.org/show_bug.cgi?id=653636 is the main bug for Billboard Testing

https://bugzilla.mozilla.org/show_bug.cgi?id=654343 is a won't fix, so some users will not get the graphical version of the billboard update.

First Round MU testing

First round of testing performed by marcia and geo on 2011-05-02

linux (geo)

  • 3.7a5 (placeholder) disabled addon: Ubuntu Firefox Modifications
  • 4.0b12 (billboard)

linux64 (geo)

  • 3.7a5 (placeholder) disabled addon: Ubuntu Firefox Modifications
  • 4.0b12 (billboard)

mac (10.6) (marcia)

  • 4.0b12 (billboard)

mac (10.5) (marcia)

  • 4.0b12 (billboard)

vista (geo)

  • 3.7a5 (placeholder)
  • 4.0b12 (billboard)

win7_64 (geo)

  • 3.7a5 (placeholder)
  • 4.0b12 (billboard)

winxp (geo)

  • 3.7a5
  • 4.0b12

win7 (time permitting)

  • 3.7a5
  • 4.0b12 (billboard)

win2k (time permitting)

  • 3.7a5
  • 4.0b12

Second Round MU Testing

What we will be checking on en-US only:

  • MU Billboard offered on <betatest> for 3.7a5 (text-only)
  • MU Billboard offered on <betatest> for 4.0b12 (graphical)
  • MU Billboard NOT offered on <betatest> for 3.7a1
  • MU installs for 3.7a5 and 4.0b12 on <betatest>
  • Test Pilot compatibility during and post update (should update to 1.1.1, display all studies)

User studies will not work in 1.0.9 due to bug 652193

Please specify if you receive the Graphical Billboard Update or the Text Version. Due to bug 654343, 3.7a3 thru to 4.0b6 will not get the Graphical Billboard update.

MUs will not be offered to FF3.7a1,a2 because of a change in size of billboard. MUs will be offered to FF3.7a3/4/5 and to FF4.0b* users --> 5.0b1.


linux (ashughes)

  • 3.7a1 en-US: PASS (see notes)
    • Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.3a1) Gecko/20100208 MozillaDeveloperPreview/3.7a1
    • Mozilla/5.0 (X11; Linux i686; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
    • Billboard: Text
    • Test Pilot: PASS
      • Feedback 1.1.1 installed after updating Firefox
    • Notes:
      • Update offered was labeled "Mozilla Developer Preview 4.0.1"
  • 3.7a5 en-US: PASS (see notes)
    • Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.3a5) Gecko/20100610 MozillaDeveloperPreview/3.7a5
    • Mozilla/5.0 (X11; Linux i686; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Billboard: Text
    • Test Pilot: PASS
      • Feedback 1.0.9 installed after updating Firefox, manually updates to 1.1.1
    • Notes:
      • Update offered was labeled "Mozilla Developer Preview 5"
  • 4.0b12 en-US: PASS
    • Mozilla/5.0 (X11; Linux i686; rv:2.0b12) Gecko/20100101 Firefox/4.0b12
    • Mozilla/5.0 (X11; Linux i686; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Billboard: Graphical
    • Test Pilot: PASS
      • Feedback 1.0.6 auto-updates to 1.0.9 after updating Firefox, manually updates to 1.1.1

linux64 (ashughes)

  • 3.7a1 en-US i686: PASS
    • Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.3a1) Gecko/20100208 MozillaDeveloperPreview/3.7a1
    • Mozilla/5.0 (X11; Linux i686 on x86_64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
    • Billboard: Text
    • Test Pilot: Feedback 1.1.1 installed after updating Firefox
    • Notes:
      • Update offered labeled "Mozilla Developer Preview 4.0.1"
  • 3.7a4 en-US i686: PASS
    • Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.3a4) Gecko/20100407 MozillaDeveloperPreview/3.7a4
    • Mozilla/5.0 (X11; Linux i686 on x86_64; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Billboard: Text
    • Test Pilot: Feedback 1.1.1 installed after updating Firefox
    • Notes:
      • Update offered labeled "Mozilla Developer Preview 5.0"
  • 4.0b9 en-US x86_64: PASS
    • Mozilla/5.0 (X11; Linux x86_64; rv:2.0b9) Gecko/20100101 Firefox/4.0b9
    • Mozilla/5.0 (X11; Linux x86_64; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Billboard: Graphical
    • Test Pilot: Feedback 1.0.6 updated to 1.0.9 with Firefox, manually updates to 1.1.1

mac (10.6) (marcia)

  • 3.7a4: PASS
    • Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.3a4) Gecko/20100407 MozillaDeveloperPreview/3.7a4
    • Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Billboard: Text
    • Test Pilot: PASS
    • Notes
      • Feedback 1.0.9 after update, manually updated to 1.1.1
  • 4.0b6 en-US: PASS
    • Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b6) Gecko/20100101 Firefox/4.0b6
    • Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Billboard: Text
    • Test Pilot: PASS
      • Feedback 1.0.3 -> 1.0.9, then had to manually update to 1.1.1 after 5.0 update applied

mac (10.5.8) (marcia)

  • 3.7a5 PASS
    • Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.3a5) Gecko/20100610 MozillaDeveloperPreview/3.7a5
    • Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Billboard: Text
    • Test Pilot: PASS
    • Notes
      • Crash after the update is being applied. Able to restart successfully (Apple Crash Reporter)
      • TP not installed in the alpha. After update goes to 1.0.9. Manually updated to 1.1.1
  • 4.0b11 PASS
    • Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:2.0b11) Gecko/20100101 Firefox/4.0b11
    • Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Billboard: Graphical
    • Test Pilot: PASS
    • Notes
    • Feedback 1.0.4->1.0.9 after update. Had to manually update to 1.1.1

vista (geo)

  • 3.7a1 FAIL, Test Pilot PASS
    • Got unexpected update: bug 654572
    • nthomas commented that a1/a2 snippets were left in betatest, shouldn't happen in beta
    • Feedback 1.0.9 after update; user studies did not load. Searched for ext. updates, got 1.1.1, user studies loaded.
  • 4.0b10 PASS, Test Pilot PASS
    • Mozilla/5.0 (Windows NT 6.0; rv:2.0b10) Gecko/20100101 Firefox/4.0b10
    • Mozilla/5.0 (Windows NT 6.0; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Graphical billboard
    • Feedback 1.0.4 before update, 1.0.9 after update. Studies did not load. Manual update found 1.1.1, user studies loaded

win7_64 (geo)

  • 3.7a5 PASS Test Pilot PASS
    • Mozilla/5.0 (Windows; U; Windows NT 6.1; WOW64; en-US; rv:1.9.3a5) Gecko/20100610 MozillaDeveloperPreview/3.7a5
    • Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Placeholder text
    • Feedback 1.0.9 after update; user studies did not load. Searched for ext. updates, got 1.1.1, user studies loaded.
  • 4.0b11 PASS Test Pilot PASS
    • Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b11) Gecko/20100101 Firefox/4.0b11
    • Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Graphical billboard
    • Feedback 1.0.9 after update; user studies did not load. Searched for ext. updates, got 1.1.1, user studies loaded.

winxp (geo)

  • 3.7a5 PASS Test Pilot PASS
    • Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a5) Gecko/20100610 MozillaDeveloperPreview/3.7a5
    • Mozilla/5.0 (Windows NT 5.1; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Placeholder text
    • Feedback 1.0.9 after update; user studies did not load. Searched for ext. updates, got 1.1.1, user studies loaded.
  • 4.0b6 PASS Test Pilot PASS
    • Mozilla/5.0 (Windows NT 5.1; rv:2.0b6) Gecko/20100101 Firefox/4.0b6
    • Mozilla/5.0 (Windows NT 5.1; rv:5.0b1) Gecko/20100101 Firefox/5.0
    • Placeholder text
    • Feedback 1.0.9 after update; user studies did not load. Searched for ext. updates, got 1.1.1, user studies loaded.

win7 (time permitting)

  • 3.7a4
  • 4.0b3

win2k (time permitting)

  • 3.7a4
  • 4.0b5

Live testing on Beta Channel

Automation

Ran automation on yesterday's test plan against beta channel, everything passed fine.

win vista (marcia)

  • 4.0b4 - 'PASS

win7xp (marcia)

  • 3.7a5 - PASS
    • Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.3a5) Gecko/20100610 MozillaDeveloperPreview/3.7a5
    • Mozilla/5.0 (Windows NT 5.1; rv:5.0b1) Gecko/20100101 Firefox/5.0
  • 4.0b11 - PASS
    • Mozilla/5.0 (Windows NT 5.1; rv:2.0b11) Gecko/20100101 Firefox/4.0b11
    • Mozilla/5.0 (Windows NT 5.1; rv:5.0b1) Gecko/20100101 Firefox/5.0

win7 (marcia)

  • 3.7a4
    • Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.3a4) Gecko/20100407 MozillaDeveloperPreview/3.7a4
    • Mozilla/5.0 (Windows NT 6.1; rv:5.0b1) Gecko/20100101 Firefox/5.0
  • 4.0b3
    • Mozilla/5.0 (Windows; Windows NT 6.1; rv:2.0b3) Gecko/20100805 Firefox/4.0b3
    • Mozilla/5.0 (Windows NT 6.1; rv:5.0b1) Gecko/20100101 Firefox/5.0

mac (10.6) (owen)

Note: got verbal everything works, didn't think to copy user agents during testing.

  • 3.7a1 (no update to 5.0b1, will likely update to 4.0.1 instead)
  • 4.0b1
  • 4.0b10
  • 4.0b12

linux (Owen) (ubuntu x64)

Note: got verbal everything works, didn't think to copy user agents during testing.

  • 3.7a3 <-- No x64 build, x32 build worked.
  • 4.0b6

Bugs filed during Test Cycle

  • bug 654572 Getting MU offered from 3.7a1 to 5.0b1 with broken billboard
Retrieved from "https://wiki.mozilla.org/index.php?title=Releases/Firefox_5b1/Test_Plan:Updates&oldid=305615"