QA/Fennec/X2N Migration/Beta

From MozillaWiki
< QA‎ | Fennec‎ | X2N Migration
Jump to: navigation, search


X2N Migration Test Plan for Beta

Google Play Update Testing

Before Beta (750819, 747534, and "test scenario 1"):
* RelEng: Verify that a multi-locale XF for tablets can be uploaded alongside many single locale FN builds 
* QA: Determine what build different localizations will receive depending on Firefox/system locale and available builds 
* QA: Verify that FN14 is offered to phone XF13 users, XF14 is offered to tablet XF13 users, and no unexpected issues occur
* QA: Test the update experience for an Android 2.0 XF13 user, now that FN14 no longer supports them

Note: We ran into some issue with posting multiple locales up on Google Play. We will only be testing multi locale XUL fennec to Single locale English Native Fennec for the first test run.

Migration Testing (Google Play is unnecessary)

Before Beta (750819, builds available):
* QA: Verify that XF13 to FN14 migrations (APK over APK) have no new perf/functional/crash regressions across multiple devices

Overview Results

Google Doc Spreadsheet of results

Version to Version Language to Language Results
ESR 10.0.4 XUL to 14 Native - Phone
Single XUL to Single Native En to En
Single XUL to Single Native JP to En
Multi XUL to Single Native Fr to En
13 Beta XUL to 14 Native - Phone
Single XUL to Single Native En to En
Single XUL to Single Native JP to En
Multi XUL to Single Native Fr to En
13 Beta Native to 14 Native - Phone
Single Native to Single Native En to En
14 XUL to 14 Native - Phone
Multi XUL to Single Native Fr to En
Single XUL to Single Native JP to En
Single XUL to Single Native En to En

Detailed Results

Phone

Firefox ESR 10.0.4 (XUL) to Firefox 14 (Native)

  • Single Locale to Single Locale
    • Smoke Test
    • L10N Test
    • Performance
  • Multi Locale to Single Locale
    • Smoke Test
    • L10N Test
    • Performance

Firefox 13 beta (XUL) to Firefox 14 (Native)

  • Single Locale to Single Locale
    • Smoke Test
    • L10N Test
    • Performance
  • Multi Locale to Single Locale
    • Smoke Test
    • L10N Test
    • Performance

Firefox 13 (Native) to Firefox 14 (Native)

  • Single Locale to Single Locale
    • Smoke Test
    • L10N Test
    • Performance

Firefox 14 (XUL) to Firefox 14 (Native)

  • Single Locale to Single Locale
    • Smoke Test
    • L10N Test
    • Performance
  • Multi XUL to Single Locale
    • Smoke Test
    • L10N Test
    • Performance

Tablet Testing

Firefox ESR 10.0.4 (XUL) to Firefox 14 (Native)

  • Single Locale to Single Locale
    • Smoke Test
    • L10N Test
    • Performance
  • Multi Locale to Single Locale
    • Smoke Test
    • L10N Test
    • Performance

Firefox 13 beta (XUL) to Firefox 14 (Native)

  • Single Locale to Single Locale
    • Smoke Test
    • L10N Test
    • Performance
  • Multi Locale to Single Locale
    • Smoke Test
    • L10N Test
    • Performance

Test Plan after Migration

Smoketests

QA Task Person/Group How Often?
Smoketests Once

Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 10.0 Test Suite, and the tablet version here. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs. QA/Fennec/X2N Migration/Beta/smoketests-results Detailed Smoketest results

Exploratory Testing

QA Assignment

QA/Fennec/X2N Migration/Beta/exploratory-results Detailed Exploratory results

Major Bugs Found

Android Permissions Testing

Verify permissions between what is currently on the store and with this beta build candidate. Details : Android Permissions

QA/Fennec/Fennec/X2N Migration/Beta/permissions-results Detailed Android Permissions Testing Results

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec/X2N_Migration/Beta&oldid=427408"