Overview

Purpose

The purpose of this wiki is to serve as a general test plan for verifying that there are no errors caused by the gradle tools migration and build configuration.

Ownership

Developer contacts:

• Nick Alexander
• Max Liu

QA contacts:

• Bogdan Surd
• Sorina Florean

Overall Status

[LANDED] Nightly 58:

[LANDED] Beta 58:

Testing Summary

• Verifying that Gradle builds do not affect any major functionality of FF.

Scope of testing

The testing will be focus on ensuring that:

• upgrading over top works
• we don't lose bookmarks
• we don't lose home screen items
• we can still share different types of content to Fennec
• send tabs
• FxA are not lost on upgrade
• Sync remains functional
• Web Push works
• Tab Queue appears functional
• verify CrashReporter and crash symbols are working
• Custom Tabs functionality
• LeanPlum Integration

Environments

Testing will be performed on following devices:

• Phones
  • 5.x - Huawei Honor 5X, Samsung Galaxy Note 4
  • 6.x - Nexus 5, Oneplus Two, Huawei P9 Lite, Sony Xperia Z5 Premium
  • 7.0 - HTC 10, Samsung Galaxy S6, Huawei P10
  • 8.0 - Nexus 6P, Google Pixel
• Tablets
  • 5.1.1 - Huawei MediaPad M2
  • 6.0.1 - Asus ZenPad 8.0 Z380KL
  • 7.0 - Samsung Galaxy Tab 3

Acceptance Criteria

• Gradle Builds do not break any major functionality in Fennec

Testing details

Test Cases

• Gradle Build Transition

Testing days

Date: 12-10-2017

• Created Test Plan and Test Cases

Date: 25-10-2017

• Full test suite:
  • https://docs.google.com/spreadsheets/d/112zRf_uOnbHT5ykGu7StnK0hdybxa30o6SXtJLqkxko/edit#gid=0
  • new bug 1411608 No Pocket stories in Gradle Builds

Date: 26-10-2017

• Tested Leanplum push notification and active dialog on Gradle build

Date: 03-11-2017

• Performed full suite of test cases for Pre-Beta SignOff
  • Status:Green
• https://testrail.stage.mozaws.net/index.php?/reports/view/681

Date: 08-01-2018

• Performed full suite of test cases for Pre-Release SignOff with Samsung Galaxy Note 4 (Android 5.0.1), HTC 10 (Android 7.0), Nexus 5 (Android 6.0.1), Samsung Galaxy Tab S3 (Android 7.0)
• Status Green so far, exept LeanPlum tests witch are scheduled for testing on 9/01.

Bug Work

Tracking bugs

• Meta bug
  • bug 1405376 - (gradle-automation-v1) [meta] Use Gradle to build Firefox for Android in automation (v1 -- work for initial landing)

4 Total; 0 Open (0%); 4 Resolved (100%); 0 Verified (0%);

Signoff

Criteria

• All test cases should be executed
• All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed

Results

Nightly testing

• feature status GREEN

Beta testing

• in progress