QA/Fennec/iOS/iOS Backup&Restore Issues

Overview

Purpose

The purpose of this wiki is to serve as a general test plan for testing the improvement and resolved iOS Backup&Restore issues.

Ownership

Developer contact: Justin D'Arcangelo (:justindarc)

QA: Catalin Suciu (:csuciu)

Overall Status

Nightly[LANDED]
Beta[NOT LANDED]

Testing Summary

Scope of testing

The testing will be focus on:

• Verifying that the Pushbox notifications are received when a tab is received
• Verifying that the sent tabs are opened when tapping on a Pushbox notification

Specifications

• None

User stories

• None

Acceptance Criteria

• Migration to a new iPhone from backup does not affect sync

Prep: - Signed into FxA and enabled Firefox Sync on "iPad A" (contained saved logins) - Not signed into FxA on "iPad B" (contained saved logins) - iTunes Backups were performed using iTunes 12.7.4.76 on macOS High Sierra 10.13.4 - iPads were wiped to factory state using "Restore iPad..." button in iTunes

 - This process downloaded the iOS 11.4 image and flashed it to wipe the device to its original "on-boarding" state

- Both iPads were running iOS 11.4

Backups: a.) Backed up "iPad A" to un-encrypted iTunes Backup

 - Clicked "Back Up Now" button in iTunes with "Encrypt local backup" un-checked

b.) Backed up "iPad A" to encrypted iTunes Backup

 - Clicked "Back Up Now" button in iTunes with "Encrypt local backup" checked

c.) Backed up "iPad A" to iCloud (with iCloud Keychain disabled)

 - Tapped "Back Up Now" button in Settings app under Accounts & Passwords > iCloud > iCloud Backup

d.) Backed up "iPad A" to iCloud (with iCloud Keychain enabled)

 - Tapped "Back Up Now" button in Settings app under Accounts & Passwords > iCloud > iCloud Backup

e.) Backed up "iPad B" to iCloud (with iCloud Keychain enabled)

 - Tapped "Back Up Now" button in Settings app under Accounts & Passwords > iCloud > iCloud Backup

Restores: a.) Restored "iPad B" with un-encrypted iTunes Backup of "iPad A"

 - Clicked "Restore iPad..." button in iTunes to factory reset the device, then selected the un-encrypted iTunes Backup to restore after reset

b.) Restored "iPad B" with encrypted iTunes Backup of "iPad A"

 - Clicked "Restore iPad..." button in iTunes to factory reset the device, then selected the encrypted iTunes Backup to restore after reset

c.) Restored "iPad B" with iCloud Backup (with iCloud Keychain disabled) of "iPad A"

 - Clicked "Restore iPad..." button in iTunes to factory reset the device, then unplugged and followed the on-boarding on the device to restore the iCloud Backup

d.) Restored "iPad B" with iCloud Backup (with iCloud Keychain enabled) of "iPad A"

 - Clicked "Restore iPad..." button in iTunes to factory reset the device, then unplugged and followed the on-boarding on the device to restore the iCloud Backup
 - After restoring the device, re-enabled iCloud Keychain in Settings (it was off after restoring)

e.) Restored "iPad B" with iCloud Backup (with iCloud Keychain enabled) of "iPad B"

 - Clicked "Restore iPad..." button in iTunes to factory reset the device, then unplugged and followed the on-boarding on the device to restore the iCloud Backup
 - iCloud Keychain was enabled automatically after restoring (presumably because the backup was of the same original device)

Results: a.) BROKEN - Restoring un-encrypted iTunes Backup to new device

 - History tab showed "Synced Devices - 4 device(s) connected" even though FxA appeared to not be logged in
 - Viewing saved logins in "Settings" resulted in an empty list and an infinite spinner
 - Was able to log into FxA successfully, but Firefox Sync was broken (as described by this bug)
   - The sync appears to complete quickly without error
   - Nothing new is synced and nothing appears under "Synced Devices"

b.) WORKED - Restoring encrypted iTunes Backup to new device

 - FxA was logged in
 - Firefox Sync was working
 - Saved logins were present in "Settings"

c.) BROKEN - Restoring iCloud Backup (with iCloud Keychain disabled) to new device

 - History tab showed "Synced Devices - 4 device(s) connected" even though FxA appeared to not be logged in
 - Viewing saved logins in "Settings" resulted in an empty list and an infinite spinner
 - Was able to log into FxA successfully, but Firefox Sync was broken (as described by this bug)
   - The sync appears to complete quickly without error
   - Nothing new is synced and nothing appears under "Synced Devices"

d.) BROKEN - Restoring iCloud Backup (with iCloud Keychain enabled) to new device

 - History tab showed "Synced Devices - 4 device(s) connected" even though FxA appeared to not be logged in
 - Viewing saved logins in "Settings" resulted in an empty list and an infinite spinner
 - Was able to log into FxA successfully, but Firefox Sync was broken (as described by this bug)
   - The sync appears to complete quickly without error
   - Nothing new is synced and nothing appears under "Synced Devices"

e.) WORKED - Restoring iCloud Backup (with iCloud Keychain enabled) to same device

 - FxA was logged in
 - Firefox Sync was working
 - Saved logins were present in "Settings"

Testing details

• Date: 10/05/2018
• Build:
• Device:
• Display:
• iOS:
  • All test cases executed:[Yes] - Whats new page N/A
    • New issues: None
    • Verified issues: None
    • Re-opened issues: None

Tracking bugs

• bug 1421082 Reports of Sync Not Working After Successful Login

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

Signoff criteria

• All test cases are executed on devices with the following display sizes: 4", 5", 5.5", 7", 9", 12.9"
  
• All test cases are executed on devices with iOS 10.x and iOS 11.x
  
• All blockers, criticals must be fixed and verified or have an agreed-upon timeline for being fixed