QA/Loop/E2E

Summary

This page documents the test strategy of verifying Firefox Hello is working end-to-end.

Meetings

Discussions of end-to-end test coverage are on-going in the weekly Loop QA meeting, Thursdays at 13:00 Pacific. As this meeting may involve partner specific information it is not always publicly accessible. Please contact Anthony Hughes to request an invitation.

Strategy

Firefox OS

Owner: Tony Chung

Status: [link to weekly report]

Tasks:

• once per week, check in with partner for a status report on testing/blockers

Dependencies:

• bug 1076708
• bug 1079974
• bug 1067437

Important Dates:

• 2014-09-29: first testrun complete on Flame 2.0, regression pass on Fire E device
• 2014-10-20: Desktop -> FxOS client testing

Server

Owner: Richard Pappalardo

Loop Client

• Verify the server-side code matches the deployment
• Spotcheck 1:1 calling has not regressed
• Spotcheck Terms of Service & Privacy Policy continues to link correctly

Loop Server

• Verify the server-side code matches the deployment
• Spotcheck 1:1 calling has not regressed
• Spotcheck Terms of Service & Privacy Policy continues to link correctly

Simplepush

Firefox Accounts

Scalability

Desktop

Owner: Anthony Hughes

Platform

Owner: Nils Ohlmeier

• Subset of WebRTC platform tests running across different OS’s, build versions, network scenarios

Automation

Owner: Nils Ohlmeier

• [ON TRACK] 1:1 calling via marionette (trello)
• Integrating platform WebRTC tests into treeherder
• Investigate getting Softvision/A-team resources to help with test development
• Investigate getting Stephen Donner's support in on-device testing
• Set up a process for report monitoring and failure escalation

Next Steps

Anthony Hughes

• document a list of what needs to be checked for each server/client/product deployment
• develop a list of what needs to be automated for each server/client/product deployment
• clarify roles, expectations, division of labour for Firefox OS
• create a weekly status update page for Firefox OS
• consider inviting Massimo Barone to the QA meeting
• confirm with Shell what B2G 2.0/2.1 exist and whether they're Mozilla's to fix before Loop release
• Find a Softvision resource to assist with test development

Nils Ohlmeier

• Review dmose's trello dashboard to make sure it’s a complete list of work to be completed

Richard Pappalardo

• Review dmose's trello dashboard to make sure it’s a complete list of work to be completed
• Investigate feasibility of developing a middleware test using a fake Loop client to test server APIs

Syd Polk

• review and escalate gaps in current coverage to Anthony Hughes

Tony Chung

• Document on-going status of Firefox OS coverage