QA/Loop/E2E

Summary

This page documents the test strategy of verifying Firefox Hello is working end-to-end.

Meetings

Discussions of end-to-end test coverage are on-going in the weekly Loop QA meeting, Thursdays at 13:00 Pacific. As this meeting may involve partner specific information it is not always publicly accessible. Please contact Anthony Hughes to request an invitation.

Strategy

Firefox OS

Owner: Tony Chung

Status: [link to weekly report]

Tasks:

• once per week, check in with partner for a status report on testing/blockers

Dependencies:

• bug 1076708
• bug 1079974
• bug 1067437

Important Dates:

• 2014-09-29: first testrun complete on Flame 2.0, regression pass on Fire E device
• 2014-10-20: Desktop -> FxOS client testing

Server

Owner: Richard Pappalardo

Loop Client

• Verify the server-side code matches the deployment
• Spotcheck 1:1 calling has not regressed
• Spotcheck Terms of Service & Privacy Policy continues to link correctly

Loop Server

• Verify the server-side code matches the deployment
• Spotcheck 1:1 calling has not regressed
• Spotcheck Terms of Service & Privacy Policy continues to link correctly

Simplepush

Firefox Accounts

Scalability

Desktop

Owner: Anthony Hughes

Platform

Owner: Nils Ohlmeier

• Subset of WebRTC platform tests running across different OS’s, build versions, network scenarios

Automation

Owner: Nils Ohlmeier

• [ON TRACK] 1:1 calling via marionette (trello)
• Integrating platform WebRTC tests into treeherder
• Investigate getting Softvision/A-team resources to help with test development
• Investigate getting Stephen Donner's support in on-device testing
• Set up a process for report monitoring and failure escalation

Next Steps

Task	Description	Owner	Status
1	Review dmose's trello dashboard to make sure it’s a complete list of work to be completed	Nils Ohlmeier & Richard Pappalardo	[ON TRACK]
2	Find a Softvision resource to assist with test development	Anthony Hughes	[BLOCKED] depends on #1
3	Investigate feasibility of developing a middleware test using a fake Loop client to test server APIs	Richard Pappalardo	[BLOCKED] depends on #1
4	Document a list of what needs to be checked for each server/client/product deployment	Anthony Hughes	[ON TRACK]
5	Develop a list of what needs to be automated for each server/client/product deployment	Anthony Hughes	[BLOCKED] depends on #4
6	Document on-going status of Firefox OS coverage	Tony Chung	[ON TRACK]

• Syd Polk
  • review and escalate gaps in current coverage to Anthony Hughes
• Anthony Hughes:
  • clarify roles, expectations, division of labour for Firefox OS
  • create a weekly status update page for Firefox OS
  • consider inviting Massimo Barone to the QA meeting
  • confirm with Shell what B2G 2.0/2.1 exist and whether they're Mozilla's to fix before Loop release