MozillaWiki

QA/Execution/Web Testing/Goals/2013/Q2: Difference between revisions

< QA‎ | Execution‎ | Web Testing
(Created page with "= High-level goals = Taken from https://intranet.mozilla.org/QA/Q22013_QAgoals *'''Goal:''' {{drop|Help support, through automation, the purchase, installation of, and refund ...")
 
 
(27 intermediate revisions by the same user not shown)
Line 1: Line 1:
= High-level goals =
= High-level goals =
Taken from https://intranet.mozilla.org/QA/Q22013_QAgoals
Taken from https://intranet.mozilla.org/QA/Q22013_QAgoals
*'''Goal:''' {{drop|Help support, through automation, the purchase, installation of, and refund of, apps in the Firefox Marketplace, on mobile}}
 
*'''Goal:''' {{new|Continue to and increase automated support for Firefox OS by growing and maintaining a reliable test suite against the leading Gaia branch}}
*'''Goal:''' {{prev|Help support, through automated smoketests, the purchase, installation of, and refund of, apps in the Firefox Marketplace, on Firefox OS}}
*'''Goal:''' {{new|Build out and support additional infrastructure and processes, to be able to}}:
**run Gaia UI Tests against v1-train and v1.0.1 (and v1.1?) concurrently
**run the above on multiple devices (2 soon-to-ship ones)


== Firefox Marketplace ==
== Firefox Marketplace ==
*'''Goal:''' {{drop|Help support, where possible through automation, the purchase, installation, and refund of, apps in the Firefox Marketplace, on mobile}}
*'''Goal:''' {{prev|Help support, where possible through automated smoketests, the purchase, installation, and refund of, apps in the Firefox Marketplace, on Firefox OS}}
** might have to be dropped, due to being feature-incomplete
**'''Dependencies:'''
**'''Dependencies:'''
*** Persona login
*** Marionette's ability to reliably automate the purchase/refund flow(s), end-to-end
*** Infrastructure necessary for testing payments multiple times per day
*** Persona login/logout/account management (in)
*** Infrastructure necessary for testing (live?) payments multiple times per day
**** Testing can be done on [https://marketplace-dev.allizom.org/ marketplace-dev.allizom.org] now, with the "testpay" system
**'''Risks:'''
**'''Risks:'''
*** feature readiness
*** Any of the blocking Marionette bugs listed in [https://bugzilla.mozilla.org/show_bug.cgi?id=801898 bug 801898] - Get GaiaTest UI smoke tests running reliably in Jenkins CI, which could affect our ability to automate
*** feature readiness/stability on the Marketplace-app side
*** stability on the vendor/payment-provider side
*** lack of a "back-door," automation-friendly way to purchase, frequently
*** Marionette's (Firefox OS) ability to automate, end-to-end
*** Marionette's (Firefox OS) ability to automate, end-to-end
**'''Tracking Bugs:'''
*** [https://github.com/mozilla/gaia-ui-tests/issues?labels=marketplace+smoketest&milestone=&page=1&state=open GitHub Issues for Marketplace]


== Firefox OS ==
== Firefox OS ==
*'''Goal:''' {{new|Continue to support and increase end-to-end automation for Firefox OS by growing and maintaining a reliable test suite which runs in a continuous-integration environment (including update testing) against the leading Gaia branch}}
*'''Goal:''' {{new|Build out and support additional infrastructure and processes, to be able to}}:
**'''Dependencies:''' Marionette ability/reliability - the unresolved dependencies expressed in [https://bugzilla.mozilla.org/show_bug.cgi?id=801898 bug 801898]
**run Gaia UI Tests against [https://github.com/mozilla-b2g/gaia/tree/v1-train v1-train] and [https://github.com/mozilla-b2g/gaia/tree/v1.0.1 v1.0.1] (and v1.1?) concurrently
**'''Risks:''' ever-changing Gaia, both as it affects the features themselves, but also Marionette (see above)
**run the above on multiple devices (Leo, Inari and/or Hamachi) - one device per branch
**'''Tracking bugs:'''
***'''Dependencies:'''
*** [https://bugzilla.mozilla.org/show_bug.cgi?id=834531 bug 834531] -  [META] Set up and house Unagis/Jenkins boxes (Mac Minis) for Gaia UI automation and update testing
**** Commercial RILs for/in Unagi/other engineering builds
***'''Risks:'''
**** Lack of commercial RIL or even engineering build for any/all of the above
***'''Tracking bugs:'''
**** .

Latest revision as of 21:20, 17 April 2013

High-level goals

Taken from https://intranet.mozilla.org/QA/Q22013_QAgoals

  • Goal: [CARRY OVER] Help support, through automated smoketests, the purchase, installation of, and refund of, apps in the Firefox Marketplace, on Firefox OS
  • Goal: [NEW] Build out and support additional infrastructure and processes, to be able to:
    • run Gaia UI Tests against v1-train and v1.0.1 (and v1.1?) concurrently
    • run the above on multiple devices (2 soon-to-ship ones)

Firefox Marketplace

  • Goal: [CARRY OVER] Help support, where possible through automated smoketests, the purchase, installation, and refund of, apps in the Firefox Marketplace, on Firefox OS
    • Dependencies:
      • Marionette's ability to reliably automate the purchase/refund flow(s), end-to-end
      • Persona login/logout/account management (in)
      • Infrastructure necessary for testing (live?) payments multiple times per day
    • Risks:
      • Any of the blocking Marionette bugs listed in bug 801898 - Get GaiaTest UI smoke tests running reliably in Jenkins CI, which could affect our ability to automate
      • feature readiness/stability on the Marketplace-app side
      • stability on the vendor/payment-provider side
      • lack of a "back-door," automation-friendly way to purchase, frequently
      • Marionette's (Firefox OS) ability to automate, end-to-end
    • Tracking Bugs:

Firefox OS

  • Goal: [NEW] Build out and support additional infrastructure and processes, to be able to:
    • run Gaia UI Tests against v1-train and v1.0.1 (and v1.1?) concurrently
    • run the above on multiple devices (Leo, Inari and/or Hamachi) - one device per branch
      • Dependencies:
        • Commercial RILs for/in Unagi/other engineering builds
      • Risks:
        • Lack of commercial RIL or even engineering build for any/all of the above
      • Tracking bugs:
        • .
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Execution/Web_Testing/Goals/2013/Q2&oldid=647576"