B2G/QA/Automation/UI/Strategy

From MozillaWiki
< B2G‎ | QA‎ | Automation‎ | UI
Jump to: navigation, search

Team Mandates

  • Establish visible subject matter expertise and impact in test and automation
  • Own, expand and maintain QA End-to-end automation
  • Increase confidence in End-to-end test automation
  • Provide expert assistance to improve all phases of UI test coverage
  • Catch problems quickly and effectively
  • Increase community support for FxOS QA
  • Increase quality of and confidence in the product

Challenges

  • Automation systems and concerns are poorly documented
  • Development and QA have different approaches and needs from UI testing
  • UI tests have had no developer support, largely because of Python implementation
  • Team has been blocked too much by cross-team dependencies
  • Test results are hidden in Jenkins
  • Test results have too many spurious failures to sheriff
  • Test results only looked at a limited number of times per day
  • The product is too unstable for end-to-end testing during development phases
  • End-to-end coverage is insufficient to guarantee a quality build
  • Community paths are poorly defined and advertised

Plan

For End-to-end:

For Integration:

For Community: