B2G/QA/WeeklyBuilds

From MozillaWiki
< B2G‎ | QA
Jump to: navigation, search

Summary

With the B2G project growing rapidly, we need a system where we can generate daily builds and qualify them on a weekly basis. This will give expectations on a common build that our partners and public can continue to test on, and give more to a unified platform of issue tracking and reporting. This will also help with the health of each milestone.

Weekly builds

Expectations

  • We're picking one build of the week (Wednesday) to "qualify" as a good build for the rest of internal consumption. Primary audience would be those that want a "known good build", or even something we'd want to give to involved partners.
  • Any critical bugs found during the weekly build qualification, should be considered a blocker and debated if the build should be qualified. Devs should give attention to these builds and treat issues as showstoppers.
    • If the weekly build is not qualified, QA will recommend the next known good build for the team to consume.

Process

  • B2G internal will retrieve builds directly from an internal FTP site. (see jgriffin for credentials)
  • The manifest file used for building will also be published alongside so Mozilla partners and Community can use to build externally.
  • QA will take a midweek build (target: Wednesdays) and will validate within 48 hours or less of receipt.
  • QA will validate on all the supported devices (Nexus S & SGS2)
  • run a set of smoketests to qualify this build (this is defined in Testplan)
  • Test results will be emailed out at the end of week to dev-b2g and dev-gaia aliases, with a recommendation from team if build has passed Quality.


Daily builds

Expectations

  • Qualifying Daily builds are primarily for dogfooding of bug fixes and new features. Smoketesting is lighter than weekly build output, and there will not be a public qualification process. but there could be bugs filed.
  • Run exploratory testing against builds. Any bugs found may not be a blocker, but should be filed and treated with urgency if it is.

Process

  • B2G internal will retrieve builds directly from an internal FTP site. (see jgriffin for credentials)
  • The manifest file used for building will also be published alongside so Mozilla partners and Community can use to build externally.
  • QA will do a light smoketest or exploratory pass on daily builds (this is defined in Testplan)
  • QA will validate on 1 of the supported devices (Nexus S or SGS2)
  • QA will self-validate daily builds, but only post results to internal consumption (tracking testplan)