Firefox:NewPlatformStandup

New Platform Standup process

History:

• 8/19/09 - initial notes from from QA team brainstorm

Dev Team Starts Exploration

• Setup repository if needed
• Setup any special directories for custom code
• Develop working prototype

First Builds

• Dev sets up, determines the SDK/tool chain,
• Dev work with RelEng to integrate into build process
• RelEng does first builds, manually installs and does basic tests. Passes to QA
• Once basic automation is done and produces reliable builds, they can go straight to QA with no RelEng manual testing.
• RelEng sets up automated sanity tests that get run on each milestone build before handover to QA. (These are the tests RelEng currently runs for Firefox milestone builds. joduinn- do you have a name fore these?)

Setup Project

• Bugzilla product, components, keywords, flags
• Establish rough schedule
• Wiki page for tracking?
• Mail list for communication
• Establish a project leader and a forum for release-drivers - need someone to drive coordination between groups and keeping good communication and status

Automated tests

• Prerequisites:
  • Need the ability to reset devices! - Preferably in a very automated way. Push a button and come back later.

• Test Dev Stands up Unit tests
  1. Talos tests - this could be first or last. For Tegra, this was first due to performance concerns. This was a tag team effort with Aki and Joel. Could use more engagement by Dev here.
  2. XPCShell - needs command line and Python on device
  3. RefTest/CrashTest family - Need the ability to install an extension and write to filesystem
  4. MochiTest family - Need to be able to write to filesystem (and install an extension?)

• These above (#2 - 4 at least) need an external server to hold tests (IT!)
• Consider chunking of tests - this is very case by case depending on platform and device
• To stand up MozMill - Must be able to install an extension
• Once reliable, documented and reproducible, hand off to RelEng

Manual Testing

• Prerequisites:
  • Ability to load builds
  • Machines/Devices available (VMs, simulators, breadboards, VMs, Lab machines, Service Packs, Seeds. ex: nVidia devices, Snow leopard, Mac Intel, touch screens (IT!)
  • Builds available in a place that is easy to find, save to download and install.

• Actions - Basic functional testing
  • Get access to DeviceAnywhere devices?
  • Setup project/branch in Litmus (QA Litmus Admin)
  • Crash capture and reporting
  • Later nightly builds
  • Supported plug-ins and support for changing plug-ins

• Further Actions - extended update/download/l10n etc testing
  • mar updates
  • data migration
  • locales
  • Weave sync

Other considerations

• Consider L10n requirements
• Accessibility requirements
• Security requirements
• Partner relationships - dev sharing, test sharing, ramping up on tools for the platform

Community Engagement

• Need information about how to install, run, and update. How to setup/clear profiles.
• Release notes/help info
• Info about how to test, what is stable, what is high priority
• Place to report feedback (textual for novices, bugzilla for others)
• Monitor uptake (nightly users, beta users total uses)