Confirmed users
14,525
edits
QA/Platform/Graphics/Features/Planning: Difference between revisions
Jump to navigation
Jump to search
QA/Platform/Graphics/Features/Planning (view source)
Revision as of 21:28, 25 July 2016
, 25 July 2016→Exploratory Phase
| Line 40: | Line 40: | ||
* Experiments on Aurora/Beta were well communicated via email | * Experiments on Aurora/Beta were well communicated via email | ||
* QA managed to achieve decent test coverage (mainly thanks to amount of time available) | * QA managed to achieve decent test coverage (mainly thanks to amount of time available) | ||
* | * Don't push fixes and experiments in very late in the cycle and try to bake this attitude in to all at the table | ||
* Hard to QA due to the size of the feature (covering almost the whole browser) – this caused some confusion at times as to what Firefox version should be used to run our Full Test set (which was run across several months) | * Hard to QA due to the size of the feature (covering almost the whole browser) – this caused some confusion at times as to what Firefox version should be used to run our Full Test set (which was run across several months) | ||
* High level of uncertainty at times as to when we want to release (including when we want it enabled on each channel) – at times this was clearly communicated via email, but then got contradictory information in meetings | * High level of uncertainty at times as to when we want to release (including when we want it enabled on each channel) – at times this was clearly communicated via email, but then got contradictory information in meetings | ||
* Need clearer communication from QA – this was hard to track for most managers not directly involved in e10s - this should have included an overall view: what was tested, what remains to be tested, future plans (what, when, on which version) | * Need clearer communication from QA – this was hard to track for most managers not directly involved in e10s - this should have included an overall view: what was tested, what remains to be tested, future plans (what, when, on which version) | ||
=== Key Takeaways === | |||
* Clearly defined and documented release criteria | |||
* Define tests as part of release criteria | |||
* Make sure all stakeholders buy-in to and respect release criteria | |||
* Make sure all work in costed from conception to development to testing to release to support | |||
* Make sure messaging is clear, concise, and uniform across all messaging | |||
* Frequent check-in of all stakeholders on progress toward release criteria | |||
* Experiment frequently, measuring against release criteria and a "control" user base | |||
* Be prepared to change release criteria as project evolves | |||
* Periodically conduct a broad-spectrum analysis to make sure blindspots are visible | |||
* Use system addon for experimentation and release roll-out | |||
* Make sure everything has a documented owner and approver | |||
* See e10s [https://wiki.mozilla.org/Electrolysis/Experiments Experiments Wiki], [https://wiki.mozilla.org/Electrolysis/Release_Criteria Release Criteria], and [https://goo.gl/AGMqvC Planning Doc] for more information | |||