overview

Socorro frequently needs to release unscheduled "patch" releases, to fix problems (bugs in Socorro, or to help other teams, such as the Firefox/Fennec breakpad bustage, see bug 637680).

We have been either building a new package or applying a patch to production. The former is pretty heavyweight, so generally IT does the latter, especially for minor config issues which require code change (as opposed to simple config override) like changing throttling (see bug 630798)

problems

• manual application of patches
  • can result in errors or inconsistent state across servers
  • bypasses our usual Jenkins unittest->staging push
    • we generally check into trunk first, so this is somewhat mitigated
  • more time-consuming and manual than our automated package install
• trunk is closed for future development from freeze time until release

proposal

1) For each each scheduled version (for example 2.2.1), create a branch at freeze time

• this opens master up for immediate future development (2.3)
• could create the branch in advance, but would rather limit the divergence between trunk and branch

2) For each major.minor.patch version (for example 2.2.2), devs check changes into trunk and merge to branch

• manual process; not all patches are applicable to both

3) Jenkins will automatically run tests, create package, and publish for staging

• Jenkins needs a very simple config change for each git branch
  • each branch has it's own job, for example:
    • https://jenkins.mozilla.org/job/socorro-master
    • https://jenkins.mozilla.org/job/socorro-release

4) The package Jenkins made for staging is available for automated production install

• totally scripted, limits potential for errors
• automatically backs up current release

------------------------trunk (future 1.7.8)--->
             \
              1.7.7----------------->
                      \         \
                    1.7.7.1   1.7.7.2