Bugzilla:Migrating to git: Difference between revisions

← Older edit

Bugzilla:Migrating to git (view source)

Revision as of 15:17, 4 May 2015

6,500 bytes added , 4 May 2015

→‎Plan: a few more things have been accomplished

Mcote

Confirmed users

1,927

edits

@@ Line 1: / Line 1: @@
-It has been proposed to move the Bugzilla source from bzr.mozilla.org to git.mozilla.org with a read-only mirror on GitHub.
+It has been proposed to use [http://git-scm.com/ Git] as Bugzilla's [http://en.wikipedia.org/wiki/Revision_control VCS] instead of [http://bazaar.canonical.com/ Bazaar], and move the Bugzilla source from bzr.mozilla.org to git.mozilla.org with a read-only mirror on GitHub.
+The migration was completed successfully on 2014/03/11. git.mozilla.org currently contains the repositories of record for Bugzilla and related code.
 == Rationale ==
@@ Line 30: / Line 32: @@
 There are a bunch of things we'll need to do to complete this.  Some can be done in parallel.
+The tracking bug for the migrations is {{bug|929685}}.
 Pre-migration:
@@ Line 37: / Line 41: @@
 # ([[Bugzilla:Moving_From_Bazaar_To_Git|DONE]]) Write migration instructions similar to https://wiki.mozilla.org/Bugzilla:Moving_From_CVS_To_Bazaar
 # (DONE, though see {{bug|967697}}) Set up repos (bugzilla and bmo) on git.mozilla.org.
-# Determine which branches should be migrated over (presumably all, but there might be obsolete branches) and estimate how long this will take.
+# (DONE, approx 1h15m) Determine which branches should be migrated over (presumably all, but there might be obsolete branches) and estimate how long this will take.
-# Do a trial migration.
+# (DONE) Do a trial migration.
-# Announce date for migration.
+# (DONE, 11 March 2014) Announce date for migration.
 Migration:
-# Disallow commits to bzr aside from those originating from the git-to-bzr script.
+# (DONE) Create git repos, restricting write access to just the account used for the migration.
-# Perform fast-import/-export migration on all (supported) branches of bugzilla and bmo.
+# (DONE) Disallow commits to bzr aside from those originating from the git-to-bzr script.
-# Commit .gitrev to bzr containing the ID of the last git commit.
+# (DONE) Perform fast-import/-export migration on all branches of bugzilla, bmo, and related repositories, excluding those listed in the section below, using the migrate.sh script.
-# Enable git-to-bzr script (per branch).
+# (DONE) Run git-to-bzr script once on all mirrored branches (see below) to initialize the .gitrev tracking file.
-# Commit .gitignore file based on .bzrignore to git, and confirm that it has been mirrored.
+# (DONE) Start running git-to-bzr periodically (every 5 minutes should be good).
+# (DONE just for bugzilla/master so far) Commit .gitignore file based on .bzrignore to git (at least to supported/active repos/branches), and confirm that it has been mirrored.
+# (DONE) Enable write access on all git repos for all authorized accounts.
 Soon after:
-# Set up repos on GitHub and configure git.m.o repos to mirror there.
+# (DONE) Set up repos on GitHub and configure git.m.o repos to mirror there.
-# Set up a travis-ci instance with a test migration to GitHub.  Note: someone out there has apparently done just this: https://travis-ci.org/dnozay/bugzilla
+# (DONE) Set up a travis-ci instance with a test migration to GitHub.  Note: someone out there has apparently done this for the unit tests (https://travis-ci.org/dnozay/bugzilla); we still need support for the Selenium tests in the qa branches.
 Eventually:
-# Turn off tinderbox when we are sure that the test results from travis-ci and tinderbox match.
+# (DONE) Turn off tinderbox when we are sure that the test results from travis-ci and tinderbox match.
 # Turn off bzr ({{bug|967642}}).
 Migration scripts are in a [https://github.com/markrcote/bugzilla-git-migration GitHub repo].
+If there are any problems during the migration, we'll abort and re-enable write access to bzr.mozilla.org as it was pre-migration.
+We will mirror the following bzr branches from git.  There are three reasons to mirror: to support existing deployment scripts, to support existing bzr-based installations, and to have tinderbox continue to test commits until we switch to travis-ci.
+For existing installations:
+* bugzilla/4.0 (switch off after 5.0 is released, at which point we can also stop mirroring to cvs)
+* bugzilla/4.2
+* bugzilla/4.4
+For tinderbox CI testing (switch off after travis-ci is set up for both packaged tests and qa tests) and deployment scripts:
+* bugzilla/trunk
+* bugzilla/qa/4.0
+* bugzilla/qa/4.2
+* bugzilla/qa/4.4
+* bmo/4.2 (also needed until deployment scripts are updated, {{bug|973298}})
+* bmo/4.2-dev (also needed until deployment scripts are updated, {{bug|973298}})
+* bmo/qa/4.2
+=== Mapping ===
+Almost all the Bazaar branches on bzr.mozilla.org will be migrated to one or more branches in various repositories on git.mozilla.org.
+All branches of the Bugzilla source will be migrated to individual branches in one git repo, /bugzilla/bugzilla.git.  Similarly, the QA branches will be migrated to bugzilla/qa.git.  Branches in extensions and misc will be migrated to repos in /bugzilla/extensions, keeping related branches together.
+The BMO source will be migrated to /webtools/bmo/bugzilla.git, and the BMO qa repo as /webtools/bmo/qa.git.
+/bzr-plugins and /securemail will not be migrated, since the former is unneeded in git, and the latter was a mistake.
+Bazaar trunk branches are migrated to git as master, following convention.  Note that not all Bazaar branches of the same project contain a trunk branch (e.g. bmo); git references may need to be updated to prevent warnings during cloning.
+Full mapping follows:
+; /bugzilla/2.*, 3.*, 4.*, trunk, sightings
+: /bugzilla/bugzilla.git (as individual branches, with trunk -> master)
+; /bugzilla/qa/*
+: /bugzilla/qa.git
+; /bugzilla/extensions/*
+: /bugzilla/extensions/*.git
+Full list of git extension repos:
+* /bugzilla/extensions/Browse.git
+* /bugzilla/extensions/browserid.git
+* /bugzilla/extensions/DescribeUser.git
+* /bugzilla/extensions/Developers.git
+* /bugzilla/extensions/ExtraValues.git
+* /bugzilla/extensions/GNOME.git
+* /bugzilla/extensions/PatchReport.git
+* /bugzilla/extensions/ProductInterests.git
+* /bugzilla/extensions/profanivore.git
+* /bugzilla/extensions/requestwhiner.git
+* /bugzilla/extensions/sitemap.git
+* /bugzilla/extensions/StockAnswers.git
+* /bugzilla/extensions/sync.git
+* /bugzilla/extensions/typesniffer.git
+* /bugzilla/extensions/vcs.git
+* /bugzilla/extensions/WeeklyBugSummary.git
+* /bugzilla/extensions/cannedcomments.git
+* /bugzilla/extensions/rest.git
+* /bugzilla/extensions/trackingflags.git
+* /bugzilla/extensions/InlineHistory.git
+* /bugzilla/extensions/securemail.git
+* /bugzilla/extensions/splinter.git
+* /bugzilla/extensions/testopia.git
+; /bugzilla/misc/*
+: /bugzilla/misc/*.git
+Full list of git misc repos:
+* /bugzilla/misc/active-installs.git
+* /bugzilla/misc/bugzilla-bugbot.git
+* /bugzilla/misc/build.git
+* /bugzilla/misc/landfill.git
+* /bugzilla/misc/tinderbox-bugbot.git
+* /bugzilla/misc/tinderbox-client.git
+; /bmo/3.*, 4.*
+: /webtools/bmo/bugzilla.git
+; /bmo/qa/*
+: /webtools/bmo/qa.git
+== Testing ==
+=== Initial Migration ===
+When the git repos are created, we can do a full migration to look for errors.  The migration script (migrate.sh) calculates the md5sum for all files in the head of each branch in both bzr and the resulting git repository and compares them along with file metadata (as reported by 'ls -l').  After the trial migration we can examine the log for differences.  Limitations: this only checks heads of branches and does not verify any commit metadata.
+The first trial migration was completed on 2014/02/12.  It was successful aside from a few files that were removed in bzr commits but present in the final git repository.  It appears that at least some of these are a result of a directory being renamed and files in that directory being removed in the same commit; bzr records this (given old name "A" and new name "B") as "R A B" followed by "D A", meaning that the delete is recorded on the original name, not the new name.  This results in the rename being successful but the delete being ignored.  This is not critical, since they can be removed after the fact, but it might be worth adjusting the fastexport translation script to account for this.
+=== Continuous Mirroring ===
+Initial testing involves creating a new bzr branch from a current one (e.g. from bugzilla/trunk), migrating that over to git, then making various changes to git and running the git-to-bzr.pl script (either regularly on cron, or manually for more fine-grained testing).  The resulting commits should have identical commit metadata and changes to file metadata and contents.  We can manually inspect for the first two and use the same process as the initial migration for the latter.
+Changes that should be tested:
+* Commits from various authors.
+* Creating a new file in an existing directory.
+* Creating a new file in a new directory (git has no concept of empty directories).
+* Modifying a file's contents.
+* Modifying a file's metadata.
+* Removing a file.
+* Removing the only file in a directory (mirror script should automatically delete the empty directory to match git's behaviour).
+* Moving/renaming a file.
+We can leave the script running on the test branch for continued testing.