Confirmed users
329
edits
Multiple Project Repositories: Difference between revisions
Jump to navigation
Jump to search
m
minor updates
(→Issues) |
m (minor updates) |
||
| Line 1: | Line 1: | ||
There have been various grumblings for a while that we ought to be using more project repositories, following the model of the TraceMonkey branch. The recent "Post-FF4 Branch organization" megathread and the faster release cycle discussion have brought the idea into the light. This page is for exploration of the idea in a more persistent format (it's gotten hard to keep track of everything covered in the thread.) | There have been various grumblings for a while that we ought to be using more project repositories, following the model of the TraceMonkey branch. The recent "Post-FF4 Branch organization" megathread and the faster release cycle discussion have brought the idea into the light. This page is for exploration of the idea in a more persistent format (it's gotten hard to keep track of everything covered in the thread.) '''Note that nothing has been decided yet. This page is not an official policy statement, just a place to enumerate concerns, propose solutions, and archive discussion.''' | ||
Note | Note: multiple project repos are not necessarily connected to a faster release cycle, but they could be part of the overall release cycle change plan. | ||
Specifically, this page talks about splitting up development into a set of project repositories, similar to the current tracemonkey repo, where changes from the repositories flow into a global repo (perhaps the current mozilla-central). For most of these, I would expect m-c changes to be merged back into the project repo fairly frequently, but that's up for discussion. | Specifically, this page talks about splitting up development into a set of project repositories, similar to the current tracemonkey repo, where changes from the repositories flow into a global repo (perhaps the current mozilla-central). For most of these, I would expect m-c changes to be merged back into the project repo fairly frequently, but that's up for discussion. | ||
= Main Advantages/Disadvantages = | |||
* When bustage is committed, fewer people are affected | * When bustage is committed, fewer people are affected | ||
* More experimental changes can be made and the problems shaken out because of the smaller sandboxes | * More experimental changes can be made and the problems shaken out because of the smaller sandboxes | ||
* Easing of the patch bottleneck when the tree gets busy (you just need to space pushes with respect to others in the same tree, not with everyone who might commit anything anywhere) | * Easing of the patch bottleneck when the tree gets busy (you just need to space pushes with respect to others in the same tree, not with everyone who might commit anything anywhere) | ||
* Commit policy can be more closely matched to the needs of a restricted area (eg churn from small changes is less harmful and so more acceptable => more conducive to refactoring, fewer change freeze periods to discourage external contributors) (Note that I would '' | * Commit policy can be more closely matched to the needs of a restricted area (eg churn from small changes is less harmful and so more acceptable => more conducive to refactoring, fewer change freeze periods to discourage external contributors) (Note that I would not expect ''review'' policies to differ between repos) | ||
* More merges are required, and more "clumps" of changes move around the system | * More merges are required, and more "clumps" of changes move around the system | ||
* Less clarity on where individual changes should go, especially for less active (new/occasional) contributors | * Less clarity on where individual changes should go, especially for less active (new/occasional) contributors | ||
| Line 20: | Line 18: | ||
* More uncertainty about when a specific change will land in a given repo or nightly | * More uncertainty about when a specific change will land in a given repo or nightly | ||
= Questions = | |||
Should all work go into project repos, and mozilla-central be a dedicated merge destination? Or should m-c live on as the default repo, and we'd just increase the number of project repos that we already have? | Should all work go into project repos, and mozilla-central be a dedicated merge destination? Or should m-c live on as the default repo, and we'd just increase the number of project repos that we already have? | ||
| Line 28: | Line 26: | ||
Do we need "urgent" sub-repositories like Linux kernel development uses? (Changes made to these repos are expected to go directly into the central repo rather than waiting on the next merge.) | Do we need "urgent" sub-repositories like Linux kernel development uses? (Changes made to these repos are expected to go directly into the central repo rather than waiting on the next merge.) | ||
= Issues = | |||
A number of issues came up during the mailing list thread. The ones I can recall are captured here and (one) response given. Please add more, and add/improve the responses. You can try to initiate discussions here, or (probably better) post to the mailing list for discussion and document the result here. | A number of issues came up during the mailing list thread. The ones I can recall are captured here and (one) response given. Please add more, and add/improve the responses. You can try to initiate discussions here, or (probably better) post to the mailing list for discussion and document the result here. | ||
== Contributors will need to maintain too many repositories == | |||
Summary: A user who wants to contribute to multiple divergent areas within the Mozilla codebase will need to have half a dozen repositories checked out. | Summary: A user who wants to contribute to multiple divergent areas within the Mozilla codebase will need to have half a dozen repositories checked out. | ||
| Line 44: | Line 42: | ||
When using multiple upstream repos, you can either do the same thing, once for each repo. Or perhaps better, pull all of the upstream repos together into one local master repo with a bunch of heads. Then, when cloning into your topic repos, use |hg clone -r <head-rev>| to make a subrepo that only has the head from the appropriate upstream repo, but can still pull updates from the master. (You can push from this subrepo normally.) The bookmarks extension might make this even better, but I haven't tried it. | When using multiple upstream repos, you can either do the same thing, once for each repo. Or perhaps better, pull all of the upstream repos together into one local master repo with a bunch of heads. Then, when cloning into your topic repos, use |hg clone -r <head-rev>| to make a subrepo that only has the head from the appropriate upstream repo, but can still pull updates from the master. (You can push from this subrepo normally.) The bookmarks extension might make this even better, but I haven't tried it. | ||
Warning: I haven't actually tried the above, so I don't know what problems may arise in practice. | Warning: I haven't actually tried the multiple-head approach above, so I don't know what problems may arise in practice. | ||
== Merge Nightmare == | |||
Summary: The merges of the project repos into mozilla-central will be messy, time-consuming, and error-prone | Summary: The merges of the project repos into mozilla-central will be messy, time-consuming, and error-prone | ||
Sayrer probably has good advice and practical experience for minimizing the pain. I'll add one more: mozilla-next | Sayrer probably has good advice and practical experience for minimizing the pain. I'll add one more: "mozilla-next" | ||
The Linux kernel uses a "linux-next" repo for precisely this purpose. See the [http://linux.f-seidel.de/linux-next/pmwiki/pmwiki.php?n=Linux-next.Announcement original announcement], but in brief, this is an automatically-maintained repository that merges together all of the upstream repos daily. If any merge fails, | The Linux kernel uses a "linux-next" repo for precisely this purpose. See the [http://linux.f-seidel.de/linux-next/pmwiki/pmwiki.php?n=Linux-next.Announcement original announcement], but in brief, this is an automatically-maintained repository that merges together all of the upstream repos daily. If any merge fails, that repo is skipped. The end result is a merger of all repos that successfully merged. Maintainers of the repos that failed merge are notified. I assume that when a merge fails, the repo maintainer pulls in the master repo's changes and manually fixes up the merge, so that the next merge is more likely to succeed (it won't even have to try to merge any of those changesets next time.) | ||
In practice, linux-next has apparently been very helpful in reducing merge problems, since new problems are discovered quickly and reported. But it is important to be aware of what this tree is ''not'': successfully landing in this tree (as in, having your tree make it through the automated merge procedure) is no guarantee of inclusion in the "real" tree. It does help give people an idea of what could be in the next version of Linux, well before those features are ready for inclusion (let alone accepted). You might think it could be useful for forward-looking QA, but it really isn't: if a bug is found in mozilla-next, there's no way to bisect it because the tree is rebuilt from scratch every day, so there's no useful revision history. Still, it is good for what it is: a way to minimize merge conflicts between | In practice, linux-next has apparently been very helpful in reducing merge problems, since new problems are discovered quickly and reported. But it is important to be aware of what this tree is ''not'': successfully landing in this tree (as in, having your tree make it through the automated merge procedure) is no guarantee of inclusion in the "real" tree. It does help give people an idea of what could be in the next version of Linux, well before those features are ready for inclusion (let alone accepted). You might think it could be useful for forward-looking QA, but it really isn't: if a bug is found in mozilla-next, there's no way to bisect it because the tree is rebuilt from scratch every day, so there's no useful revision history. Still, it is good for what it is: a way to minimize merge conflicts between an assortment of upstream repositories (80 or so in the case of Linux). | ||
== Spreading QA Thin == | |||
Summary: We don't have enough QA bandwidth if it's split up among the multiple project repos | Summary: We don't have enough QA bandwidth if it's split up among the multiple project repos | ||
| Line 68: | Line 66: | ||
I don't understand our QA setup well enough to say anything more intelligent here. So far, I'm not seeing any fundamental reason why QA would be worse with project repos. | I don't understand our QA setup well enough to say anything more intelligent here. So far, I'm not seeing any fundamental reason why QA would be worse with project repos. | ||
== Spreading philor Thin == | |||
Drawing and quartering philor? | Drawing and quartering philor? | ||
| Line 80: | Line 78: | ||
(An an aside, I should point out it's really not philor's personal responsibility to star everyone's oranges. He's providing an incredibly valuable service for us all. Is it because he has a generous heart, or because he's batshit insane? Or both?) | (An an aside, I should point out it's really not philor's personal responsibility to star everyone's oranges. He's providing an incredibly valuable service for us all. Is it because he has a generous heart, or because he's batshit insane? Or both?) | ||
== Too hard to watch changes == | |||
Summary: Right now you can watch the mozilla-central pushlog. With project repos, you'd need to either watch them all, or get giant clumps of changes when they hit mozilla-central. | Summary: Right now you can watch the mozilla-central pushlog. With project repos, you'd need to either watch them all, or get giant clumps of changes when they hit mozilla-central. | ||
Create a unified pushlog view | Proposed solution: Create a unified pushlog view. You even get added functionality of being able to filter by repo. | ||
== Contributor Confusion == | |||
Summary: A prospective contributor won't know where to make changes | Summary: A prospective contributor won't know where to make changes | ||
| Line 92: | Line 90: | ||
This is a real problem I don't have a quick answer to. I think there's probably a flip side, though -- if the contributor ''does'' find the repo or IRC channel corresponding to the area being modified, they'll be able to work in a smaller, less overwhelming environment. The rules and procedures for contributing are pretty complex, and right now you have to either not care, or spend a lot of time in advance figuring out exactly what you're supposed to do. If you know you're only going to affect the dozen or so people active in a particular area, it's a lot less intimidating to get involved. But I still haven't addressed the original issue. | This is a real problem I don't have a quick answer to. I think there's probably a flip side, though -- if the contributor ''does'' find the repo or IRC channel corresponding to the area being modified, they'll be able to work in a smaller, less overwhelming environment. The rules and procedures for contributing are pretty complex, and right now you have to either not care, or spend a lot of time in advance figuring out exactly what you're supposed to do. If you know you're only going to affect the dozen or so people active in a particular area, it's a lot less intimidating to get involved. But I still haven't addressed the original issue. | ||
== Fragmenting The Nightly Tester Community == | |||
Mozilla currently benefits from a (relatively) small but dedicated nightly tester community. It is unclear what impact having multiple repositories churning out different nightly builds will do to that community and for early regression testing. Potentially, this could lead to delays during the beta release cycle due to more regressions being discovered nearer to the intended release date. | Mozilla currently benefits from a (relatively) small but dedicated nightly tester community. It is unclear what impact having multiple repositories churning out different nightly builds will do to that community and for early regression testing. Potentially, this could lead to delays during the beta release cycle due to more regressions being discovered nearer to the intended release date. | ||