User:AxelHecht/Building l10n: Difference between revisions

Jump to navigation Jump to search

User:AxelHecht/Building l10n (view source)

Revision as of 18:38, 20 November 2008

112 bytes removed ,  20 November 2008
→‎Buildbot: update to mercurial and state of the art
(→‎Fixing stuff: new l10n-merge)
(→‎Buildbot: update to mercurial and state of the art)
 
Line 87: Line 87:
There are a few fuzzy terms here, I'll go into detail in some not-so random order.
There are a few fuzzy terms here, I'll go into detail in some not-so random order.


''build'' means repackaging an existing binary, and we should run our source checks against the source of that build. Finding out which one that is would be {{bug|398954}}.
''build'' means repackaging an existing binary, and we should run our source checks against the source of that build. There is a 'ident' target in locales/browser/Makefile.in to get that, at least for hg builds.


''whenever you have to'' is another interesting question, I don't have a full answer to that. The goal would be "when there's a good reason to build this app for this locale". For the l10n side of things, this is involved, but with a clear answer. When a localizer checks in, build all products that that check-in affected. More on that below.
''whenever you have to'' is another interesting question, I don't have a full answer to that. The goal would be "when there's a good reason to build this app for this locale". For the l10n side of things, this is involved, but with a clear answer. When a localizer checks in, build all products that that check-in affected. More on that below.


For en-US, things are more confuzzled in my head. Let's take a check-in to /cvsroot. That may or may not affect localization, if it does, it would impact all locales for all apps that include that localizable string. Butt. At the time of that check-in, there's no build with that change yet. I think there's some value in running compare-locales at the point, but it's not uber-big. I guess for en-US, we really need to figure out which builds we want to repackage, and what their source is. Right now, firefox nightlies would be good again, for an actively evolving line of development, being more agile is key. Like, if localizers only have 24 hours to fix a late-l10n issue, we shouldn't wait for the next nightly to tell them. Not sure how much horse power a full repack run on each nightly would take, and for how many products we want to cough that up. Right now, linux cycles around 1.5 hours on l10n, dep en-US builds are around 4 per hour. Just doing a full compare-locales run is a lot quicker, though. Maybe compare-locales tip against tip, and l10n-merge against the time-stamped source of the build.
For en-US, things are more confuzzled in my head. Let's take a check-in to mozilla-central. That may or may not affect localization, if it does, it would impact all locales for all apps that include that localizable string. Butt. At the time of that check-in, there's no build with that change yet. I think there's some value in running compare-locales at the point. I guess for en-US, we really need to figure out which builds we want to repackage, and what their source is. Right now, firefox nightlies would be good again, for an actively evolving line of development, being more agile is an option. Like, if localizers only have 24 hours to fix a late-l10n issue, we shouldn't wait for the next nightly to tell them. Not sure how much horse power a full repack run on each nightly would take, and for how many products we want to cough that up. Right now, linux cycles around 1.5 hours on l10n, dep en-US builds are around 4 per hour. Just doing a full compare-locales run is a lot quicker, though. Maybe compare-locales tip against tip, and l10n-merge against the time-stamped source of the build. (Editors note, the last two sentences are old, in particular the times.)


=== Lessons learned ===
=== Lessons learned ===
Line 109: Line 109:
=== Current arch on l10n.m.o ===
=== Current arch on l10n.m.o ===


Right now, I'm running a bonsai poller per branch and rep, so, as I'm only running trunk for l10n right now, two bonsai pollers. There is a single scheduler, which is using helper objects (''Dispatcher'') to map those changes to tree/application/locale tuples to be built.
Right now, I'm running a bonsai poller per branch and rep, so, as I'm only running trunk for l10n right now, two bonsai pollers. For Mercurial, there is a poller for each hg repo (mozilla and comm-central at this point). There is a single scheduler, which is using helper objects (''Dispatcher'') to map those changes to tree/application/locale tuples to be built.


Then the scheduler submits non-mergeable build requests, which the respective builders (linux-langpack only for now) pick up. They then ask the Scheduler which tree/application/locale is actually pending. That way I work around the lack of custom build properties. That's implemented by a specific Build class, and that's communicating those settings back down to the steps by setting those as build properties, too.
Then the scheduler submits non-mergeable build requests, which the respective builders pick up. The scheduler sets build properties to communicate the locale and application/branch combination to the respective builders.
 
; drawback : The build queue is now state of the Scheduler, which doesn't persist state. Thus, no reconfig.


[[Image:L10nBuildbotOverview.png]]
[[Image:L10nBuildbotOverview.png]]
Line 119: Line 117:
=== Outlook ===
=== Outlook ===


In this scheme, application and locale and such are only available as build properties, that is, finding the latest 15 German builds is hillariously costly in standard buildbot. Thus I added a status plugin that stores state in a real database. I intend to extend that to match buildbot better, and then use that database to actually persist the status of pending builds, so the scheduler can go dumb again, and things would be restart and reconfig friendly.
In this scheme, application and locale and such are only available as build properties, that is, finding the latest 15 German builds is hilariously costly in standard buildbot. Thus I added a status plugin that stores state in a real database. I intend to extend that to match buildbot better, and then use that database to actually persist the status of pending builds, so the scheduler can go dumb again, and things would be restart and reconfig friendly.
AxelHecht
Confirmed users, Bureaucrats and Sysops emeriti
2,976

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/117543"

Navigation menu