Confirmed users
920
edits
Partnering:Projects:BYOB:PilotBuildQueueDetails: Difference between revisions
Partnering:Projects:BYOB:PilotBuildQueueDetails (view source)
Revision as of 17:22, 7 May 2010
, 7 May 2010→Build queue implementation details
LesOrchard (talk | contribs) (Created page with '== Overview == This is a summary of the moving parts involved in the PHP/MySQL based build queue in use for the pilot launch of BYOB. __TOC__ == Repack states == The workflow …') |
LesOrchard (talk | contribs) |
||
| (2 intermediate revisions by the same user not shown) | |||
| Line 7: | Line 7: | ||
== Repack states == | == Repack states == | ||
The workflow status of a repack is state machine, where each state describes what actions are currently available and to which states it may transition next. | The workflow status of a repack is state machine, where each state describes | ||
what actions are currently available and to which states it may transition | |||
next. A history log of these state transitions is recorded in the repack | |||
data, along with comments. | |||
{| border="1" cellpadding="4" width="100%" | {| border="1" cellpadding="4" width="100%" | ||
| Line 18: | Line 21: | ||
| requested || user has requested a build and review || cancelled; started; failed; pending | | requested || user has requested a build and review || cancelled; started; failed; pending | ||
|- | |- | ||
| cancelled || user has cancelled a repack build || edited; requested; deleted | | cancelled || user has cancelled a repack build and review request || edited; requested; deleted | ||
|- | |- | ||
| started || build has started || failed; pending | | started || build has started || failed; pending | ||
| Line 26: | Line 29: | ||
| pending || build completed, review pending || cancelled; released; rejected | | pending || build completed, review pending || cancelled; released; rejected | ||
|- | |- | ||
| rejected || build rejected | | rejected || build rejected for public release || edited; requested; deleted | ||
|- | |- | ||
| released || build | | released || build approved for public release || reverted | ||
|- | |- | ||
| reverted || public release of build taken down || edited; requested; deleted | | reverted || public release of build taken down || edited; requested; deleted | ||
| Line 41: | Line 44: | ||
* Repacks get copied on edit as an unreleased version when in these states: | * Repacks get copied on edit as an unreleased version when in these states: | ||
** released | ** released | ||
== Build queue tasks in BYOB == | == Build queue tasks in BYOB == | ||
In reaction to certain state changes, build tasks get enqueued into a PHP/MySQL | In reaction to certain state changes, build tasks get enqueued into a PHP/MySQL | ||
based offline work queue that gets exhausted every 5 minutes or so. The | based offline work queue that gets exhausted at least every 5 minutes or so. The | ||
current build queue tasks are: | current build queue tasks are: | ||
| Line 66: | Line 54: | ||
* Enqueue triggered by repack transitioning to '''requested''' state. | * Enqueue triggered by repack transitioning to '''requested''' state. | ||
* Fire up the repack build script | * Process: | ||
* Move the built repack to private review hosting | ** Fire up the repack build script | ||
* If successful, transition the repack to '''pending''' state. | ** Move the built repack to private review hosting | ||
* If failed, transition the repack to '''failed''' state. | * Result: | ||
** If successful, transition the repack to '''pending''' state. | |||
** If failed, transition the repack to '''failed''' state. | |||
=== Release build === | === Release build === | ||
* Enqueue triggered by repack transitioning to '''released''' state. | * Enqueue triggered by repack transitioning to '''released''' state. | ||
* Move a built repack from private review to public release hosting | * Process: | ||
** Move a built repack from private review to public release hosting | |||
* Result: | |||
** In any case, repack remains in '''released''' state. | |||
** (No failure case currently implemented.) | |||
=== Delete build === | === Delete build === | ||
* Enqueue triggered by repack transitioning to '''deleted''' state. | * Enqueue triggered by repack transitioning to '''deleted''' state. | ||
* Delete a built repack from both private and public hosting | * Process: | ||
** Delete a built repack from both private and public hosting | |||
* Result: | |||
** In any case, the repack data has been deleted from the BYOB database. | |||
** (No failure case currently implemented.) | |||
== Build queue implementation details == | |||
The build queue consists of: | |||
* [https://svn.mozilla.org/projects/byob/trunk/modules/messagequeue/config/schema-mysql.sql a MySQL table listing tasks to be performed]; | |||
* [https://svn.mozilla.org/projects/byob/trunk/bin/exhaust-queue.sh-dist a shell script] that runs in a crontab or [https://svn.mozilla.org/projects/byob/trunk/bin/org.mozilla.byob.cron.plist-dist as an OS X launchd scheduled task]; | |||
** every 5 minutes or so, it sets a lock to prevent concurrent runs and exhausts the queue | |||
* [https://svn.mozilla.org/projects/byob/trunk/modules/messagequeue/controllers/messagequeue.php a PHP controller that processes the queue], using [https://svn.mozilla.org/projects/byob/trunk/modules/messagequeue/models/messagequeue.php a model library] to perform the work. | |||
The web app uses this build queue by: | |||
# [https://svn.mozilla.org/projects/byob/trunk/application/hooks/builds.php Registering DeferredEvent callback handlers with unique IDs] - these do the actual work | |||
# [http://viewvc.svn.mozilla.org/vc/projects/byob/trunk/application/models/repack.php?revision=66815&view=markup#l882 Firing Events] that correspond with [http://viewvc.svn.mozilla.org/vc/projects/byob/trunk/application/hooks/builds.php?revision=66819&view=markup#l14 registered DeferredEvent handlers] | |||
# Firing Events results in enqueued tasks, which the queue processor handles via calls to registered DeferredEvent callbacks. | |||