User:Bhearsum/Balrog/Administration

From MozillaWiki
Jump to: navigation, search

This page describes and documents the web-based administration interface for AUS3.

Overview

The administration interface for AUS3 will be a separate application from the client interface, possibly running on a separate server. It will be the only part of AUS3 with read/write access to the database. Authentication will be handled through LDAP (whether that's done by the web server or by the application is TBD).

Use Cases

Build Machines

Build machines have a relative small set of interactions with the server. The following use cases demonstrate them all:

Releases

At the start of the release process a new release is created in the AUS database. Its entry contains all platform independent information: release name, versions, filenames, bouncer products, etc. No platform information (including a list of platforms to be built) is included in the initial entry. If the release already exists an error will be raised.

At the end of each en-US and l10n repack job the newly created build is added to the AUS database, associated with the release created in the previous step. If the build already exists an error will be raised.

Once all builds for a release have been added to the database the betatest and releasetest channel rules for that stream will be updated to point at the newly created release.

Making the beta & release channels live will not be done by automation, and thus aren't covered here.

Nightly Updates

When a branch that receives nightly updates is set-up, a rule will be added that maps all users on its nightly channel to a release identified by $product-$channel-latest (Eg, Firefox-nightly-latest)1. This is a one time set-up and requires no additional maintenance down the road.

When a new nightly build is ready to be published, the build machine will add it to the AUS database in two places: the $product-$channel-latest release, as well as a $product-$channel-$buildid release. Once the -latest release has been updated the new update is live.

Web Administration

Use Cases

In contrast to build machines, the human-driven interface is capable of modifying all parts of the AUS database. These use cases are highly dependent on UI, so are only bullet points for now:

  • Turn off updates for a specific channel, branch, or globally
  • Adjust throttling for a specific rule or across a whole branch
  • Change an update channel to point at a different release
  • Add rules to block a specific architecture
  • Add, modify, or delete a custom rule
  • Add, modify, or delete a release
  • Add, modify, or delete a permission for a known or unknown user
  • View history of changes to rules, releases or permissions
  • Rollback to previous versions of a rule, release or permission.

UI

Permissions

permissions.html

permissions.html is the entry point for managing user permissions. It...

  • Contains a list of all known users (which is to say, all unique usernames in the permissions table), with links to user_permissions.html for each of them.
  • Has a form that a new username can be entered into, which redirects to user_permissions.html for the new user upon submission.

Because AUS doesn't perform authentication, no details besides username are needed when "creating" a new username.

user_permissions.html

user_permissions.html allows for creation, editing, and deletion of permissions for a specific user. It...

  • Lists existing permissions, and allows for deletion or editing of their options.
    • Only known options for the permission are listed.
  • Allows new permissions to be given to the user
    • Only known permissions are listed

Logging & Rollback

Logging and rollback of will be handled by a shadow tables with two additional fields: username and timestamp. All rows in this table must have a username and timestamp logged. This schema will end up using more space than strictly necessary, but the simplistic nature makes implementation easy. Rules are small enough pieces of data that their shadow table's size should still be negligible. Releases are a big bigger, but we can reduce the size of shadow entries by only storing differences or changes in their JSON blobs, which should be of trivial size. Additionally, it should be a rare case that releases need updating, so their shadow table will not have many rows in it.

When an INSERT to the a table is requested a new row will be inserted into the shadow table containing the rule_id (update_paths) or release name (releases). All other fields (besides username & timestamp) will be NULL. When an UPDATE or DELETE is requested, a new row will be created in the shadow table with identical information to the row being updated.

With the schema being so simplistic there's a lot of onus on the client to present the history in a clear and concise way. I envision a few possible views to make this easy:

  • Simple undo ("Revert the last change to the rules")
  • Rollback by day/time ("Revert the rules table to the state it was in at noon on August 1st")
  • List of recent changes, allowing the user to select the ones they want to revert
    • When the details link is clicked, the differences between the current state of the rule and the state before the change is shown. (similar for releases)
[ ] Push 6.0 to the release channel (details)
[ ] Push 6.0 to the beta channel (details)
[ ] Push 3.6.20 to the beta channel (details)
[ ] Push 6.0b5 to the beta channel (details)
[ ] Push 6.0b5 to the test channels (details)
[ ] Push 3.6.20 to the test channels (details)

[SUBMIT]
  • Wiki-history-like form that lets you compare/revert to arbitrary states.

The above is by no means a final list or must-have, just some ideas.

Open questions:

  • How are updates to shadow table done? Through triggers or by the application?
  • Should the application have write access to this table? If so, how do we prevent accidental (or intentional) removal of history?
  • How long should we keep shadow table entries for? A month? A year? Forever? This largely depends on whether we care about having long term history or just the ability to roll back quickly.

Authentication/Accounts

Authentication through LDAP will be handled by the web server and passed onto AUS3. AUS3 will maintain a database of usernames and permissions. Permissions will be granted through ACLs. There will be one entry in the ACL for every URL in the API that supports PUT, POST, or DELETE, as well as an entry to allow/disallow user management.

Most humans who use AUS3 will likely have permissions for everything except user management.

Build machines will also have an account they must use, and will only be allowed to make changes to a limited set of things:

  • /releases/[name]
    • For creating new nightly or release builds
  • /releases/[name]/builds/[platform]/[locale] PUT
    • For adding details of builds and repacks
    • This method can implicitly create required intermediaries, should we grant those too, for clarity?
  • /rules/[id] POST
    • For updating test channel rules during a release.

API

Both the build machine client and the web administration API will make use of the REST interface described below. A 500 response is possible for any method, so not listed for all of them. Most error responses (4xx and 5xx) will have an 'errmsg' entity in their response. The body of all responses will be a JSON object as described for each resource below.

URI Method Arguments Description Response status Response body
/releases GET Retrieves all release names 200 - Releases returned JSON object whose keys are names of releases and values are an object with the following data: product, version and release data
204 - No releases
/releases/[name] GET Retrieve the release identified by "name" 200 - Release returned JSON object with the following data: product, version, release data
404 - Release doesn't exist
/releases/[name] PUT details - the JSON blob associated with this release Update the release with the supplied JSON data. If non-existent, it is created. Be careful when using this for existing releases that already have builds added to them -- the JSON blob given here overrides all data for the release, not just platform independent data. 200 - Release modified
201 - Release created
400 - The details value is malformed or NULL errmsg
/releases/[name] DELETE Delete the release 200 - Release deleted
404 - Release does not exist errmsg
/releases/[name]/rollback PUT releases_log_id - Id of an entry in the releases shadow table. Must be an entry with a matching release name. Rollback the release to the specified log entry 200 - Rule modified
400 - releases_log_id doesn't exist or release name doesn't match errmsg
/releases/[name]/builds GET Retrieve all builds from a release 200 - Builds returned JSON object with build data - top-level keys are platforms
204 - No builds
404 - Release does not exist errmsg
/releases/[name]/builds/[platform]/[locale] GET Retrieve a specific build from a release 200 - Build returned JSON object with build data
404 - Release, platform, or locale doesn't exist errmsg
/releases/[name]/builds/[platform]/[locale] PUT details - the JSON blob associated with this build Update the build with the supplied JSON data. If any of the named release, platform, or locale don't exist, they will be created. If the specific build already exists, it will be overwritten with the supplied information. 200 - Build modified
product 201 - Build created
version 400 - The details value is malformed or NULL. errmsg
/releases/[name]/builds/[platform]/[locale] DELETE Delete the build 200 - Release deleted
404 - Build doesn't exist
/releases/log GET name - Only retrieve log entries for a release with this name Retrieve history of changes to rules 200 - Log entries returned JSON object whose keys are releases_log_id's and values are an object with the following data: release name/version/product/data, username and timestamp
product - Only retrieve log entries for a release whose product is this
version - Only retrieve log entries for a release whose version is this 204 - No log entries
startdate - Only retrieve rules newer than this date/time
enddate - Only retrieve rules older than this date/time
/rules POST priority Create a new rule with the supplied arguments. 201 - Rule created
mapping
throttle
update_type
product
version
channel
buildTarget 400 - One or more of the arguments is invalid errmsg
buildID
locale
osVersion
distribution
distVersion
headerArchitecture
comment
/rules GET Same as /rules POST Retrieve all of the rules that match the specified arguments. Any missing arguments are regarded as wildcards, therefore a request with no parameters returns all rules. 200 - Rules returned JSON object whose keys are rule_id's and values are the same as the arguments described above
204 - No rules
/rules/[id] GET Retrieve the rule 200 - Rule returned Same as a single entry in /rules GET.
404 - Rule does not exist
/rules/[id] PUT Same as /rules POST Update the rule with the supplied arguments. Any missing arguments will be considered NULL. 200 - Rule modified
400 - One or more of the arguments is invalid errmsg
/rules/[id] POST Same as /rules POST Update the rule with any arguments passed. Any missing arguments will not be changed. 200 - Rule modified
400 - One or more of the arguments is invalid errmsg
404 - Rule does not exist
/rules/[id] DELETE Delete the rule 200 - Rule deleted
404 - Rule does not exist
/rules/[id]/rollback PUT rules_log_id - Id of an entry in the rules shadow table. Must be an entry with a matching rule id. Rollback the rule to the specified log entry 200 - Rule modified
400 - rules_log_id doesn't exist or rule_id doesn't match errmsg
/rules/log GET All arguments to /rules POST are valid here Retrieve history of changes to rules 200 - Log entries returned JSON object whose keys are rules_log_id's and values are an object with all rule data, username, and timestamp.
startdate - Only retrieve rules newer than this date/time 204 - No log entries
enddate - Only retrieve rules older than this date/time
/users GET Retrieve all users referenced in the Permissions table 200 - Usernames returned List of usernames in the permission table, in JSON format
204 - No usernames found
/users/[username]/permissions GET Retrieve all permissions that the named user has. 200 - Permissions returned List of permissions for the named user, in JSON format.
204 - No permissions found
/users/[username]/permissions/[permission] GET Retrieve the options for a specific perimssion 200 - Options returned JSON object whose keys are option names and values are option values.
404 - Permission does not exist
/users/[username]/permissions/[permission] PUT options - a JSON object whose keys are option names and values are option values Update the permission with the supplied arguments. If it does not exist, it will be created 200 - Permission modified
201 - Permission created
400 - Options argument is invalid errmsg
/users/[username]/permissions/[permission] DELETE Delete the permission 200 - Permission deleted
404 - Permission does not exist

Footnotes

1. $product-$channel-latest only works in a world where no channel names overlap, which we should be in once 1.9.2 is dead. Doing it this way ensures that we nightly updates work after version bumps happen *and* we don't need to maintain a branch name <-> version mapping a la existing AUS.