Confirmed users
1,927
edits
Auto-tools/Projects/Pulse: Difference between revisions
no edit summary
No edit summary |
|||
| Line 1: | Line 1: | ||
== Mozilla Pulse == | == Mozilla Pulse == | ||
https://pulse.mozilla.org/ | |||
Mozilla currently has a ton of different systems that are inter-connected via polling, screen scraping, email, and other brittle methods. To make their lives easier community members often build tools on top of this house of cards, adding yet another level of scraping and polling. Many systems don't even export important data for others to scrape and use, preventing better tools from being written. | Mozilla currently has a ton of different systems that are inter-connected via polling, screen scraping, email, and other brittle methods. To make their lives easier community members often build tools on top of this house of cards, adding yet another level of scraping and polling. Many systems don't even export important data for others to scrape and use, preventing better tools from being written. | ||
| Line 13: | Line 13: | ||
=== System Description === | === System Description === | ||
Pulse isn't any one thing. At its heart, it is a RabbitMQ system with a particular configuration and a set of conventions for using it. Pulse follows the pub-sub pattern, in which publishers send messages to topic exchanges, and consumers create queues bound to these exchanges in order to subscribe to the publishers' messages. The [https://pypi.python.org/pypi/MozillaPulse mozillapulse] Python package provides classes for existing publishers, consumers, and messages so you can quickly build Pulse applications. | Pulse isn't any one thing. At its heart, it is a RabbitMQ system with a particular configuration and a set of conventions for using it along with a management tool, [[Auto-tools/Projects/Pulse/PulseGuardian|PulseGuardian]], to make Pulse as automated and self-serve as possible. Pulse follows the pub-sub pattern, in which publishers send messages to topic exchanges, and consumers create queues bound to these exchanges in order to subscribe to the publishers' messages. The [https://pypi.python.org/pypi/MozillaPulse mozillapulse] Python package provides classes for existing publishers, consumers, and messages so you can quickly build Pulse applications. | ||
=== Contributing === | |||
[http://mzl.la/1pc2iGd Browse] the list of open, unassigned mentored Pulse bugs to see how you can contribute! | |||
To set up a local system for development, see the [https://hg.mozilla.org/automation/mozillapulse/file/tip/HACKING.md HACKING.md] file included in the mozillapulse source. | |||
=== Status === | === Status === | ||
At the moment, only | At the moment, only BuildBot messages (BuildMessage, TestMessage) and [[BMO/ChangeNotificationSystem|SimpleBugMessages]] are being published to Pulse. | ||
There used to be two other publishers, which have been disabled: | There used to be two other publishers, which have been disabled: | ||
| Line 30: | Line 36: | ||
* Protocol used to talk to the broker is [http://en.wikipedia.org/wiki/AMQP AMQP]. | * Protocol used to talk to the broker is [http://en.wikipedia.org/wiki/AMQP AMQP]. | ||
* Messages are in JSON. | * Messages are in JSON. | ||
* For | * For the Python mozillapulse package, the underlying library currently used to talk AMQP is [http://kombu.readthedocs.org/ Kombu]. | ||
=== Road Map === | === Road Map === | ||
See the [http://mzl.la/1pc2F3M prioritized bug list] for all open issues. | |||
==== Website ==== | ==== Website ==== | ||
* {{bug|1017957}} Merge above in with PulseGuardian; no point in having two websites. | * {{bug|1017957}} Merge above in with PulseGuardian; no point in having two websites. | ||
* Indicate current | * Indicate current Pulse status (at least just up/down). | ||
* (Maybe) Display published messages on the | * (Maybe) Display published messages on the Pulse website (mostly decorative but also an example of use in the browser). | ||
==== Management ==== | ==== Management ==== | ||
* Intelligently handle queues that start filling up. | * (Almost done!) Intelligently handle queues that start filling up. | ||
** See [[Auto-tools/Projects/Pulse/PulseGuardian|PulseGuardian]]. | ** See [[Auto-tools/Projects/Pulse/PulseGuardian|PulseGuardian]]. | ||
| Line 52: | Line 55: | ||
** {{bug|1013980}} Enable SSL by default in clients. | ** {{bug|1013980}} Enable SSL by default in clients. | ||
** Close non-SSL port eventually? | ** Close non-SSL port eventually? | ||
* Move to a tighter permission model. See the Security Model section below. | * Move to a tighter permission model. See the Security Model section below. | ||
| Line 78: | Line 76: | ||
* Only the user that created a particular queue should be allowed to consume from it. | * Only the user that created a particular queue should be allowed to consume from it. | ||
Since exchange and queue permissions go together, we'll need exchange and queue naming conventions mixed with restrictive permissions. Each | Since exchange and queue permissions go together, we'll need exchange and queue naming conventions mixed with restrictive permissions. Each user will be restricted to a particular exchange and queue naming prefix. Many users will be either consumers or publishers, but for simplicity, each user can do both. Users will have full permissions on <code>"^exchange/<username>/.*$"</code> and <code>"^queue/<username>/.*$"</code>. They will also have read permissions to exchange/*. This will both prevent users from writing to other users' exchanges as well as prevent them from consuming from other users' queues. For convenience, if a consumer creates a nondurable queue, mozillapulse can assign a random suffix to the user's standard queue name prefix, i.e. <code>queue/<username>/<random string></code>, since the user wouldn't be able to create nor access a completely random server-assigned name. | ||
Note that this doesn't prevent a consumer from creating an exchange named as a queue, since the permission model doesn't distinguish between queues and exchanges, and consumers need the ability to create queues. This is not particularly problematic, since no one would have permission to use that exchange. | Note that this doesn't prevent a consumer from creating an exchange named as a queue, since the permission model doesn't distinguish between queues and exchanges, and consumers need the ability to create queues. This is not particularly problematic, since no one would have permission to use that exchange. | ||