Confirmed users
1,927
edits
Auto-tools/Projects/Pulse: Difference between revisions
Jump to navigation
Jump to search
Added spec
(Start with text instead of title) |
(Added spec) |
||
| Line 13: | Line 13: | ||
=== System Description === | === System Description === | ||
Pulse isn't any one thing. At its heart, it is a RabbitMQ system with a particular configuration and a set of conventions for using it along with a management tool, [[Auto-tools/Projects/Pulse/PulseGuardian|PulseGuardian]], to make Pulse as automated and self-serve as possible. Pulse follows the pub-sub pattern, in which publishers send messages to topic exchanges, and consumers create queues bound to these exchanges in order to subscribe to the publishers' messages. | Pulse isn't any one thing. At its heart, it is a RabbitMQ system with a particular configuration and a set of conventions for using it along with a management tool, [[Auto-tools/Projects/Pulse/PulseGuardian|PulseGuardian]], to make Pulse as automated and self-serve as possible. Pulse follows the pub-sub pattern, in which publishers send messages to topic exchanges, and consumers create queues bound to these exchanges in order to subscribe to the publishers' messages. | ||
==== Specification ==== | |||
Pulse is a managed [https://www.rabbitmq.com/resources/specs/amqp0-9-1.pdf AMQP 0-9-1] | |||
service with [https://www.rabbitmq.com/extensions.html RabbitMQ extensions] for publishing messages from Mozilla | |||
infrastructure. The aim is provide hooks that subscribers can | |||
use to integrate and extend Mozilla infrastructure. | |||
===== Authentication ===== | |||
Pulse credentials are managed and issued by [[Auto-tools/Projects/Pulse/PulseGuardian|PulseGuardian]], | |||
available at https://pulse.mozilla.org. This service SHALL issue | |||
an ''accessToken'' for any ''clientId'' that is registered | |||
with ''authorized'' email address. | |||
The accessToken is strictly secret and MUST NOT be shared | |||
publicly. The clientId is not secret. When establishing an AMQP | |||
connection, the clientId and accessToken MUST be used as | |||
''username'' and ''password'', respectively. | |||
===== Authorized Users ===== | |||
Pulse is intended to be open to all Mozillians who want to | |||
extend or integrate with Mozilla infrastructure. To guard against | |||
abuse PulseGuardian users MUST authenticate via Persona. PulseGuardian SHOULD verify that users have a vouched Mozillians profile. | |||
===== Publishers ===== | |||
Publishers MUST name ''exchanges'' in the form | |||
<code>exchange/<clientId>/<name></code>; attempts to name an exchange | |||
otherwise SHALL result in an authorization error. Exchanges MUST | |||
be ''topic exchanges'' and they MUST be declared ''durable''. | |||
Messages MUST contain a UTF-8-encoded [http://tools.ietf.org/html/rfc7159 JSON] payload, and | |||
their <code>Content-Type</code> MUST be <code>application/json</code>. | |||
Messages SHOULD NOT be larger than 8 kB; deviations may be | |||
feasible for low-traffic exchanges. Messages MUST NOT contain | |||
secret or sensitive information; all exchanges and messages | |||
SHALL be considered public. | |||
A message SHOULD carry a ''routing key'', in which fields have a | |||
fixed index from the left. Additionally, a message MAY be | |||
''cced'' to multiple routing keys, using the RabbitMQ | |||
[https://www.rabbitmq.com/sender-selected.html ''Sender-selected Distribution''] extension. | |||
Messages SHOULD be ''durable'' and SHOULD be published over | |||
RabbitMQ [https://www.rabbitmq.com/confirms.html ''confirm-publish'' channels]. Otherwise, the | |||
documentation MUST clearly reflect that messages from the | |||
given exchange do not exhibit deliver ''at-least-once'' semantics. | |||
===== Subscribers ===== | |||
Subscribers MUST name ''queues'' in the form | |||
<code>queue/<clientId>/<name></code>; attempts to name a queue otherwise | |||
SHALL result in an authorization error. Queues MAY ''consume'' | |||
from any exchange prefixed <code>exchange/</code>; attempts to consume | |||
from any other exchange SHALL result in an authentication error. | |||
Subscribers MAY limit the size of their queues using the RabbitMQ | |||
[https://www.rabbitmq.com/maxlength.html ''Queue Length Limit''] extension. Subscribers MUST NOT let | |||
their queues grow unbounded; if left unattended, ''Pulse'' | |||
SHALL notify the owner by email. Additionally, ''Pulse'' MAY delete | |||
a queue which exceeds defined limits. Subscribers SHOULD specify a | |||
prefetch limit using the RabbitMQ [https://www.rabbitmq.com/consumer-prefetch.html ''Consumer Prefetch'' limit] extension. | |||
Subscribers SHOULD use either ''durable'' queues or | |||
''auto-delete'' queues. Implementors are recommended to aim | |||
for deliver-''at-least-once'' semantics. | |||
===== Appendix A: Everything in Bullet Points ===== | |||
This is a summary of the above. | |||
Pulse: | |||
* MUST offer registration at <code>pulse.mozilla.org</code> | |||
* MUST support [https://www.rabbitmq.com/resources/specs/amqp0-9-1.pdf AMQP 0-9-1] and these RabbitMQ extensions: | |||
** [https://www.rabbitmq.com/confirms.html Confirms] | |||
** [https://www.rabbitmq.com/consumer-prefetch.html Consumer Prefetch] | |||
** [https://www.rabbitmq.com/maxlength.html Queue Length Limit] | |||
** [https://www.rabbitmq.com/sender-selected.html Sender-selected Distribution] | |||
* SHOULD exhibit deliver-''at-least-once'' semantics | |||
* MAY delete queues that grows beyond ''Pulse'' defined limits | |||
* SHALL notify owner by email when a queue grows close to ''Pulse''-defined limits. | |||
Publishers: | |||
* SHOULD use [https://www.rabbitmq.com/confirms.html confirm-publish channels] | |||
Exchanges: | |||
* MUST be named <code>exchange/<clientId>/<name></code> | |||
* MUST be topic exchanges | |||
* MUST be durable | |||
Messages: | |||
* MUST be UTF-8-encoded [http://tools.ietf.org/html/rfc7159 JSON] | |||
* MUST carry <code>application/json</code> as <code>Content-Type</code> | |||
* SHOULD be durable | |||
* SHOULD be less than 8 KiB (for good performance) | |||
* MAY be CC'ed to [https://www.rabbitmq.com/sender-selected.html multiple routing keys] | |||
* MUST NOT contain private or sensitive information | |||
* SHOULD have a routing key where fields have a fixed index from the left | |||
Subscribers: | |||
* SHOULD specify a [https://www.rabbitmq.com/consumer-prefetch.html ''consumer prefetch'' limit] | |||
Queues: | |||
* MUST be named <code>queue/<clientId>/<name></code> | |||
* MAY have a [https://www.rabbitmq.com/maxlength.html limited length] | |||
* MUST not grow unbounded | |||
=== Let's Use It === | === Let's Use It === | ||