|
|
| (16 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| = Overview =
| | #REDIRECT[[Services/Sagrada/Queuey]] |
| | |
| The '''Message Queue''' project is part of [[Services/Sagrada|Project Sagrada]]
| |
| , providing a service for applications to queue messages for clients.
| |
| | |
| = Project =
| |
| | |
| == Engineers ==
| |
| | |
| * Ben Bangert
| |
| | |
| == User Requirements ==
| |
| | |
| === Phase 1 ===
| |
| | |
| The first version of the MQ will focus on providing a useful service for the
| |
| [[Services/Notifications|Notifications project]] to enqueue messages for many
| |
| queues and for clients to poll for messages on a given queue.
| |
| | |
| No message aggregation is done in this version, if a client needs to get
| |
| messages for more than one subscription it must poll each queue and aggregate
| |
| them.
| |
| | |
| The first version could be considered a '''Message Store''' rather than a
| |
| ''queue'' as it supports a much richer set of query semantics and does not let
| |
| public consumers remove messages. Messages can be removed via the entire queue
| |
| being deleted by the App or by expiring.
| |
| | |
| Requirements:
| |
| | |
| * Service App can create queues
| |
| * Service App can add messages to queue
| |
| * Messages on queues expire
| |
| * Clients may read any queue they are aware of
| |
| | |
| === Phase 2 ===
| |
| | |
| The second version allows authenticated applications to queue messages and for
| |
| clients to consume them. This model allows for a worker model where jobs are
| |
| added to a queue that multiple clients may be watching, and each message
| |
| will be given to an individual client.
| |
| | |
| Optionally, the client can 'reserve' a message to indicate it would like it,
| |
| and if the client does not verify that it has successfully handled the message
| |
| it will be put back in the queue or to a fail/retry queue if desired.
| |
| | |
| Requirements:
| |
| | |
| * Service App can create queues
| |
| * Service App can add messages to queue
| |
| * Authenticated clients may consume a message from a queue
| |
| * Authenticated clients may mark a message for consumption with a reservation TTL
| |
| | |
| == Proposed API (Phase 1) ==
| |
| | |
| For the first version, applications allowed to use the '''Message Queue'''
| |
| will be given an application key that must be sent with every request, and
| |
| their IP must be on an accepted IP list for the given application key.
| |
| | |
| === Internal Apps ===
| |
| | |
| '''POST /new'''
| |
| | |
| Creates a new queue, takes the queue name as POST param ''queue_name''.
| |
| | |
| '''DELETE /{queue_name}'''
| |
|
| |
| Deletes a given queue created by the App.
| |
| | |
| '''POST /{queue_name}/clear'''
| |
|
| |
| Clears the given queue of all messages, without deleting it.
| |
| | |
| '''POST /{queue_name}'''
| |
| | |
| Create a message on the given queue. Contents is expected to be
| |
| a JSON object.
| |
|
| |
| Raises an error if the queue does not exist.
| |
| | |
| | |
| === Clients ===
| |
| | |
| '''GET /{queue_name}'''
| |
|
| |
| Returns all messages for the queue. Can optionally be passed
| |
| one of several query parameters:
| |
|
| |
| ''since_timestamp'' - All messages newer than this timestamp
| |
| ''limit'' - Only return N amount of messages
| |
|
| |
| Messages are returned in order of newest to oldest.
| |
| | |
| | |
| == Use Cases ==
| |
| | |
| === Notifications ===
| |
| | |
| The new version of [[Services/Notifications/Push|Notifications]] will use the
| |
| MQ to store messages for users from various websites and allow users to poll
| |
| the MQ directly for new notifications they have allowed.
| |
| | |
| * Internal app add's messages to a queue by queue name
| |
| * Internal app can delete queues by name
| |
| * Devices can read a queue by name
| |
| * Devices can read the X most recent messages
| |
| * Devices can read messages since XXX
| |
| * Messages are encoded as JSON
| |