|
|
| (27 intermediate revisions by 4 users not shown) |
| Line 1: |
Line 1: |
| == Planning Questions ==
| | #REDIRECT[[Services/Sagrada/Queuey]] |
| | |
| What is the simplest MQ system and features to add from there?
| |
| | |
| *Put a single message on a queue for later retrieval in a FIFO manner, non-persistent, via a RESTful service to push and poll for messages (send XX many at once per client instruction?)
| |
| *Transient messages (Semi-durable, higher throughput at the possible loss of some recent messages in a crash)
| |
| *Persistent messages (Accessible until consumed)
| |
| *Prioritizing of messages? Or allow for multiple queues and have the client prioritize where it pulls messages from?
| |
| *Allow messages to go unacknowledged to allow a client to 'time-out' on handling it?
| |
| *Message push (persistent MQ connection), reduces latency but increases complexity. Pub/Sub or load-balanced on multiple connected clients?
| |
| | |
| Trade-offs of implementation
| |
| | |
| *Non-persistent allows use of memcached for higher throughput
| |
| *Fully persistent (i.e., backed by a db) would be much slower
| |
| | |
| Crypto
| |
| | |
| *Payload could be encrypted at the client layer
| |
| *If routing keys are used (RabbitMQ style brokering), those could be encrypted as well
| |
| | |
| Apps should be able to create both durable and temporary queues on-demand, as well as assign a TTL to a queue for temporary use.
| |
| | |
| Authentication
| |
| | |
| *maybe [[Services/AppKeys|AppKeys]] would work, depending on if only the App could send messages... might need something else if end-user clients can directly push messages via Javascript
| |
| | |
| Message Handling
| |
| | |
| *Should the MQ Service do anything beyond merely holding messages and allowing connections for their retrieval?
| |
| *Simplest has no built-in handling, app would need to poll MQ service for new messages
| |
| *Allow a message payload to contain a callback to a RESTful service (URL + POST payload)?
| |
| | |
| Queue Workers?
| |
| | |
| *Introduces additional challenge of requiring Apps to have the code the worker should call for a message accessible to the MQ Worker Service
| |
| *Feels like a separate service from the MQ Service (though highly related)
| |
| *Requires worker pools on machines, worker watchers, resource monitoring, worker killers for excessive resource usage
| |
| *Provide 'built-in' queue workers for delayed common tasks? i.e. sending email, update caches.
| |