Services/MessageQueuing

Planning Questions

What is the simplest MQ system and features to add from there?

• Put a single message on a queue for later retrieval in a FIFO manner, non-persistent, MQ service polling
• Transient messages (Semi-durable, higher throughput at the possible loss of some recent messages in a crash)
• Persistent messages (Accessible until consumed)
• Prioritizing of messages? Or allow for multiple queues and have the client prioritize where it pulls messages from?
  
• Allow messages to go un-acknowledged to allow a client to 'time-out' on handling it?
• Message push (persistent MQ connection), reduces latency but increases complexity. Pub/Sub or load-balanced on multiple connected clients?

Trade-offs of MQ implementations

• Levels of durability, and their throughput characteristics
• Prioritization of messages
• Out of band task vs. remote actor

Crypto

• Payload could be encrypted at the client layer
• If routing keys are used (RabbitMQ style brokering), those could be encrypted as well

Apps should be able to create both durable and temporary queues on-demand, as well as assign a TTL to a queue for temporary use.