They solve overlapping problems in genuinely different ways

Kafka, RabbitMQ, and Amazon SQS all move messages from a producer to a consumer, which makes them look interchangeable in a "which message broker should I use" search. They aren't. Kafka is a distributed, partitioned commit log built for high-throughput event streaming with replay. RabbitMQ is a traditional message broker built around flexible routing and per-message delivery guarantees. SQS is a managed queue service built for simplicity and zero operational overhead. Picking based on brand recognition rather than these underlying models is the most common broker-selection mistake we see.

Ordering guarantees

Kafka

Strict ordering within a partition, no ordering guarantee across partitions. You get ordering for a given key (e.g. all events for one orderId) by making sure that key always maps to the same partition - which Kafka does automatically by default.

RabbitMQ

Ordering within a single queue with a single consumer is guaranteed. Add multiple concurrent consumers on the same queue (common for throughput) and per-message ordering is no longer guaranteed across them.

SQS standard queues make no ordering guarantee at all - messages can arrive out of order. FIFO queues fix that, but cap throughput at 300 messages/second per message group (3,000/sec with batching) and trade some of SQS's operational simplicity for that guarantee.

Delivery semantics

Broker      Native guarantee           What that means in practice
----------  -------------------------  --------------------------------------------
Kafka       At-least-once (default)    Exactly-once achievable within Kafka-to-Kafka
                                        flows via idempotent producers + transactions,
                                        not guaranteed once a consumer's side effects
                                        (a DB write, an email) leave Kafka.
RabbitMQ    At-least-once (with acks)  Consumer must ack after successful processing.
                                        Unacked messages redeliver on consumer crash.
SQS         At-least-once              Standard queues can occasionally double-deliver
                                        even without a crash, by design of the service.

None of the three gives you true end-to-end exactly-once delivery once a consumer's side effect leaves the messaging system - that's exactly why the outbox pattern and idempotency keys exist as separate, broker-independent concerns. Choosing a broker doesn't remove the need for either.

Throughput and retention

Kafka is built to sustain very high throughput - hundreds of thousands of messages per second per broker is realistic - and it retains messages for a configured time window (or indefinitely) regardless of whether they've been consumed, which is what makes event replay possible: a new consumer can join and read from the beginning of a topic. RabbitMQ and SQS both remove a message once it's been successfully consumed and acknowledged - there's no built-in replay. If "reprocess everything from last Tuesday" needs to be a real capability, that alone pushes toward Kafka.

Operational overhead - the part that gets underweighted

Kafka: real infrastructure to run

Self-hosted Kafka means brokers, ZooKeeper or KRaft, partition rebalancing, and disk capacity planning as an ongoing job. Managed options (Confluent Cloud, MSK) remove most of this at a real dollar cost - which is usually the right trade for a team without dedicated platform engineers.

SQS: near-zero ops, by design

No servers, no capacity planning, no cluster to patch - it's a managed AWS API. The trade-off is you're accepting SQS's throughput ceiling, its lack of replay, and being tied to AWS as the operating environment.

RabbitMQ sits in between: self-managed like Kafka, but a materially smaller operational surface - typically a 3-node cluster with mirrored queues is enough for most workloads, and there's no partition-rebalancing complexity to reason about.

A practical decision framework

Need                                          Pick
--------------------------------------------  ------------------------
Event replay / multiple independent readers   Kafka
  of the same stream (analytics + audit +
  a new service, all reading the same events)
Complex routing (topic exchanges, priority     RabbitMQ
  queues, delayed delivery, dead-lettering
  built into the broker)
Already all-in on AWS, want zero ops, and      SQS
  don't need replay or sub-100ms latency
Very high sustained throughput (100K+ msg/s)   Kafka
Small team, no dedicated platform/infra role   SQS (or managed RabbitMQ)

Key takeaways

  • Kafka is a distributed log built for high-throughput streaming and replay - not a drop-in "message queue" in the traditional sense.
  • RabbitMQ offers flexible routing and per-message delivery control with materially less operational surface than self-hosted Kafka.
  • SQS trades throughput ceiling and lack of replay for near-zero operational overhead - a legitimate trade for small teams already on AWS.
  • None of the three delivers true exactly-once processing once a side effect leaves the broker - idempotency keys are still required regardless of choice.
  • Choose based on replay needs, ordering requirements, and who's actually going to operate the thing - not on which broker shows up first in search results.
If nobody on your team can articulate why Kafka was chosen over the alternatives beyond "it's what everyone uses," that's worth revisiting before it becomes the piece of infrastructure nobody wants to touch.