Nobody can pay you.
Checkout is timing out. The card processor is fine. The database is bored. You start pulling threads, and forty minutes later you find it: the loyalty points service ran out of memory.
Loyalty points. A feature nobody would notice was missing for a week. It has stopped every sale in the shop.
Most Kafka-vs-RabbitMQ articles open with throughput charts, as if you were buying a phone. You cannot pick between them until you know what problem you have. So we start with the Friday night, and the tools arrive when they are needed.
How you got here
Nobody designed it this way. It grew, one reasonable decision at a time.
Version one of checkout did exactly one thing: charge the card. Twenty lines. You understood it completely.
Then the business had ideas, and each idea was one more line in that same function.
Three things went wrong at once, and only one of them woke you up on Friday.
The customer waits
They want to see “order confirmed”. Instead they are waiting on an analytics database in another region.
One break breaks all
A feature nobody would miss for a week takes down the till. There is no fuse between them.
Everything edits one file
Seven teams, one function, and a merge conflict every Friday afternoon.
Checkout knew about seven services. That was the bug — not the memory leak.
The fix: stop telling, start announcing
Here is the flip, and it is smaller than it sounds.
Checkout stops telling seven services what to do. It says one thing, out loud, to nobody in particular — “Order 1234 was placed” — and then it stops. It does not wait. It does not know who is listening, or whether anyone is.
Everyone who cares listens on their own. The email service hears it and sends an email. The warehouse hears it and starts packing. Loyalty hears it whenever it feels well enough.
That is event-driven architecture. Not a framework, not a product — a change in who knows about whom.
The old way, checkout knew about seven services. The new way, checkout knows about nothing. It announces a fact and walks away.
But do you actually need this?
Here is the honest part, and the part most articles skip. Event-driven is not free. You are trading one problem you can see for several you cannot, and plenty of systems should never do it.
So do not ask “is this modern?” Ask whether you have the symptoms.
- Every new feature means editing the same one file.
- Your main flow is slow, and most of the wait is optional work.
- One small service going down stops people paying you.
- Another team keeps asking, “can we get a copy of that data?”
- You wrote a cron job that polls a table to spot changes.
- You need an answer right now. “Is this card valid?” cannot be announced and forgotten.
- It must all succeed or all fail. Moving money wants a database transaction, not five services hoping.
- Small app, one person. You would add a broker to run and messages arriving out of order, to solve a problem you do not have.
- “Eventually” is unacceptable. For a moment the order exists and the points do not. If that is intolerable, stop here.
That cron job deserves a note, because it is the clearest tell of all. A job that runs every minute asking “has anything changed?” is a system begging to be told instead of having to ask.
A good rule: go event-driven when the number of people interested in a fact is growing — not when the traffic is.
Meet the middleman
If checkout announces things without knowing who listens, something must stand in the middle and hold the message. That is a message broker.
Its job is small, and it is the whole point: take a message from one place, keep it safely, and give it to whoever should get it — even if that listener is currently face down on the floor.
You have more than two choices.
RabbitMQ
Handing out jobs. Flexible routing, very mature, running in ten minutes. Also speaks MQTT and STOMP, which matters more than you think the day a device needs to talk to it.
Kafka
Keeping a permanent, replayable record of everything that happened.
Redis Streams
Very fast, and probably already in your stack. Lighter guarantees.
AWS SQS / SNS
Nothing to run at all. Hard to beat if you already live on AWS.
NATS
Small, fast, simple. Lovely for service-to-service chatter.
For most teams it comes down to the first two. So what actually separates them?
A to-do list, and a diary
Forget the logos. Underneath, these are two different shapes — and almost every real difference falls out of that.
Four consequences follow, and they are the whole comparison.
No. Once a worker says “done”, it is gone. To replay you must have saved it somewhere else first — at which point you have hand-built a small, worse diary.
One queue, one worker: perfect order. Add a second worker for speed and you have just given it up.
The broker does — and this is its superpower. Send order.created.eu and it lands in three queues by pattern match, the producer knowing none of them. One line of config.
Yes. This one is urgent. That one expires in ten minutes. This one failed three times — dead-letter it for a human.
It pushes. The broker hands messages to consumers and tracks who got what. Convenient — and it means the broker decides your pace.
A smart broker, simple consumers. Routing, delivery tracking and retries are the broker’s job. Your consumer just does the work.
Yes — move the bookmark back. Ship a listener with a bug, fix it, replay last week, damage undone. This one ability is why big companies standardise on it.
Order is kept per key — all of order 1234 in sequence. The catch: order and speed are the same dial, and the partition count is a decision you get stuck with.
You do, in your own code. Kafka has topics and nothing else. Anything cleverer is your problem.
No. None of it. Retention is per topic. There is no such thing as a high-priority offset.
Consumers pull. They ask for the next batch when ready, which is exactly why a slow reader cannot be overwhelmed and can catch up later.
A simple broker, smart consumers. Kafka stores bytes in order and little else. Tracking your place — and replaying — is your side’s job.
Four scenes where the choice is obvious
Enough theory. Here is each one being the obvious right answer.
A user uploads a 40 MB video
It needs transcoding. Twenty workers sit around one queue, each grabbing the next job when it comes free. One fails? Retry it three times, then dead-letter it so a human can look. Nobody will ever want to “replay all the videos from March”.
Why it fits: somebody has to do this, once, and then it is over.
A password reset, stuck behind a newsletter
You just queued 200,000 marketing emails. Now someone locked out of their account requests a reset. In RabbitMQ, that message is marked high priority and jumps the whole lot. In Kafka it waits behind all 200,000, because offset 200,001 comes after offset 200,000.
Why it fits: individual messages need individual treatment.
Fraud, spotted across three countries in ten minutes
The same card, five purchases, in this sequence, in three countries. Answering that needs order and history — not a message that vanished the moment somebody read it. And the analytics team is reading the exact same stream for something entirely different.
Why it fits: the message is a fact, and more than one system cares.
The new team that wants last year’s data
A team forms in March and wants to build something from everything that happened since January. They point a reader at the beginning of the topic and let it run. Nobody else is asked to do anything. No migration, no export, no meeting.
Why it fits: one of the interested parties did not exist yet.
Three ways this goes wrong
Every honest recommendation includes the cost. These are the ones that ruin days.
The 4,000 emails that never came back
Your email listener has a bug. It marks each message done, then quietly discards it. Four thousand order confirmations, gone. You find the bug and fix it in ten minutes.
Now what? The messages were acknowledged and deleted. They are not coming back. Your only recovery is whatever you happened to write into your own database.
With a log this is: move the bookmark back, restart, done.
The bug that only appears when you speed things up
One queue, one worker: perfect order. The queue backs up, so you add four more workers. Now two “address updated” events for the same customer are processed in parallel — and the older one finishes last and wins.
Intermittent, invisible in testing, and horrible to find. Ordering and parallelism are a trade, and adding workers silently spends it.
One poisonous message stops the whole line
In a queue, a job that keeps failing goes to the dead-letter pile and everything else carries on. In Kafka, a message that cannot be processed blocks its partition — every message behind it waits.
The fix is retry topics and a dead-letter topic, which you design, build and maintain yourself. RabbitMQ ships that in a checkbox.
“Exactly-once” does not mean what you hope
Kafka advertises exactly-once semantics, and teams read that as “I can stop worrying about duplicates.” The guarantee is real, but it applies inside Kafka — reading a topic, processing, writing back to a topic.
The moment your consumer writes to something else — Postgres, Stripe, an email provider — you are outside the guarantee. Crash between “charged the card” and “committed the offset” and the card gets charged twice on restart.
You still need idempotent consumers. Exactly-once buys you less than the name suggests.
And the quiet one that catches most teams: Kafka is heavy for small jobs. You wanted background emails. You now think about partitions, consumer groups, offsets, rebalancing and retention — and you run a cluster. A single RabbitMQ node runs on a laptop and does the job. Choosing Kafka for an email queue is a real mistake, and a common one.
The thirty-second answer
Two questions get you almost all the way.
Notice what the question is not about: speed. People reach for Kafka believing they have too much traffic. Put the numbers next to each other and see how rarely that is true.
Kafka genuinely is about a hundred times faster. Both ceilings are still so far above a normal application that throughput is almost never the deciding factor — and picking on it is how teams end up running a cluster to send password resets.
If you pick a queue and then start writing every message to a database “just in case” — you needed a log. You have built one by hand, badly.
Most real systems use both
The idea that one replaces the other is the actual mistake.
A completely ordinary shape looks like this: Kafka carries facts between services; RabbitMQ carries jobs inside them.
The failure mode was never picking the wrong broker. It is picking one without ever deciding which of the two your messages are — and then spending a year hand-building the other one’s behaviour on top of it.
And on the Friday night, it would have made no difference which one you had chosen. Both would have kept the shop open.
Where to read more
Three pieces worth your time, and what each adds beyond this one.
