Skip to main content
Version: Next

RabbitMQ vs. Apache Kafka®

Most existing comparisons of these two systems were written years ago by someone who sells one of them. This one is written by the team that builds RabbitMQ, so bear that in mind — but we have tried hard to describe Kafka the way its own documentation describes it, and we are explicit about the things Kafka does that RabbitMQ does not. If anything here reads as unfair to you, we would genuinely like to know — tell us.

The short version: the two systems started from opposite ends of the problem and have been growing towards each other for years. RabbitMQ built its reputation as the default choice for messaging and task queueing; Kafka built its reputation as the default choice for event streaming. RabbitMQ then gained a durable, replicated log in 2021, and Kafka gained queue semantics in 2026. Their high-level capabilities now overlap substantially.

If you are looking for a high-level summary and want to skip the technical details, you can jump directly to the use cases.

The line that used to divide them​

For a long time the advice was easy to give:

RabbitMQ for messaging and queueing, Kafka for streaming.

It was good advice. It is now out of date on both sides.

RabbitMQ has had a real log since 3.9 (2021). Streams are an append-only, replicated, disk-based log with non-destructive reads, offset-based positioning, retention policies and a dedicated protocol built for throughput. RabbitMQ 3.11 (2022) added super streams — partitioned streams — along with single active consumer, giving ordered, partition-parallel consumption. Streams have been a mature, widely deployed feature for several release series, not a recent bolt-on. This is not a queue wearing a log costume; it is a log, designed as one, with the same mechanical sympathies that make Kafka fast.

Kafka has had queue semantics since 4.2 (2026). KIP-932 "Queues for Kafka" introduced share groups, production-ready as of Kafka 4.2. Share consumers acquire individual messages under a time-limited lock, acknowledge them one at a time, and can outnumber the partitions they read from. Kafka's own documentation is refreshingly direct about why this was needed: "some traditional messaging workloads are not a good fit for consumer groups."

The idea had been circulating for years. In April 2020 the RabbitMQ team sketched its own design for competing readers over a stream — strikingly close in shape to share groups in Kafka, four years before KIP-932 was filed. We claim no influence; the log imposes similar constraints on everyone who builds this.

So the honest framing is not "log versus queue". It is: two mature systems, each of which now does both things, each still carrying the assumptions of where it started. Kafka's queue semantics are a consumption mode layered on a log. RabbitMQ's queue is a queue, and RabbitMQ's log is a log.

Two models, side by side​

How RabbitMQ organises data​

RabbitMQ splits the problem in two. Publishers send to an exchange. Bindings — rules attached to the exchange — decide which queues or streams receive a copy. Exchanges and bindings are just metadata living inside RabbitMQ that decide on the routing algorithm and therefore define the routing and messaging topology.

Behind the exchange you choose a queue type. Queue types are genuinely different data structures storing your messages:

TypeStorageReadsBest at
Quorum queueRaft-replicated log, fsynced before confirmDestructiveWork distribution where data safety and high availability matters
Classic queueLocal, per-message persistenceDestructiveTransient, high-churn, single-node queues
StreamReplicated append-only logNon-destructiveFan-out, replay, large backlogs, high throughput
JMS queue (VMware Tanzu RabbitMQ)Raft-replicated log, fsynced before confirmDestructive or non-destructiveJMS applications needing selectors or queue browsers
MQTT QoS 0 queueNoneDestructiveLow-latency fire-and-forget fanouts to MQTT QoS 0 subscribers

A single RabbitMQ cluster runs all of these at once.

How RabbitMQ streams organise data​

Streams deserve their own picture, because this is where the comparison with Kafka is closest — and where readers coming from Kafka usually want a mapping.

A super stream is a logical stream partitioned into individual streams. Roughly: a super stream corresponds to a Kafka topic, and a stream to one of its partitions. The mapping is not exact, and the difference is worth understanding: a RabbitMQ stream is a first-class, individually named object that exists on its own, whereas a Kafka partition is a subordinate of a topic and has no independent identity. You can create, bind, and consume a single RabbitMQ stream without any super stream in sight.

Two things in that diagram are worth calling out.

Stream protocol publishers write straight to the stream. A client using the RabbitMQ Stream protocol discovers which node hosts the stream leader and publishes to it directly, with no exchange and no routing step in the path — the same shape as a Kafka producer writing to a partition leader, and for the same performance reason.

Everything else can still use the exchange. A stream is also integrated as an ordinary "queue" type, so a device publishing over MQTT, a microservice publishing over AMQP 1.0, and a legacy application publishing over AMQP 0-9-1 can all land messages in the same stream by way of a binding. You get the log's throughput without forcing every producer in your estate onto one protocol. Kafka has no equivalent: to write to a Kafka topic, you have to speak the Kafka protocol.

KafkaRabbitMQDifference that matters
TopicSuper streamA super stream is a logical grouping of real, independently usable streams
PartitionStreamA stream is a first-class named object; a partition is not
OffsetOffsetBroker-side offset tracking in both
Record batchChunkSame idea, same purpose: amortise everything
Consumer groupSingle active consumer per partitionSame ordered, partition-parallel consumption
Producer to partition leaderStream protocol client to stream leaderSame; RabbitMQ also accepts writes via exchanges from other protocols

How Kafka organises data​

A Kafka topic is split into partitions. A partition is an ordered, immutable sequence of messages on disk. Producers append to the end; each message gets a monotonically increasing offset. Messages with the same key land in the same partition, which is how Kafka gives ordering guarantees for related events. Consumers track their position by offset and can rewind or replay at will.

Routing happens in the producer: the client picks the partition, usually by hashing a key. The broker does not inspect the message. Consumption parallelism in a consumer group is bounded by the partition count, because each partition goes to at most one consumer in the group. Share groups lift that bound — that is largely what they exist for.

This is an excellent design for a log, and it is why Kafka is so hard to beat at replaying terabytes of history.

Where the two designs agree: streaming​

This section exists because comparisons usually skip it, and skipping it produces a wrong mental model. RabbitMQ streams and Kafka partitions solve the same problem in strikingly similar ways.

Batching as the unit of work. Kafka's design document explains that its protocol is built around a "message set" abstraction so that network requests can group messages together: "The server in turn appends chunks of messages to its log in one go, and the consumer fetches large linear chunks at a time." RabbitMQ streams use exactly this approach: a stream segment is made of chunks, and a chunk is made of messages. Both systems also support compressing a batch once rather than each message individually — RabbitMQ calls these sub-entry batches.

One binary format from producer to disk to consumer. Kafka standardised its message format across producer, broker and consumer so data moves between them without re-encoding. RabbitMQ streams do the same, using the AMQP 1.0 message format as both the on-disk and on-wire representation.

Zero-copy reads. Because the on-disk format is the on-wire format, both brokers hand a region of a log file straight to a socket with the sendfile system call, skipping several copies through user space. Both lose that optimisation under TLS.

Page cache over heap. Neither broker keeps the log in its own process memory. Both write to the filesystem and let the operating system's page cache do what it is good at. A caught-up consumer is served from RAM without reading from disk.

Publisher-side deduplication. Kafka's idempotent producer assigns each producer an ID and deduplicates using a sequence number, so a retry after an ambiguous failure does not create a duplicate entry in the log. RabbitMQ streams offer the same guarantee through named producers and publishing IDs.

Raft for cluster metadata. Kafka replaced ZooKeeper with KRaft. RabbitMQ replaced Mnesia with Khepri, likewise built on Raft. Both projects concluded independently that cluster metadata deserves a real consensus protocol.

If you have internalised how Kafka achieves its throughput, you already understand how RabbitMQ streams achieve theirs.

Where the two designs differ: queueing​

With the shared design philosophies of streaming in mind, let's take a look at how RabbitMQ and Kafka diverge around queuing.

RabbitMQ messages are independent items of work​

tip

In Kafka, a message is a position in a shared log. In RabbitMQ, a message is an independent item of work.

That does not mean Kafka only tracks an offset: share groups durably track a delivery state and a delivery count for every message.

It also does not mean RabbitMQ reads your payload. For performance reasons a message body is opaque bytes to RabbitMQ exactly as it is to Kafka. What RabbitMQ parses is the message envelope: headers, annotations and properties, which is what makes broker-side routing, filtering, TTL, priorities, and scheduled delivery possible.

What it does mean is coupling. A Kafka message's fate is tied to its neighbours'. Its position in the log is fixed, a share group's progress is anchored at a start offset that advances only once every earlier message has reached a terminal state, and the bytes stay on disk until the topic's retention policy removes them. A RabbitMQ message has none of those ties: it can be held back, rescheduled, rerouted, or deleted on its own, without reference to the messages around it.

Quorum Queues use Raft all the way down to the individual message​

A quorum queue is a replicated state machine built on Raft. Distributed consensus is about getting different nodes to agree on a value, and in Raft that value is an ordered log of operations. Kafka uses Raft for cluster metadata. RabbitMQ uses it for that and for the queue itself.

The operations in a quorum queue's Raft log are individual, per-message events:

  • enqueue this message
  • this consumer granted credit — assign messages
  • that message was acknowledged — delete it
  • that message was rejected — dead-letter it
  • message processing failed — increase the delivery count and re-queue
  • hold that message until 14:32:10

Each one is replicated to a majority of members and flushed to disk before it takes effect, which means delivery state in RabbitMQ is as durable as the message itself.

That has a concrete payoff during failures. If the node hosting a quorum queue leader dies, the new leader resumes with exactly the same view: which messages were acquired by which consumer, how many attempts each has had, and which were parked or scheduled for later. Consumers pick up where they left off. In Kafka, the delivery state and counter are durable, but the acquisition — which consumer currently holds a message — is state in the share-partition leader's memory. A leader change therefore returns in-flight messages to Available to be delivered again. That is correct at-least-once behaviour, and for most workloads it is fine; however, it does mean a Kafka broker failover produces a burst of redeliveries, which matters if the work is expensive to repeat.

The RabbitMQ per-message toolkit​

This is where Kafka is lacking features that RabbitMQ provides. None of these are niche; they are the day-to-day mechanics of running work through a queue reliably. Each one needs the broker to make a decision about one message independently of the rest.

Message delays. A publisher can send a message that the queue accepts, replicates and confirms immediately but hides from consumers until a chosen time.

Delayed retry. A quorum queue sets a failed message aside and makes it available again after a backoff derived from its delivery count. A consumer can override that per message with an exact timestamp — so when a downstream API rate-limits one tenant with a Retry-After header, that single message is rescheduled for precisely when the API is ready while everyone else's work continues at full speed.

Message deferral. A consumer can park a message under a token of its choosing and request that exact message back later, on demand, when an external condition occurs — a supplier confirmation, a file landing, an approval. The message is held durably by the queue rather than in consumer memory.

Message TTL. Individual messages can have distinct TTLs expiring on their own schedule.

32 strict priority levels. An urgent message overtakes the backlog.

Dead-letter routing. A rejected message is re-routed through an exchange, so the failure path has the same routing power as the success path: different failure categories to different queues, owned by different teams.

The modified outcome. A consumer can add or change message annotations on the way back — recording why it failed, when, and on which consumer. The next consumer to receive the message sees them. Better still, when dead-lettering to a headers exchange, those annotations drive the routing, so the consumer can effectively choose which dead-letter queue its failure lands in: one queue for transient faults worth retrying, another for business-logic errors a human must inspect, another for unknown schemas.

Poison message handling. A message that keeps failing is dead-lettered instead of looping forever.

Consumer timeouts. A stuck consumer's messages are returned and handed to a healthy one.

Single active consumer and consumer priorities. Control over which consumer gets the next message, including hot-standby patterns.

Message interceptors. RabbitMQ can run your custom logic on every message — stamping, validating, annotating.

Disk that tracks your backlog. An acknowledged message is deleted and its space eventually reclaimed, so a queue converges towards empty. A Kafka partition's disk usage tracks your retention window instead: completed work is still stored, and still costs money, until retention expires.

What Kafka share groups do and do not change​

Share groups close a real gap. It is worth being precise about what they do, because the marketing around them is not.

They do give you: many consumers per partition, per-message acknowledgement, durable per-message delivery counting with a configurable limit, a 30-second default acquisition lock released automatically if a consumer dies, and explicit release / reject / renew delivery states. If your requirement is "spread work items across a variable pool of consumers and retry the failures", that is now a thing Kafka can do.

What Kafka share groups do not change is the storage layer underneath:

  • Acknowledging does not delete. The message stays in the partition until retention removes it, so you size disk for your retention window rather than your backlog.

  • No per-message TTL, priorities, delays, deferral, or consumer-annotated returns.

  • No dead-letter routing. Rejecting archives the message for that share group; where it should go and how it gets reprocessed is yours to build.

  • Head-of-line blocking is reduced, not eliminated. A share group's in-flight window runs from the share-partition start offset to the last offset fetched, and that whole span is capped — 2000 offsets by default (share.partition.max.record.locks). The start offset advances only once every earlier message has reached a terminal state, and messages above an unfinished one cannot leave the window even after they are acknowledged. So a single message that is still in flight — whether it keeps failing or is simply slow — pins the start offset while completed work piles up behind it, and once the span reaches the cap no further messages are fetched from that partition at all. The stall lasts as long as that one message stays unfinished. In RabbitMQ an unacknowledged message never blocks the delivery of another one: other consumers keep receiving work, and a problem message can be dead-lettered, delayed or parked out of the delivery path altogether.

  • Every read is served by the partition leader. A regular consumer can be steered to a nearby follower instead (KIP-392), if the operator configures replica.selector.class and the consumer sets client.rack. That KIP exists because reading from a distant leader is expensive:

    consumers are limited to fetching only from the leader, so there is no easy way to leverage locality in order to reduce expensive cross-dc traffic.

    Share groups do not get that escape hatch — the ShareFetch request carries no rack, its response carries no preferred read replica, and the broker always reads from the leader. In contrast, in RabbitMQ, a quorum queue delivers from whichever member is local to the consumer, leader or follower, with nothing to configure. Quorum queue consumers therefore read from the node they are connected to, which spreads read load off the leader, and — for a cluster stretched across data centers — keeps the delivery path inside the local site.

  • No server-side filtering. Neither consumer groups nor share groups can express "only give me the messages I care about"; every consumer receives everything in its assigned partitions and discards what it does not want, paying full network and deserialisation cost for the privilege.

  • Records remain opaque to the broker. No interceptors, no broker-side selectors, no routing on content.

Durability: what "acknowledged" actually means​

Martin Kleppmann writes in Designing Data-Intensive Applications:

Perfect durability does not exist: if all your hard disks and all your backups are destroyed at the same time, there’s obviously nothing your database can do to save you. [...] In practice, there is no one technique that can provide absolute guarantees. There are only various risk-reduction techniques, including writing to disk, replicating to remote machines, and backups — and they can and should be used together.

Therefore, a system's durability guarantee depends on which of these risk-reduction techniques are switched on. Both projects document their defaults clearly, so this comparison is easy to make fairly, and the difference is substantial.

Kafka relies on replication alone, by default. Its documentation is unambiguous:

Note that durability in Kafka does not require syncing data to disk, as a failed node will always recover from its replicas. We recommend using the default flush settings which disable application fsync entirely.

The per-topic flush.messages setting can force an fsync per message, but the documentation advises against it:

In general we recommend you not set this and use replication for durability

Be clear about what this means in practice. When a Kafka producer receives an acknowledgement with acks=all, the message is in the page cache of every in-sync replica. It is not necessarily on any physical disk. Kafka's default configuration does not flush the log itself. Instead, it relies on the operating system to write those pages to disk, meaning it can take several seconds by default before the data is physically stored.

The durability guarantee therefore holds as long as failures are independent. One machine dying is survivable, because the others still have the data and will eventually flush it.

warning

If every in-sync replica in your on-prem data center loses power at once, then messages Kafka has already acknowledged to the publisher - but not yet fsynced - are lost.

Kafka's design assumes hardware failures are independent, and a correlated power event is exactly what breaks that assumption. Spreading replicas across racks or availability zones makes it unlikely, and for most cloud deployments that is a perfectly reasonable trade — which is why it is the default.

That mitigation assumes separate zones with independent power. Many on-premises and private-cloud deployments run from a single facility on a single utility feed, where no amount of rack- or software-level separation delivers that — so it is worth confirming with your infrastructure team whether your "zones" are genuinely on separate power before relying on it.

For payments, orders, or anything with a legal or financial consequence, this should be a conscious decision rather than an inherited one.

RabbitMQ quorum queues do both, by default. A quorum queue issues a publisher confirm only once a majority of its Raft members have written and flushed the message to disk. Pull the power on the whole data centre and the confirmed messages are still there when it comes back. This is as durable as a practical system gets.

RabbitMQ streams make the same trade as Kafka, and we should say so. Streams do not explicitly fsync; like Kafka, they rely on replication plus the operating system's writeback, and they carry the same correlated-failure exposure.

What differs is having the choice at all. In Kafka, adopting queue semantics does not change the durability story: share groups consume from ordinary topics, so a work queue is stored exactly like an event stream and inherits the same page-cache exposure — with the discouraged flush.messages/flush.ms settings as the only levers. That matters because queueing workloads, such as orders, payments and jobs you must not drop, are usually the ones where durability matters most. In RabbitMQ you choose per destination: a stream where Kafka's trade is the right one, a quorum queue where it is not, in the same cluster.

Why fsync per message is cheap in RabbitMQ and expensive in Kafka​

Kafka's documentation gives two reasons for discouraging flush.messages:

The drawback of using application level flush settings is that it is less efficient in its disk usage pattern (it gives the OS less leeway to re-order writes) and it can introduce latency as fsync in most Linux filesystems blocks writes to the file whereas the background flushing does much more granular page-level locking.

Both points are true, and the architecture explains why they bite at scale.

In Kafka, a log is per partition. The flush check sits in the append path, inside that partition's own append lock, and each flush syncs the segment plus its offset, time and transaction indexes. Nothing coordinates flushing across partitions, because there is nothing shared to coordinate on: every partition is its own directory of segment files. Enabling per-message fsync therefore costs a separate flush for every partition. This is not an oversight — Kafka's design treats replication as the durability mechanism, so the flush path was never meant to be a hot one.

RabbitMQ starts from the opposite assumption, and its Raft library, Ra, is explicitly a multi-Raft implementation: one node runs thousands of independent Raft clusters — one per quorum queue — and they share a single write-ahead log (WAL). The WAL is a batching server: it collects operations arriving from many different quorum queues, writes them to the shared WAL file in one write, calls fsync once, and only then notifies each queue that its entries are durable. Because every quorum queue appends to the same file, a single fsync makes writes durable for all of them at once, so the number of fsync calls tracks the batching cadence rather than the number of queues. In Kafka, where each partition has its own files, one fsync cannot cover many partitions however the calls are scheduled.

tip

RabbitMQ is designed for all three: many queues, data safety and throughput at the same time. With thousands of quorum queues, every message is replicated and written to physical storage before it is acknowledged.

Kafka's architecture cannot give you all three: pick at most two, and the one you give up is usually data safety.

Disaster recovery​

VMware Tanzu RabbitMQ adds continuous schema and data replication to standby clusters in other data centres, so a site can be promoted after a disaster. Kafka's equivalent, MirrorMaker 2, ships with Apache Kafka and is built on Kafka Connect; richer multi-region capabilities come from commercial Kafka vendors.

Replicatedfsynced before confirmSurvives a site-wide power lossCross-site standby
Kafka topic — streaming and queue semanticsYesNo, discouraged by defaultNo, acknowledged messages can be lostMirrorMaker 2
RabbitMQ streamYesNo, same trade as KafkaNo, same exposure as KafkaWarm Standby Replication
RabbitMQ quorum queueYesYesYesWarm Standby Replication

Broker-side routing and filtering​

In Kafka, routing is a producer-side concern: the client hashes a key to choose a partition, and consumers subscribe to whole topics. Anything more selective happens in the consuming application. Consumer groups and share groups have no filters — it is why "selective fan-out to interested parties" tends to turn into topic sprawl on Kafka.

tip

One of the main strengths of RabbitMQ is its powerful model consisting of exchanges, bindings, and queues enabling flexible broker-side routing.

RabbitMQ puts routing in the broker, where it can be changed by clients dynamically at runtime without redeploying the client or broker.

Exchanges define the routing algorithm. RabbitMQ ships several built-in exchange types including direct, topic (supporting wildcard patterns like orders.eu.*.priority), fanout, headers, local random, consistent-hash - and more. Exchange-to-exchange bindings let you compose them into more complex topologies: For example, a JMS client subscribing to a JMS topic with a JMS message selector will create an exchange-to-exchange binding by using a topic exchange as the first filtering stage and the JMS message selector exchange as the second filtering stage.

Exchange types are an extension point - you can develop and deploy your own custom exchange types.

For streams, where consumers read a shared log rather than a private queue, RabbitMQ supports broker-side filtering too (full details):

  1. Bloom filters let the broker skip whole chunks on disk without decoding them — very cheap, and very effective when a stream carries many logical sub-topics.
  2. AMQP filter expressions match on specific message properties, and can be written as SQL expressions.

The practical consequence: when you have thousands of distinct event kinds, Kafka pushes you towards more topics or more partitions, each with its own metadata, file handles and replication overhead. RabbitMQ lets you keep one stream and filter server-side, so consumers pay network cost only for messages they actually want.

VMware Tanzu RabbitMQ extends the same idea to JMS queues with JMS message selectors, evaluated on the broker.

A worked example: orders.#​

Suppose a consumer wants every message about an order — any region, any priority. In RabbitMQ that is one binding to a topic exchange with the pattern orders.#, declared by the consumer at runtime. Publishers neither know nor care, and no existing consumer is affected.

Kafka has no direct equivalent, because there is nothing for the pattern to match: the broker does not look at a message's key or headers when deciding what to deliver. There are three workarounds, and each costs something.

  1. Split the data across many topics — orders.eu.priority, orders.us.standard and so on — then subscribe with a topic-name regex: consumer.subscribe(Pattern.compile("orders\\..*")). This genuinely works and is the closest analogue. But the producer has to commit to the taxonomy at write time, every combination becomes a topic carrying its own partitions, files, metadata and replication overhead, and adding a dimension means creating a new generation of topic names. Note too that pattern subscription belongs to the regular consumer: KafkaShareConsumer accepts only an explicit list of topic names, so Kafka's queueing mode cannot subscribe by pattern at all.
  2. One topic, filter in the consumer — straightforward, but every consumer fetches and deserialises every message in order to throw most of them away, paying the full network and CPU cost of the messages it does not want.
  3. A stream processing job — read the topic, filter, write a derived topic. Correct and idiomatic, but it is another application to deploy and monitor, a second copy of the data on disk, and additional end-to-end latency.

The underlying difference is who decides. In Kafka, selection happens at the topic level, and only there: the producer's choice of topic name fixes at write time what a consumer can subscribe to. In RabbitMQ, the consumer declares what it is interested in whenever it likes, and the broker does the matching.

Protocols and clients​

Kafka has one wire protocol. It is well documented and widely implemented. It is not an independent or formal standard. However, it has become a "de facto" standard because other "Kafka-compatible" streaming platforms have emerged in recent years. The officially maintained client is the Java one. Some commercial vendors support additional Kafka clients.

tip

RabbitMQ treats multi-protocol support as a feature: One broker for all your messaging use cases. Numerous clients in numerous languages are officially supported.

ProtocolRabbitMQApache Kafka
AMQP 1.0 (ISO/IEC and OASIS Standard)YesNo
AMQP 0-9-1YesNo
MQTT 3.1, 3.1.1, 5.0 (ISO/IEC and OASIS Standard)YesNo
STOMPYesNo
RabbitMQ Stream protocolYesNo
Kafka protocolNoYes
WebSocket (IETF standard)Yes: AMQP 1.0 over WebSocket, MQTT over WebSocket, and STOMP over WebSocketExternal community proxies only
JMS (Java Message Service API)Yes: supported in VMware Tanzu RabbitMQ over AMQP 1.0No
HTTPManagement HTTP APIAdmin API; REST proxies are vendor products

In RabbitMQ, a message published over MQTT from an IoT device can be consumed over AMQP 1.0 by a backend service and read again from a stream over the RabbitMQ Stream Protocol by an analytics job. RabbitMQ handles the protocol conversions transparently broker-side and documents them precisely.

This matters most at the edges of an architecture. If you are terminating IoT traffic, serving browsers, or integrating a Java estate that speaks JMS, RabbitMQ does it in the broker you already run. With Kafka each of those is a separate component — an MQTT broker, a WebSocket proxy, a JMS bridge — to deploy, secure, and monitor.

Security​

Both systems are enterprise-grade here, and neither should be a reason to rule the other out. The differences are in granularity and in what comes built in.

RabbitMQApache Kafka
Transport encryptionTLS on every protocol, including inter-nodeTLS on client and inter-broker listeners
AuthenticationSASL: Username/password, mTLS, x.509 certificates, LDAP, OAuth 2.0 / OIDC, pluggable backendsSASL: GSSAPI (Kerberos), PLAIN, SCRAM-SHA-256/512, OAUTHBEARER; mTLS
AuthorisationPer-vhost and per-user permissions supporting regular expressions, topic authorisations, ACLsACLs per resource, prefixed ACLs
Tenant isolationVirtual hosts: namespaces with their own exchanges, queues, permissions and limitsTopic naming conventions plus prefixed ACLs plus quotas
Resource protectionPer-vhost and per-user connection, and queue limitsClient and broker quotas, request rate throttling
Credential managementManagement UI, HTTP API, CLI, definitions import/exportCLI, Admin API

Two things are worth drawing out. First, virtual hosts are a real boundary, not a convention: a user with access to the payments vhost cannot see, name, or collide with anything in analytics, and you can export and re-import a vhost's entire topology as a definitions file. Kafka's documented multi-tenancy approach is a hierarchical topic naming structure enforced by prefixed ACLs — workable, widely used, and rather more of a convention than a boundary. Second, topic authorisation lets RabbitMQ authorise on the routing key, so permissions can follow your event taxonomy in addition to your queue names.

VMware Tanzu RabbitMQ adds FIPS 140-2 compliance, audit logging of internal events such as who deleted a queue, and continuous CVE scanning of both RabbitMQ and its dependencies.

Performance​

tip

Both systems can move more messages per second than the overwhelming majority of applications will ever produce.

"RabbitMQ is slow" is the most common misconception that exists in the marketplace. Let’s objectively review some numbers to see if this is indeed fact or fiction.

Throughput on a log is governed by how full the chunks are, in both systems. A stream fed one message at a time is slow for the same reason a Kafka partition fed one message at a time is slow. The Delivery Optimization blog post shows a 20x difference between well-batched (high ingress) and poorly-batched (low ingress) streams.

The RabbitMQ vs. Kafka experiments point the same way: for streaming, RabbitMQ Streams and Kafka land in the same ballpark on both throughput and latency. Neither has a structural speed advantage over the other here — expected, given how much of the underlying design the two share.

Treat any head-to-head benchmark you read elsewhere with that in mind. The two systems expose very different knobs — batch and chunk sizes, acks/confirms settings, replication factors, compression, disk and network setup — and a benchmark that tunes one side hard while leaving the other on defaults is comparing tuning effort, not what the systems are capable of. Always be sure to read and understand the configuration before the result.

Operational considerations​

A management UI in the box. RabbitMQ ships a management plugin with a web UI, an HTTP API and the rabbitmqadmin CLI — inspect queues, trace messages, change policies, export and re-import entire topologies. Apache Kafka ships CLI tools; web UIs are third-party or commercial. VMware Tanzu RabbitMQ adds a Stream Browser for inspecting stream contents by offset or timestamp from the UI.

Windows. Kafka's documentation says:

We have seen a few issues running on Windows and Windows is not currently a well supported platform.

tip

RabbitMQ supports Windows as a first-class platform.

Some organisations — often in finance, manufacturing, healthcare and the public sector — run their on-premises infrastructure under a Windows Server standard as a matter of policy. Where that policy applies, it decides the question before any other criterion is considered.

Kubernetes. RabbitMQ runs on Kubernetes and has official Kubernetes Operators maintained by the RabbitMQ team. Kafka's most popular operator, Strimzi, is an excellent CNCF project, but it is not part of Apache Kafka.

Deployment surface. Both run on bare metal, VMs, containers, on-premises and in every major cloud, and both are offered as managed services by multiple vendors.

The runtime​

Kafka is implemented in Java and runs on the Java Virtual Machine (JVM). RabbitMQ is implemented in Erlang and runs on the BEAM.

The JVM is one of the great pieces of systems engineering: a superb JIT compiler, decades of optimisation, world-class profiling and debugging tooling, an enormous library ecosystem, and by far the largest talent pool of any managed runtime. For general-purpose server software it is an outstanding default, and Kafka's team has used it expertly.

But a message broker is not general-purpose server software. It is critical infrastructure: when your broker stops, every application that depends on it stops with it. For infrastructure in that position, the property that matters more than any other is that it stays up — under load, under partial failure, under misbehaving clients, for years at a time. And for that, you do not want an excellent general-purpose runtime. You want a runtime built specifically for staying up.

That is exactly what the BEAM is. The Erlang/OTP virtual machine — also the foundation of Elixir — was built at Ericsson to run telephone switches that were not permitted to stop. Today it also features a highly effective JIT compiler, but more importantly, it gives RabbitMQ properties that matter enormously for a broker:

  • Fault isolation. Every connection, session and queue is its own lightweight Erlang process with its own heap, sharing no memory and communicating only by message passing — the actor model, with roots in Hoare's communicating sequential processes (CSP). When one process crashes, its supervisor restarts it in microseconds while everything else carries on undisturbed. A malformed frame from one badly-behaved client is a non-event, not an incident.
  • Scalability. Erlang processes are cheap to create and destroy, so a node can host millions of them. One process per connection and per queue is not a design compromise; it is the natural unit.
  • Parallelism without locks. Give a RabbitMQ node more CPU cores and the work spreads across them: connection processes parsing frames, session processes routing, queue processes dispatching to consumers, writer processes serialising frames to the socket — all in parallel, with no application-level locking.
  • Garbage collection that stays local. The scheduler is pre-emptive, so one busy queue cannot starve the others. More importantly, there is no shared heap to collect: garbage collection runs per process on its own small heap, and that is true of both minor and major (fullsweep) collections. A major GC pauses the one process it belongs to, not the broker. So a node's tail latency does not fall off a cliff as total memory grows, and GC tuning is not a prerequisite for going to production.
tip

Fault tolerance, scalability and parallelism are properties RabbitMQ inherits from its runtime, not features it had to build. The choice to use Erlang and BEAM was deliberate. If we were starting RabbitMQ from scratch today, we would choose it again.

Kafka's own documentation acknowledges the other side of this trade. It is precisely why Kafka keeps data out of the heap and in the page cache, noting that

  1. The memory overhead of objects is very high, often doubling the size of the data stored (or worse).
  2. Java garbage collection becomes increasingly fiddly and slow as the in-heap data increases.

That is a sound and well-executed workaround. It is still a workaround for a problem the BEAM does not have, to begin with.

Monitoring​

RabbitMQ exposes Prometheus metrics natively. The broker serves a /metrics endpoint directly, and the project maintains a set of Grafana dashboards to go with it. Point Prometheus at your nodes and you have observability with no additional software in the path.

Kafka reports metrics through JMX. Getting them into Prometheus means either running a JMX exporter agent inside every broker JVM, or enabling remote JMX and scraping that. Either way there is real operational cost:

  • An extra moving part per broker — with its own version and failure modes, inside the process you are trying to keep healthy.
  • A mapping file to maintain — JMX metric names do not map cleanly onto Prometheus names, so you keep a set of regex rewrite rules. Kafka 4.2 renamed metrics to standardise on the kafka.COMPONENT convention, so those rules, and every dashboard and alert built on them, needed revisiting.
  • Security you have to add — Kafka's documentation notes that remote JMX, and JMX authentication, are both disabled by default.

Both approaches end in a Grafana dashboard. Only Kafka requires an agent and a mapping file in between.

What RabbitMQ does that Kafka does not​

What Kafka does that RabbitMQ does not​

These are real, they are not small, and if you need one of them then you need Kafka. We are aware of each and several are on our long-term roadmap; we are not going to attach dates to that here.

  • Tiered storage. Kafka can offload older log segments to object storage (KIP-405), decoupling retention from broker disk. One nuance worth knowing: Apache Kafka provides the framework but, in its own words, "doesn't provide an out-of-the-box RemoteStorageManager implementation" — the actual backend comes from a vendor or a third-party project. RabbitMQ streams are bounded by local disk.
  • Log compaction. Kafka can retain the last value per key indefinitely, turning a topic into a durable changelog you can rebuild state from — the backbone of change data capture. RabbitMQ streams retain by size and age only.
  • The stream processing ecosystem. Apache Flink, Apache Spark, Iceberg integrations, ksqlDB and a long tail of connectors treat Kafka as the default source and sink. Kafka Connect itself is part of Apache Kafka, though many production-grade connectors and the Schema Registry are vendor products rather than Apache Kafka components. RabbitMQ has a Spark connector in VMware Tanzu RabbitMQ, and Shovel and Federation for broker-to-broker movement, but this is currently not approaching Kafka's breadth here.
  • Kafka Streams. A Java stream processing library — joins, windows, aggregations, state stores, exactly-once — running in your Java application with no separate cluster.

What's the best broker for my use case?​

With the mechanics on the table, here is how they cash out per workload. Most people arrive at this page with a use case in mind, so this is the short answer — and every entry traces back to something explained above.

Use caseRecommendationWhy
Task and job queues, background workersRabbitMQExclusive assignment per message, retries with backoff, dead-lettering, priorities. Kafka has no native answer to "retry this one job in 90 seconds".
Priority handling, SLAs, expiry, scheduled deliveryRabbitMQ32 strict priorities, per-message TTL, delayed delivery, deferral. Kafka has none of these.
Request/reply between services (RPC)RabbitMQEfficient Direct reply-to, exclusive queues, per-message correlation. Kafka needs response topics and correlation logic you write yourself.
Microservice event notificationsRabbitMQTopic exchanges route by pattern; a new subscriber is a binding. In Kafka, selective fan-out means new topics or client-side filtering.
Order processing, payments, anything you cannot loseRabbitMQQuorum queues replicate and fsync before confirming. Kafka's default is replication only.
IoT / device telemetry ingestRabbitMQNative MQTT for millions of device connections, into the same cluster that serves your backends. Kafka needs a separate MQTT broker and a bridge.
Per-tenant / per-device / per-workflow isolationRabbitMQThousands of cheap independent queues. Every Kafka equivalent costs a partition.
Migrating off legacy JMS brokersRabbitMQJMS support, with broker-side selectors and queue browsers in VMware Tanzu RabbitMQ.
Deploying on Windows ServerRabbitMQA first-class platform on desktop and server editions. Kafka's own documentation says Windows "is not currently a well supported platform".
Streaming with tens of thousands of logical subjects (event types, customers, SKUs)RabbitMQBroker-side Bloom filters let one stream serve many narrow subscriptions. A dedicated Kafka topic per logical subject does not scale well, and client-side filtering wastes network bandwidth reading everything to keep little.
Event streaming with replay and multiple independent readersEitherStreams and Kafka topics do the same job with the same techniques.
Exactly-once read-process-write pipelinesEitherKafka's transactional producer commits the result and the consumer offset together. RabbitMQ streams achieve the same outcome. Neither covers side effects on external systems.
Website activity tracking, metrics firehose, log aggregationEitherBoth handle millions of messages per second. Pick on ecosystem and operations, not throughput.
Stateful stream processing: joins, windows, aggregationsEitherRabbitMQ has an Apache Spark connector. However, Kafka has the broader ecosystem of integrations including Flink and Kafka Streams.
Retaining months or years of history cheaplyKafkaTiered storage to object storage. RabbitMQ streams are bounded by local disk.
Event sourcing with a keyed changelog you rebuild state fromKafkaLog compaction. RabbitMQ streams retain by size and age only.

If your list of ticks is entirely in the RabbitMQ or Either column, you do not need Kafka, and running it anyway is a second distributed system to staff, secure and upgrade for no return.

Mythbusters: A few old claims worth retiring​

"Use Kafka for streaming and RabbitMQ for queueing."​

True in 2019. Since RabbitMQ 3.9 streams and Kafka 4.2 share groups, both systems do both. What survives is how much the broker knows about each message — and there RabbitMQ is far ahead.

"RabbitMQ can't handle high throughput."​

A single stream provides a throughput of several million messages per second. This folklore was formed long before streams existed and has not been true since.

"RabbitMQ deletes messages as soon as they are consumed, so you cannot replay them."​

True for queues, false for streams. RabbitMQ streams are an append-only log that retain messages based on size or time limits, allowing consumers to rewind and replay history exactly as they do with Kafka topics.

"RabbitMQ needs a lot of RAM because it keeps messages in memory."​

Not on modern RabbitMQ. Both classic queues and quorum queues do not keep message bodies in memory, streams rely on the page cache exactly as Kafka does. The following blog post explains this in detail: How Are The Messages Stored? Not in Memory!.

"Queues for Kafka means you don't need RabbitMQ any more."​

Share groups add cooperative consumption, per-message acknowledgement and durable delivery counting on top of a log — genuinely useful, and enough for some work queues. What they do not add is the per-message toolkit: per-message TTL, priorities, delivery delays, delayed retry, deferral, a routable failure path, broker-side routing and filtering, custom message interceptors, modification of message headers, or reclaiming disk when the work is done; and they still cannot fully avoid head-of-line blocking. Crucially, Kafka has weaker durability guarantees. If none of that matters for your workload, Kafka's queue semantics will serve you well. If it does, the gap is not close.

"Only Kafka can do exactly-once processing."​

Both systems can make a read-process-write loop produce each result downstream once, and neither does it by executing the processing step only once. Kafka commits the result and the consumer offset in one producer transaction; RabbitMQ streams make the resume point part of the result, using the source offset as the publishing ID so that deduplication discards a replayed result. Different designs, same guarantee — which RabbitMQ calls effectively-once, the more honest name.

"Erlang is a liability — it's hard to troubleshoot."​

The language a broker is written in is not something its users interact with. Applications use client libraries in Java, .NET, Python, Go and more; operators use the management UI, CLI tools and Prometheus metrics. Diagnosing a production problem means reading logs, metrics and queue state — not broker source code. Erlang expertise is our job, not yours, and the Erlang runtime is one of RabbitMQ's biggest advantages.

"RabbitMQ's replication is bolted on."​

Kafka's design document writes:

Other messaging systems provide some replication-related features, but, in our (totally biased) opinion, this appears to be a tacked-on thing, not heavily used, and with large downsides: replicas are inactive, throughput is heavily impacted, it requires fiddly manual configuration, etc.

That was a fair description of RabbitMQ's old classic mirrored queues, which is why we entirely removed them in RabbitMQ 4.0. Quorum queues — production-ready since 2019 — are Raft-based, active, replicated-by-default data structures, and Jepsen tests run against quorum queues continuously. Quorum queues provide all three: scalability (thousands of queues), strong durability guarantees (replication and fsync), and high throughput — something Kafka cannot — by design.

Conclusion​

Apache Kafka is a well-engineered event streaming platform, and there are jobs it does better than any other messaging solution in the market. If you need log compaction, tiered storage, or the Kafka Streams Java library, use Kafka.

For everything else, our honest assessment is that RabbitMQ is the better default, and increasingly so.

It does both halves of the job. Its streams match Kafka's log design technique for technique — chunks, zero-copy reads, page cache, direct-to-leader publishing, deduplication — at several million messages per second, and they have been production technology since 2021. Its queues do things Kafka cannot: replicate per-message delivery state through Raft, route messages through exchanges, per-message TTL and priorities, hold a message until a chosen delivery time, retry one message in 3 minutes without pausing the others, park a message until an external condition occurs, and give the disk space back when the work is done.

RabbitMQ is more durable where durability counts: quorum queues replicate and fsync before confirming, so a data-centre power failure does not silently destroy messages your publishers were told were safe.

RabbitMQ is easier to operate. Prometheus metrics and a management UI in the box rather than a JMX agent per broker and a third-party console. Real virtual hosts instead of naming conventions. Five protocols instead of one, so IoT devices, browsers and JMS applications connect to the broker you already run rather than to three more components. First-class Windows support, which for a large number of enterprises is not a preference but a requirement.

And RabbitMQ runs on a virtual machine built for systems that are not allowed to stop.

Our recommendation: start with RabbitMQ, and add Kafka when you hit something only Kafka does. After all, one solution is cheaper to operate than two different solutions.

Learn more about RabbitMQ Streams​

info

Found something on this page that is out of date or unfair to Kafka? Kafka moves quickly and so do we. Please start a discussion — we would rather fix it than defend it.