Interview frequency: …
Short answer
RabbitMQ provides flexible, complex message routing (smart broker, dumb consumer) and deletes a message after it is processed. Kafka is a high-speed sequential event log that retains messages for long periods while consumers track their own position (dumb broker, smart consumer).
Explanation
RabbitMQ is excellent for disparate background tasks and asynchronous request processing that needs precise routing (routing keys, topic exchanges) and message priorities. Kafka is designed for big-data streams and millions of events per second: events are written to sequential partitions on disk, and subscribers can read messages or replay them from the beginning at any time without deleting them.
Code example
# RabbitMQ: a work queue — the broker deletes the message after the consumer acknowledges it
def handle(channel, method, props, body):
send_email(body)
channel.basic_ack(method.delivery_tag)
channel.basic_consume(queue="emails", on_message_callback=handle, auto_ack=False)
# Kafka: an append-only log — consumers track their own offset and can replay history
from kafka import KafkaConsumer
consumer = KafkaConsumer("orders", group_id="billing", auto_offset_reset="earliest")
for record in consumer:
process(record.value) # the record stays in the log; another consumer group can read it againCommon mistakes
- Using Kafka for simple individual background tasks, adding huge operational infrastructure complexity without a need.