Pub/Sub (Publish-Subscribe)

A message queue is like a postal service: one sender, one recipient. The sender hands the package to the queue, a delivery worker picks it up, and it goes to exactly one destination.

Pub/Sub is like a radio broadcast. One station transmits, and every radio tuned to that frequency receives it simultaneously. The sender doesn't know who's listening, doesn't care how many listeners there are, and doesn't send a separate copy to each one.

That distinction — point-to-point vs fan-out — is what separates message queues from pub/sub, and understanding it determines which one you reach for.

Pub/Sub vs Message Queues

In a message queue, if you have 3 workers consuming from a queue, each message goes to exactly one of them. That's ideal for distributing work — you don't want 3 workers all trying to send the same email.

In pub/sub, if you have 3 subscribers to a topic, each message goes to all 3. That's ideal for events — when a user signs up, you want the email service, the analytics service, and the onboarding service to all know about it independently.

Topics and Subscriptions

The core abstraction in pub/sub is the topic — a named channel that publishers write to and subscribers read from.

Publishers are completely decoupled from subscribers. The analytics team can add a new subscriber to the "order.placed" topic without touching anything in the order service. The order service doesn't know or care who's listening.

Press enter or space to select a node. You can then use the arrow keys to move the node around. Press delete to remove it and escape to cancel.

Press enter or space to select an edge. You can then press delete to remove it or escape to cancel.

algobase.dev

Pub/Sub is the broadcast model. The Order Service publishes one event to the topic and does not know or care how many subscribers are listening. Every subscriber receives an independent copy — email, inventory, and fraud services all get the event simultaneously and process it at their own pace. This is the key difference from a message queue, where each message goes to exactly one consumer. The loose coupling is what makes pub/sub powerful: the Fraud team can subscribe to order.placed events without asking the Order team to change anything. You can add or remove subscribers at any time without touching the publisher. This is the architecture behind Netflix's event pipeline, Stripe's payment notifications, and most modern event-driven microservice systems.

1 / 10.5×1×2×3×Playground

Pub/Sub fan-out — one event published, all subscribers receive a copy independently

This loose coupling is one of pub/sub's biggest advantages. You can add new consumers without modifying producers — critical for large organizations where different teams own different services.

Real Systems

Apache Kafka

Kafka is the dominant pub/sub system at scale. It's built as a distributed commit log — events are written to topics, partitioned across brokers, and retained for a configurable period (days, weeks, forever). Consumers read from topics at their own pace and track their position (offset) independently.

Key Kafka properties:

• Durability: events are persisted to disk, replicated across brokers
• High throughput: LinkedIn (who built it) handles millions of events per second
• Replay: consumers can re-read events from any point in history
• Consumer groups: multiple consumers in the same group share partitions (queue semantics); multiple independent groups each get all events (pub/sub semantics)

Kafka is the right choice for event streaming, audit logs, data pipelines, or anything where you want event replay or very high throughput.

Google Cloud Pub/Sub

Google's fully managed pub/sub service. Handles global scale automatically with no cluster management. Messages are retained for up to 7 days. Supports push subscriptions (Google calls your endpoint) and pull subscriptions (you poll). Integrates natively with BigQuery, Dataflow, and Cloud Functions.

Use it for: GCP-native architectures, serverless event processing, global-scale messaging without operational burden.

AWS SNS (Simple Notification Service)

Amazon's fan-out messaging service. A topic can have many subscriptions — SQS queues, Lambda functions, HTTP endpoints, email addresses, SMS numbers. SNS is often paired with SQS in AWS architectures: SNS fans out to multiple SQS queues, each of which is consumed by a different service (the "SNS + SQS" pattern).

Use it for: AWS-native event fan-out, multi-protocol notifications, triggering multiple services from one event.

Redis Pub/Sub

Redis has a built-in pub/sub mechanism. Publishers push to a channel, all subscribers get the message immediately. It's simple and fast — but ephemeral. If no subscriber is connected when a message is published, it's lost. Redis Pub/Sub has no persistence, no message history, no consumer groups.

Use it for: real-time broadcasting where message loss is acceptable — live dashboard updates, gaming events, presence notifications. Not for critical business events.

Use Cases

Event Streaming

Kafka was built for this. Every action in your system becomes an event in a stream: user clicked, order placed, payment processed, item shipped. These events flow through Kafka, and any service that cares can subscribe and react.

Netflix uses Kafka to process billions of events per day — playback events, recommendation signals, A/B test data, infrastructure metrics. All flowing through a central event stream.

Notifications at Scale

When a social media post goes viral, millions of users might need to be notified. A pub/sub system fans that event out to notification workers that handle push notifications, email digests, and in-app alerts, all independently, all at scale.

Analytics Pipelines

Clickstream data, page views, API calls, feature usage — all published to topics, consumed by analytics services that write to data warehouses (BigQuery, Snowflake, Redshift). The application code doesn't need to know anything about the analytics stack; it just publishes events.

Cache Invalidation

When a product's price changes in the source-of-truth database, a "price.updated" event is published. All caching services subscribed to that topic invalidate or refresh their cached price — without the price service needing to know which caches exist.

Microservice Communication

In a microservices architecture, services can communicate via events rather than direct API calls. The order service publishes "order.placed". The inventory, email, analytics, and fraud services all subscribe and react independently. Adding a new service that cares about orders means subscribing to the topic — zero changes to the order service.

Ordering and Exactly-Once: The Hard Parts

Pub/sub ordering is tricky. Kafka guarantees ordering within a partition, but events in the same topic spread across multiple partitions may arrive out of order at a subscriber. The common solution: use a partition key (like user ID or order ID) so all events for a given entity go to the same partition.

Exactly-once delivery in distributed pub/sub is notoriously difficult. Kafka added exactly-once semantics in version 0.11 for transactional producers, but it requires careful configuration. Most systems settle for at-least-once delivery and design consumers to be idempotent.

Summary

Pub/Sub is the broadcast model of async communication — publishers write events to topics, and every subscriber gets a copy independently. Unlike message queues (where each message goes to one consumer), pub/sub enables fan-out: one event triggers many independent reactions. Kafka is the gold standard for high-throughput, durable event streaming. Google Pub/Sub and AWS SNS offer managed options for cloud-native architectures. Redis Pub/Sub works for lightweight real-time broadcasting where message loss is tolerable. The killer use cases are event-driven microservices, analytics pipelines, notification systems, and audit logs — anywhere you want multiple parts of a system to react to the same event without tight coupling.