Design a Chat System

A chat app sounds easy — "send a message from A to B." The tricky part is the receiving side. The web's normal rule is that the client always starts the conversation; the server only answers. So how does the server push a new message to someone who didn't ask for it right now? Get that one idea right — plus where you store messages and how you show who is online — and you have a design like Facebook Messenger that handles 1-on-1 chat, small groups, and an online dot for 50 million daily users.

Step 1: Decide what we are building

Agree on the exact chat type before you draw anything. Designing for group chat when the interviewer wanted 1-on-1 is a common mistake. For a Messenger-style app, here is the agreed list of features:

• 1-on-1 chat that feels fast (low delay)
• Small group chat — up to 100 people
• An online indicator (the green dot)
• Multiple devices — the same account logged in on phone and laptop at once
• Push notifications (alerts when the app is closed)
• Text only, under 100,000 characters per message; chat history is kept forever

Back-of-the-envelope estimation

These rough numbers explain every choice we make later. ("Back-of-the-envelope" just means quick math, not exact.)

Daily active users        = 50,000,000 DAU
Connections at once       ≈ 1,000,000 (peak, rough)
Memory per connection     ≈ 10 KB
Memory for all of them    = 1,000,000 × 10 KB ≈ 10 GB on one box (in theory)
Messages per day (industry) ≈ 60 billion (Messenger + WhatsApp)
Reads vs writes (1-on-1)  ≈ 1:1

That 10 GB number is a trap, not a goal. Yes, one machine could physically hold all the connections. But putting everything on one server is a bad idea — if that one box goes down, the whole app goes dark. Say the number out loud, then reject it right away.

Step 2: How do clients and servers talk?

Clients never message each other directly. Each one connects to a chat service in the middle that passes messages along. Sending is the easy half — a normal request from the client works fine. The hard half is receiving: the server cannot start an HTTP request on its own, so how does a brand-new message reach a client that is just sitting there?

Because WebSocket works both ways, we use it for sending too. Using one channel for both sending and receiving keeps the client and server simpler. Everything else (sign-up, login, profile) stays as plain request-and-response over HTTP. WebSocket is only for the real-time part.

Step 3: High-level design

Sort the system into three groups:

• Stateless services — login, signup, profile, and service discovery (finding the right server). These keep no per-user memory, sit behind a load balancer, and are easy to scale.
• Stateful service — the chat servers. They are stateful because each client pins one WebSocket to one server and stays on that server the whole time it is connected.
• Third-party integration — push notifications (covered in the notification-system chapter).

                 ┌──────────────┐
   Client ──WS──▶│ Chat Servers │──▶ ID Generator (message_id)
                 └──────────────┘──▶ Message Sync Queue ──▶ KV Store (history)
   Client ──────▶ API Servers (login/signup/profile)        │
   Client ──WS──▶ Presence Servers (online/offline)         └─▶ Chat Server 2 ──WS──▶ recipient
                 Notification Servers (push) ◀───────────────── (if offline)

Chat servers handle real-time messages. Presence servers track who is online or offline. API servers do everything else. Notification servers send alerts. The key-value store (a simple database that looks things up by key) saves chat history, so a user who was offline sees her full history when she comes back.

Storage: why a key-value store

Two kinds of data live here. General data (profiles, settings, friend lists) goes in a normal relational database (copied and split across machines for safety and scale). Chat history is the interesting one, and how it is used points us to a key-value store:

• A huge volume (~60 billion messages a day across the industry)
• Mostly only recent chats are read, but search and "jump to" need to grab any message at random
• Reads and writes happen about equally (≈ 1:1)
• Relational database indexes get slow over a giant pile of old data, while key-value stores stay fast and add machines easily

Real products prove it works: Messenger uses HBase, Discord uses Cassandra.

Message IDs

Every message needs an ID that sets its order, so the ID must be unique and sortable by time (newer messages get bigger IDs). You can't rely on a created_at timestamp, because two messages can land on the exact same time.

Step 4: Deep dive

Service discovery

Its job: when a client logs in, hand it the best chat server to use (based on location, how busy each server is, and so on). A tool like Apache ZooKeeper keeps a list of all chat servers and picks one:

1. User A logs in; the load balancer sends her to an API server.
2. The API server checks who she is, then asks service discovery for the best chat server.
3. Service discovery answers with, say, Chat Server 2.
4. User A opens a WebSocket to Chat Server 2.

1-on-1 message flow

This is the path the animation walks through:

1. User A sends the message to Chat Server 1.
2. Chat Server 1 gets a message_id from the ID generator.
3. It drops the message onto the message-sync queue (the recipient's inbox).
4. The message is saved in the KV store.
5. (a) If User B is online, send it on to Chat Server 2 (where B is connected); (b) if offline, fire a push notification.
6. Chat Server 2 pushes the message to User B over her open WebSocket.

Multi-device sync

Each device remembers a cur_max_message_id — the latest message ID it has already seen. A message is new for a device when it is addressed to that user and its ID in the KV store is bigger than that device's cur_max_message_id. So a phone and a laptop on the same account each pull exactly what they missed, on their own.

Small group chat

When you send to a group, the message is copied into each member's inbox (B's inbox, C's inbox, and so on). Then each person only has to check their own inbox — simple, and cheap while groups are small. This "copy on send" works fine up to ~100 people (WeChat caps groups at 500). For very large groups, copying once per member gets too expensive.

Online presence and the heartbeat

The naive plan — "mark offline the moment the connection drops, online when it comes back" — flickers badly. Picture a phone going through a tunnel. The fix is a heartbeat.

To tell friends about a status change, the system uses publish/subscribe (one writer announces; many readers listen). Each pair of friends shares a channel, so when A's status changes she announces it on her channels A-B, A-C, A-D, and those friends get the update over WebSocket. This is fine for short friend lists. A 100,000-member group would fire 100,000 updates per change, so for big groups you instead fetch status only when needed — when a user opens the group or refreshes the list.

Wrap-up

The chat system boils down to a few parts, each with a clear job:

If you have time left, the natural add-ons are media files (compress them, store in the cloud, make thumbnails), end-to-end encryption (WhatsApp), caching messages on the client, edge caches near users (Slack's Flannel), and error handling — if a chat server dies, service discovery hands clients a new one, and a retry/queue system resends any dropped messages.

Practice

Check that the message flow and the receiver problem make sense.