Two computer programs often need to talk to each other. There are several ways to do this. Picking the right one matters: it changes how fast your system feels, how complex it is to build, and how much it costs to run.
The animation on the right walks through three ways to send updates between systems — polling (ask, ask, ask), WebSockets (one open pipe where the server pushes), and a message queue (Publisher → Broker → Consumer).
Ask and answer: REST and gRPC
The most common pattern is simple: one side asks a question, the other side answers. This is called request/response. Two popular tools do this.
REST
Talk over normal web requests (HTTP) and send the data as JSON (plain text anyone can read). REST works everywhere, is easy to read, and can be cached. It is the default choice for public APIs and mobile apps. REST is stateless — each request carries everything the server needs to answer it.
gRPC
Send data as small binary messages (a packed format called Protocol Buffers) over HTTP/2. It is faster and strongly typed (the message shape is fixed and checked). It is popular for calls between your own internal services, where you control both sides. It also supports streaming (continuous push from server, client, or both at once).
REST vs. gRPC in practice
| REST + JSON | gRPC + Protobuf | |
|---|---|---|
| Payload size | Bigger (it is text) | Smaller (binary, about 5–10× less) |
| Speed | Baseline | About 2–5× faster to pack/unpack |
| Type safety | Weak (OpenAPI is optional) | Strong (generated from a .proto file) |
| Streaming | Workaround needed (SSE, chunked) | Built in (4 streaming modes) |
| Browser support | Works directly | Needs a grpc-web proxy |
| Human readable | Yes | No (you need tools to read it) |
Simple rule: use REST for public APIs and browsers; use gRPC for internal service-to-service calls when speed matters.
Getting updates in real time
Sometimes a client needs fresh data the moment it appears. The hard part: a normal web request only works when the client asks first. The server cannot just speak up on its own. Here are the ways to deal with that.
Polling
The client keeps asking "anything new?" on a timer. Very easy to build. The problem: most answers are "no", so you waste a lot of requests. It scales badly.
Long polling
The server holds the request open and only answers once it actually has new data (or a timeout is hit). Fewer empty answers than plain polling. It works through firewalls and proxies and needs no special protocol.
WebSockets
One connection that stays open and works both ways. The server can push data the instant it is ready — the client never has to re-ask. Great for chat, live feeds, online games, and shared editing.
Server-Sent Events (SSE)
A one-way push from server to client over a normal web connection. Simpler than WebSockets when you only need server → client data (live dashboards, alerts). It reconnects on its own if dropped.
Comparison
| Protocol | Direction | Use case | Complexity |
|---|---|---|---|
| HTTP polling | Client → Server | Simple status checks | Low |
| Long polling | Client → Server (held open) | Notifications, chat | Medium |
| WebSockets | Both ways | Chat, live gaming, shared editing | High |
| SSE | Server → Client | Dashboards, feeds | Low |
Why we step from polling to WebSockets
Picture a chat app. Sending a message is easy: the client opens a web connection (and reuses it with keep-alive, so it does not redo the slow setup each time) and tells the server to deliver the message. The hard part is receiving a message. Normal web requests are started by the client, so the server has no built-in way to push a new message to you. The usual path interviewers want you to walk is polling → long polling → WebSocket. Each step fixes a real flaw in the one before it.
Polling has the client ask "any new messages?" again and again. This can get expensive — the server spends work answering "no" most of the time. Ask too often and you waste effort; ask too rarely and messages show up late.
Long polling is the next step: the client opens a request and the server keeps it open until it has new data or a timeout fires. As soon as the client gets an answer it opens a new request. Fewer wasted round trips, but it has real downsides worth saying out loud:
- The sender and the receiver may land on different servers. Web servers are usually stateless and a load balancer spreads requests around, so the server that gets a new message may not be the one holding the open connection to the person who should receive it.
- The server cannot reliably tell when a client has dropped off.
- It is still wasteful — even a user who barely chats keeps reopening a request after every timeout.
WebSocket fixes all three. It is one connection that stays open and works both ways. It starts as a normal web request and is then "upgraded" with a quick handshake to the ws:///wss:// protocol. After that, the server can push the instant data is ready. WebSockets use ports 80 and 443 (the same ports as web traffic), so they pass through most firewalls and proxies. And because the pipe already works both ways, you may as well use it for sending too — one protocol both directions keeps the code simpler on both sides.
| Polling | Long polling | WebSocket | |
|---|---|---|---|
| Connection | New request each time | Held open until data/timeout | One that stays open, upgraded from HTTP |
| Server push | No (client re-asks) | Sort of (when data arrives) | Yes (instant) |
| Wasted requests | Many empty answers | Fewer, but reopens on timeout | None |
| Routing problem | — | Sender/receiver may hit different servers | — |
| Detect a dropped client | n/a | Poor | Built in |
WebSocket connections are stateful and not free. A rough rule: about 10KB of server memory per connection. At 1 million open connections that is about 10GB just to hold the sockets — before you process a single message. This is why the chat part of a system is its own dedicated service that needs careful connection management and its own scaling plan, separate from your normal stateless API servers.
Do not jump straight to "use WebSockets." Walk the steps: polling wastes requests → long polling holds the connection open, but the sender and receiver can land on different stateless servers and you cannot tell if the client dropped → WebSocket stays open, works both ways, and rides ports 80/443 through firewalls. Then add the cost: "open connections are stateful — about 10KB each, so about 10GB for a million — so the chat tier scales on its own, apart from the stateless API tier." Naming the trade-off, not just the winner, is the senior signal.
Webhooks
The reverse of polling, but between servers. Instead of you asking, the other service calls your URL when something happens (for example, "payment succeeded"). Your service must be reachable from the internet. Add an HMAC signature check so you can prove the call really came from them.
Decoupling with message queues
Message queue
A broker (Kafka, RabbitMQ, SQS) sits between a producer (the sender) and consumers (the readers). The producer drops off a message and moves on; consumers handle it when they are ready.
Queues give you three things: decoupling (sender and reader do not need to be online at the same time), buffering (soak up sudden traffic spikes), and retries. The cost: extra infrastructure to run, and messages are handled a little later instead of right away.
Point-to-point vs. publish/subscribe
Point-to-point (queue)
Each message goes to exactly one consumer. Several consumers compete for messages, so the work spreads naturally across them. Use it for handing out tasks (resize an image, send an email).
Publish/subscribe (topic)
Each message goes to every subscribed consumer, each on its own. Use it when several services all need to react to the same event (for example, "order placed" → billing, inventory, email).
Kafka vs. RabbitMQ vs. SQS
| Kafka | RabbitMQ | AWS SQS | |
|---|---|---|---|
| Model | Log-based (keep messages, replay them) | Classic queue (read, then delete) | Managed queue |
| Throughput | Very high (millions/sec) | High | High (managed) |
| Ordering | In order per partition | Configurable | Best effort (FIFO queues exist) |
| Replay | Yes (up to the retention limit) | No | No |
| Best for | Event streaming, audit logs, analytics | Task queues, RPC-style | Simple decoupling on AWS |
Kafka is the right pick when you need replay (several consumers reading the same stream on their own), very high throughput, or an event-driven design where the events themselves are the source of truth.
How microservices talk to each other
When you split one big app (a monolith) into many small services, you need a plan for how those services call each other.
Synchronous communication
Service A calls Service B and waits for the answer (REST, gRPC). Easy to follow, but it creates temporal coupling — if B is slow, A is slow; if B is down, A fails too.
Asynchronous communication
Service A drops an event and keeps going. B handles it on its own schedule (through a queue or topic). More resilient, but harder to debug and trace.
API Gateway sits at the front door of your services. It handles login checks, rate limiting, SSL termination, routing requests to the right service, and sometimes combining several responses into one — so each service does not have to.
Service mesh (Istio, Linkerd) provides mutual TLS between services, automatic retries, circuit breaking, and tracing — all as infrastructure, so you do not write that code in every service.
Circuit breaker pattern
When a service you depend on starts failing, blindly retrying it makes things worse — the failures pile up and spread.
Circuit breaker
Wrap calls to a dependency in a small state machine. Closed = calls go through as normal. If too many fail → Open (fail fast for a few seconds instead of waiting). After a pause, go to Half-open and try one request; if it works → back to Closed.
This stops one slow database from dragging down your whole API tier. Libraries like Netflix Hystrix and Resilience4j do this for you.
Match the tool to the need out loud: ask and wait for an answer → REST/gRPC; push live to clients → WebSockets; an event arriving from another server → webhooks; decouple, soak up spikes, do background work → a message queue. Choosing on purpose beats reaching for "just use REST" every time.
Sending to many people: fanout on write vs. read
Sometimes one event must reach lots of people — a post going to every follower's feed, or a message going to every member of a group. You have to decide when to do the delivery work: when the post is written, or when each feed is read. This is the fanout decision, and it is one of the most common "now make it scale" follow-ups in feed and chat interviews.
Fanout on write (push model)
Do the work at write time. The moment a user posts, the system looks up their followers and writes the post into every follower's feed cache right away. Reads are fast because each feed is already built — the reader just grabs a ready-made list.
Fanout on read (pull model)
Do the work at read time, on demand. Nothing is pushed when the post is written. When a user opens their feed, the system pulls recent posts from everyone they follow and merges them. Writes are cheap, but reads are slow because the feed is built fresh each time.
The trade-offs are exact mirror images:
| Fanout on write (push) | Fanout on read (pull) | |
|---|---|---|
| Feed read | Fast (already built) | Slow (built per request) |
| Write cost | High (one write per follower) | Cheap (store the post once) |
| Hotkey problem | Yes — a user with millions of followers means millions of writes | No — nothing is pushed |
| Inactive users | Wasted work — feeds built for people who may never log in | No waste — only built when someone reads |
Two problems make pure push fall apart at scale:
- The hotkey problem. A celebrity with millions of followers means one post turns into millions of feed writes — slow, costly, and a load spike on every post.
- Wasted work for inactive users. Building feeds for people who rarely (or never) log in burns resources for nothing.
The senior answer is a hybrid: use push for most users (so the common case — reading a feed — stays fast), and switch to pull for celebrities and accounts with huge follower counts (so one post does not cause millions of writes). Their followers fetch that content on demand at read time and merge it in. Consistent hashing helps spread the fanout work so no single machine becomes a hotspot.
Lead with the hybrid and explain the split: "Push for normal users keeps reads fast; pull for celebrities avoids the hotkey explosion where one post becomes millions of writes — and it stops us wasting compute building feeds for people who never log in." Saying which users go down each path — not just "use a hybrid" — shows you really thought about how the load spreads.
Notifications: staying reliable across many channels
A notification system takes one event and sends it out across very different channels — iOS push (APNS), Android push (FCM), SMS (Twilio/Nexmo), email (Sendgrid/Mailchimp) — and each one is a third-party service you do not control. Two things matter here: keep the channels isolated from each other, and have a clear reliability story, because the rule is never lose a notification.
Per-channel message queues
Give each notification type its own queue (an iOS push queue, an Android push queue, an SMS queue, an email queue), each drained by its own pool of workers. The point is fault isolation: if Twilio goes down, SMS events pile up in the SMS queue while iOS, Android, and email keep flowing. One provider's outage cannot stall the others.
Reliability — do not lose data. Notifications can be late or arrive out of order, but never lost. Two mechanisms guarantee this:
- Save a notification log in a database before (or as) events are queued, so an event survives a crash and can be sent again.
- Retry on failure. When a third-party service rejects a send, put the event back on the queue and retry up to a set limit; if it keeps failing, alert the developers.
Exactly-once delivery is impossible in a distributed system. You can save, retry, and isolate, but because the system is spread across many machines, duplicates will happen sometimes. Never promise exactly-once — it is a known impossibility, and claiming it is a red flag.
At-least-once + dedupe on event ID
Since you cannot have exactly-once, aim for at-least-once delivery (retry until the receiver confirms) and make the consumer idempotent (safe to run twice): when a notification event arrives, check its event ID against a list of IDs you have already handled. Seen it before → throw it away. New → send it and record the ID. At-least-once delivery plus dedupe is the practical stand-in for the exactly-once guarantee you cannot actually give.
Before sending, the system also checks the user's notification settings (whether they opted in for each channel) and applies frequency capping — over-notifying is the fastest way to make users turn notifications off entirely, which is worse than a missed send.
When asked about reliability, say it plainly: "Exactly-once delivery is impossible in a distributed system, so I design for at-least-once and dedupe on the event ID to make consumers idempotent. I save a notification log so nothing is lost in a crash, retry failed sends, and give each channel its own queue so a Twilio outage does not take down email." Knowing that exactly-once is a fallacy — not just unbuilt — is a strong senior signal.
Rate limiting
Rate limiting caps how many requests a client can make. Every public API needs it: to block denial-of-service (DoS) attacks, cut infrastructure cost, enforce fair use, and stop a buggy client from flooding you.
Where to put it:
- API Gateway — best for limiting across many services; apply it before traffic reaches them.
- Server middleware — full control over the algorithm; back it with a shared Redis counter.
- Client-side — unreliable; clients can fake requests; never the only line of defense.
Rate limiting algorithms
Token bucket
A bucket holds up to N tokens and refills at a steady rate. Each request spends one token; a request with no token is rejected. This allows short bursts up to the bucket size. Used by Amazon and Stripe.
Leaking bucket
Requests line up in a FIFO queue, and the queue drains at a fixed rate no matter how fast they come in. This smooths traffic to a steady output — good for payment systems where steady throughput matters. A sudden burst fills the queue and extra requests are dropped.
Fixed window counter
Split time into fixed windows (say, 1 minute) and count requests per window. Simple and light on memory, but it has a boundary burst flaw — a client can send double the limit by bunching requests at the end of one window and the start of the next.
Sliding window log
Store the timestamp of each request in a sorted set (in Redis). On each new request, drop timestamps older than the window, then count what is left. Very accurate, but storing a timestamp per request uses a lot of memory at scale.
Sliding window counter
A mix of the two: take the previous window's count, weighted by how much it overlaps the current
rolling window. count = current_window_count + prev_window_count × overlap_fraction. One counter
per window — light on memory and reasonably accurate.
Practical default: token bucket for most APIs (it allows bursts); sliding window counter when you need strict per-minute accuracy without the memory cost of the log approach.
Response headers to include:
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1735689600
Retry-After: 30 (only when 429 is returned)
Rate limiting across many API servers needs a shared counter — local in-memory counters drift
apart from server to server. Use Redis INCR with EXPIRE for a simple shared counter, or Redis
sorted sets for a sliding window log. The atomic increment makes sure no request is double-counted
across servers.
Idempotency — the key to safe retries
When a network call fails, you do not know if the server actually got it. Retrying is only safe if the operation is idempotent — it gives the same result whether you run it once or many times.
- GET, DELETE, PUT — idempotent by nature.
- POST — not idempotent by default; use an idempotency key (the client makes a UUID and sends it in a header; the server uses that key to drop duplicates).
This matters most for payment APIs, creating orders, and any write that must not happen twice.
Practice
Check your understanding of how systems talk to each other.
1. What is the main problem with plain polling?
2. Why is a WebSocket better than long polling for a chat app?
3. What does a message queue give you?
4. Why do you use pull (fanout on read) for celebrities instead of push?
5. Why can you not promise exactly-once notification delivery?