Twitter is the news feed problem plus three extras that are worth knowing separately: the follow graph, trending topics, and search.
Requirements
Functional
- Post a tweet (280 chars, optional media).
- Follow and unfollow users.
- Home timeline (people you follow) and user timeline (one author).
- Like, retweet, reply.
- Trending topics and search.
Non-functional
- Timeline load under 200 ms.
- Eventual consistency is acceptable for the timeline; a tweet arriving a second or two
late is fine.
- Extremely read-heavy, with an extreme celebrity skew.
300M monthly users, 150M daily
6,000 tweets/second average, ~15,000 at peak
300,000 timeline reads/second
Storage: 500M tweets/day × 300 bytes ≈ 150 GB/day of text, plus mediaThe two timelines
Distinguish them early — they have completely different mechanics:
- User timeline — one author's tweets. A simple indexed query on
(author_id, created_at), cheap, no fanout needed.
- Home timeline — a merge across everyone you follow. This is the hard one.
Timeline fanout
The hybrid described in news feed:
Ordinary user tweets → async job pushes the tweet id into each
follower's Redis list, trimmed to ~800 entries
Celebrity tweets → no fanout; pulled at read time
Home timeline read → precomputed list ∪ live query for followed celebrities
→ merge by time → hydrate ids → returnThe threshold is a tuning knob, typically tens of thousands of followers. Everything else in the design serves this decision.
Storage and sharding
tweets tweet_id (snowflake), user_id, text, media_ids, created_at, reply_to
follows follower_id, followee_id, created_at
likes user_id, tweet_id, created_at
counters tweet_id → like_count, retweet_count, reply_countTweet ids are Snowflake ids — 41 bits of millisecond timestamp, 10 bits of machine id,
12 bits of sequence. Two properties matter: they are generated without coordination, and
they are time-sortable, so "the newest tweets" is a range scan on the primary key and the timeline merge can sort by id alone.
Shard tweets by tweet_id. Sharding by user_id seems natural but concentrates a
celebrity's entire history — and its read traffic — on one shard.
Counters are stored separately from the tweet row. A viral tweet gets millions of
concurrent like increments; keeping the count in the tweet row makes that row a lock hotspot. Use an atomic counter in Redis with periodic flushes, or a sharded counter (N sub-counters summed on read).
The follow graph
Hundreds of billions of edges. Queried in both directions — "who do I follow" on read, "who follows me" on fanout — so store both adjacency directions, sharded by the queried user id. Cache the follower list of high-fanout accounts, since it is read on every one of their tweets.
Unfollow does not need to purge the timeline immediately; filtering at hydration time is acceptable and far cheaper.
Trending topics
Not a database query — a streaming aggregation.
tweets → Kafka → stream processor (Flink/Spark)
→ extract hashtags and entities
→ count over a sliding window (e.g. 5 min, 1 hour)
→ compare against a baseline rate to find SPIKES, not just volume
→ top-K per region → cache, refreshed every minuteTwo points to make. First, trending is about acceleration, not volume — otherwise the same permanently popular tags win forever. Second, exact counts over a firehose are unnecessary: Count-Min Sketch gives approximate frequencies in fixed memory, which is the standard answer for top-K over a stream.
Search
A separate inverted index, not the primary store. Tweets flow from the write path into Elasticsearch, sharded by time so recency queries hit few shards. Ranking blends text relevance with engagement and recency. Real-time search means the index must be near-live, which is why the pipeline is streaming rather than batch.
Follow-ups to expect
- Hot tweet — a viral tweet read millions of times: cache it at the edge, and use
sharded counters.
- Timeline for a new user — no follows yet; fall back to recommended accounts.
- Media — uploaded directly to blob storage via a signed URL, served through a CDN;
tweets store only ids.
- Deletes — mark deleted and filter at hydration rather than rewriting millions of
timeline lists.
- Replies and threads — a parent pointer, with the conversation assembled at read time.
- Multi-region — timelines served locally; writes replicated asynchronously, accepting
brief cross-region lag.
Track this problem on the System Design sheet.
Frequently asked
How does Twitter's home timeline work?
With hybrid fanout. When an ordinary user tweets, an async job pushes the tweet id into each follower's precomputed Redis list. Accounts with very large followings are excluded from fanout, and their tweets are fetched live at read time and merged into the precomputed list. That keeps the common read to a single cache lookup while avoiding hundred-million-write fanouts.
Why does Twitter use Snowflake IDs?
They are generated without coordination — each machine builds an id from a timestamp, its machine id and a local sequence number — and they sort by time, so 'newest first' is a range scan on the primary key and timeline merges can sort on the id alone with no extra column.
How are trending topics computed?
As a streaming aggregation, not a query. Tweets flow through Kafka into a stream processor that extracts hashtags and counts them over a sliding window, comparing against a baseline so the result reflects acceleration rather than raw volume. Count-Min Sketch keeps the counting approximate but fixed-memory, and the top-K per region is cached and refreshed every minute or so.