Design a News Feed

The 30-second version

Your home timelineHome timeline (feed)The list of posts one user sees when they open the app: posts from everyone they follow, newest first or ranked. Different from a user's own profile, which shows only their own posts. (the feed) is the screen you see when you open a social app. Behind it sits a simple-sounding job: "show Bob the newest posts from the 300 accounts he follows". At a few thousand users, one database query does it. At hundreds of millions, the question of when that work happens decides almost everything else.

A feed system has to fan out somewhere: either the reader fans out to everyone they follow, or each post fans out to everyone following its author.

The answer, in one paragraph

Feeds are opened far more often than people post, so most designs push: when you post, background workers copy the post's ID into each follower's precomputed feed. That breaks for the few accounts with millions of followers (the Celebrity (hot-account) problemCelebrity (hot-account) problemWhen one account has millions of followers, a single post triggers millions of writes or reads at once. Designs that work for normal users break for these few, so they get special handling.), so those accounts are pulled instead and mixed in when the reader opens the app. On top of that sit a few more pieces: a cache of feeds, a ranking step, and a plan for when parts of it fail.

The route through this page

What are we building?

Functional: what users can do

1. Post a short text post (photos and videos are stored elsewhere; see the note below).
2. Follow and unfollow other accounts. Following is one-way: Bob can follow a pop star without her following him back.
3. Read the home feed: posts from everyone you follow. Version 1 is newest first; later (§09) it's ranked by "most interesting".
4. Scroll: the feed comes in pages of about 20 posts, and "load more" continues where you stopped.
5. See new posts quickly: within a few seconds is fine. Nobody needs a live, to-the-millisecond stream (that's what Design a Chat System is about).

Out of scope here: likes and comments themselves, search, notifications (Design a Notification System), ads, and recommending accounts you don't follow.

Non-functional: how well

That last-but-one row is a deliberate trade. A feed doesn't need every reader to see a post at the same instant; different followers seeing it a few seconds apart is fine. That puts feeds low on the consistency ladder, close to "eventual" (CAP §05 shows the whole ladder). One exception is worth keeping: you should always see your own new post right away (§11 shows the trick).

The data

Three kinds of data, shown as tables of a relational database (any database would do):

CREATE TABLE users (
  id        bigint PRIMARY KEY,
  name      text NOT NULL
);

CREATE TABLE posts (
  id         bigint PRIMARY KEY,          -- time-sortable: bigger id = newer (§07)
  author_id  bigint NOT NULL REFERENCES users(id),
  body       text NOT NULL,
  created_at timestamptz NOT NULL DEFAULT now()
);

CREATE TABLE follows (
  follower_id bigint NOT NULL REFERENCES users(id),  -- Bob
  followee_id bigint NOT NULL REFERENCES users(id),  -- the account Bob follows
  PRIMARY KEY (follower_id, followee_id)
);

The follows table is the follow graphFollow graphThe data that records who follows whom: one row (follower, followee) per follow. Two questions are asked of it all the time: “whom does Bob follow?” and “who follows Ana?”.. Every design on this page asks it two questions over and over: "whom does Bob follow?" (needed to read a feed by pulling) and "who follows Ana?" (needed to deliver Ana's post by pushing).

The API

Three endpoints cover the requirements:

POST /posts                 { "body": "hello" }              → 201 { "id": 9001 }
PUT  /follows/{userId}                                       → 204   (DELETE to unfollow)
GET  /feed?limit=20&cursor=… → 200 { "posts": [...], "next_cursor": "…" }

201 means Created and 204 means No Content (it worked, nothing to send back). cursor is an opaque "continue from here" token for scrolling, explained in §08.

How big is it? A rough estimate

All numbers below are approximate and invented for a large, typical social app. They're not any real company's figures.

Step 1: the inputs

• 200 million daily active users (DAU)DAU (daily active users)How many different people use the app at least once on a given day. The usual starting number for a capacity estimate: 200 million DAU × 10 feed opens each = 2 billion feed loads a day..
• Each opens their feed about 10 times a day.
• 1 in 10 users posts once a day.
• An account has 200 followers on average. That average hides a lot: most have fewer, and a handful have 50 million.
• One post is about 1 KB (text, IDs, timestamps; media lives elsewhere, §02).
• A day has 86,400 seconds. Traffic isn't spread evenly, so the busiest hour is about 3× the average: the peak factorPeak factorHow much busier the busiest hour is than the daily average. Traffic follows people's days (evenings, big events), so designs plan for the average times 2–5..

Step 2: reads and writes per second

Feed loads: 200,000,000 users × 10 = 2,000,000,000 loads a day. 2,000,000,000 ÷ 86,400 ≈ 23,000 loads per second on average, ≈ 70,000/s at peak.

New posts: 200,000,000 × 1/10 = 20,000,000 posts a day. 20,000,000 ÷ 86,400 ≈ 230 posts per second, ≈ 700/s at peak.

So reads outnumber writes 2,000,000,000 ÷ 20,000,000 = 100 to 1. The feed is very read-heavy.

Step 3: the fan-out numbers

Here's where the two strategies from §01 differ. Suppose we push every post into every follower's feed:

20,000,000 posts × 200 followers = 4,000,000,000 feed inserts a day. 4,000,000,000 ÷ 86,400 ≈ 46,000 inserts per second (≈ 140,000/s at peak). Busy, but each insert is tiny.

Now one celebrity with 50,000,000 followers posts once. That single post means 50,000,000 inserts. At the normal rate of 46,000/s, that's 50,000,000 ÷ 46,000 ≈ 1,100 s ≈ 18 minutes of the whole system's normal work, for one post. Keep that number in mind for §06.

If we pull instead, each feed load reads the recent posts of everyone the reader follows. Following 200 accounts on average: 23,000 loads/s × 200 = 4,600,000 lookups per second, about 14 million at peak. That's the cost of pull, paid on every single feed open.

Step 4: storage

Posts: 20,000,000 posts × 1 KB = 20 GB a day. × 365 ≈ 7 TB a year (before keeping extra copies for safety).

Precomputed feeds (if we push): keep the newest 800 post IDs per user. An ID is a 64-bit number = 8 bytes. 800 × 8 bytes = 6,400 bytes ≈ 6.4 KB per user. 6.4 KB × 200,000,000 users ≈ 1,300,000,000,000 bytes ≈ 1.3 TB of raw IDs. With the cache's own bookkeeping, plan for roughly twice that. Too big for one machine, comfortable for a few dozen (§07).

Bandwidth: a page of 20 posts × 1 KB = 20 KB. 23,000 pages/s × 20 KB ≈ 460 MB/s ≈ 3.7 Gbit/s of feed text on average. Images and video go through the CDN, not through these servers.

Play with the inputs

Change the inputs and watch which outputs explode. Try raising average followers to 1,000, or DAU to 1 billion.

Version 1: one database query, and what breaks

The simplest feed: ask the database

One app server, one database with the three tables from §02. The feed is one query:

SELECT p.id, p.author_id, p.body, p.created_at
FROM follows f
JOIN posts p ON p.author_id = f.followee_id   -- posts by people Bob follows
WHERE f.follower_id = $1                      -- $1 = Bob's user id
ORDER BY p.id DESC                            -- newest first (bigger id = newer)
LIMIT 20;

Plus one more index, so the database can find "this author's newest posts" without scanning the whole table:

CREATE INDEX posts_by_author ON posts (author_id, id DESC);

This is fan-out on read, done by the database: for every feed load, it walks Bob's follow list and, for each account, jumps into the index to find that account's newest posts. Then it merges them and keeps the top 20.

At 10,000 daily users that's a handful of queries a second. It works, it's always up to date, and there's nothing to keep in sync. Ship it.

Now grow the traffic

Drag the slider from 10,000 users up to 200 million. Each step shows what breaks (highlighted in red) and the piece that gets added to fix it.

What changed at each step, in words

• At 1 million users (100× version 1): about 1,000,000 × 10 ÷ 86,400 ≈ 120 feed loads a second, ~350 at peak (worked out as in §03). Each load visits ~200 followees, so that's ~70,000 index lookups a second, and the one database runs out of CPU, almost all of it spent on reads. Add a Load balancerLoad balancerA component that spreads incoming requests or connections across several servers, so no single one is overloaded and a dead one can be skipped. with more app servers (Load Balancing), a few read replicasReplicaA full copy of a database kept on another machine and updated as the original changes, for reading load and for taking over if the original dies. of the database, and a CacheCacheA small, fast store of answers you already computed or fetched, so the next request for the same thing skips the slow work. The hard part is knowing when a cached answer is out of date. for posts (Caching §03 shows the usual cache-aside pattern). The query is unchanged; it just runs on more machines. (Replicas lag behind the main database by a moment, Replication §06, which is fine here; §02 allowed that.)
• At 20 million users (another 20×): ~7,000 loads a second at peak, ~1.4 million lookups. Each feed load is still one heavy query: ~200 index jumps plus a merge inside the database. More replicas add capacity, but they don't make that one query faster, so p99 latency stays high and climbs at peak. The fix is a different idea: build each user's feed ahead of time, in the background, so reading it is a single lookup. That's fan-out on write, §05.
• At 200 million users: the precomputed feeds no longer fit on one cache server, and a single celebrity's post floods the background workers. Shard the feeds, and treat celebrities differently (§06–§07).

Fan-out on read vs fan-out on write

The two strategies, side by side

• Fan-out on read (pull): when Bob opens the app, look up the newest posts of each of the F accounts he follows, merge them, and return the top 20. Posting is one write. Version 1 from §04 is this.
• Fan-out on write (push): when Ana posts, a background worker adds the post's ID to the stored feed of each of her N followers. Opening the app is one read of your own stored feed.

With fan-out on readFan-out on read (pull)Building a feed when the reader opens it: look up the recent posts of everyone they follow and merge them. Posting is cheap (one write); every feed open is expensive., the reader does the collecting. With fan-out on writeFan-out on write (push)Copying a new post's ID into the stored feed of every follower at the moment it is posted. Feed opens become one cheap read; a post costs one write per follower., each post is delivered ahead of time into a precomputed feed, usually called the user's timeline. Both are correct; they just put the work in different places.

The cost, as formulas

Let F be how many accounts a reader follows, and N how many followers an author has.

Now multiply by how often each happens, using the numbers from §03 (F ≈ N ≈ 200 on average):

• Pull, per second: 23,000 loads × 200 lookups = 4,600,000 lookups/s, all on the path where a user is waiting.
• Push, per second: 230 posts × 200 followers = 46,000 inserts/s, done in the background where nobody waits.

Push does about 100× less work here, because reads outnumber posts 100 to 1. It also moves the work off the critical path: Bob's feed loads fast even if delivery takes a few seconds, which §02 allowed. That's why most large feeds push by default.

Try it

Pick a strategy, then press the buttons. Ana is a normal account; the star is not. Watch the dots: each one is a write or a lookup.

Notice the star's row in push mode. We'll come back to it in §06.

How push works in practice

Writing the post and delivering it are separate steps. The post API saves the post, publishes a small "post created" message to a Message brokerMessage brokerA server that accepts messages from producers and hands them to consumers later — RabbitMQ, Kafka, SQS, NATS. It decouples who sends from who handles, and buffers bursts., and answers the user right away. A pool of fan-out workers reads those messages and does the slow part.

The workers form a Consumer groupConsumer groupA set of consumer processes that share the work of reading one stream or queue: each message (or Kafka partition) goes to exactly one member of the group.: several copies of the same program sharing one stream of messages, each message handled by one of them (Queues §06 explains how they split the work). Here's the heart of one worker, with the follow graph and the timeline cache replaced by in-memory maps so it runs on its own:

package main

import "fmt"

const timelineCap = 5 // real systems keep hundreds (say 800); 5 keeps the output short

// In-memory stand-ins: the follow graph, and the timeline cache (Redis in production).
var followers = map[string][]string{
	"ana": {"bob", "cy", "dee"},
	"bob": {"ana"},
}
var timelines = map[string][]int64{} // user → post IDs, newest first

// fanOut runs once per "post created" event taken from the queue.
func fanOut(author string, postID int64) {
	for _, f := range followers[author] { // one write per follower ← the cost of push
		t := append([]int64{postID}, timelines[f]...) // add at the front (newest first)
		if len(t) > timelineCap {
			t = t[:timelineCap] // cap the list: drop the oldest
		}
		timelines[f] = t
	}
}

func main() {
	for id := int64(101); id <= 106; id++ {
		fanOut("ana", id) // ana posts six times
	}
	fanOut("bob", 107)
	fmt.Println("bob's timeline:", timelines["bob"])
	fmt.Println("ana's timeline:", timelines["ana"])
}

Output (Go, then Node):

bob's timeline: [106 105 104 103 102]
ana's timeline: [107]

bob's timeline: [ 106, 105, 104, 103, 102 ]
ana's timeline: [ 107 ]

Bob's timeline holds only the newest 5 of Ana's six posts: post 101 was pushed out the end. Real timelines are capped the same way, just at a few hundred IDs (§07).

Details a real worker needs

• Followers come in pages. An author with 5,000 followers is read from the follow graph 1,000 at a time, and each batch of inserts goes to the cache in one round trip.
• Messages can arrive twice. Brokers usually promise At-least-once deliveryAt-least-once deliveryA delivery promise where a message is never lost but may arrive more than once (for example, re-sent after a crash). Receivers must therefore handle duplicates safely. delivery: after a crash, a message is handed out again. Inserting the same post ID twice must not show the post twice. The usual fix is cheap: the read path's merge skips duplicate IDs (§06), so a repeated insert does no harm. Making a repeated operation harmless is what IdempotentIdempotentDoing it twice has the same effect as doing it once. “Set balance to 50” is idempotent; “add 50 to balance” is not — so only the first is safe to retry blindly. means (Idempotency §08 covers consumers in depth).
• Skip people who never come back. Pushing into the timeline of someone who hasn't opened the app in a year is wasted work. Many designs only push to recently active users and rebuild a timeline by pulling when an inactive user returns (§10).

The celebrity problem and the hybrid

What one celebrity post does to push

From §03: one post by a 50-million-follower account is 50,000,000 timeline inserts. §03 measured that against the system's normal load (46,000 inserts/s): 18 minutes' worth of normal work. A real fleet has headroom above normal, so ask a second question: how long does it take a fleet sized well above normal, say 1,000,000 inserts a second (an assumed round number)? Then:

• The last follower gets the post 50,000,000 ÷ 1,000,000 = 50 seconds after it was posted, even if the fleet does nothing else.
• Everyone else's posts wait in the queue behind it. A normal user's post that arrives just after the star's is delayed by those same 50 seconds.
• Stars post more than once, often at the same moment (a big match, an awards night). Five of them posting in the same minute is 250 million inserts, and 4 minutes of backlog.

On top of that, many of those 50 million followers won't open the app today, so much of that work is wasted. This is the Celebrity (hot-account) problemCelebrity (hot-account) problemWhen one account has millions of followers, a single post triggers millions of writes or reads at once. Designs that work for normal users break for these few, so they get special handling.: a design that's great for the typical account is terrible for the few huge ones.

Pull is cheap for exactly these accounts

Flip the formulas from §05 around. Pulling one celebrity costs each reader one extra lookup of "that account's newest posts". A reader follows only a handful of celebrities, so the extra read cost is small. Push for the same account costs N writes per post, with N in the millions.

So: hybrid fan-outHybrid fan-outPush for normal accounts, pull for a short list of accounts with huge follower counts. At read time the server merges the pushed feed with the few pulled lists..

1. Mark accounts above a follower threshold (say 100,000; you tune it by measuring) as "celebrities".
2. Fan-out workers skip celebrity posts. Normal posts are pushed as before.
3. When Bob opens his feed, the server reads his pushed timeline, and the recent posts of the few celebrities he follows, and merges them.

Try all three

Set the star's follower count and how many stars Bob follows. Compare the cost table at the bottom: hybrid keeps every column small.

Merging at read time

Each celebrity's recent posts are a list sorted newest first, and Bob's pushed timeline is one too once it's sorted (more on that below the code). Combining a few sorted lists into one sorted list is a merge: repeatedly take the newest item from the fronts of the lists. Post IDs grow over time (§07), so "newest" simply means "biggest ID".

Watch it on Bob's lists first. A copy of 117 was delivered twice and arrived late, so step 1 sorts his timeline, and you'll also see the duplicate skipped:

package main

import (
	"fmt"
	"slices"
)

// mergeNewest merges lists that are each sorted newest-first (biggest ID first)
// and returns the first `limit` distinct IDs. Fine for a handful of lists; use a heap for many.
func mergeNewest(limit int, lists ...[]int64) []int64 {
	pos := make([]int, len(lists)) // how far we have read into each list
	seen := map[int64]struct{}{}   // IDs already written out
	var out []int64
	for len(out) < limit {
		best := -1
		for i, l := range lists { // find the list whose next ID is the newest
			if pos[i] < len(l) && (best == -1 || l[pos[i]] > lists[best][pos[best]]) {
				best = i
			}
		}
		if best == -1 {
			break // every list is used up
		}
		v := lists[best][pos[best]]
		pos[best]++
		if _, dup := seen[v]; dup {
			continue // already written: skip the duplicate
		}
		seen[v] = struct{}{}
		out = append(out, v)
	}
	return out
}

func main() {
	// Bob's pushed timeline is in ARRIVAL order: a redelivered copy of 117 arrived last, so it's first.
	arrived := []int64{117, 120, 117, 109, 104}
	bobTimeline := slices.Clone(arrived)
	slices.Sort(bobTimeline)     // 104 109 117 117 120
	slices.Reverse(bobTimeline)  // 120 117 117 109 104: now newest-first
	popStar := []int64{119, 110} // author lists are written in ID order already
	footballer := []int64{118, 103}
	fmt.Println(mergeNewest(5, bobTimeline, popStar, footballer))
	fmt.Println(mergeNewest(5, arrived, popStar, footballer), "← without sorting first")
}

Output (Go, then Node):

[120 119 118 117 110]
[119 118 117 120 110] ← without sorting first

[ 120, 119, 118, 117, 110 ]
[ 119, 118, 117, 120, 110 ] ← without sorting first

Two things this code doesn't assume. First, Bob's pushed timeline is only roughly newest-first. LPUSH puts the newest arrival at the front, and arrivals get out of order: fan-out workers run in parallel, and a message redelivered after a crash (§05) arrives late. That's why arrived starts with a late copy of 117. The merge needs sorted input, so the feed service sorts the timeline IDs first, which is cheap for a few hundred numbers. The second output line shows what happens if you skip that: 120 lands in the middle. Second, duplicates are skipped with a seen-set, which works wherever the copies sit. Duplicates come from that late redelivery, and from an account that was just switched from push to pull (see the callout at the end of this section): its older posts are still in followers' timelines and in its pulled list.

Each output item scans every list's front, so this costs about K steps per item for K lists. That's fine for Bob's 1 timeline + 2 stars. With hundreds of lists (pure pull, §04) you'd keep the fronts in a max-heap (a heap whose top is always the largest item), which costs about log K per item. Heaps §09 builds the same merge for lists sorted smallest-first with a min-heap; flip the comparison and it becomes ours.

The celebrity's own list becomes a hot key

Under hybrid, millions of readers ask for the same thing: "the pop star's newest posts". Whichever cache server holds that key gets hammered while the others idle. That's a Hot key / hot spotHot key / hot spotA single key (or a narrow range) that gets far more traffic than the rest, like a celebrity's profile. Because one key always lives on one shard, that shard overloads while the others idle. (Sharding §06). Two common fixes:

• Copy it widely. Keep several copies of each celebrity's recent-posts list on different cache servers, or a copy in each app server's memory that's refreshed every second or two. A second of staleness is fine (§02).
• Keep it short. The merge only needs a celebrity's last few dozen posts (say 50), not their whole history, so the hot list stays small. A reader who scrolls deeper than that falls back to reading older posts from the post store, slower but rare.

Storing timelines: capped lists of IDs

Why IDs and not copies

If Ana's post were copied in full into 200 timelines and she then fixed a typo, all 200 copies would need updating. An ID is 8 bytes and never changes. Edits, deletes and like counts live in one place, the post store, and every timeline sees them the moment the post is looked up (§08 calls this lookup hydrating).

IDs that sort by time

If bigger IDs always mean newer posts, then "sort by newest" is just "sort by ID", and two sorted lists of IDs can be merged (§06) without fetching any timestamps.

A database counter (1, 2, 3, …) has this property but needs one central counter that every new post must go through. Large systems instead use a Snowflake IDSnowflake IDA 64-bit ID made of a timestamp, a machine number and a per-millisecond counter, invented at Twitter. Unique without coordination, sortable by time, and half the size of a UUID.: a 64-bit number whose top bits are the current time in milliseconds, followed by a machine number and a small per-millisecond counter. Any server can create one without asking anyone, and they sort roughly by time. Only roughly: two machines' clocks can disagree by a few milliseconds or more, so two posts made in the same instant on different servers may come out in either order (Clocks explains why). For a feed that's fine. UUIDv7UUIDv7A UUID whose first 48 bits are the current time in milliseconds, followed by random bits. New IDs sort roughly by creation time, which keeps B-tree indexes compact. works the same way at 128 bits. Design a URL Shortener builds an ID generator step by step.

The timeline as a capped list

RedisRedisA popular in-memory data server: a fast key–value store and cache that also offers counters, sorted sets, pub/sub channels and streams (an append-only log for messages). (a popular in-memory data server) has a list type that fits exactly. A fan-out worker adds a new ID to the front and trims the list to 800 entries; the feed reads a range from the front:

> LPUSH timeline:bob 1860000000000000101
(integer) 1
> LPUSH timeline:bob 1860000000000000102 1860000000000000103
(integer) 3
> LTRIM timeline:bob 0 799
OK
> LRANGE timeline:bob 0 19
1) "1860000000000000103"
2) "1860000000000000102"
3) "1860000000000000101"

Why cap at 800? Almost nobody scrolls back through hundreds of posts, and a hard cap keeps memory predictable: 800 × 8 bytes ≈ 6.4 KB per user, as in §03. If someone does scroll past the end, fall back to pulling older posts from the post store, slower but rare. (800 is the cap Twitter described in its QCon San Francisco 2012 talk "Timelines at Scale", §06; any few hundred works.)

Spreading timelines over many cache servers

About 2 TB of timelines (§03, with overhead) won't fit on one server. Split them by user: hash(userId) % number of shards picks the cache server, so all of Bob's timeline lives on one shard and a feed load touches one server. That's ordinary hash shardingShardOne piece of a split-up dataset, together with the database server that stores it. “Tenant 42 is on shard 3” means its rows live on that server. (Sharding §03); use Consistent hashingConsistent hashingA way to map keys (user IDs, cache keys) to servers so that adding or removing one server moves only about 1/N of the keys, instead of reshuffling almost all of them as hash(key) % N does. so adding a server only moves a small share of users (Load Balancing §05).

Two more savings that real designs make:

• Only active users get a timeline. Someone who hasn't opened the app recently has no cached timeline at all. Workers skip them, and their timeline is rebuilt by pulling when they come back (§10). §03 already sized the cache for the 200 million daily active users only; without this rule you'd pay for every registered account, which is often several times more.
• The timeline is a cache, not the truth. The real data is in the post store and the follow graph. Any timeline can be thrown away and rebuilt from those two, which is what makes losing a cache server survivable (§10).

Follow and unfollow

When Bob follows Ana, her old posts aren't in his timeline: they were pushed before he followed. A small background job backfills them: it reads Ana's ~20 most recent post IDs (from her author list, below), reads Bob's list, merges the two with the §06 merge, and writes the result back in one pipeline. A Redis list can't insert into the middle cheaply, which is why it's read-merge-write.

When Bob unfollows Ana, her IDs are still in his list. The list holds bare IDs, so finding which ones are Ana's means checking each ID's author, or comparing against Ana's author list. That's real work, so it runs in the background, and meanwhile the read path drops posts from authors Bob no longer follows (§08).

Each author's own list of post IDs

Several pieces of this design need "this author's newest posts": pulling a celebrity (§06), the follow backfill above, and rebuilding a lost timeline by pulling (§10). In version 1 (§04) that was an index on the posts table. But at this scale the posts table is itself split over many database servers, usually by post ID so that hydrating a batch of IDs goes straight to the right servers. Then "Ana's newest posts" would mean asking every server and combining the answers (a scatter-gather), which gets slower as servers are added.

So keep one more small list per author: their own most recent post IDs, newest first, often called the user timeline (as opposed to the home timeline). It's written once per post, by the post API, and stored in the cache sharded by author ID, so author:ana lives on one server. Every "this author's newest posts" question becomes one read of that list, followed by the usual batch hydration by ID.

Assembling one page of feed

Step through one request

Bob opens the app. He follows 2 celebrities, and his pushed timeline holds the IDs fan-out workers put there. Watch each panel change:

1async function getFeed(bob, cursor) {
2  if (cursor) return readSnapshot(cursor);  // "load more": next slice of the saved ranking
3  const pushed = await timelines.read(bob);
4  const stars  = await follows.celebritiesOf(bob);
5  const pulled = await Promise.all(stars.map((s) => authorList(s)));
6  const ids    = mergeNewest(50, sortNewest(pushed), ...pulled);  // skips duplicates
7  const posts  = await postStore.getMany(ids);  // ONE batch call
8  const shown  = posts.filter((p) => visibleTo(bob, p));
9  const ranked = rerank(score(shown, bob));
10  const page   = ranked.slice(0, 3);
11  return { page, cursor: saveSnapshot(bob, ranked, 3) };
12}

Hydrate in one batch, not one lookup per post

Turning IDs into full posts is called hydrating them: hydrationHydrate (IDs into objects)Turning a list of IDs into the full records they point to (text, author name, like count) by looking them up, ideally in one batch request. Feeds store IDs, not copies of posts, so an edit shows up everywhere. adds the text, author name, like count and media URLs. (Same word as the front-end term for making server HTML interactive, different meaning.)

The tempting way is a loop: for each of 20 IDs, ask the post cache for that post. That's 20 network round trips, one after another, and a textbook N+1 problemN+1 query problemFetching a list with 1 query, then running 1 more query for each of its N items (each post's author, each order's items). The page makes N+1 round trips to the database where 1 or 2 would do.. Instead, ask for all 20 at once: Redis MGET post:120 post:119 …, or in SQL SELECT … FROM posts WHERE id = ANY($1) with the list of IDs as one parameter. One round trip, whatever the page size (N+1 §06 shows this fix in detail).

The batch comes back as a map from ID to post, so the code walks the original ID list to keep the timeline's order, and skips IDs that came back empty: those posts were deleted.

Pagination with a cursor

"Load more" must continue exactly where the reader stopped, even though new posts keep arriving at the top. Counting ("skip the first 20") breaks: if 3 new posts arrive while Bob reads page 1, page 2 starts 3 posts too early and he sees them twice.

Instead the server returns a cursorCursor (pagination)An opaque token the server hands out with each page that means “continue after here”. The client sends it back unchanged to get the next page; only the server knows what's inside (often the last row's sort key). that means "older than post 118". Because IDs sort by time, the next page is simply "the next IDs smaller than 118". New posts at the top don't shift anything. This is Keyset paginationKeyset paginationPaging by “rows after the last one I showed” (WHERE key > $last ORDER BY key LIMIT 20) instead of OFFSET. The index jumps straight to the right spot, so every page costs the same., the same idea as in API Design §07.

package main

import "fmt"

type Post struct {
	ID   int64
	Text string
}

// The post store. In production: a cache in front of the posts database.
var postsDB = map[int64]Post{
	120: {120, "lunch"}, 119: {119, "new single!"}, 118: {118, "goal!"},
	117: {117, "hello"}, 110: {110, "tour dates"}, // 109 was deleted
	104: {104, "first post"},
}

// fetchMany is ONE round trip for many IDs (Redis MGET, or SQL WHERE id = ANY($1)).
func fetchMany(ids []int64) map[int64]Post {
	found := map[int64]Post{}
	for _, id := range ids {
		if p, ok := postsDB[id]; ok {
			found[id] = p
		}
	}
	return found
}

// feedPage returns one page of posts older than `before` (0 = start from the newest),
// plus the cursor for the next page. ids is the merged timeline, newest first.
func feedPage(ids []int64, before int64, size int) ([]Post, int64) {
	start := 0
	if before != 0 {
		for start < len(ids) && ids[start] >= before { // skip what the reader has seen
			start++
		}
	}
	end := min(start+size, len(ids))
	pageIDs := ids[start:end]

	found := fetchMany(pageIDs) // ← one batch call, not one call per ID
	page := make([]Post, 0, len(pageIDs))
	for _, id := range pageIDs { // keep the timeline's order
		if p, ok := found[id]; ok { // a deleted post is simply missing: skip it
			page = append(page, p)
		}
	}
	var next int64
	if end < len(ids) {
		next = ids[end-1] // "continue after the last ID on this page"
	}
	return page, next
}

func texts(ps []Post) []string {
	out := []string{}
	for _, p := range ps {
		out = append(out, p.Text)
	}
	return out
}

func main() {
	ids := []int64{120, 119, 118, 117, 110, 109, 104}
	var cursor int64 // 0 = first page
	for range 3 {
		var page []Post
		page, cursor = feedPage(ids, cursor, 3)
		fmt.Printf("%q next cursor: %d\n", texts(page), cursor)
	}
}

Output (Go, then Node):

["lunch" "new single!" "goal!"] next cursor: 118
["hello" "tour dates"] next cursor: 109
["first post"] next cursor: 0

[ 'lunch', 'new single!', 'goal!' ] next cursor: 118
[ 'hello', 'tour dates' ] next cursor: 109
[ 'first post' ] next cursor: null

Look at page 2: two posts, not three. Post 109 was deleted, so hydration dropped it. Short pages are normal in feeds. Either accept them (the app just asks for more as the reader scrolls) or over-fetch a few extra IDs and trim to the page size. Notice also that cursor 109 is the ID of the deleted post. That's fine: the cursor is a position in the ID order, not a post that must exist.

The filter step

Between hydration and the response, drop anything Bob shouldn't see:

• Deleted posts (missing from the batch, or marked deleted, §10).
• Posts by authors Bob has unfollowed, blocked or muted since the IDs were pushed (§07). The check is against Bob's current follow, block and mute lists, cached per user.
• Posts that became private or were removed for breaking rules.

Filtering at read time is what makes the eventual consistency from §02 safe: the timeline may be a little out of date, but nothing the reader shouldn't see gets through.

Ranking: from newest-first to most-interesting-first

Why rank at all

Bob follows 300 accounts and opens the app twice a day. Between visits, those accounts post 150 times. Newest-first shows him the last 20 and buries his sister's baby photo from this morning under 20 posts from a news account that posts every 10 minutes. Ranking fixes that by putting the posts he's most likely to care about first.

Stage 1: candidate generation

Candidate generationCandidate generationThe first stage of a ranked feed: cheaply collect a few hundred posts that might be worth showing (recent posts from people you follow, plus a few others), before the more expensive scoring step. cheaply collects a few hundred posts that might be worth showing: the newest few hundred IDs from Bob's timeline plus the celebrities he follows (exactly the merged list from §06 and §08), maybe a few popular posts from accounts his friends follow. This step must be fast and generous. Anything it misses can never be shown.

Stage 2: scoring

Hydrate the candidates (§08) and give each one a number. Real systems learn the formula from data with machine learning; we'll skip how that's trained and just use a hand-written formula with three ingredients:

• Recency: newer is better, fading over hours.
• AffinityAffinityA number (say 0 to 1) for how close a reader is to an author, estimated from past behaviour: how often they like, comment on, message or visit that author. Used as one input to ranking.: how close Bob is to the author, from how often he likes, comments on or visits their posts. Computed in the background and stored per (reader, author) pair.
• Engagement: how much everyone else is reacting. A comment usually counts more than a like.

package main

import (
	"fmt"
	"math"
)

type Candidate struct {
	ID       int64
	AgeHours float64 // how old the post is
	Affinity float64 // 0..1: how much this reader interacts with the author
	Likes    int
	Comments int
}

type Weights struct{ Recency, Affinity, Engagement float64 }

func score(c Candidate, w Weights) float64 {
	recency := 1 / (1 + c.AgeHours/6)                             // 1.0 now, 0.5 at 6 h, 0.2 at 24 h
	engagement := math.Log10(1+float64(c.Likes+3*c.Comments)) / 4 // ~0..1; 10,000 → 1.0
	return w.Recency*recency + w.Affinity*c.Affinity + w.Engagement*engagement
}

func main() {
	w := Weights{Recency: 1, Affinity: 1, Engagement: 1}
	for _, c := range []Candidate{
		{ID: 1, AgeHours: 0.5, Affinity: 0.1, Likes: 3},                  // brand new, from an acquaintance
		{ID: 2, AgeHours: 5, Affinity: 0.9, Likes: 12, Comments: 4},      // your best friend
		{ID: 3, AgeHours: 20, Affinity: 0.2, Likes: 9000, Comments: 900}, // viral, but old
	} {
		fmt.Printf("post %d: %.2f\n", c.ID, score(c, w))
	}
}

Output (Go and Node print the same):

post 1: 1.17
post 2: 1.79
post 3: 1.45

Your best friend's 5-hour-old post (2) wins, the viral post (3) comes second despite being 20 hours old, and the brand-new post from an acquaintance (1) comes last.

Stage 3: re-rank for variety

Pure scores often put four posts from the same close friend in a row. The last stage adjusts the top of the list with simple rules: no author twice in a row, mix in a video between text posts, don't show the same link twice. It's a quick pass over the 20–50 best posts, not a re-score.

Try it

Move the weights and watch the feed reorder. Then turn on "no author twice in a row" and see what moves.

Why ranking makes caching harder

A newest-first timeline is the same list for every request until a new post arrives, so it caches well and pages cleanly with the "older than ID 118" cursor from §08. A ranked feed is different:

• Scores change on their own. Recency decays every minute and engagement grows as people react, so yesterday's ranking is wrong today even if no one posted.
• It's personal. Affinity is per reader, so there's no shared answer to cache across users; a ranked feed costs a scoring pass per user per visit.
• "Older than ID 118" no longer means "the next page". A ranked list isn't sorted by ID. The usual fix is to rank once when Bob opens the app, store that ranked list of IDs for a few minutes under a session ID, and make the cursor mean "session abc, position 20". Pull-to-refresh starts a new ranking.

That's why the stored timeline stays chronological: it's the cheap, cacheable candidate source. Ranking happens on top of it, at read time, on a few hundred candidates.

When things fail

Pick a failure and see what users notice, how the system recovers, and which number on the dashboard tells you it's happening:

1. A fan-out backlog

A celebrity who isn't flagged yet (say, someone who went viral this morning) posts, and 20 million inserts land in the fan-out queue. Every post behind it waits. Users see nothing broken; they just see new posts minutes late.

How to spot it: Consumer lagConsumer lagHow far a subscriber of a message broker is behind the newest message — e.g. 5,000 messages or 3 minutes. Growing lag means the subscriber is too slow., measured in seconds (how old is the oldest message nobody has handled yet?), not just in message count. A queue holding 10 million inserts is fine if workers clear it in about 10 seconds (at the 1,000,000 inserts/s fleet from §06); a queue of 10,000 that hasn't moved for a minute is not.

How to fix and prevent it:

• Split big fan-outs. One "deliver post 9001 to 20 million followers" message becomes 2,000 messages of 10,000 followers each, so every worker in the Consumer groupConsumer groupA set of consumer processes that share the work of reading one stream or queue: each message (or Kafka partition) goes to exactly one member of the group. can help.
• Two lanes. Posts from authors with few followers go into a fast queue; huge fan-outs go into a bulk queue. A normal user's post never waits behind a celebrity's.
• Scale workers on lag. Add workers when lag in seconds passes a threshold, and remove them when it drops (Worker Pools covers sizing and backpressure).
• Flag hot accounts automatically (§06) when their follower count or fan-out time crosses the line.

2. A deleted or edited post

Bob's timeline holds IDs, so an edit shows up everywhere the moment the post is next hydrated. Only the cached copy of the post itself must be updated or dropped (Caching §05).

A delete is harder: the ID is in 200 timelines (or 50 million). Removing it from all of them is another fan-out. Instead, mark the post as deleted in the post store, a TombstoneTombstoneA marker meaning “this key was deleted”, left in place of actually erasing it. An LSM tree writes one because it can't edit its immutable files (compaction removes both later); an open-addressing hash table leaves one in the slot so lookups for later keys keep probing past it., and let hydration drop it (§08). A cleanup job may remove the ID from timelines later, or not at all: it falls off the end of the 800-entry cap eventually. For legal removals that must happen fast, the tombstone is what makes it immediate: the post stops being served everywhere at once, because every read goes through the post store.

3. An unfollow

Bob unfollows Ana; her IDs are still in his timeline. The read-path filter (§08) checks Bob's current follow list and hides them right away, and a background job removes them from his list. This is the general pattern: the timeline may be stale; the read path enforces the current truth.

4. A timeline cache server dies

With 50 cache shards, losing one wipes the timelines of about 1 in 50 users. Nothing is lost for good: timelines are a cache built from the post store and the follow graph (§07). On a miss, the feed service rebuilds that user's timeline the slow way, by pulling from the author lists of the accounts Bob follows (§07), writes it back, and serves it. Users on that shard see one slow feed load, then normal service.

Two traps:

• A stampede. A million users miss at once, and a million pull-rebuilds hit the post store together. That's a Cache stampedeCache stampedeWhen a popular cached item expires and many requests miss at the same moment, all rush to the database to rebuild it, and the database falls over. Also called thundering herd or dog-piling. (Caching §07). Limit how many rebuilds run at once (a worker pool), merge identical rebuilds for the same user with Request coalescingRequest coalescingMerging identical requests that are in flight at the same time, so only one does the slow work and the rest share its result. Go's singleflight and a JS map of promises both do this., and serve a degraded feed (just the celebrity posts, or a "popular now" list) to whoever is waiting.
• The half-filled list. While Bob's timeline is missing, a fan-out worker pushes a new ID for him. LPUSH creates a brand-new list with one entry, and the next feed load sees a timeline that exists, so it never rebuilds. Bob's feed now shows one post. Workers should use LPUSHX, which only adds to a list that already exists, leaving the rebuild to create it. One small gap remains: a post fanned out while the rebuild is running is dropped by LPUSHX and isn't in the rebuild's pull either. So the rebuild re-reads the newest few posts of the accounts Bob follows right after writing the list, and merges in anything it missed.

5. Slow post store

If hydration is slow, the whole feed is slow (§08 hydrates every page). Give the batch lookup a strict TimeoutTimeoutA limit on how long to wait for something (a connection, a response) before giving up with an error, so a slow or dead peer can't hang your program forever., return whatever posts came back in time, and let the app fetch the rest on scroll. Degrade gracefullyGraceful degradationDesigning a system so that when a part fails, it keeps working with less — showing a default list instead of personalized recommendations — instead of failing completely.: a feed with 15 posts instead of 20 is a non-event; a spinner is not. (Reliability & SRE §08 covers mitigating first during an incident.)

What to monitor, and the trade-offs that remain

The dashboard

The first two are the ones to promise as SLOsSLO (service level objective)A reliability target you measure, like “99.9% of requests succeed in under 300 ms over 30 days”. The measured number (here, the share of fast successful requests) is the SLI; the 0.1% you're allowed to miss is the error budget. ("99% of feed loads under 300 ms"; "99% of posts delivered to followers within 10 s"). The rest are early warnings. Observability §09 shows how to turn an SLO into an alert that doesn't fire on every blip.

See your own post immediately

§02 allowed followers to see a post a few seconds late. But if you post and your post isn't on your own feed when the app refreshes, it looks like posting failed, and you post again. The common fix: on every feed load, also merge in your own last few posts, read straight from the post store, just like a celebrity's list in §06. One small extra lookup gives every user read-your-own-writesRead-your-writesA guarantee that after you change something, your own later reads show the change. Other users may still see the old value for a moment. for the one case where they'd notice (Replication §07 covers the same guarantee for database replicas).

The trade-offs, in one table

Check yourself

Work out the answer before you click. Q1 is §05, Q2 §06, Q3 §07, Q4 §08, Q5 §10, Q6 §09.

Cheat sheet