The home timeline looks like the simplest screen in the world. It's a list of recent posts from people you follow, newest first. The trouble is the word follow. One person posts, and that post has to appear in the timelines of everyone who follows them. For most people that's a few hundred timelines. For a pop star it's a hundred million.
This lab is about that one problem: getting posts into timelines fast, for everybody, without the biggest accounts melting the system. Work through it from a blank page, and try to decide each step before you read on.
The brief
You're building the home timeline for a service like Twitter. People post short messages (call them tweets). Each person follows other accounts. When you open the app, you see the latest tweets from the accounts you follow.
Who uses it?
- Readers open the app many times a day and scroll. Most people mostly read.
- Posters write tweets. Most accounts post rarely. A small number post constantly.
- Celebrities are posters with millions of followers. There are only a few thousand of them, but they shape the design.
In scope: posting a tweet, following and unfollowing, and reading your home timeline in reverse-chronological order.
Out of scope: search, trends, direct messages, notifications, media upload (assume images live in object storage behind a CDN and a tweet stores only a link), ads, and ranking by a machine-learned model. We'll mention where ranking would plug in, but the timeline here is newest first.
Requirements
Functional
- Post a tweet of up to 280 characters.
- Follow and unfollow an account.
- Read your home timeline: tweets from accounts you follow, newest first, with pagination.
- Read a user timeline: one account's own tweets.
Non-functional
- Timeline reads are fast: p99 under 200 ms at the server. This is the screen people open most.
- Timelines are highly available. Showing a slightly stale timeline is far better than an error.
- New tweets show up quickly: within about 5 seconds for most followers. A minute for a celebrity's followers is acceptable.
- Tweets are durable. A posted tweet is never lost.
- Eventual consistency is fine. If you see a tweet a few seconds after your friend does, nobody notices.
That last line is a big gift. You don't need every follower to see a tweet at the same instant, which frees you to do the work in the background.