Open a social app and the first screen is a feed. It looks like a list, but behind it is a small search engine that runs every time you pull to refresh. Thousands of posts from friends, pages and groups could go there. You'll look at about ten. Picking those ten, in the right order, in a fraction of a second, for hundreds of millions of people, is the whole problem.
The Twitter and Instagram labs solved how posts get delivered to followers, and the Reddit lab ranked a feed with a formula over votes and time. This lab starts where those stop: turning a pile of candidates into a ranked page, with a model, inside a strict time budget, without ever showing you the same post twice.
The brief
You're building the home feed for a social network like Facebook. People have friends, follow pages, and join groups. The feed mixes posts from all of them, plus some posts from accounts you don't follow yet that you're likely to enjoy, plus a few ad slots.
Who uses it?
- Viewers open the app several times a day and scroll for a minute or two. They expect something new and interesting every time.
- Posters are friends, pages (businesses, creators, news outlets) and group members. Their posts compete for a viewer's attention.
- Ranking engineers change the model every week or two and need clean training data.
In scope: collecting candidates from every source, ranking them with a machine-learned model, serving pages of the feed, recording what each viewer saw and did, and feeding that back into training.
Out of scope: how posts are delivered into per-user inboxes (the Twitter and Instagram labs cover it), the ad auction (we reserve slots and an ads service fills them), the mathematics of the model, comments, and media upload.
Requirements
Functional
- Return the first page of a ranked feed when the app opens or the viewer pulls to refresh.
- Serve further pages as the viewer scrolls, with no duplicates and no gaps.
- Mix candidates from friends, followed pages, groups and recommended accounts, with room for ad slots.
- Never show a post the viewer has already seen recently, unless something new happened on it.
- Record impressions (a post was actually on screen) and actions (like, comment, share, hide) for training and analytics.
Non-functional
- Fast: the first page in under 400 ms at p99, server side. It's the first thing people see.
- Always something: if ranking is slow or broken, show a worse feed, never an empty screen.
- Fresh: a friend's post should be rankable within about a minute of being posted, and a post going viral should be noticed within a few minutes.
- Consistent within a session: scrolling down must not reshuffle what the viewer already passed.
- Trainable: what the model saw when it ranked a post must be recorded exactly, so it can be trained on the same thing.
The last one sounds like an analytics detail. It turns out to shape the feature store.