File sync looks like "upload a file, download it elsewhere". Then someone edits a 2 GB video on a train, a laptop wakes up after two weeks offline, and two people save the same spreadsheet at the same moment. In this lab you'll design a file sync service from a blank page, modelled on a product like Dropbox. The goal is a design where every device ends up with the same files, and where changing one byte of a big file doesn't mean uploading the whole thing again.
The brief
People install a client on their laptops and phones. It watches a folder. Anything they put in that folder shows up on every other device they own, and in the folders of people they share with. They can also browse files on the web.
In scope:
- Upload, download and sync of files across a user's devices.
- Sending only the parts of a file that changed.
- Detecting and handling conflicting edits.
- Shared folders between users.
- File version history (restore last month's copy).
Out of scope:
- Real-time co-editing inside a document (see the Google Docs lab).
- Search inside file contents, previews and thumbnails.
- Billing, quotas and admin consoles.
Requirements
Functional
- A user can add, edit, rename, move and delete files in a synced folder.
- Changes appear on the user's other online devices within seconds.
- An offline device catches up when it reconnects.
- Shared folders sync between all members.
- If two devices change the same file at once, neither edit is lost.
- A user can restore a previous version of a file for 30 days.
Non-functional
- Never lose data. A file the client reports as synced must survive disk, server and data-centre failures. This beats everything else.
- Efficient: editing part of a big file should upload roughly the changed part, not the whole file.
- Fast enough: small changes visible on other devices within ~5 seconds.
- Resumable: a 10 GB upload over hotel Wi-Fi must survive disconnects.
- Scale: hundreds of millions of users and exabytes of data.