Discord is a chat app, so its central table is easy to describe: messages, in channels, in order. What makes it hard is volume and skew. In 2017 Discord stored "billions" of messages on a 12-node Cassandra cluster. By early 2022 it was trillions of messages on 177 nodes, and the team was being paged for the database far too often. In May 2022 they finished moving it to ScyllaDB.
This note follows Discord's two engineering posts, five years apart. Together they're one of the best public records of a key-value and wide-column store at scale: how the data model was chosen, what went wrong as it grew, and how the move was done without taking chat down.
What they had
Discord started on MongoDB. Stanislav Vishnevskiy's 2017 post explains that around November 2015, at about 100 million stored messages, the data and its index no longer fitted in RAM and latencies became unpredictable. They wanted a database that scaled linearly, failed over on its own, needed little care and performed predictably, and they chose Cassandra.
The data model is a textbook Cassandra design:
PRIMARY KEY ((channel_id, bucket), message_id)
- The partition key is the channel plus a time bucket, so one channel's messages live together, but no partition grows forever. Each bucket covers about 10 days, sized to keep partitions under roughly 100 MB.
- The clustering key is
message_id, a Snowflake ID, so messages within a partition are already sorted by time.
At the time of that post, Discord ran a 12-node cluster with a replication factor of 3, each node holding nearly 1 TB of compressed data, and was handling well past 120 million messages a day.