Module 20 / Databases in depth

Module 20 · lesson 9 of 10 · Read 9 min

Columnar, wide-column and graph stores

When rows in tables are the wrong shape for the data.

Everything in this module so far assumed one shape of data: rows in tables, read and written a few at a time. That shape fits Tiffin's core beautifully. Place an order, update a status, show a customer their last 20 orders. A relational database like Postgres is the right home for it (see Module 2 on choosing a database).

But some questions don't fit that shape. "What was the average delivery time per city, per hour, over the last 90 days?" touches hundreds of millions of rows. "Store every rider's GPS ping, 37,500 a second, forever" is more writes than one Postgres primary wants to take. "Which accounts share a phone, a card or a device with this fraudster, up to four steps away?" is a chain of joins of unknown length.

This lesson covers three kinds of store built for those three shapes: columnar, wide-column and graph. The aim is to recognise the shape of a problem, not to learn each product.

How a row store lays out data

A row store (Postgres, MySQL) keeps each row's columns together on disk. A page of the orders table holds a few dozen complete orders: id, customer, restaurant, city, total, status, timestamps, all side by side.

That's ideal when you want whole rows: "fetch order 9001" reads one page and has everything. It's wasteful when you want one column of many rows. To average delivery_minutes over 300 million orders, a row store reads every column of every order, even though you need two of the forty.