It's 8:12pm on a Tuesday. Tiffin's dashboards turn red. CPU on every app server is at 90%, menu pages take four seconds to load, and the on-call engineer assumes it's the dinner rush. It isn't. Real customers are sending about the same traffic as last Tuesday. The extra load comes from a single IP address asking for restaurant menus 10,000 times a second. A competitor is copying Tiffin's prices.
Nothing was broken in the code. The servers did exactly what they were built to do: answer every request that arrived. That's the problem. Nobody had decided how much any one client was allowed to ask for.
This module is about making that decision, and making it stick when Tiffin runs on many servers.
What a rate limit is
A rate limit is a rule that says how many requests one client may make in a period of time, and what happens to the requests over that number.
"100 requests per minute per user" is a rate limit. So is "3 OTP messages per phone number per 10 minutes". Every rate limit has the same three parts:
| Part | Question | Tiffin example |
|---|---|---|
| Key | Who is being counted? | User ID, IP address, phone number, API key |
| Limit | How many, over how long? | 300 requests per minute |
| Action | What happens when they go over? | Reply with an error and ask them to wait |
The rest of this module fills in those three columns. The next two lessons are about how to count (windows and buckets). Then we look at where to count and what to send back, and finally how to keep one shared count when Tiffin runs on eight servers.
Who sends too many requests
It's tempting to think of rate limiting as "stopping hackers". Attackers are only one of four groups, and often not the biggest.
Bots and scrapers
The 8:12pm incident. Someone writes a script that walks every restaurant ID and downloads every menu, every few minutes. They're not trying to break anything. They just don't care what it costs you. Scrapers often spread across dozens of cloud IPs so no single address looks too busy.
Attackers
These want something specific:
- Password guessing. Try thousands of passwords against one account, try one common password against thousands of accounts (password spraying), or replay email and password pairs leaked from other sites (credential stuffing).
- OTP guessing. Tiffin logs people in with a 4-digit code sent by SMS. There are only 10,000 possible codes.
- Coupon guessing. Try
TIFFIN10,TIFFIN20,RAHUL50and thousands more against the "apply coupon" endpoint until one works. - SMS pumping. Ask Tiffin to send OTPs to thousands of premium-rate numbers the attacker gets paid for. Tiffin pays for every SMS.
Your own bugs
This one surprises people. Some of the worst floods come from your own app. A new release has a bug where the order-status screen refetches every time it re-renders, and it re-renders every time it refetches. Each phone now sends 20 requests a second instead of one every 10 seconds. Multiply by 40,000 phones that updated overnight.
A partner can do the same thing. A restaurant chain connects its point-of-sale system to Tiffin's API, and a retry loop in their code hammers you every time your server is slow, which makes your server slower.
Normal people doing odd things
The customer who keeps pulling to refresh while waiting for their biryani. A person can manage maybe 2 or 3 refreshes a second. If you see one account sending 50 requests a second, it's not a hungry human. It's a stuck button, a script, or a bug. Either way, it shouldn't be allowed to slow down everyone else.
What's at stake
Rate limits protect four different things. It helps to know which one you're protecting, because it changes the numbers you pick.
Capacity
Say Tiffin's eight app servers together handle about 7,800 requests a second at the comfortable 70% of their knee (8 x 1,400 x 0.7, from Module 4). At the dinner peak, real customers send about 3,600. That leaves about 4,200 spare. One scraper at 2,000 requests a second eats half of it. Two more and real customers start waiting.
A per-client limit means no single client can take more than a small slice. If no user may exceed 5 requests a second, it takes 840 such clients to use up the spare 4,200.
Money
Every OTP is an SMS, and every SMS costs money. Say Tiffin pays ₹0.25 per message. An attacker who triggers 200,000 OTPs in a night costs Tiffin ₹50,000. Nothing crashed. The bill just arrived.
The same goes for anything you pay per call for: maps lookups, payment checks, AI features. A limit there is a spending cap.
Security
A 4-digit OTP has 10,000 possible values. With no limit, a script making 100 guesses a second tries every code in 100 seconds. With a limit of 5 attempts per code, the chance of a lucky guess is 5 in 10,000, or 0.05%. The limit turns a certain break-in into a very unlikely one.
Fairness
Even when nobody is malicious, one heavy client can slow down others who share the same servers, database or partner API. Limits keep one tenant from crowding out the rest.
What rate limiting is not
Rate limiting is one tool in a box. It's worth knowing its neighbours so you don't ask it to do their jobs.
| Problem | Better tool | Why rate limiting alone isn't enough |
|---|---|---|
| Everyone is busy at once (real dinner rush) | More servers, caching (Modules 3 and 4) | Every client is under its limit. The total is still too much. |
| The whole system is overloaded right now | Load shedding (Module 12) | Load shedding protects total capacity. Rate limits protect against one client. |
| A retried "place order" creates two orders | Idempotency keys (Module 12) | Even two requests can do damage. A limit of 10 doesn't help. |
| A determined attacker with 10,000 IPs | Bot detection, CAPTCHA, a WAF | Each IP stays under the limit. |
That first row matters most. Rate limiting doesn't add capacity. If real traffic triples, a rate limit just turns slow pages into error pages. Its job is to stop one client from taking more than its share.
How do you pick a number?
Start from what real clients actually do, not from a round number that feels safe.
Look at a week of logs and count requests per user per minute. For Tiffin you might find:
- The median customer sends 8 requests a minute while using the app.
- The 99th percentile sends 45.
- The 99.9th percentile sends 90 (they're comparing restaurants fast).
- A handful of accounts send over 2,000. Those are scripts.
A good limit sits well above the honest heavy users and far below the abusers. Here, 300 per minute leaves more than 3x headroom for the busiest real customer and still cuts the scripts down by 85% or more.
Two rules of thumb:
- Be generous on reads, strict on expensive or risky actions. Browsing a menu is cheap. Sending an SMS costs money. Guessing a coupon code is an attack.
- A limit that blocks real customers is a bug. A customer who sees "too many requests" at 8pm while trying to order dinner will open a competitor's app. Err on the generous side and tighten later with data.
Name the protection
For each Tiffin endpoint, say which of the four things (capacity, money, security, fairness) a limit mainly protects, and whether the limit should be loose or strict.
GET /restaurants/:id/menuPOST /auth/otp(send an OTP by SMS)POST /cart/coupon(apply a coupon code)POST /partner/orders(a restaurant chain's POS system pushing order updates)
Show one answer
- Capacity. Menus are cheap and cached, so the limit can be loose (hundreds per minute per user). Its job is to stop scrapers, not browsers.
- Money, and security. Each call costs an SMS and could be used for pumping. Strict: a few per phone number per 10 minutes.
- Security. Strict on failed attempts: a real customer tries one or two codes, not fifty. Count wrong codes per user, maybe 10 per hour.
- Fairness and capacity. One partner's buggy retry loop shouldn't slow other partners or customers. Give each partner API key its own limit, sized from their real order volume with plenty of headroom.
What to remember
- A rate limit has three parts: the key (who is counted), the limit (how many per how long) and the action (what happens when they go over).
- Floods come from bots, attackers, your own buggy releases and partners, and occasionally odd human behaviour. Plan for all of them.
- Limits protect capacity, money, security and fairness. Knowing which one you're protecting tells you how strict to be.
- Rate limiting stops one client taking too much. It doesn't add capacity and doesn't replace caching, scaling, load shedding or idempotency.
- Pick limits from real traffic: well above honest heavy users, far below abusers. A limit that blocks real customers is a bug.