Where to limit

You know how to count. Now three practical questions. Where in the request's path does the counting happen? What do you count by: IP address, user, phone number? And when a client goes over, what exactly do you send back?

Get these wrong and a perfectly good algorithm still fails. Count by IP in India and you'll block a whole college hostel because one student wrote a script. Reply with the wrong status code and Tiffin's own app will retry instantly and make things worse.

Layers, from the edge inward

A request to Tiffin passes through several places that could reject it. Each sees different information and costs a different amount to reach.

The rule of thumb: reject as early as you can, but only with information you trust at that layer. A flood of 50,000 requests a second from one IP should die at the edge before it costs you anything. "This user has tried 10 wrong coupons in an hour" can only be judged inside the coupon code, because only it knows the coupon was wrong.

Most real systems use several layers. Tiffin's managed load balancer from Module 4 can't count requests per IP, so an nginx reverse proxy in front of each app server holds a loose per-IP limit as a seatbelt against floods. (Behind the load balancer, nginx has to read the client's IP from X-Forwarded-For, trusting it only from the balancer, or it would count the balancer's address.) The app has per-user limits. A few sensitive features have their own strict limits. In Module 11 we'll give this a proper home in an API gateway, a single front door that handles limits, authentication and routing for many services.

Choosing the key

The key decides who shares a budget. Pick it wrong and either abusers slip through or innocent people get punished together.

IP address

The only key you have for requests that aren't logged in. Cheap and available everywhere. But an IP is not a person.

In India this is a big deal. Mobile carriers use carrier-grade NAT: they put many subscribers behind a single public IPv4 address because there aren't enough addresses to go round. Hundreds or even thousands of phones can share one IP at the same moment. Office networks, college hostels and some home broadband do the same on a smaller scale.

So a per-IP limit of 60 requests a minute could block a whole neighbourhood of Tiffin customers at 8pm because they share an IP with someone browsing fast. Per-IP limits must be generous, a seatbelt against floods, not a tool for fairness.

IPv6 has the opposite problem. A single phone can get a huge block of addresses and change them freely, so a script can rotate through millions of IPs. Limit IPv6 clients by their /64 prefix (the part of the address the network assigns), not the full address.

User ID

Once someone is logged in, this is the best key. One person, one budget, no matter which network they're on. An abuser can create more accounts, but that costs them a phone number and an OTP each time, which you also limit.

API key

For partners like a restaurant chain's point-of-sale system. Each partner gets their own key and a limit sized to their real volume. A big chain with 200 outlets gets more than a single café. This is also where tiers live: a partner on a higher plan gets a bigger bucket.

The thing being attacked

Some limits make sense only on the target. To stop OTP guessing, limit attempts per phone number, because the attacker can switch IPs and devices but not the number they're attacking. To stop SMS pumping, limit OTP sends per phone number and per IP and in total per hour, because the attacker uses thousands of numbers.

Per-feature limits

A general limit of 300 requests a minute is fine for browsing. It's useless for login. At 300 a minute, an attacker could try 300 OTP codes a minute. Sensitive features need their own, much smaller limits, on top of the general one.

Tiffin's feature limits:

• Send OTP: 3 per phone number per 10 minutes, 20 per IP per hour (generous, because many phones share one carrier IP), and an alarm if total sends go above 3x the normal rate.
• Verify OTP: 5 wrong attempts per code. After that, the code is dead and a new one must be requested.
• Apply coupon: 10 wrong codes per user per hour.
• Place order: 10 per user per 10 minutes. Nobody orders dinner 11 times in 10 minutes.

Notice the coupon limit counts only failures. A customer applying the same valid code twice shouldn't use up their allowance. Only the coupon code knows whether the code was wrong, which is why this limit must live inside the feature.

What to send back

When a client is over the limit, reply with HTTP 429 Too Many Requests. Not 500 (that says your server broke), and not 503 (that says your server is overloaded for everyone). 429 tells the client: you specifically are asking too fast, and the server is fine. Clients and libraries react to those codes differently.

Tell the client when to come back, and how much budget it has:

HTTP/1.1 429 Too Many Requests
Retry-After: 12
X-RateLimit-Limit: 300
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 12
Content-Type: application/json

{
  "error": "rate_limited",
  "message": "You're going a little fast. Try again in a few seconds.",
  "retry_after_seconds": 12
}

• Retry-After is a standard header, in seconds (or a date). Every client should honour it.
• The X-RateLimit-* headers are a widely copied convention, not a standard, and they vary: GitHub's X-RateLimit-Reset is a Unix timestamp, while the example above sends seconds to wait. An IETF draft is standardising similar RateLimit headers, and the exact format is still settling. Pick one style, document it, and be consistent.
• Sending the remaining budget on successful responses too lets well-behaved clients slow down before they hit the wall.

The body should be readable by humans. The Tiffin app shows that message instead of a generic "Something went wrong".

Other actions besides rejecting

Rejecting isn't the only option:

• Delay. Queue the request briefly instead of refusing it, like nginx's burst without nodelay. Good for partner integrations that can wait.
• Degrade. Serve a cached menu instead of a fresh one. The client still gets an answer and the database is spared.
• Challenge. Ask for a CAPTCHA. Humans pass, most scripts don't.
• Log only. Run a new limit in shadow mode first: count, record who would have been blocked, but let everything through. You'll see whether real customers would be hit before any of them are. Always do this for a new limit.

The client's side

Your own app must behave well when it gets a 429. If the Tiffin app retries immediately, a rate limit becomes a retry storm. The app should wait at least Retry-After, and spread retries out with backoff and a bit of randomness so thousands of phones don't retry at the same instant. Module 12 covers retries and backoff properly.