It's 8:20pm. Support says customers are complaining that placing an order "takes forever". The dashboards show checkout p99 latency has gone from 400 ms to 4 seconds. In the monolith, you'd open the slow request's log, read the stack trace and probably find the slow query in a few minutes.
Now checkout touches the gateway, Orders, Catalogue, Payments, Fraud and Accounts. Each has its own logs on its own machines. Which one is slow? Is it slow for everyone, or only for some restaurants? This lesson covers the three tools that answer those questions: logs, metrics and traces, and how to make them work across many services.
Three kinds of signal
| Signal | Answers | Example |
|---|---|---|
| Metrics | Is something wrong, and how much? | Checkout p99 is 4 s, errors 2% |
| Logs | What happened in one place? | "Payment provider returned 503 for order 81" |
| Traces | Where did the time go for one request? | 3.6 s of 4 s was spent waiting on Fraud |
You need all three. Metrics tell you there's a fire. Traces tell you which room. Logs tell you what's burning.