A rider delivers an order to a basement flat at 20:04. There's no signal down there. The rider app marks the order delivered, holds the event, and sends it when the rider is back on the street at 20:07.
Tiffin's dashboard counts "deliveries per minute". Which minute does this delivery belong to: 20:04, when it happened, or 20:07, when Tiffin heard about it? And if the dashboard already showed a final number for 20:04 at 20:05, what should it do now?
That's the whole lesson in one example. Streams never end, so to count anything "per minute" you have to cut them into pieces, and the pieces have to deal with events that turn up late.
Two kinds of time
Every event has at least two timestamps.
| Time | Meaning | Example |
|---|---|---|
| Event time | When the thing actually happened | 20:04, at the customer's door |
| Processing time | When the stream processor saw it | 20:07, after the phone reconnected |
Counting by processing time is easy: look at the clock when the event arrives. But the answer depends on network delays, not on what happened. If Kafka has a 5-minute hiccup, the dashboard shows zero deliveries for 5 minutes, then a huge spike. Nothing like that happened on the road.
Counting by event time gives the true answer, but it creates a new problem: you never know for sure whether all the events for 20:04 have arrived.
Event time is when something happened in the real world. Processing time is when your system noticed. For anything that describes the real world, count by event time.
Event time comes from a clock you may not trust. A rider's phone can be minutes off (Module 12 covered why clocks lie). A common compromise: the app records how long ago the event happened by its own clock, and the server subtracts that from its own receive time. That's only as good as the phone's clock over a short gap, which is usually good enough.
Sign in free to keep reading
A free account opens every lesson, all 34 labs and the cloud pages, and saves your progress across devices. Sign in with Google: no payment, no card.