Sooner or later someone at Tiffin says "let's add AI to support". A customer types "where is my order?" or "can I get a refund for a cold biryani?", and instead of a form, they get a helpful answer in a few seconds.
This module is about building that kind of feature well. Not how models are trained, but how to treat one as a part of your system. And from a system designer's point of view, a large language model (LLM) is a strange dependency: it is slow, it charges by the word, it sometimes says different things to the same question, and it can be down. You already know how to design around slow, costly, flaky services (Module 12 covers timeouts and retries). This lesson gives you the numbers you need to do it for models.
A model is a remote function that writes text
Strip away the hype and an LLM call looks like any other API call:
POST /generate
{ "model": "large-v3", "messages": [...], "max_output_tokens": 400 }
You send text in. You get text back. Under the hood, the model reads your whole input, then writes its answer one small piece at a time, each piece chosen based on everything before it. That "one piece at a time" detail explains most of what follows: why answers stream, why long answers are slow, and why the same question can get different answers.
Most teams call a hosted model over HTTPS. Some run open models on their own GPUs. The design questions are the same either way: how much text goes in, how much comes out, how long it takes and what it costs.