This lab puts the module's pieces together - token budgets, retrieval, hybrid search, caching and evals - at a size where the easy answers stop working.
At each step, try to decide before you read on. If you're arriving here directly, skim the RAG and hybrid search lessons in this module first. This lab assumes you know what chunks, embeddings, HNSW, BM25 and re-ranking are.
The brief
Tiffin is now a large company. Employees waste hours hunting for answers spread across old support tickets, restaurant contracts and internal wikis. You're building Ask Tiffin: an internal assistant where any employee types a question and gets a short answer with links to the exact sources.
What's in the corpus:
| Source | Documents | Average size | Tokens |
|---|---|---|---|
| Resolved support tickets | 7 million | 600 tokens | 4.2 billion |
| Restaurant partner documents (contracts, compliance PDFs) | 2 million | 4,000 tokens | 8 billion |
| Internal docs (policies, runbooks, wiki) | 1 million | 2,800 tokens | 2.8 billion |
| Total | 10 million | 15 billion |
Requirements:
- 30,000 employees, about 10 questions each per working day: 300,000 questions a day, peaking around 25 a second.
- First words of the answer within 2 seconds at p95. Full answer within 10 seconds.
- Every answer cites its sources. If the sources don't answer it, say so.
- Permissions are strict. A support agent sees tickets and policies. A partner manager sees contracts only for the restaurants they manage. Legal sees all contracts. Nobody may ever see content they can't open directly.
- About 150,000 documents are added or changed a day, mostly new tickets. New content is searchable within 15 minutes. Lost access must take effect within 5 minutes.
- Model spend under an illustrative $3,000 a day.
Out of scope: writing or editing documents, actions like "issue a refund" (that's the agents lesson), and image understanding beyond extracting text from PDFs.