Watch on YouTubeWatch on Instagram
About this video
Paste the whole handbook, or search it first? I measured both: here's what RAG really does.
How retrieval-augmented generation finds the right page, keyword vs meaning search, why combining them lost on my data, why finding the right paragraph still isn't answering, and what RAG looks like in 2026.
Episode 8 of Learn AI in Under 3 Minutes, level 2: using models well.
Measured for this video on a 2020 MacBook Air. Zephyra Air is made up (episode 7's airline), so no model could know its rules: a 312-paragraph handbook with near-miss rules side by side and a long tail of airports, fares and fee codes. 20 traveller questions, 14 in everyday words, 6 naming an exact code or place. Searches: keyword (BM25), meaning (Qwen3-Embedding-0.6B, open weights) and both fused by rank; the right paragraph was in the top 3 for 14, 18 and 16 of 20. Answers by Qwen3.5-0.8B (open weights, February 2026; greedy, thinking off) given: the question only (0/20); the whole handbook, about 22,804 tokens per question (15/20, 309 s per answer); the top 3 paragraphs by keyword (14/20), by meaning (16/20, about 270 tokens, 4 s), by both (12/20); only the right paragraph (19/20). An answer counts if it has the handbook's number and no number that is not in the question or the right paragraph. One small model and one made-up handbook are an example, not a benchmark.
Sources: Lewis et al., "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (Facebook AI Research, UCL, NYU; arXiv:2005.11401, NeurIPS 2020). Anthropic, "Introducing Contextual Retrieval" (September 19, 2024): contextual embeddings plus contextual BM25 cut failed top-20 retrievals by 49%, and by 67% with reranking; under 200,000 tokens, include the whole knowledge base in the prompt. Boris Cherny, who created Claude Code, on replacing its early vector database with agentic search (interviews, 2025).
0:00 Paste it all, or search?
0:29 How RAG works
1:08 Keyword or meaning
1:39 Found isn't answered
1:58 RAG in 2026
2:25 My take
Music: Voice Over Under by Kevin MacLeod, CC BY 3.0, via Wikimedia Commons (https://commons.wikimedia.org/wiki/File:Voice_Over_Under_(ISRC_USUAN1600001).mp3)
Follow @PranavInnovates for the tech story that matters, every day.
What the video says
I gave a small AI an airline handbook: three hundred and twelve paragraphs, twenty questions. With the question alone, zero right. Pasting in the whole handbook: fifteen. But every answer took five minutes. Then a search pulled out three paragraphs per question. Sixteen right, in under four seconds, reading eighty-four times fewer tokens. That's RAG: retrieval-augmented generation. How the search works, which search won, and why finding the page isn't enough. Last episode, the right page on the desk took a model from zero to thirteen. But a whole handbook won't fit every desk. So who fetches the right page? That's the idea in a twenty twenty paper from Facebook AI, Patrick Lewis and team: retrieve first, then generate. Step one, cut the handbook into paragraphs. Step two, put each one on a meaning map, like episode two's word map. Similar meaning, nearby points. Step three, the question lands on the map too: my suitcase didn't show up. The search grabs its three nearest paragraphs. Step four, those go on the desk, and the model answers from them. Sixty-five euros a day. Right. But look at the words. I said suitcase, didn't show up. The handbook says delayed baggage. Not one word in common. So which search finds it? Keyword, or meaning? Meaning. Keyword search came back with business bags and bicycles. Over all twenty, the right paragraph made the top three fourteen times by keyword, eighteen by meaning. Both combined, the usual advice? Sixteen. Worse than meaning alone, on my data. And finding isn't answering. Handed only the right paragraph, the model got nineteen. With meaning search, sixteen. Every point between is lost at the search. My nine-year-old is flying solo. The search brought the rule for teenagers flying alone, and the model quoted it. Forty-nine euros. The answer: eighty-five. So the fixes go after the search. Anthropic adds a line of context to every chunk, mixes in keyword search and re-ranks: sixty-seven percent fewer failed retrievals. And coding agents skip the map. Claude Code dropped its vector index. The agent searches with grep, reads, and searches again. Anthropic's rule: under two hundred thousand tokens, big models can just read it all. Bigger than that, or paying per token? Search first. My take: when RAG gets it wrong, I look at what the search returned before I touch the model. Usually the right paragraph never made it to the desk. Try it this week. Write twenty real questions, mark the paragraph that answers each, and count how often your search puts it in the top three. That's your ceiling. Next: models that think before they answer, and what that thinking costs you. Episode nine.