This is the lab behind Episode 11. It exists to make one uncomfortable finding reproducible: grow the knowledge base and retrieval quality can fall without anything looking broken. Clone it, run the four retrieval arms against the golden set, then add documents and watch the numbers move. The same shape, a frozen golden set plus a harness that scores every change, is what I use as release control for agents at work.
Adding more documents to a knowledge base can make retrieval worse, and the answers still sound confident. Teams find out from users, not from a test.
A fictional knowledge base (product docs, policies, incident reports, requirements, test-run notes), a frozen golden set of 50 questions across easy, hard-paraphrase, multi-hop and unanswerable tiers, each mapped to the document that answers it, and a zero-key retrieval harness with four arms so the comparison runs on a laptop.
A regression gate for retrieval you can run before every knowledge-base change. Episode 11, AI Evals Explained, uses it to show the dilution effect live.