The filesystem is the last unindexed corpus
Naive RAG over a hard drive will bankrupt you - and what to do instead
There is a moment in every "let's add RAG" conversation where someone says, with total confidence, "we'll just embed all their documents." I used to nod. Now I flinch. Because I have watched what "just embed everything" does to a bill, and I want to write down, plainly, why the hard drive is a different animal from every other corpus we feed a model - and what to do instead.
The asymmetry nobody prices in
1nicorn reads a small business owner's inbox, decides which emails deserve a reply, drafts those replies in the owner's voice, and generates the documents the conversation implies: invoices, proposals, NDAs, consulting agreements, statements of work. To do that well, it helps enormously to know things the business already knows - their standard rate, the way they word a limitation-of-liability clause, the name they use for a recurring client.
Sources
- Claude Code searches code with grep and glob on demand instead of embedding the repo
- Counter-argument: grep-only retrieval can burn model tokens (Milvus)
- BEIR: plain BM25 keyword search is a strong zero-shot baseline (Thakur et al., NeurIPS 2021)
- Recent work: fancy chunking barely beats fixed-size chunking
- Lost in the Middle: long-context stuffing is costly and unreliable (Liu et al., TACL 2024)
- Windows Recall: researchers read the decrypted store from the user's own session (CSO Online)