The broken telephone in your AI's memory
Why a relationship AI rots while it still sounds fluent, and the rule that tells you when to rebuild
An AI that drafts your email is only as good as its memory of the person on the other end. Most systems build that memory once, then patch it one message at a time, forever. That is a mistake, and it has a name from a children's game: the broken telephone.
The failure is almost impossible to catch, because it does not look like a failure. The model stays fluent the whole way down. The sentences are smooth, the tone is right, and the facts are slowly, quietly wrong. A dashboard will not flag it. The first person to notice is the customer on the receiving end of a reply that confidently misremembers them.
I build enterprise software by running a lot of Claude task windows, and one of the things those windows do is keep a memory of every correspondent a business talks to. I got the memory design wrong the first time in exactly this way. This is what I learned about what a memory should be made of, and when to throw it away and build it again.
Sources
- LLM as a Broken Telephone: iterative distortion compounds with chain length (Mohamed et al.)arxiv.org
- Semantic drift from lossy re-encoding, with the preference-fact mutation example (SSGM)arxiv.org
- Lost in the Middle: U-shaped accuracy, mid-context facts used far worse (Liu et al.)cs.stanford.edu
- Retrieval method moves accuracy far more than write strategyarxiv.org
- Generative Agents: retrieval by recency x importance x relevance (Park et al.)arxiv.org
- Zep/Graphiti: bi-temporal facts with explicit validity intervals (Rasmussen et al.)arxiv.org
- LSM-tree compaction triggers on a bound, not a clock (Sarkar et al., VLDB)vldb.org
- RAG freshness: ~15-20% retrieval loss over a stale gap, recall 0.92 -> 0.74tianpan.co
- Anthropic pricing, used for the rebuild cost modelclaude.com