Set a Rebuild Threshold for AI Memory: stop patching a dossier forever

Level
intermediate
Time to complete
25 min
Prerequisites
4

What you will buildA touched-and-thin trigger that rebuilds a per-person memory from source only when the patches have piled up. About 25 minutes.

What You Will Build

You will build a small decision that runs before your AI updates its memory of a person: should this dossier be rebuilt from the original history, or patched cheaply like every other day?

By the end you will have a should_rebuild function that fires only when a dossier is both touched (new mail arrived) and thin (the daily patches have piled up past a threshold). Brand-new correspondents always build from scratch. Everyone else rides the cheap patch until they trip the rule. You will also have an honest way to replace the default threshold with a number measured on your own data, instead of trusting mine.

This is small code. The hard part is not the function. It is knowing which three numbers to track and when to trust them.

Prerequisites

  • A per-person memory file (a dossier) that you currently build once and update incrementally.
  • A source store you can re-read in full - the original emails or messages, not just the dossier.
  • A place to persist a few counters per person, next to each dossier.
  • An existing "build from scratch" routine and an existing "merge one message" routine. This tutorial wires a trigger around them; it does not replace them.

Tip

If you do not keep the original messages anywhere, stop here and fix that first. A rebuild re-reads the source. If the only copy of the history is the patched dossier, you can never rebuild clean, and this entire pattern is impossible. Keep the raw source.

Mental Model

Think of each daily patch as a photocopy of a photocopy. One copy is fine. The tenth is still readable but the edges are wrong. At some point you go back to the original document and make a fresh copy. The rebuild threshold is the rule that says when.

There are two reasons a dossier goes bad. The first is drift: every merge rewrites a rewrite, and the meaning slowly warps even when no new fact was added. The second is bloat: even with zero drift, the file grows, and a model reading a long file skims the middle, so your best facts get buried where they barely get read.

Databases solved the "when to rebuild" question decades ago with log-structured merge trees. They take cheap writes all day and compact them into a clean level only when the number of un-merged pieces crosses a bound. Not on a clock. On a count. That is exactly the rule you are about to build: rebuild when the patches earned it, not when a timer went off.

Steps

Six steps. The first two are free; the trigger logic unlocks with Supporter. Follow them in order - each one builds on the state record from the first.

Track the three numbers that tell you a dossier is thin

Next to each dossier, persist a tiny state record. Everything downstream reads from this. Initialize it the first time you build a correspondent from scratch.

# one record per correspondent, saved beside the dossier
memory_state = {
    "merge_count": 0,            # K: patches since the last full rebuild
    "last_rebuild": "2026-09-01",
    "new_mail_since_rebuild": 0,
    "dossier_chars": 0,          # length of the live dossier
    "distinct_facts": 0,         # facts counted at the last rebuild
    "redundancy_baseline": 1.0,  # chars-per-fact captured at the last rebuild
}

Then make your existing daily merge update these numbers every time it folds in a message:

def patch(correspondent, message):
    dossier = merge_message(load_dossier(correspondent), message)  # your cheap merge
    state = load_state(correspondent)
    state["merge_count"] += 1
    state["new_mail_since_rebuild"] += 1
    state["dossier_chars"] = len(dossier)
    save_dossier(correspondent, dossier)
    save_state(correspondent, state)

You know it worked when, after a few days of patches, merge_count and new_mail_since_rebuild climb and last_rebuild stays fixed.

The three signals you track per correspondent: merges since rebuild, days since rebuild, and bloat.

Write the touched check

A dossier that got no new mail has had no new drift injected into it. There is nothing to fix, so you never rebuild it on mail volume alone. This check is one line, and it is the cheap gate that keeps you from rebuilding idle relationships.

def is_touched(state):
    return state["new_mail_since_rebuild"] > 0

You know it worked when a correspondent you have not heard from since the last rebuild returns False, and one who wrote yesterday returns True.

Sources

  1. LSM-tree compaction triggers on a bound, not a clock (Sarkar et al., VLDB)vldb.org
  2. Semantic drift from lossy re-encoding across merges (SSGM)arxiv.org
  3. RAG freshness: cosine drift bands and the stale-index recall droptianpan.co
  4. Lost in the Middle: growth alone buries facts (Liu et al.)cs.stanford.edu