Build an Estimator That Does Not Follow the Herd

Level
pro
Time to complete
30 min
Prerequisites
3

What you will buildA runnable Python reference for a format picker that rides the standard without collapsing into a monoculture. About 30 minutes, standard library only.

What You Will Build

You will build a small Python program that decides which document format to suggest next - an estimate layout, a proposal shell, an invoice style - and keeps getting better as real documents get sent and accepted. The hard part is not picking the current winner. Any argmax does that. The hard part is riding the winning format while refusing to converge onto it so hard that the next good format never gets a chance. A picker that always serves the current best is a picker that will slowly make every document look the same, and sameness is where recommender systems go to die (Chaney and colleagues, RecSys 2018; Shumailov and colleagues, Nature 2024).

So you will build the de-correlated version. It scores each format on a decayed, confidence-bounded signal, serves with a bandit that always keeps a door open for something new, and trips an alarm if the field starts collapsing toward one answer. The whole thing is about 120 lines of standard-library Python, and it runs on your laptop.

Prerequisites

  • Python 3.9 or newer on your machine. No third-party packages - math and random from the standard library are enough.
  • Comfort reading a dataclass and a loop. You do not need any statistics background; the two formulas you need are explained where they appear.
  • A text editor and a terminal. You will create one file, standards_engine.py, and run it a few times.

Tip

The weights, half-life, and floors below are starting values, not laws. Treat them as a surface you tune per document type: a social post trend turns over in days, a contract standard in months.

Mental Model

Think of every document format as a contestant, and every real action a user takes as a vote with a different weight. A document you generated that nobody touched is barely a whisper. An edit a human kept is the format surviving contact. A send is someone staking their name on it. A downstream win - an accepted quote, a paid invoice - is the market itself voting, and it counts the most. Only humans and the market get to vote; the machine's own untouched output never does. That one rule is what keeps you out of the loop where a model trains on its own output and collapses (Shumailov, Nature 2024).

Each contestant carries two running numbers. One is its standing: a decayed tally of those weighted votes, so an old champion fades unless it keeps earning. The other is its acceptance posterior: a Beta distribution that answers "how often does this actually land, and how sure are we?" You serve by sampling from those posteriors (Thompson, 1933) instead of always taking the leader, and you keep a hard floor of random exploration underneath. That floor is the open door. It is why the herd never fully closes.

Steps

Weight the signal

Create standards_engine.py. Start with the vote table and the constants you will tune. Every weight here is a design decision: generation is near-zero because it is not evidence a human wanted anything, and acceptance is highest because it is the market, not the machine, talking.

import math
import random
from dataclasses import dataclass

SIGNAL_WEIGHTS = {
    "generated": 0.1,     # produced, but nobody acted on it (weak)
    "edit_kept": 0.5,     # a human left a generated block untouched
    "edit_reverted": -0.5,
    "sent": 1.0,          # the user staked their name on it
    "accepted": 3.0,      # the market accepted it (highest)
    "rejected": -1.5,
}
POSITIVE = {"sent", "accepted"}
NEGATIVE = {"edit_reverted", "rejected"}

HALF_LIFE_DAYS = 45.0   # a winning format fades this fast if it stops earning
GAMMA = 0.9             # sub-linear attachment: <1 keeps a live field
EPSILON_MIN = 0.10      # exploration floor, never 0
GRACE_TRIALS = 50       # protect a newborn variant from early variance
MIN_TRIALS = 30         # anti-fluke: a format must earn volume to be crowned
MARGIN = 0.15           # a challenger must clearly beat the incumbent
K_WINDOWS = 5           # and beat it for this many windows in a row
DIVERSITY_FLOOR = 0.6   # below this, monoculture alarm
LCB_QUANTILE = 0.10     # promote on the pessimistic 10% bound

You know this step worked if the file imports cleanly: run python -c "import standards_engine" and get no output and no error.

Score a format with a decaying memory

Add the state object and the update. The decay is the whole trick against lock-in: on every event you multiply the old standing by a half-life factor before adding the new vote, so popularity has to be renewed to survive. Preferential attachment is the motor (Barabasi and Albert, 1999); the decay is the brake that keeps QWERTY-style lock-in from freezing the field.

@dataclass
class FormatState:
    format_id: str
    S: float = 0.0        # decayed, quality-weighted standing
    alpha: float = 1.0    # Beta prior for acceptance
    beta: float = 1.0
    trials: int = 0
    last_ts: float = 0.0
    win_windows: int = 0

def _decay(dt_days):
    return math.exp(-math.log(2) * max(0.0, dt_days) / HALF_LIFE_DAYS)

def on_event(st, signal, ts_days):
    w = SIGNAL_WEIGHTS[signal]
    d = _decay(ts_days - st.last_ts)
    st.S = st.S * d + max(0.0, w)     # decayed cumulative advantage
    st.alpha *= d
    st.beta *= d
    if signal in POSITIVE:
        st.alpha += w
    elif signal in NEGATIVE:
        st.beta += abs(w)
    if signal == "sent":
        st.trials += 1    # one trial per document that actually went out
    st.last_ts = ts_days

You know this worked if you can feed one format two events and watch its standing rise: f = FormatState("A"); on_event(f, "sent", 0); on_event(f, "accepted", 0); print(f.S, f.alpha) should print a standing near 4.0 and an alpha near 5.0.

Sources

  1. Thompson sampling: balancing exploit and explore (Thompson, Biometrika 1933)academic.oup.com
  2. Finite-time analysis of the multi-armed bandit, UCB (Auer, Cesa-Bianchi and Fischer, ML 2002)link.springer.com
  3. Preferential attachment produces winner-heavy popularity (Barabasi and Albert, Science 1999)science.org
  4. Algorithmic confounding increases homogeneity and decreases utility (Chaney, Stewart and Engelhardt, RecSys 2018)dl.acm.org
  5. AI models collapse when trained on recursively generated data (Shumailov et al., Nature 2024)nature.com