Model Evolution II: Test-Time Training and the Continual Learning Problem

Post 5 asked where the training signal comes from; this post asks when the weights change — and the two most radical answers bracket the question. Test-time training updates weights during inference, per input, then throws the update away; continual learning updates weights forever and keeps them. One treats adaptation as disposable, the other as permanent — and between them sits the oldest unsolved problem in neural networks: learning the new without trampling the old.

Test-time training: adaptation as a disposable good

The standard contract of deployment is train-then-freeze: whatever the input, the same weights serve it. Test-time training (TTT) breaks the contract surgically — when an input arrives, run a few gradient steps on an auxiliary objective built from that input (no labels needed: reconstruct masked pieces of it, enforce prediction consistency across augmentations, minimize prediction entropy), serve the answer with the adapted weights, then discard the adaptation. The model meets each input slightly specialized for it.

Why this earns its place on the map: it attacks distribution shift, the standing failure of frozen models — the input that’s unlike training data (corrupted, novel domain, a genuinely new puzzle structure) is exactly where frozen weights are worst and exactly where a self-supervised signal from the input itself helps most. The pattern has a notable record on abstraction-and-reasoning benchmarks, where per-task adaptation on a handful of demonstration pairs substantially outperforms frozen inference: the demonstrations are a tiny training set, and TTT actually uses them as one. On the taxonomy’s axes, TTT is the strangest point in the whole map: the weights evolve (deepest locus) but the loop closes inside a single task and persists for zero time (narrowest closure). Self-evolution with the blast radius of a cache entry — the safety profile of a harness tweak at the power level of training. The costs are equally clear: inference now carries a training bill (gradient steps per input), serving infrastructure must handle per-request weight copies, and the auxiliary objective is a proxy that can mis-adapt — entropy minimization happily makes a model confidently wrong on garbage input, so production TTT gates adaptation behind a shift detector and falls back to frozen weights when the signal looks degenerate.

Continual learning: adaptation that has to live together

Keep the updates instead of discarding them and you inherit the problem online machine learning has studied for decades, with a neural-network twist that has a brutal name: catastrophic interference. Gradient descent on new data moves all the weights that help the new task, indifferent to what else those weights were doing — and in a network, every capability is smeared across shared weights. Train on this month’s data and last year’s competence silently degrades; nothing errors, nothing logs, the model just quietly forgets. The field’s three mitigation families map cleanly onto how they spend resources: replay (keep or generate samples of old data, mix them into every update — simple, effective, costs memory and raises retention questions), regularization (estimate which weights matter to old capabilities, penalize moving them — elegant, no stored data, estimates degrade over long sequences), and isolation (give new learning its own parameters — adapters, routed experts — so old pathways are untouched; forgetting drops toward zero, parameters grow, and the router becomes the new thing that can fail). Production systems increasingly treat isolation-plus-replay as the default, because the two failure modes are at least visible: parameter count and buffer size show up on a dashboard; smeared-weight forgetting doesn’t.

The stability–plasticity frame

The deep tension under both halves is the stability–plasticity dilemma: a system plastic enough to learn from today is, by the same machinery, unstable enough to forget yesterday. TTT resolves it by fiat — perfect stability (nothing persists) purchased at the price of re-learning everything every time. Continual learning buys plasticity and pays in guardrails. Biology’s hint — and a live research direction mirrored in agent memory designs — is that the dilemma dissolves under two systems on two timescales: fast, disposable adaptation that accumulates episodes, and slow, gated consolidation that integrates them into long-term structure during protected downtime. Read this blog’s memory series against this post and the architecture rhymes: episodic store as the fast system, distillation into prompts and skills as consolidation — harness evolution as continual learning’s low-risk understudy, holding what the weights can’t yet safely absorb. That’s the bridge the final posts build on: in integrated systems, the harness accumulates and the weights consolidate — each locus covering the other’s weakness.

The takeaway: the when axis of model evolution runs from test-time training — per-input weight adaptation via self-supervised objectives, discarded after serving: the deepest locus at the narrowest closure, powerful against distribution shift and priced in inference-time compute and mis-adaptation risk — to continual learning, where kept updates collide with catastrophic interference and are managed by replay, regularization, and isolation. Underneath both sits the stability–plasticity dilemma, and the emerging resolution is two-timescale design: fast disposable adaptation plus slow gated consolidation, with the harness (memory, skills) serving as the safe fast layer while weight consolidation stays rare and heavily gated.

Key takeaways

Further reading

Sources & References