Overfitting and Generalization — The Central Problem of Machine Learning

Post 1 named generalization as ML’s obsession; this post makes it precise. Everything here — why you never trust training accuracy, why simpler is often better, why you hold data back — exists to answer one question: will this model work on inputs it has never seen? Getting this wrong is the most common and most expensive mistake in applied ML, including in LLM evaluation.

Overfitting vs underfitting

A model can fail to generalize in two opposite ways:

• Overfitting — the model learns the training data too well, capturing not just the real pattern but the noise, quirks, and accidents of that particular sample. It gets near-perfect training accuracy and poor accuracy on new data, because it memorized specifics that don’t generalize. An overfit model is too complex for the signal available — like a student who memorized the practice exam’s answers instead of learning the subject.
• Underfitting — the model is too simple to capture the real pattern, so it does poorly on both training and new data. A straight line fit to a curved relationship underfits.

The art of ML is navigating between them to the sweet spot: complex enough to capture the true signal, simple enough not to memorize the noise. This is the bias-variance trade-off: - Bias is error from overly simplistic assumptions (underfitting) — the model can’t represent the truth. - Variance is error from over-sensitivity to the training sample (overfitting) — the model changes wildly with different data. - Lowering one tends to raise the other, and the goal is the balance that minimizes total error on unseen data. Every regularization technique is a deliberate trade of a little more bias for a lot less variance.

The train/test split: never trust training accuracy

The single most important practice in all of ML follows directly: evaluate on data the model never saw during training. You split your data: - Training set — the model learns from this. - Test set — held back, untouched during training, used once to estimate real-world performance. - often a validation set too — for tuning choices (hyperparameters, model selection) without contaminating the final test estimate.

The iron rule: training accuracy is meaningless as a measure of quality, because a sufficiently complex model can memorize the training set perfectly while being useless on new data. Only performance on held-out data tells you whether the model generalized. Violating this — evaluating on data the model saw, or letting test data leak into training (data leakage) — produces impressively wrong results and is the most common way ML projects fool themselves. Cross-validation (rotating which slice is held out, averaging the results) makes the estimate more robust when data is limited. This discipline is exactly why LLM benchmarks obsess over contamination — a model that saw the test questions in training isn’t being measured, it’s being caught cheating.

Fighting overfitting: regularization and friends

Once you can detect overfitting (train accuracy ≫ test accuracy), you fight it with a toolkit, all variations on “prefer simpler models”:

• Regularization — add a penalty for complexity to the loss function, so training is pushed to use smaller/fewer weights unless the data really justifies them. L1 regularization (Lasso) can drive weights to zero (feature selection); L2 (Ridge) shrinks them. This is the most general anti-overfitting tool, and it’s a direct, tunable dial on the bias-variance trade-off.
• More data — the most reliable cure. More examples make it harder to memorize and easier to find the real pattern; a model that overfits on 1,000 examples may generalize fine on 100,000.
• Simpler models / fewer features — reduce capacity to memorize (shallower trees, fewer parameters, feature selection from post 2).
• Early stopping — stop training when validation performance stops improving, before the model starts memorizing (standard in neural nets).
• Ensemble methods — averaging many models (post 4’s random forests) reduces variance, a built-in regularizer.
• For neural nets specifically, dropout (randomly ignoring units during training) and data augmentation (synthetically expanding the training set) are the field’s workhorses.

The unifying principle is Occam’s razor applied to models: among models that fit the data comparably, prefer the simpler one, because it’s more likely to have captured the real pattern rather than the noise, and therefore more likely to generalize. Every technique here is a way to enforce that preference.

The throughline: ML’s goal is generalization — performance on unseen data — and its central failure is overfitting (memorizing training noise: great train accuracy, poor new-data accuracy), with underfitting (too simple to capture the signal) as the opposite error. The bias-variance trade-off frames the balance (bias = too-simple error, variance = over-sensitive error; lower total error on unseen data is the target). The non-negotiable practice is the train/test split — training accuracy is meaningless, only held-out performance counts, and leakage/contamination is how projects (and LLM benchmarks) fool themselves — with cross-validation for robustness. Fight overfitting with regularization (penalize complexity — the tunable bias-variance dial), more data, simpler models, early stopping, ensembles, and (for nets) dropout/augmentation — all enforcing Occam’s razor: prefer the simpler model that fits comparably.

Key takeaways

• The goal of ML is generalization (performance on unseen data), never training performance; overfitting (memorizing training noise → great train accuracy, poor new-data accuracy) is the central failure, underfitting (too simple → poor on both) the opposite.
• The bias-variance trade-off: bias = error from too-simple assumptions (underfit), variance = error from over-sensitivity to the training sample (overfit); lowering one raises the other, so target the balance minimizing total unseen-data error.
• The non-negotiable practice: evaluate on held-out data (train/test, plus a validation set for tuning) — training accuracy is meaningless, and data leakage/contamination (test data seen in training) is the most common way projects fool themselves (and why LLM benchmarks obsess over contamination); use cross-validation when data is limited.
• Fight overfitting with regularization (penalize complexity — L1/Lasso zeros weights, L2/Ridge shrinks them; the tunable bias-variance dial), more data (most reliable), simpler models/fewer features, early stopping, ensembles, and for neural nets dropout and data augmentation.
• The unifying principle is Occam’s razor for models: among comparably-fitting models, prefer the simpler one — it’s likelier to have learned the real pattern, not the noise, and therefore to generalize.

Further reading

• Overfitting — memorizing noise instead of learning the pattern
• Bias–variance tradeoff — the balance that governs generalization