← All guides
Analysis
Overfitting: when models fool themselves
Given any historical dataset, you can construct a rule that 'explains' it perfectly. That rule will still be worthless, because it was built by memorizing noise. Overfitting is the reason backtests must be out-of-sample.
Fit is not prediction
A strategy with a dozen parameters can be tuned until it nails every past draw. The only honest test is one it never saw during tuning — the future. Every backtest on this site enforces that rule: a strategy picks for a draw using only the draws before it.
The symptom
A strategy that looks great on history and collapses out of sample is overfit. The prediction lab shows the expected result on a fair lottery: models that fit the past beautifully still match future draws exactly as often as random selection.