What is Overfitting?
Overfitting is tuning a trading rule so closely to past data that it captures that period's noise instead of a repeatable pattern, so it fails on new data.
How it works
It happens when many versions of a rule are tried and the best-looking one is kept. With enough variations, some will look good by luck alone. Guards include keeping a holdout period the rule never touches, correcting the p-values for the number of rules tried (as the Benjamini-Hochberg method does), and checking that the result survives small changes to the settings.
What it can't tell you
Corrections reduce the problem and do not remove it. A rule that passes every check can still be a false positive, and one that fails can still be a real but small effect.
How Tickfloor uses it
Tickfloor corrects every strategy result for the number of strategies tested, and publishes how many survive.
Does it work as a strategy?
This is a concept more than a trading rule, so there is no single strategy to score. It matters for reading the results below.
Tickfloor's research desk has backtested 517 strategies, most against an equal-weight benchmark of the same assets rebalanced daily that pays no costs (the strategies pay theirs). After correcting for the 598 tests run (Benjamini-Hochberg), 0 passed. That does not prove no strategy works. These are historical diagnostics, not validation under Testing Standard v2: the backtest harness predates that standard and has not been re-run to meet it.
General information only. It doesn't consider your objectives, finances or needs. Tickfloor holds no financial services licence and never places trades.
Common questions
What is Overfitting?
Overfitting is tuning a trading rule so closely to past data that it captures that period's noise instead of a repeatable pattern, so it fails on new data.
How does Overfitting work?
It happens when many versions of a rule are tried and the best-looking one is kept. With enough variations, some will look good by luck alone. Guards include keeping a holdout period the rule never touches, correcting the p-values for the number of rules tried (as the Benjamini-Hochberg method does), and checking that the result survives small changes to the settings.
What are the limits of Overfitting?
Corrections reduce the problem and do not remove it. A rule that passes every check can still be a false positive, and one that fails can still be a real but small effect.
How does Tickfloor use Overfitting?
Tickfloor corrects every strategy result for the number of strategies tested, and publishes how many survive.