Sample size
Last updated
Sample size is the number of independent trades or observations behind a result. The fewer there are, the more a result can be luck.
How sample size is calculated
The uncertainty of an average shrinks with the square root of the number of observations. A win rate measured on 25 trades could easily be 15 points away from the true rate. Trades that overlap in time or move together carry less information than the raw count suggests, so seasonal rules with one signal a year have very small samples.
How it is read
A common rule of thumb is to want at least a few dozen independent trades before reading anything into an average. More is better.
Common mistakes
- Counting bars or days as trades.
- Quoting a percent from 10 trades.
- Forgetting that a result found by searching needs a bigger sample than one set in advance.
What Tickfloor tested
How we counted
Tickfloor's research desk has backtested 1,346 strategies, net of modelled trading costs. After correcting for the 1,778 tests run, 0 passed. That does not prove no strategy works. These are results from our older test setup, which was built before our current testing rules. They have not been redone under those rules.
Claim tests on Sample size
- Santa Claus Rally (claim test)
- The January Effect (claim test)
- Sell in May (claim test)
- Bitcoin Halving Cycle (claim test)
Lessons that cover it
Related concepts
General information only. It doesn't consider your objectives, finances or needs. Tickfloor holds no financial services licence and never places trades.