Tickfloor

Sample size

Last updated

Sample size is the number of independent trades or observations behind a result. The fewer there are, the more a result can be luck.

How sample size is calculated

The uncertainty of an average shrinks with the square root of the number of observations. A win rate measured on 25 trades could easily be 15 points away from the true rate. Trades that overlap in time or move together carry less information than the raw count suggests, so seasonal rules with one signal a year have very small samples.

How it is read

A common rule of thumb is to want at least a few dozen independent trades before reading anything into an average. More is better.

Common mistakes

  • Counting bars or days as trades.
  • Quoting a percent from 10 trades.
  • Forgetting that a result found by searching needs a bigger sample than one set in advance.

What Tickfloor tested

1,346strategies tested across all our research
0passed the luck check
How we counted

Tickfloor's research desk has backtested 1,346 strategies, net of modelled trading costs. After correcting for the 1,778 tests run, 0 passed. That does not prove no strategy works. These are results from our older test setup, which was built before our current testing rules. They have not been redone under those rules.

Claim tests on Sample size

Lessons that cover it

Related concepts

General information only. It doesn't consider your objectives, finances or needs. Tickfloor holds no financial services licence and never places trades.

Keep reading