Testing an idea honestly before you use it
A backtest runs a rule on old prices to see how it would have done. Learn to test one fairly, spot lucky results, and try it live in a small way you can undo.
Module 15, in the Risk, testing and funded accounts stage. 8 lessons.
Lessons
- Testing on prices the strategy never sawSplit time into training, checking and test windows without leaking future information.
- Why testing many ideas finds lucky onesSee why trying lots of versions throws up lucky winners, and how the Deflated Sharpe Ratio adjusts for it.
- Going live and checking it still worksPlan a small paper test you can undo, with health checks and honest limits.
- Benchmarks and the dumb controlChoose the right comparison for any strategy claim, including the cheapest rule that shares its mechanism.
- Costs inside a backtestAdd spread, slippage, fees and funding to a test and watch the result change sign.
- Split-half, concentration and regimes: is it a few lucky trades?Run two cheap robustness checks and spot a result that lives in one period or a handful of trades.
- Preregistration: write the test down before you lookWrite a oneāpage test plan a friend could run without you, before touching the data.
- Data that lies: frozen prices, missing values and unadjusted splitsCatch silent data errors before they reach a signal or a backtest.
Concepts covered
- Out-of-sample testing
- Look-ahead bias
- Walk-forward analysis
- Multiple testing
- Overfitting
- Sharpe ratio
- Backtest
- Benchmark
- Random-entry control
- Trading costs
- Bid-ask spread
- Slippage
Related modules
Keep going
General information only. It doesn't consider your objectives, finances or needs. Tickfloor holds no financial services licence and never places trades.