Testing an idea honestly before you use it

A backtest runs a rule on old prices to see how it would have done. Learn to test one fairly, spot lucky results, and try it live in a small way you can undo.

Module 15 of 26, in the stage "Risk, testing and funded accounts". 8 lessons.

Lessons

1. Testing on prices the strategy never sawSplit time into training, checking and test windows without leaking future information. 18 min.
2. Why testing many ideas finds lucky onesSee why trying lots of versions throws up lucky winners, and how the Deflated Sharpe Ratio adjusts for it. 18 min.
3. Going live and checking it still worksPlan a small paper test you can undo, with health checks and honest limits. 18 min.
4. Benchmarks and the dumb controlChoose the right comparison for any strategy claim, including the cheapest rule that shares its mechanism. 18 min.
5. Costs inside a backtestAdd spread, slippage, fees and funding to a test and watch the result change sign. 18 min.
6. Split-half, concentration and regimes: is it a few lucky trades?Run two cheap robustness checks and spot a result that lives in one period or a handful of trades. 18 min.
7. Preregistration: write the test down before you lookWrite a one-page test plan a friend could run without you, before touching the data. 16 min.
8. Data that lies: frozen prices, missing values and unadjusted splitsCatch silent data errors before they reach a signal or a backtest. 16 min.

Concepts covered

• Out-of-sample testing
• Look-ahead bias
• Walk-forward analysis
• Multiple testing
• Overfitting
• Sharpe ratio
• Backtest
• Benchmark
• Random-entry control
• Trading costs
• Bid-ask spread
• Slippage

Keep going

• Previous module: Funded accounts: how prop firm challenges work
• Next module: Fees, borrowed money, and who holds your crypto
• All 26 modules and 157 lessons
• Open the first lesson in the course

General information only. It doesn't consider your objectives, finances or needs. Tickfloor holds no financial services licence and never places trades.