Performance Methodology
How we test what we build.
How results are defined, measured and disclosed — published first, so every future result can be checked against it.
Version 1.0 ·
Evidence classes.
A system climbs them in order: idea, causal check, backtest, out-of-sample or walk-forward, robustness, execution-engine validation, dry run, live.
- Backtest evidence — Historical simulation only. Not forward evidence.
- A historical simulation of the complete system, including costs. It shows what the rules would have done; it is not forward evidence.
- Out-of-sample evidence — Held-out historical evidence produced under the declared process.
- Historical data that was held out while the system was designed and tested only once under the declared process. Re-using it to tune the system turns it back into a backtest.
- Dry run evidence — Forward execution without real capital.
- The system running forward in time on the execution engine against live market data, without real capital. Fills are simulated by the engine.
- Live evidence — Real capital, real orders, real fills.
- Real capital, real orders and real fills on a real venue. Only this class is a track record.
The standard in practice.
Inside Investment Intelligence, research strategies show backtest and forward tracking apart — and stay labelled research.
Definitions
Costs, capital, universe, data, benchmarks, risk and versions — stated before any result is published.
- Every result is net of trading fees at the stated rate for the venue and account type used.
- A slippage assumption is stated with each simulated result. Dry-run fills come from the execution engine; live results use actual fills.
- For perpetual futures, funding payments are included where the result spans funding times.
- Each result states the starting capital, the leverage allowed and the position-sizing rule.
- Returns are measured on the stated capital, not on the margin actually used, so higher leverage cannot hide behind a small denominator.
- The universe is defined as it was at each point in the test, including instruments that were later delisted.
- Results built only on instruments that still exist today are labelled as such, because that choice inflates them.
- A decision uses only information available at its decision time. Signals are evaluated on closed bars; orders act after the bar that produced them.
- Data that is revised later — fundamentals, index membership, corrected prices — is used as first known, where that history exists. Where it does not, the result says so.
- A benchmark is chosen for the market the system trades — for example, holding the main asset of that market — and stated with the result.
- A benchmark is context, not a target the result was fitted to.
- Maximum drawdown is the largest fall in equity from a previous peak to a later trough, as a percentage of that peak, measured on the equity curve including open positions.
- Drawdown duration is the time from that peak until equity recovers to it, or until the end of the period if it has not.
- The return for the actual period is always shown.
- An annualised figure is added only when the period is at least one year; shorter periods are not extrapolated.
- Return is never shown without maximum drawdown. Volatility, worst trade, worst period and expected shortfall are added where the sample supports them.
- Profit factor, expectancy, win rate and payoff ratio are shown together; none is meaningful alone.
- When trades cluster around a few events, the effective number of independent observations is stated, and the share of the result from the best trades is shown.
- Gaps, stale periods and suspect data are detected and reported with the result, not silently filled.
- Forced exits caused by data or venue problems are counted, not removed.
- Simulated fills cannot capture every effect of real order books, especially for large orders or thin markets.
- Past results — of any evidence class — do not guarantee future results. Market regimes change, and a system can stop working.
- Out-of-sample evidence is only as good as the discipline of not re-using it.
- Each published result shows when it was last updated, the strategy version and the methodology version it follows.
- When the methodology changes, its version changes, and results are labelled with the version they were produced under.
Interested in the private beta?
Investment Intelligence is in an invitation-only beta for a small number of evaluators.
