§ Methodology

The math behind the verdict.

Every metric on the dashboard, every test in the robustness panel, spelled out. The point is not to impress — it is to make the numbers falsifiable. If you can't explain the metric, you can't trust it.

§ 01

Robustness

A backtest is one sample from the strategy's distribution of possible outcomes. Robustness testing asks: if I sampled again, how different would the answer look? The four tests here approach that question from different angles.

§ 01.01

Monte Carlo simulation

The original trade sequence is one of many possible orderings. Monte Carlo shuffles the order (preserving the PnL of each trade) thousands of times and re-computes the equity curve for each shuffle. Read the distribution of outcomes — the worst-case drawdown at the 95th percentile is the relevant number, not the original drawdown.

Limit: Monte Carlo here destroys serial correlation. If the strategy's edge comes from streak-following behavior, MC understates it. Use the bootstrap with block size > 1 (§ 01.03) to preserve serial structure.

§ 01.02

Walk-Forward analysis

Split the data sequentially into in-sample (IS) and out-of-sample (OS) windows. This is a holdout-stability check — no parameters are re-optimised between windows (true walk-forward optimisation would re-fit the strategy on each IS window). Walk-Forward Efficiency compares OOS to IS performance:

WFE = (avg OS return-per-trade / avg IS return-per-trade) × 100

Per-trade returns are used so the larger IS window isn't credited with more return simply for holding more trades — comparing window totals would penalise the strategy for the train/test split rather than for any real edge decay. A WFE above 70% suggests the edge survives unseen data; below 50%, suspect overfitting. Five non-overlapping 70/30 windows by default — adjustable in the dashboard.

§ 01.03

Bootstrap resampling

Distinct from Monte Carlo: bootstrap samples trades with replacement. A single trade can appear twice in one resample and not at all in another. The result is a confidence interval on every metric — not just a "worst case" number.

Block bootstrap. Setting block size > 1 samples contiguous runs of trades. This preserves serial-correlation structure that simple bootstrap destroys. If serial correlation (§ 04.02) is meaningful, use a block size of 5–20.

CI_95(metric) = [pct(metric_dist, 2.5%), pct(metric_dist, 97.5%)]
§ 01.04

Stress injection

What if execution decays? Stress injection applies worst-case modifications to the trade list in a fixed order:

  1. Skip best % — remove the top X% of winning trades (those "lucky" home runs you couldn't have anticipated)
  2. Add random losses — inject K losses sized at the strategy's average loss
  3. Extra loss % — multiply existing losses by (1 + X%) to model deeper drawdowns
  4. Slippage pips — subtract a per-trade execution cost

The order matters. Compare the original metrics to the stressed metrics; a strategy that loses its edge after 2 pips of slippage was probably never tradable in size.

§ 01.05

Regime decomposition

Bucket trades two ways: by calendar (year / quarter / month) and by rolling-volatility tercile (low / mid / high stdev of PnL over the trailing N trades). Compute return, win rate, and drawdown per bucket.

The "top contributors" table reveals where PnL is concentrated. If 60% of the strategy's PnL comes from a single quarter, the edge isn't durable — it's a regime artifact.

§ 02

Risk metrics

§ 02.01

Value at Risk (VaR 95%)

The maximum expected loss with 95% confidence. If VaR is $1,000 per trade, there is a 5% probability the next trade loses more than $1,000. Computed as the 5th percentile of the per-trade PnL distribution.

§ 02.02

Conditional VaR (Expected Shortfall)

When things go wrong (the worst 5% of trades), how bad on average? CVaR is the mean of the per-trade PnL distribution conditioned on being worse than VaR. Captures tail risk that VaR misses.

§ 02.03

Risk of Ruin

The probability the account loses all its capital given the strategy's edge. We express the per-trade expectancy as an edge in units of the average loss (the risk staked per trade), and the starting capital as a count of those units:

edge = expectancy / avg_loss;   units = capital / avg_loss
RoR = ((1 − edge) / (1 + edge)) ^ units,   bounded to [0, 1]

A positive edge makes the base < 1, so more capital (more units) drives ruin toward zero; a zero-or-negative edge makes ruin certain (1). This always lies between 0% and 100% — unlike the naïve ((1−W)/W) form, which exceeds 1 (an impossible probability) whenever the win rate is below 50%. Anything above 5% is a flashing warning; above 20% means stop trading.

§ 03

Position sizing

§ 03.01

Kelly Criterion

The optimal fraction of capital to risk per trade for long-run geometric growth.

K = W - (1 - W) / R

Where W is win probability and R is the win/loss ratio. Practical traders use half Kelly (K/2) to reduce volatility — full Kelly is mathematically optimal but emotionally untradable.

§ 03.02

Optimal F

Ralph Vince's Optimal F: the fraction of capital that maximises geometric growth over the actual distribution of past trades — not just the average win rate and ratio. We search f ∈ (0, 1] for the value that maximises the Terminal Wealth Relative:

HPR_i = 1 + f × (pnl_i / |worst loss|);   TWR = ∏ HPR_i
Optimal F = argmax_f  TWR
§ 04

Statistical distribution

§ 04.01

Z-Score (runs test)

Wald–Wolfowitz runs test on the win/loss sequence — a run is a streak of same-sign results. Fewer runs than chance (negative Z) means results cluster: wins follow wins, losses follow losses (trend persistence). More runs than chance (positive Z) means they alternate (mean reversion).

Z = (runs − E[runs]) / √var
E[runs] = 2·n₁·n₂ / n + 1
var = 2·n₁·n₂·(2·n₁·n₂ − n) / (n²·(n − 1))

where n₁ = winning trades, n₂ = losing trades, n = n₁ + n₂.

§ 04.02

Serial correlation

Pearson correlation of trade i's PnL with trade i+1's PnL. Near zero means trades are independent; far from zero means past trades predict the next one.

§ 04.03

Skewness

Asymmetry of the PnL distribution. Negative skew (frequent small wins, occasional massive losses) is the "picking up pennies in front of a steamroller" profile — popular and dangerous.

§ 04.04

Kurtosis (excess)

Tailedness. High excess kurtosis means more outlier events than a normal distribution predicts. Strategies with high kurtosis blow up in ways VaR doesn't anticipate.