Backtest runs and comparison
Two runs side by side: what changed, what it moved, and what carries it.
/backtests is the record of everything you have verified: manual sessions on one side, automated runs on the other. This chapter is about judging one run against another.
Two lists#
Two tabs at the top:
- Manual — sessions built by hand in the chart, with headline statistics above them
- Automated — every run the assistant produced
The active tab lives in ?view=, so a reload or a shared link lands on the same list. Search is cleared when you switch, because the two lists search different fields and carrying a query across would silently filter the other side.
Every run is kept#
Change a parameter and run again; the previous attempt does not disappear. The Automated list keeps every run, which is what makes “did this change make it better?” answerable.
Opening a run shows its rules, metrics, trades and stability readings. Deleting one removes that record, not the strategy.
Comparing two runs#
Select two rows (Compare on the row, or the selection bar) and the footer reads 2 of 2 selected; press Compare. The comparison has three parts:
A comparison where exactly one assumption changed (e.g. the stop loss), so one CHANGED row is visible, with the metric differences and the fragility line in frame.
What they ran with#
Starting balance, commission, spread, leverage, financing, settlement currency and window. Rows that changed are highlighted and marked CHANGED — this section answers “what was actually different”.
What came out#
Two columns of metrics plus a difference: net P&L, return, win rate, profit factor, max drawdown, trades, and more. Positive differences read green, negative red.
How fragile each was#
Concentration and resampling for each run. The one with the higher number but a result carried by five trades is the more fragile of the two.
The footer states the engine version each run used, and says so explicitly when they differ — results either side of an engine change are not a continuation of the same experiment.
From a run back to the chart#
Open in chart on a result card materializes the run as a manual session: same rules, same trades, but you can step through it and see where each entry landed. It is the last check that the engine's reading of the rules matches yours.
The materialized session appears in the Manual list with an Automated badge.
When to distrust a result#
A few questions worth asking every time:
- Fewer than 20 trades: the distribution readings are hidden because a small sample's shape is noise
- The best five trades carry most of the net result: the strategy depends on a few extreme events
- Resamples end down often: the result is sensitive to where the window starts
- A small parameter change flips the result: you are fitting this history, not finding an edge
None of these is a verdict. They are reasons not to treat a result as a stable return.
Next#
Where symbols come from, and how to bring your own data: “Symbols and custom data”.