Trade Nothing, Prove Everything: The Signal-Only Probation Tiereric

This is the third post in a series. The first argued that indicators, strategies, and alpha are three different layers with three different jobs. The second turned the taxonomy into architecture, and made a claim I promised to back up: that the probation tier — a strategy running fully live while trading nothing — is the single highest-value feature in our codebase.

This post is the receipts.

Fully live, trading nothing

Mechanically, signal-only deployment is almost embarrassingly small: one boolean on a deployed strategy. With the flag set, the policy runs the complete live path — real-time bars as they arrive, real multi-timeframe consultation, real entry decisions, real exit doctrine, real notifications pushed to my phone — and at the last possible moment, places no order.

The phrase to notice is complete live path. This is not paper trading in a separate simulator, and the distinction is the entire point. A separate simulator is a second implementation of your system, and a second implementation is a second set of bugs. The first post in this series called divergence between "the thing you validated" and "the thing you trade" a fraud you commit against yourself; a standalone paper-trading harness reintroduces that fraud through the back door, with better intentions. In probation, the code that decides is the code that will trade. The only difference between probation and production is one flag — which means everything probation proves, it proves about the real system.

A backtest answers would this have worked? Probation answers a different and humbler question: does this actually work — this code, this data feed, this clock, this market, right now? It is remarkable how often the answer to the first question is yes and the second is no. Every one of the following stories is a case where the backtest had no opinion and probation did.

What probation catches that backtests cannot

Stale inputs. The previous post mentioned a monthly-timeframe series that had silently stopped updating for almost a year. Here is the part I left out: the way you find that fault, in practice, is probation. Nothing crashed and no monitor fired — the tell was silence. A policy that consults the monthly level and should fire occasionally had never fired at all, and signal-only deployment is what converts "it has never fired" from an unremarkable fact into a question that must be answered. When we chased the silence, we found the frozen series: the stale bar happened to read "trending," a state that particular policy could never align with, so it had been structurally dead since the day it was deployed. A backtest over the same period, using clean historical data, showed the policy firing happily. Backtests run on the data you should have. Probation runs on the data you do have.

Code-path divergence. While validating a new policy recently, we discovered that every backtest on one of our production servers had been quietly dying partway through the run — an exit-callback signature mismatch between two builds of the same portfolio library, so the first simulated position to close crashed the simulation. The live trading path was untouched; only the validation path was broken, and only on that machine. Sit with the irony: the tool whose job is to certify strategies was the thing that was wrong, and it failed in a way that produced clean-looking, quietly truncated results rather than an error. If your validation and your execution are different code — different builds count — then what you are validating is a fiction that resembles your system. The only durable defence is the one probation embodies: make the live path itself the thing that gets exercised, and treat any disagreement between the backtest and a live-path replay of the same history as a five-alarm bug, not a rounding curiosity.

Data-feed reality. Two smaller specimens from the same collection. One of our index CFD feeds only quotes the middle of the trading session — the official open and close, where the most dramatic bars print, simply do not exist on that feed. And on one server, a confirmation timeframe's history did not reach back far enough, so entry signals fired and then expired unconfirmed — not because the market rejected them, but because the bars needed to judge them were absent. Neither fact lives in any backtest dataset. Both change what a strategy actually experiences. The market data you backtest and the feed you trade are not the same object, and only one of them pays you.

The refusal machinery, proven live. The freshest story, and my favourite kind: days after we deployed a new trend-drift policy signal-only on the indices, it fired a live long trigger — and then its higher-timeframe confirmation rolled over and killed the signal before it graduated into a would-be trade. That is not a failure report. A signal that fires and then gets refused for the right reason is probation gold: it exercises the entry logic, the confirmation logic, and the refusal path, end to end, on live data, and it costs nothing. You cannot buy that certainty from a backtest, because the backtest runs the same code with the same assumptions baked into both sides of the test.

The funnel is the report card

While a policy is on probation, we grade it on a funnel: every decision it makes falls through four stages, and each stage answers a different question.

The discipline is that every stage gets explained, not just counted. Zero signals is fine if the market offered no setups; it is a defect if it traces to a frozen series. A pile of expired signals is fine if the confirmations genuinely failed; it is a defect if the confirmation bars were simply missing. The funnel does not tell you whether the policy is good. It tells you whether the policy is real — and only a real policy deserves the question of goodness.

Graduation: the actual criteria

The bar a policy must clear before the flag comes off is unglamorous, and that is deliberate. Ours is:

• A sample floor, counted in signals, not months. A quiet policy that fires four times a quarter needs more wall-clock than an active one; eight live signals validate nothing — the sample-size theatre warning from the first post applies to probation too.
• Live/backtest parity. Replaying history through the live path must reproduce the backtest's episodes — same events, same windows. Where they disagree, one of them is lying, and probation exists to find out which.
• Every silence explained. Any gap where the policy did not fire must trace to "no qualifying setups," never to "an input was broken and nobody noticed."
• An A/B against the incumbent. A new policy does not merely have to work; it has to beat, or usefully complement, whatever currently owns its scope — measured on the same bars.
• Graduate narrow. This is the one people resist. When our trailing-stop policy earned real capital, it earned it on exactly one symbol, at one timeframe, in one direction — the single scope where a corrected-implementation A/B proved it — and it remains signal-only everywhere else to this day. The instinct after a good probation is to switch everything on. The correct move is to scale scope before you scale size: one market, one timeframe, one direction, then earn the next scope the same way the first was earned.

Notice what is not on the list: conviction, elegance, how good the equity curve looked in the backtest, or how much I personally like the idea. The first two posts were about building a court that can kill your favourite idea. Graduation criteria are that court's sentencing guidelines, written down before the trial.

The door swings both ways

The part that makes probation a tier rather than a gate: demotion uses the same doorway. When a live policy decays — and edges decay; that is what being an edge means — it does not get deleted. It goes back to signal-only, where it keeps running the full live path, keeps generating evidence about itself, and costs nothing while it does. Sometimes the decay is the market moving on, and the ledger gets a dated entry saying so. Sometimes the "decay" turns out to be a broken input, and the policy comes back. Either way the answer is data, not memory.

A strategy in our system is therefore never really "in" or "out." It occupies a scope: trading real capital where it has earned it, on probation everywhere else, demotable at any time, re-promotable only by the same criteria as before. The shelf of policies from the last post is really a shelf of probation records.

The price is patience

Signal-only deployment converts opinion into data at exactly zero financial risk. That sentence sounds like a free lunch, so let me name the real cost: time. A probation takes weeks where a backtest takes minutes, and the discipline it demands is the least fashionable one in trading — waiting, while a thing you believe in trades nothing, until the ledger agrees with you or does not.

Every mechanism in this series points the same direction: the system's job is to move belief out of the trader's head and into a place where it can be counted. The backtest counts the past. Probation counts the present. Capital is what you allocate to the intersection.

Next in this series: how edges decay — what a dying policy actually looks like in the ledger, how long we let it bleed before demotion, and why "it stopped working" is a measurement, not a mood.