
{/* banner prompt: Matte clay-render 3D illustration, 16:9, soft studio shadows, simple cream background. A small warm-lit theatre stage where a single sage-green clay robot performs a full juggling routine with tiny candlestick-chart cards, spotlit and mid-performance, facing rows of completely empty terracotta theatre seats. In the front row sits one solitary cream clay judge figure holding a clipboard and pencil, attentively taking notes. Heavy terracotta stage curtains at the sides, a soft single spotlight beam, warm muted palette of sage green, cream and terracotta, clean geometry, no text, gentle depth of field. */}

This is the third post in a series. The [first](/blog/2026/09/13-indicators-measure-strategies-decide-alpha-is-what-survives/) argued that indicators, strategies, and alpha are three different layers with three different jobs. The [second](/blog/2026/09/14-a-trading-system-in-three-layers-measure-decide-account/) turned the taxonomy into architecture, and made a claim I promised to back up: that the probation tier — a strategy running fully live while trading nothing — is the single highest-value feature in our codebase.

This post is the receipts.

## Fully live, trading nothing

Mechanically, signal-only deployment is almost embarrassingly small: one boolean on a deployed strategy. With the flag set, the policy runs the complete live path — real-time bars as they arrive, real multi-timeframe consultation, real entry decisions, real exit doctrine, real notifications pushed to my phone — and at the last possible moment, places no order.

The phrase to notice is *complete live path*. This is not paper trading in a separate simulator, and the distinction is the entire point. A separate simulator is a second implementation of your system, and a second implementation is a second set of bugs. The first post in this series called divergence between "the thing you validated" and "the thing you trade" a fraud you commit against yourself; a standalone paper-trading harness reintroduces that fraud through the back door, with better intentions. In probation, the code that decides is the code that will trade. The only difference between probation and production is one flag — which means everything probation proves, it proves about the real system.

A backtest answers *would this have worked?* Probation answers a different and humbler question: *does this actually work — this code, this data feed, this clock, this market, right now?* It is remarkable how often the answer to the first question is yes and the second is no. Every one of the following stories is a case where the backtest had no opinion and probation did.

## What probation catches that backtests cannot

**Stale inputs.** The previous post mentioned a monthly-timeframe series that had silently stopped updating for almost a year. Here is the part I left out: the way you *find* that fault, in practice, is probation. Nothing crashed and no monitor fired — the tell was **silence**. A policy that consults the monthly level and should fire occasionally had never fired at all, and signal-only deployment is what converts "it has never fired" from an unremarkable fact into a question that must be answered. When we chased the silence, we found the frozen series: the stale bar happened to read "trending," a state that particular policy could never align with, so it had been structurally dead since the day it was deployed. A backtest over the same period, using clean historical data, showed the policy firing happily. Backtests run on the data you *should* have. Probation runs on the data you *do* have.

**Code-path divergence.** While validating a new policy recently, we discovered that every backtest on one of our production servers had been quietly dying partway through the run — an exit-callback signature mismatch between two builds of the same portfolio library, so the first simulated position to close crashed the simulation. The live trading path was untouched; only the validation path was broken, and only on that machine. Sit with the irony: the tool whose job is to certify strategies was the thing that was wrong, and it failed in a way that produced clean-looking, quietly truncated results rather than an error. If your validation and your execution are different code — different builds count — then what you are validating is a fiction that resembles your system. The only durable defence is the one probation embodies: make the live path itself the thing that gets exercised, and treat any disagreement between the backtest and a live-path replay of the same history as a five-alarm bug, not a rounding curiosity.

**Data-feed reality.** Two smaller specimens from the same collection. One of our index CFD feeds only quotes the middle of the trading session — the official open and close, where the most dramatic bars print, simply do not exist on that feed. And on one server, a confirmation timeframe's history did not reach back far enough, so entry signals fired and then *expired unconfirmed* — not because the market rejected them, but because the bars needed to judge them were absent. Neither fact lives in any backtest dataset. Both change what a strategy actually experiences. The market data you backtest and the feed you trade are not the same object, and only one of them pays you.

**The refusal machinery, proven live.** The freshest story, and my favourite kind: days after we deployed a new trend-drift policy signal-only on the indices, it fired a live long trigger — and then its higher-timeframe confirmation rolled over and killed the signal before it graduated into a would-be trade. That is not a failure report. A signal that fires and then gets refused *for the right reason* is probation gold: it exercises the entry logic, the confirmation logic, and the refusal path, end to end, on live data, and it costs nothing. You cannot buy that certainty from a backtest, because the backtest runs the same code with the same assumptions baked into both sides of the test.

## The funnel is the report card

While a policy is on probation, we grade it on a funnel: every decision it makes falls through four stages, and each stage answers a different question.

![The probation funnel: signals in, most refused for measured reasons, graduates out](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/09/trade-nothing-prove-everything-the-signal-only-probation-tier/funnel.png)

<Table data={
    {
        headers: ['Funnel stage', 'Question it answers', 'A bad number usually means'],
        rows: [
            ['Signals fired', 'Does the policy fire at all on live data?', 'A dead or stale input, an impossible condition, or a thesis whose setup never occurs'],
            ['Confirmed', "Do signals survive the policy's own checks?", 'Confirmation data missing or short, or checks tuned so tight nothing passes'],
            ['Expired / refused', 'Does the refusal machinery reject for market reasons?', 'Refusals tracing to broken inputs rather than to the market'],
            ['Hypothetical outcome', 'Would the confirmed signals have paid?', "The honest no - recorded in the ledger, and no capital paid for the lesson"],
        ]
    }
} />

The discipline is that every stage gets *explained*, not just counted. Zero signals is fine if the market offered no setups; it is a defect if it traces to a frozen series. A pile of expired signals is fine if the confirmations genuinely failed; it is a defect if the confirmation bars were simply missing. The funnel does not tell you whether the policy is good. It tells you whether the policy is *real* — and only a real policy deserves the question of goodness.

## Graduation: the actual criteria

The bar a policy must clear before the flag comes off is unglamorous, and that is deliberate. Ours is:

- **A sample floor, counted in signals, not months.** A quiet policy that fires four times a quarter needs more wall-clock than an active one; eight live signals validate nothing — the sample-size theatre warning from the first post applies to probation too.
- **Live/backtest parity.** Replaying history *through the live path* must reproduce the backtest's episodes — same events, same windows. Where they disagree, one of them is lying, and probation exists to find out which.
- **Every silence explained.** Any gap where the policy did not fire must trace to "no qualifying setups," never to "an input was broken and nobody noticed."
- **An A/B against the incumbent.** A new policy does not merely have to work; it has to beat, or usefully complement, whatever currently owns its scope — measured on the same bars.
- **Graduate narrow.** This is the one people resist. When our trailing-stop policy earned real capital, it earned it on exactly one symbol, at one timeframe, in one direction — the single scope where a corrected-implementation A/B proved it — and it remains signal-only everywhere else to this day. The instinct after a good probation is to switch everything on. The correct move is to scale *scope* before you scale *size*: one market, one timeframe, one direction, then earn the next scope the same way the first was earned.

Notice what is *not* on the list: conviction, elegance, how good the equity curve looked in the backtest, or how much I personally like the idea. The first two posts were about building a court that can kill your favourite idea. Graduation criteria are that court's sentencing guidelines, written down *before* the trial.

## The door swings both ways

The part that makes probation a tier rather than a gate: demotion uses the same doorway. When a live policy decays — and edges decay; that is what being an edge means — it does not get deleted. It goes back to signal-only, where it keeps running the full live path, keeps generating evidence about itself, and costs nothing while it does. Sometimes the decay is the market moving on, and the ledger gets a dated entry saying so. Sometimes the "decay" turns out to be a broken input, and the policy comes back. Either way the answer is data, not memory.

A strategy in our system is therefore never really "in" or "out." It occupies a scope: trading real capital where it has earned it, on probation everywhere else, demotable at any time, re-promotable only by the same criteria as before. The shelf of policies from the last post is really a shelf of probation records.

## The price is patience

Signal-only deployment converts opinion into data at exactly zero financial risk. That sentence sounds like a free lunch, so let me name the real cost: time. A probation takes weeks where a backtest takes minutes, and the discipline it demands is the least fashionable one in trading — waiting, while a thing you believe in trades nothing, until the ledger agrees with you or does not.

Every mechanism in this series points the same direction: the system's job is to move belief out of the trader's head and into a place where it can be counted. The backtest counts the past. Probation counts the present. Capital is what you allocate to the intersection.

*Next in this series: how edges decay — what a dying policy actually looks like in the ledger, how long we let it bleed before demotion, and why "it stopped working" is a measurement, not a mood.*


dfb4e0fa8acd0e045b28f48a6d5206fcf68ff5f36ae6d63b0d677c34a8c27a38

How a signal-only probation tier validates trading strategies on live data at zero risk, and the graduation criteria that let a policy trade real capital.

Backtests are rehearsals. Before a strategy touches real capital in our system, it runs fully live and trades nothing. What the signal-only probation tier catches that backtests structurally cannot, the funnel we grade it on, and the graduation criteria we actually use.

Trade Nothing, Prove Everything: The Signal-Only Probation Tier


{/* banner prompt: Matte clay-render 3D illustration, 16:9, soft studio shadows, simple cream background. A tidy miniature factory line in three connected stations, left to right: first a sage-green measuring station with calipers, dials and a small candlestick chart on an easel; second a terracotta sorting station where chunky clay arms route little rounded blocks through branching gates onto different tracks; third a cream accounting station with a small balance scale and two bins, one holding a few glowing golden blocks and one holding many dull grey blocks. A thin conveyor belt links all three stations. Clean geometry, warm muted palette of sage green, cream and terracotta, no text, no people, gentle depth of field. */}

In [the last post](/blog/2026/09/13-indicators-measure-strategies-decide-alpha-is-what-survives/) I argued that *indicator*, *strategy*, and *alpha* are not three words for the same thing but three different layers with three different jobs: indicators measure, strategies decide, alpha is what survives honest accounting. Several readers asked the natural follow-up: fine as a taxonomy — but what does a trading *system* built that way actually look like?

This post is the answer. The claim is stronger than "layers are a nice way to think." The claim is that the layering is an **architecture**, that each layer has a concrete set of components and a rule it must never break, and that the rate at which your whole operation finds real edges depends less on the brilliance of any one idea than on how cleanly the layers are separated. Edges are not found; they are *farmed* — and the three layers are the farm.

![The three layers of a trading system: measurement, decision, accounting](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/09/a-trading-system-in-three-layers-measure-decide-account/layers.png)

## Layer 1: the measurement layer

**What it consists of:** the data pipeline (bars at every timeframe you consult, gap-checked and freshness-checked), and the indicator library — every instrument computed on every bar of every series, all the time, whether or not anything downstream is listening. Moving averages, bands, oscillators, trailing stops, plus whatever house instruments you have built. In our system this layer runs from the 15-minute series up through hourly, daily, weekly and monthly, for every symbol, continuously.

**Its function:** report events. Never judge them.

**The rule it must never break:** an indicator is not allowed to know whether its event worked out. The moment you let outcome-awareness leak into an instrument — tuning it to "avoid false signals," suppressing a crossover because the last three lost — you have destroyed the layer boundary, and everything downstream is now measuring a mixture of the market and your opinions. The false signals are not a defect. They are the raw material the next layer exists to judge, and the layer after that exists to count.

Because the components are boring, the failure modes at this layer are boring too — and that is exactly what makes them dangerous. Two dominate:

- **Silently wrong.** An indicator implementation that drifts from its canonical definition still draws a plausible line on the chart. The last post covered this one: verify each instrument against the reference once, like calibrating a thermometer, then leave it alone.
- **Silently stale.** This is the system-level version, and it bites harder. We recently discovered that a monthly-timeframe series on one of our servers had quietly stopped updating almost a year earlier. Nothing crashed. Nothing warned. Every policy that consulted the monthly level was voting with a ten-month-old opinion — and one class of policies that required monthly agreement had been silently dead the whole time, because the frozen bar happened to read "trending," a state those policies could never align with. A measurement layer needs freshness monitoring the way a lab needs calibration stickers: every series carries an implicit "last verified current" date, and a policy consulting a stale series should be told, not humoured.

**Where a trader should focus here:** correctness and coverage, *once each*, plus a standing freshness check. This layer rewards diligence, not creativity. If you find yourself spending your evenings retuning indicator parameters, you are farming the wrong field.

## Layer 2: the decision layer

**What it consists of:** a portfolio of small policies — strategies — each one a single hypothesis about how to map measurements to actions. Each policy owns its entry conditions, its exit doctrine, its stop placement, and its right of refusal. Critically, each policy is *small*: one idea, one file, one name in the ledger.

**Its function:** judgment. This is the only layer where an opinion about the market is allowed to exist.

**The rule it must never break:** a policy consumes measurements; it never re-measures, and it never grades its own homework. Timeframes have a contract — the higher timeframes decide direction and regime, the lower timeframes time the entry — and a policy that lets a 15-minute signal overrule a daily trend has broken it.

The payoff of keeping policies small and separate is that the *same instruments* can serve contradictory hypotheses simultaneously, and the system does not have to pick a winner in advance. A worked example from our own book, using nothing more exotic than Bollinger Bands and Parabolic SAR:

Our workhorse band policy trades the cycle a band system sees: expansion, trend, exhaustion, reversion. It is structurally good at that shape — and structurally blind to one specific, expensive shape: the failed breakdown that V-reverses and melts up. Price breaks the lower band, looks like a crash, snaps back through the middle band within a few bars, and then trends for months without ever offering the pullback the policy waits for. We measured thousands of these events, and the numbers were blunt: at the moment the band policy would first consider the recovered trend tradeable, price sat a couple of ATRs beyond anything it would chase, and a quarter of the resulting runs never revisited the middle band at all. No parameter setting fixes that. It is not a tuning problem; it is the wrong hypothesis for that shape.

The instructive part is what the fix was *not*. We did not touch the bands, and we did not bolt a "melt-up exception" onto the band policy — that would have been two hypotheses wearing one name, unjudgeable at the accounting layer. Instead: a different instrument already in the measurement layer (Parabolic SAR, which flips on the recovery bar essentially every time, precisely *because* it judges nothing) and a new, separate policy that consumes the same measurements at higher timeframes and rides the calm drift the band cycle reads as "nothing happening." Two policies, one instrument bench, each individually accountable.

And the tempting shortcut — skip the policy, just buy every failed breakdown, since the pattern looks so obviously bullish on a chart? We measured that too, across nine hundred-plus events: 24% win rate, zero expectancy under every conditioning we tried. The pattern is real; the naive trade on it is worthless. Which brings us to the layer that told us so.

**Where a trader should focus here:** this is where creative effort belongs — but creativity expressed as *separable hypotheses*, not as features accreted onto one ever-growing strategy. The question "what would a policy for this market shape look like?" is productive. The question "how do I make my main strategy also handle this?" usually is not.

## Layer 3: the accounting layer

**What it consists of:** four components, in escalating order of realism.

- **The backtest through the live code path.** Not a separate vectorized approximation of your strategy — the actual policy code, fed historical bars the way live bars arrive. Any divergence between "backtest logic" and "live logic" is a fraud you are committing against yourself, and it will be discovered at the worst possible price.
- **A probation tier: signal-only deployment.** Every new policy in our system carries a flag that lets it run fully live — consuming real-time measurements, forming real decisions, logging real signals — while trading nothing. It is the single highest-value feature in the codebase. Backtests answer "would it have worked?"; probation answers "does it actually fire, on live data, when the code, the data feed, and the market are all being themselves?" The new drift policy above went out signal-only; it will earn capital or it will not, and either way the answer will be data.
- **Ground-truth labels.** For recurring patterns, a store of hand-labelled or rule-labelled outcomes ("was this breakout real?") that candidate policies can be scored against without waiting months of wall-clock time.
- **The negative-results ledger.** When an idea fails measurement, the result gets recorded where the code lives. The cheapest experiment is the one you do not re-run, and the most expensive opinion is the one you re-believe six months later.

**Its function:** decide what survives — and, just as importantly, notice what has stopped surviving. Edges decay. A layer that can only admit policies and never expel them is a museum, not an accounting department.

**The rule it must never break:** it must be *able to kill your favourite idea*, and it must apply the same test to every candidate. The moment a beloved policy gets a friendlier backtest window or a waived probation period, the layer stops producing information.

**Where a trader should focus here:** skepticism. Not the performative kind — the procedural kind, where the test is fixed before the idea is loved.

## Why the separation is what finds the edges

Here is the part that took me longest to internalise. The layering is usually sold as hygiene — nice, clean engineering. But the real payoff is economic: **separation collapses the cost of running experiments, and edge-finding is an experiment-count game.**

- **Fixed layer 1 + uniform layer 3 = cheap layer 2.** When measurement is shared and accounting is standardised, a new hypothesis costs one small policy file — not a new system. This is precisely the WorldQuant lesson from the last post, scaled to a retail codebase: they froze the policy contract so that thousands of alpha expressions could be measured on an assembly line. You freeze the instrument bench and the courtroom so that dozens of policies can be.
- **Failures become attributable.** When a layered system loses, you can say *which layer* failed: a stale series (measurement), a broken thesis (policy), or a correct decision that lost anyway (variance — the only acceptable kind of loss). An unlayered system offers only "the strategy lost," which is unactionable, so the trader responds the only way the architecture allows: retuning indicator parameters. That is the indicator graveyard, explained structurally.
- **Probation converts opinion into data at zero risk.** A signal-only tier means the gap between "I believe this" and "the ledger says this" is a few weeks of patience instead of a few percent of the account.
- **The edge becomes a portfolio property.** No single policy has to be brilliant. A shelf of small, separately-accountable policies, each with a modest measured edge and each individually killable, is more robust than one grand unified strategy — because when the market changes shape, you retire a policy, not a system.

## Where the effort goes

<Table data={
    {
        headers: ['Layer', 'Consists of', 'Function', 'Your effort'],
        rows: [
            ['1. Measurement', 'Data pipeline, indicator library, freshness checks', 'Report every event, judge nothing', 'Diligence: verify once, monitor staleness forever'],
            ['2. Decision', 'A portfolio of small, separable policies with their own exits and refusal rights', 'Judgment: map measurements to actions', 'Creativity: one hypothesis per policy'],
            ['3. Accounting', 'Live-path backtests, signal-only probation, ground-truth labels, negative-results ledger', 'Decide what survives and what has stopped surviving', 'Skepticism: fix the test before you love the idea'],
        ]
    }
} />

Most traders allocate effort in almost exactly the wrong proportions: the vast majority at layer 1 (cycling indicators and settings), a little at layer 2 (rules copied from the indicator's folklore), and none at layer 3. A working allocation is closer to the inverse — measurement verified once and then trusted, decisions where the imagination goes, accounting where most of the discipline and a surprising amount of the time goes.

The edge, when it comes, will not look like a secret indicator. It will look like a small, slightly boring policy — sitting on instruments everyone has, judged by a court you built yourself — that keeps surviving the count while its siblings get retired around it. The system that can notice that, cheaply and repeatedly, *is* the edge.

*Next in this series: what the probation tier looks like in practice — how a signal-only policy earns its way to real capital, and the graduation criteria we actually use.*


26fe9688db1d22f9378865fd7386cca8e8aa2638befa3d9b34996ce6268cb4f2

How to architect an algo trading system in three layers - measurement, decision, accounting - and why the separation is what makes edges findable.

Our last post named the three layers: indicators measure, strategies decide, alpha is what survives. This one is about building an algo trading system around them - what each layer contains, what it must never do, and why the separation itself is what makes edges findable.

A Trading System in Three Layers: Measure, Decide, Account


{/* banner prompt: Matte clay-render 3D illustration, 16:9, soft studio shadows, simple cream background. Three tidy workbench stations left to right: first a sage-green measuring instrument station with calipers, a thermometer and a small candlestick chart being measured; second a terracotta decision station with a branching flowchart of chunky clay arrows choosing between paths; third a minimal scale weighing one small glowing golden nugget against a heap of dull grey pebbles. Clean geometry, warm muted palette of sage green, cream and terracotta, no text, no people, gentle depth of field. */}

Ask ten traders what the difference is between an indicator and a strategy and you'll get ten variations of "the strategy is the indicator plus some rules." Ask where the alpha is and most will point back at the indicator — a better MACD setting, a smarter RSI threshold, the right moving average pair.

After years of building and, more importantly, *measuring* trading systems, I've come to a different model. The three words name three different layers, with three different jobs, and three different failure modes. Most losing systems aren't built from bad indicators. They're built from a confusion about which layer is supposed to be doing which job.

## An indicator is a measurement instrument

RSI doesn't have an opinion about your trade. It's a ratio of recent up-moves to down-moves, compressed into 0–100. MACD is the distance between two moving averages, and the momentum of that distance. Parabolic SAR is a trailing curve that accelerates under a trend. Bollinger Bands are a rolling mean with a volatility envelope.

That's all they are: **measurements**. A thermometer, not a doctor.

This framing sounds obvious, but it has two consequences people skip.

**First: a measurement instrument must be *correct*, and you should verify it the way you'd verify a thermometer — against a reference.** Indicators have canonical definitions, and real implementations drift from them more often than anyone admits. Wilder's Parabolic SAR is a good example, because it's deceptively fiddly: the update rule clamps the stop against the *prior two* bars' extremes, freezes the acceleration on the reversal bar, and seeds the flip at the old extreme point. Miss one of those details and you still get a plausible-looking dotted line on the chart — it hugs trends, it flips at pullbacks — but it disagrees with the canonical value on a large fraction of bars and, crucially, it *flips at different times*. Every strategy sitting on top inherits that error silently. We once traced a chronically underperforming trend system to exactly this: not a bad idea, a mis-measuring instrument. The fix wasn't a new strategy; it was making the thermometer read the true temperature.

The test is cheap: implement the indicator twice — once your way, once as a line-for-line port of the reference definition — run both over a few thousand bars, and demand zero disagreement. If you can't be bothered, at least diff your values against a charting platform's. A strategy built on an unverified indicator is a building inspection performed with a broken level.

**Second: an indicator fires on every event, including the false ones — by design.** RSI crosses 70 in every strong trend, repeatedly, all the way up. MACD crosses its signal line dozens of times in a ranging week. A Bollinger breakout bar looks identical whether it's the start of a 5% move or a one-bar fakeout. The instrument's job is to report the event, not to judge it. If you find yourself trying to tune an indicator until it "stops giving false signals," you're asking the thermometer to prescribe the medicine. Judgment is the next layer's job.

## A strategy is a policy

A strategy consumes measurements and produces decisions — and the decisions are the whole game. The same MACD crossover event can anchor a dozen different strategies depending on the policy wrapped around it:

- **Context**: do we act on a bullish crossover in any regime, or only when a higher timeframe agrees? A crossover on the hourly *against* a falling daily trend is a different trade from the same crossover *with* it. In our measurements, this single conditioning decision routinely swings a signal population from net-negative to net-positive — with the indicator untouched.
- **Timing versus deciding**: fast timeframes are excellent at *timing* an entry and terrible at *deciding* its direction. A policy that lets a 15-minute signal pick the direction while the 4-hour trend disagrees is letting the second hand of the watch overrule the calendar.
- **Exits**: two strategies with identical entries and different exits are different strategies — usually *more* different than two strategies with different entries and the same exits. An exit policy that snaps shut on every one-bar wobble will bleed a trend-following entry dry; an exit that only responds to its own thesis breaking will hold through noise and give back tails. Neither is "right"; each belongs to a different kind of trade, and mixing them up is one of the most expensive silent errors in system design.
- **Proportion**: where is the stop, relative to the structure that justified the entry? If a breakout bar is so extended that the logical stop sits several ATRs away, the signal may be "valid" and the trade still unacceptable. A policy has to be allowed to refuse a true signal.

Notice that none of these decisions live inside MACD. You cannot fix a policy problem by re-tuning a measurement — and almost every "indicator graveyard" thread you've read is a catalogue of policy problems misdiagnosed as indicator problems.

## Alpha is what survives honest accounting

So where's the alpha? Not in the indicator — everyone has RSI. Not in the strategy skeleton — crossover-plus-filter is public knowledge. Alpha lives in the *mapping* between what you measure and what you do, and it only counts if it survives honest measurement.

The clearest public evidence for that claim comes, ironically, from the place that industrialised the word: WorldQuant's famous *101 Formulaic Alphas* paper, which published a hundred and one real production alpha expressions. Look at two of the simplest:

```txt
Alpha#101:  (close - open) / ((high - low) + 0.001)

Alpha#6:    -1 * correlation(open, volume, 10)
```

Alpha#101 is just "where in its daily range did the stock close" — a normalized candle-body. Alpha#6 is the ten-day correlation between opening price and volume, negated. Squint and these are *indicators* — small, honest measurements, no more mystical than RSI. Nobody at a chart-trading forum would call them edges.

What makes them alphas is everything wrapped around the formula, which the paper's framework standardises so thoroughly it barely mentions it: each expression is computed **cross-sectionally** across thousands of stocks, converted to a rank, traded as a dollar-neutral portfolio — long the top of the rank, short the bottom — and rebalanced on a fixed schedule with costs accounted. The formula is layer one. The rank-long-short-rebalance contract is layer two, *fixed and identical for every alpha*. And because layer two never varies, layer three — the honest accounting — becomes an assembly line: thousands of candidate expressions can be measured the same way, kept if they survive, discarded or watched as they decay when the crowd finds them. WorldQuant's real invention wasn't any formula in that paper. It was freezing the policy layer so that measurement could be mass-produced.

That's the same lesson in institutional form: the expression is cheap; the mapping and the accounting are the asset. The same point shows up at retail scale with any charting indicator, and the best illustration I can offer is a study we ran on the classic Bollinger squeeze — the folklore that a breakout from unusually narrow bands is more likely to be "real." It's chart-reading orthodoxy, and every trader's eyes confirm it. We labelled about 12,000 historical band breakouts as real or failed (did the move travel a couple of ATRs before giving the break back?) and conditioned on how narrow the bands were at the break, measured properly as a z-score against recent history.

The result, in precision/recall terms:

<Table data={
    {
        headers: ['Question', 'Answer from ~12,000 breakouts'],
        rows: [
            ['Are narrow-band breakouts more likely to succeed?', 'No. Success rate was flat (~44%) at every narrowness threshold.'],
            ['Do narrow-band breakouts travel further in absolute terms?', 'No. Wide-band regimes are volatile regimes; they move more points.'],
            ['Do narrow-band breakouts travel further relative to their own bands?', 'Yes — strongly and monotonically. Deep squeezes travelled about 1.1 band-widths; bloated bands about 0.4.'],
        ]
    }
} />

Read that carefully, because it explains the folklore *and* the losing trades. On a chart, a squeeze breakout looks dramatic — the move dwarfs the coil that preceded it — and that relative drama is real and measurable. But a squeezed band-width is small in absolute points, so the probability of profit and the size of profit are no better. The eye is measuring in band-widths; the account is denominated in dollars.

Here's the layer lesson: the indicator (band-width) contained genuine information. Consumed as an **entry filter** — "only trade narrow-band breakouts" — it produced *zero* alpha; we swept every threshold and the precision never lifted. Consumed as **geometry** — sizing targets and stops in band-widths conditioned on the squeeze — the same measurement became useful. Same instrument, same numbers; the alpha appeared or vanished depending entirely on the mapping. And even then, the first mechanical implementation of the "useful" version lost money, because a target placed at the *median* expected travel amputates the right tail — and in trend trading, the right tail pays for everything. The measurement was right, the first policy consuming it was wrong, and only measurement told us which was which.

## The accounting has to be honest

That phrase — *what survives honest accounting* — is doing a lot of work, so let me name the two frauds we catch ourselves committing most often.

**Lookahead by timeframe.** When a strategy consults a higher timeframe, the daily bar your backtest sees at 2 pm *includes 2 pm's price action*. Live, the completed daily bar you can actually consult is yesterday's. We once validated a higher-timeframe filter retrodictively — it separated winners from losers almost perfectly — then re-tested it with only the bars a live system could have seen. It not only stopped working; it deleted the best trade in the sample and made everything worse. The filter's "predictive power" was the trade's own outcome leaking backwards through an in-progress bar. Anything that looks like a perfect discriminator deserves suspicion in proportion to its perfection.

**Sample-size theatre.** Eight trades cannot validate a rule; they can only inspire one. The discipline that saved us more than once: when a small sample suggests a rule, implement it, run it against the *full* history through the live-path simulator, and be genuinely prepared for the answer to be no. Roughly half the time, it is no — and every "no" is a strategy you didn't bleed out on live.

There's a quieter corollary worth stating: when your test says a mechanism doesn't work, *record the negative result where the code lives*. Six months later, someone — probably you — will have the same clever idea, and the cheapest experiment is the one you don't re-run.

## The working model

If you want the whole post in four lines:

<Table data={
    {
        headers: ['Layer', 'Job', 'Failure mode'],
        rows: [
            ['Indicator', 'Measure one thing correctly, on every bar, fakeouts included', 'Silently wrong implementation; tuned to "avoid false signals"'],
            ['Strategy', 'Policy: context, timing vs deciding, exits, proportion, refusal', 'Policy problems misdiagnosed as indicator problems'],
            ['Alpha', 'The edge left after honest, lookahead-free, adequately-sampled measurement', 'Retrodictive perfection; median targets amputating the tail; eight-trade proofs'],
        ]
    }
} />

The practical upshot is a change in where you spend effort. Most people spend it at layer one, cycling through indicators and settings. The measured reality is almost exactly inverted: verify layer one *once* against the canon and then leave it alone; spend your creative effort at layer two, where context and exits live; and spend your *skeptical* effort at layer three, where every idea — especially the beautiful ones — goes to be counted.

RSI is not your edge. What you do the third time RSI crosses 70 in a strong uptrend — and whether you ever actually measured it — might be.


dfd593d9cddbaead9ee0c92e275dfac41a1ba4078b870ba11b219c688525eafc

The working relationship between indicators, strategies, and alpha, illustrated with MACD, RSI, PSAR and Bollinger Bands.

Traders use indicator, strategy, and alpha almost interchangeably. They are three different layers — and most losing systems come from confusing them. A field guide with MACD, RSI, PSAR, Bollinger Bands and two of WorldQuant's 101 formulaic alphas.

Indicators Measure, Strategies Decide, Alpha Is What Survives


For a couple of years, my trading strategies told me things by email. Every entry, every close, every alert — a little SMTP message into my inbox. It worked, in the sense that the information arrived. But email is a filing cabinet, and a trading signal is an interrupt. I had been routing interrupts through a filing cabinet.

Last week I finally stopped. My strategy fleet now publishes its signals to [TYO Notify](https://trymq.tyo.com.au) — private push notifications, delivered the second the strategy decides — and I don't think I'll ever go back.

## The inbox was the wrong place for a signal

Here is what actually happened when a setup fired. The strategy sent an email. Gmail fetched it on its own schedule, ran it through filters, and eventually surfaced a bold unread count. I'd notice the little **(1)** a few minutes later, open the message, and read a paragraph of text to find out what symbol it was, which direction, and at what price. By then the candle that triggered it had often closed and the next one was well underway.

Two things were wrong, and neither is Gmail's fault:

- **Latency.** Email is a pull medium dressed up as a push one. There is no guarantee about *when* you see it, and "a few minutes" is an eternity when a signal has a shelf life measured in seconds.
- **No structure.** An email is a wall of text. To know whether to care, I had to open it and read. A signal isn't prose; it's a handful of fields — symbol, direction, level, price — and I wanted to read it the way I read a clock: at a glance, without opening anything.

## What I actually wanted

- **Instant.** Pushed the moment the strategy commits, not whenever a mail server gets around to it.
- **Glanceable.** Symbol, direction, timeframe, price — visible on the lock screen without unlocking, colour-coded so long and short are obvious before I've read a word.
- **Private.** My signals are my edge. I did not want them flowing through a third-party push service that logs every payload it forwards.
- **Mine.** I run the broker, so there is no per-notification pricing, no metered "premium" tier, and no account I can be deplatformed from.

<Table data={
    {
        headers: ['', 'Email', 'TYO Notify'],
        rows: [
            ['Latency', 'Whenever the mail server delivers', 'Sub-second push'],
            ['What you see', 'A paragraph you must open', 'A colour-coded card at a glance'],
            ['Who can read it', 'Your mail provider, in plaintext', 'Only a device holding the topic key'],
            ['Who runs it', "Someone else's servers", 'The broker you own']
        ]
    }
} />

## Enter TYO Notify

TYO Notify is ntfy-compatible push built into our own message broker, tyo-mq. Publishing a notification is a plain authenticated HTTP POST; subscribing is your phone. If you already speak [ntfy](https://ntfy.sh), you already know how to talk to it.

Two features made it the right home for signals:

**Private topics.** A topic starts public, but you can *claim* it with a device key — after that, only a holder of that key can publish to it or read from it. My signal topics are locked to my phone's key. Nobody else can post a fake signal, and nobody else can read mine.

**Structured cards.** When I publish, I put the signal's fields in the ntfy `Tags` header — `symbol`, `dir`, `level`, `price`, `sl`, `tp` — and the app renders them as a card instead of a line of text: a coloured accent bar (green for long, red for short), the timeframe chip, and the price, front and centre. That is the whole point. I can read a signal off the lock screen the way I read the time.

![A TYO Notify trading-signal notification in the Android shade: a green accent bar marking a long, the symbol MPL, an M15 timeframe chip, and the price with stop-loss and take-profit — the whole thing readable at a glance without opening it.](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/09/i-moved-my-trading-signals-from-email-to-tyo-notify/signal-card-shade-crop.png)

Publishing one looks like this:

```sh
curl \
  -H "Authorization: Bearer $TOKEN" \
  -H "Tags: symbol=ETHUSD,dir=long,level=h1,price=2450,sl=2400,tp=2600" \
  -d "ETHUSD long h1 @2450" \
  https://freemq.tyo.com.au/notify/tyostocks-signals
```

That's it. No SMTP, no template, no HTML. A POST with the fields in a header, and a card lands on my phone.

![The tyostocks-signals topic inside the TYO Notify app: a stream of structured signal cards — green accent bars for longs, red for shorts — each showing the symbol, an M15 timeframe chip, and the price with stop-loss and take-profit.](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/09/i-moved-my-trading-signals-from-email-to-tyo-notify/signals-in-app.png?height=680)

## The migration itself was almost boring

The best part: switching over was nearly nothing. My strategy processes already had a "notify me" seam; it used to build an email, now it builds a POST with a bearer token. I flipped it across the live fleet without a restart and without a gap — the last email signal and the first push signal were minutes apart, and I didn't miss a single alert in between.

## It followed me to the desktop

I spend the trading day on a Linux desktop, not staring at my phone. So over the weekend I built a small companion: a system-tray client. You sign in once with your TYO ID, it pulls your topics and your device key from your encrypted backup vault, and from then on it pops a native desktop toast — the same colour-coded card — the instant a signal fires. Sign-in to first toast was about a minute, and now the alerts find me wherever I'm working.

That the desktop client could exist at all is a nice property of the design: because the key that owns a private topic is *yours* and portable, any device you trust can read your topics. The phone isn't special.

## And it's on the Play Store

TYO Notify is rolling out on Google Play now — the same app I use for my own signals, not a stripped-down demo. Point your own scripts, cron jobs, alerting rules, or trading strategies at it, or self-host the broker; it's the same tyo-mq underneath either way.

## The takeaway

Email is a filing cabinet: durable, searchable, permanent, and completely indifferent to time. That is exactly what you want for a receipt and exactly what you don't want for a signal. If something in your life fires an alert that matters in the next ten seconds — a trade, a deploy, a form submission, a server on fire — stop mailing it to yourself. Push it.

{/* banner prompt: A 16:9 retro technical diagram poster in a 1970s print style, bold clean typography, illustrating a concept: a labeled 'SERVER' block on the left connects by a tangled, sluggish line to a bulky 'EMAIL' filing cabinet, while a second crisp straight arrow runs from the same server to a 'PHONE' showing a structured colour-coded notification card marked 'INSTANT SIGNAL'; limited flat palette of off-white paper, burnt orange, and deep teal; subtle screen-print halftone texture, generous negative space, no photorealism. */}


49931c2a79b44c42e1934cf8846bd6f96f8bb49fdd77f77d2839954482f4116c

Why I moved my automated trading-signal alerts from email to push notifications with TYO Notify: private topics, structured signal cards, and a message broker I own.

Trading signals don't belong in an inbox. I moved my strategy fleet's alerts from email to TYO Notify — private, structured push notifications I actually own — and here's why email was the wrong tool and what the switch looks like.

I Moved My Trading Signals From Email to TYO Notify


# Building Blocks of Algorithmic Trading: Collecting End-of-Day Data

{/* # Algorithmic Trading Series #1: Collecting End-of-Day (EOD) Data — The Foundation of a Trading System */}

Algorithmic trading begins not with signals or models, but with **data** — specifically, clean, consistent, and reliable **End-of-Day (EOD)** market data. Whether you're building a trend-following system or a multi-factor screener, your models are only as good as the data they run on. In this first article of our algorithmic trading series, we explore the essential sources of EOD data and demonstrate how to store and access it efficiently using **Redis**, a high-performance in-memory database.

## Why EOD Data?

EOD data — which includes open, high, low, close prices and volume (OHLCV) at the end of each trading day — is the **bedrock** for many algorithmic trading strategies, especially those that don’t require high-frequency execution. It's cleaner, more accessible, and less volatile than intraday data, making it ideal for:
- Backtesting strategies
- Screening assets
- Generating entry/exit signals
- Risk and portfolio management

## Where Can You Get EOD Data?

Here are several common and reliable ways to collect EOD data:

### 1. **Professional Market Data Providers**
If data cost is not an issue, consider subscribing to a professional provider:
- [**Quandl (now part of Nasdaq Data Link)**](https://data.nasdaq.com/publishers/QDL) – Offers premium and free datasets.
- [**EOD Historical Data**](https://eodhd.com/) – Affordable, easy-to-integrate REST API with global coverage.
- [**Tiingo**](https://www.tiingo.com/) – Clean, high-quality financial data with generous free API tiers.

These providers often include dividend and split adjustments, which are essential for accurate backtesting.

### 2. **Investment-Oriented Websites**
Web scraping from investment portals is possible, but not always recommended for production systems:
- **Investing.com** – Offers EOD data for equities, indices, forex, and commodities. 
- **Yahoo Finance** – Widely used; its unofficial API can be accessed using libraries like `yfinance`.
- [**Market Index**](https://www.marketindex.com.au/) – Australian oriented investment website with free ASX EOD data provided.

When using Investing.com, you might come across the Python library `investpy`. While it offers a convenient interface, it hasn’t worked reliably in my experience:

```shell
Error retrieving data for TSLA: ERR#0015: error 403, try again later.
```

Luckly "investing.com" still provides a function on their website to export the historial data for many instruments. So the worst case scenario is to download the data manualy from it. 

### 3. **Broker APIs**
Many brokers provide EOD data via REST or WebSocket APIs:
- **Interactive Brokers (IBKR)** – Offers EOD historical data via their TWS API.
- **Alpaca** – Provides free stock EOD data through their REST API.
- **IG**, **TD Ameritrade**, **OANDA**, **Polygon.io**, and others also offer options.

Always check for:
- Historical limits
- Frequency caps
- Adjustments for splits/dividends

The problem with broker APIs is that each broker has their own unique interface and data structure. Therefore, you may need to write custom code to handle the data retrieval and storage. 

### 4. **Trading Platforms**
Platforms like **MetaTrader 5 (MT5)** offer built-in historical data:
- MT5 can export EOD data directly from chart history.
- Supported by most brokers
- You can use Python with MetaTrader5 API (via `MetaTrader5` Python package) to automate downloads.

```python
from MetaTrader5 import *
initialize()
rates = copy_rates_range("EURUSD", TIMEFRAME_D1, start, end)
```
Using MetadataTrader5 is a great way to get EOD data, but be aware of the limitations:
- **On available to Windows users**: The python library for the MetaTrader5 platform is only available on Windows, so you need to run your code on a Windows machine or use a virtual machine.
- **Limited to Forex and CFDs**: Not all assets are available.
- **Data Quality**: Ensure the data is clean and adjusted for splits/dividends.

Personally, I prefer using the `MetaTrader5` API for retrieving EOD trading data. While its coverage of tradable assets is somewhat limited, it offers more than enough instruments to get started. Unlike some broker-specific APIs, MetaTrader 5 is widely adopted by traders, supported by many brokers, and backed by a large community.

A Python script will be provided to help you download EOD data directly from the MetaTrader5 platform. Setting up a demo account is also straightforward, making it easy to begin testing right away.


## Storing EOD Data with Redis

Once you've collected your data, you need fast, structured access for backtesting and model development. **Redis**, an in-memory key-value store, is an excellent choice for its speed and simplicity.

### Why Redis?
- Ultra-fast read/write
- Easy to structure time-series data
- Built-in expiration and memory management
- Great for small-to-medium-sized datasets or caching layer

### Redis Data Schema 

For EOD data, we can use a key, subkey, value structure:
- **Key**: `[PREFIX]:symbol` (e.g., `nasdaq:AAPL`)
- **Subkey**: `date` (e.g., `20250510`)
- **Value**: JSON string of OHLCV data
- **Example**: `nasdaq:AAPL:20250510` -> `{"open": 180.12, "high": 182.55, "low": 179.70, "close": 181.30, "volume": 25300000}`

Since instruments traded on different markets or exchanges can share the same symbol, it's important to define a **unique key** for each asset. To achieve this, we introduce a **prefix** for each symbol, separated by a colon.

The format is:  
`<EXCHANGE_PREFIX>:<SYMBOL>`

Examples of common prefixes:
- `nasdaq:AAPL` — Apple Inc. on the NASDAQ
- `nyse:IBM` — IBM on the New York Stock Exchange
- `asx:BHP` — BHP Group on the Australian Securities Exchange
- `crypto:BTCUSD` — Bitcoin as a cryptocurrency
- `forex:EURUSD` — Euro to US Dollar exchange rate
- `metals:XAUUSD` — Gold to US Dollar exchange rate
- `energies:XBRUSD` — Brent Crude Oil to US Dollar exchange rate
- `indices:SPX500` — S&P 500 Index

You can define the prefix based on the exchange or asset class you're working with. This convention ensures consistency and avoids symbol collisions across your dataset.


{/* You can use symbol-date as a Redis key and store OHLCV as a JSON string: */}

{/* ```python
import redis
import json

r = redis.Redis(host='localhost', port=6379, db=0)

symbol = "AAPL"
date = "2025-05-10"
key = f"{symbol}:{date}"

data = {
    "open": 180.12,
    "high": 182.55,
    "low": 179.70,
    "close": 181.30,
    "volume": 25300000
}

r.set(key, json.dumps(data))
```

To retrieve:

```python
raw = r.get("AAPL:2025-05-10")
record = json.loads(raw)
print(record["close"]) */}

### Run Redis Locally
To run Redis locally, you can use Docker Compose. Here’s a simple command to get started:

If you are on Linux, you can use the following command to run Redis locally:
```
cd redis
docker-compose up -d
```
If you are on Windows, you can contact me if you need help to run Redis locally.

## From MetaTrader to Redis: Storing EOD Data Efficiently

### Downloading EOD Data from MetaTrader5
Now it is time to get our hands dirty and start collecting EOD data. I hope you are comfortable with command line (`Command Prompt`) and Python. 

As discussed, we prefer to use the MetaTrader5 API to download EOD data. As Python is the only language supported by MetaTrader5, we have to use the Python with `MetaTrader5` Python package to download EOD data and store it in Redis.

I have made a simple script to download EOD data from MetaTrader5 and save them into files.

You need to have the both Python and MetaTrader5 platform installed and running on your machine. You can download the MetaTrader5 platform from the official website.

Then, download the code [repository](https://github.com/tyolab/tyosis-import) from GitHub:
```
git clone https://github.com/tyolab/tyosis-import.git
cd tyosis-import
pip install -r requirements.txt
```

To run the data download script, ensure you have MetaTrader 5 installed and running, and that you are logged into your trading account. Then for example, to download EOD data for XAUUSD (Gold) from 2025-01-01 to today:
```
python utils\mt5.py XAUUSD 2025
```

The content of downloaded looks like this:
```csv
symbol,date,open,high,low,close,volume
XAUUSD,02/01/2025,2624.78,2660.34,2621.7,2657.66,198834
XAUUSD,03/01/2025,2657.49,2665.26,2636.63,2639.46,169658
XAUUSD,06/01/2025,2641.42,2649.44,2614.37,2635.92,229184
XAUUSD,07/01/2025,2636.07,2664.16,2632.88,2648.79,205976
XAUUSD,08/01/2025,2648.42,2669.91,2645.29,2661.36,194517
XAUUSD,09/01/2025,2661.5,2678.11,2655.45,2669.91,151068
XAUUSD,10/01/2025,2670.79,2697.79,2663.96,2688.75,244597
XAUUSD,13/01/2025,2689.1,2693.39,2656.55,2662.87,289738
XAUUSD,14/01/2025,2663.47,2678.0,2659.89,2676.74,189931
XAUUSD,15/01/2025,2678.15,2697.32,2669.31,2696.63,176113
XAUUSD,16/01/2025,2694.86,2724.67,2689.89,2713.7,194921
XAUUSD,17/01/2025,2713.75,2717.32,2699.23,2702.2,172786
XAUUSD,20/01/2025,2702.94,2713.68,2689.31,2707.06,169458
XAUUSD,21/01/2025,2706.95,2745.93,2702.88,2744.69,224338
```

A detailed explanation of how to use the script is provided below:
```
python utils\mt5.py <symbol> <year> 
```

Replace `<symbol>` with the trading symbol you want to download data for (e.g., EURUSD) and `<year>` with the year you are interested in (e.g., 2023). The script will then connect to your MetaTrader 5 platform, download the end-of-day historical data for the specified symbol and year, and save it to a text file named YYYY.txt within a directory structure based on the market and symbol (e.g., `data/Forex/EURUSD/2023.txt`). If the file already exists, the script will append new data to it, starting from the day after the last recorded date.

If you want to download data for the last 20 years of Gold (XAUUSD), you can run the following command:
```
for /L %i in (2005,1,2025) do python utils\mt5.py XAUUSD %i
```

Now we have the EOD data stored in text files. The next step is to load this data into Redis.

### Loading EOD Data into Redis

To load the EOD data into Redis, we can use the `utils/import.py` script provided in the repository. This script will read the text files and store the data in Redis using the key structure we defined earlier.

For example,

```
python utils/import.py --key-prefix "metals:" /data/Metals/XAUUSD/2025.txt 
```

If you would like to import the last 20 years of Gold (XAUUSD) data, you can run the following command:
```
for /L %i in (2005,1,2025) do python utils/import.py --key-prefix "metals:" /data/Metals/XAUUSD/%i.txt 
```

### Checking the Data in Redis
```shell
redis-cli
>hgetall metals:XAUUSD
  1) "20250307"
  2) "{\"O\": 2911.85, \"H\": 2930.43, \"L\": 2896.95, \"C\": 2910.82, \"V\": 164598}"
  3) "20250127"
  4) "{\"O\": 2768.22, \"H\": 2772.58, \"L\": 2730.49, \"C\": 2740.11, \"V\": 264527}"
  5) "20250315"
  6) "{\"O\": 2983.94, \"H\": 2988.8, \"L\": 2981.73, \"C\": 2986.03, \"V\": 2279}"
  7) "20250211"
  8) "{\"O\": 2908.07, \"H\": 2942.65, \"L\": 2881.57, \"C\": 2897.44, \"V\": 252658}"
  9) "20250206"
 10) "{\"O\": 2867.64, \"H\": 2873.29, \"L\": 2834.22, \"C\": 2855.5, \"V\": 198609}"
 11) "20250116"
 12) "{\"O\": 2694.86, \"H\": 2724.67, \"L\": 2689.89, \"C\": 2713.7, \"V\": 194921}"
 13) "20250321"
 14) "{\"O\": 3044.99, \"H\": 3047.41, \"L\": 2999.37, \"C\": 3023.0, \"V\": 119857}"
```

## Best Practices

- **Normalize your data**: Adjust for splits/dividends if applicable.
- **Consistency**: Store all assets using the same structure.
- **Backup**: Regularly export Redis snapshots (RDB) or enable AOF for persistence.
- **Metadata**: Store additional info like timezone (if you need data with smaller timeframe like h4, h1, or m15 etc), data source, and update timestamp.

## Final Thoughts

Before building signals, portfolios, or trading bots, **build your data foundation**. Collecting clean EOD data and storing it efficiently gives you the confidence that your backtests are meaningful and your signals trustworthy.

Now that you've gathered EOD data for your chosen instruments, you have a solid foundation for backtesting your trading strategies. You can start exploring different indicators, rules, and risk management techniques to see how they would have performed historically. Remember to thoroughly evaluate your backtests and consider factors like transaction costs and slippage to get a realistic assessment of your strategy's potential.

{/* In the next part of this series, we’ll explore how to **backtest a simple moving average crossover strategy** using the data we've just stored in Redis. This will involve retrieving the data, calculating moving averages, and simulating trades based on crossover signals.

Stay tuned for a step-by-step guide on implementing and evaluating this strategy! */}

 {/* In the next part of this series, we’ll explore how to **backtest a simple moving average crossover strategy** using the data we've just stored. */}


<ContactFooter />

f0476efad7639d237009834981fd9ba52a74e9dda1f9c8a4971b3226e6358999

In this article, we explore the essential sources of EOD data and demonstrate how to store and access it efficiently using Redis, a high-performance in-memory database.

Building Blocks of Algorithmic Trading: Collecting End-of-Day Data



# Building a RAG System with Onyx for Better Trading Strategies

In the fast-paced world of trading, information is power—but only if you can access and synthesize it effectively. Over the years, I’ve collected an extensive library of trading materials: research reports, technical guides, strategy notes, economic models, and annotated charts. The challenge has always been how to organize this knowledge and use it dynamically. [**Onyx**](https://www.onyx.app/), a LLM Chat UI that can be used for building Retrieval-Augmented Generation (RAG) systems.

In this article, I’ll walk through how I used Onyx to turn a trading knowledge base into a responsive AI assistant that not only helps traders but also supports them in crafting better strategies.


## What Is Onyx?

[Onyx](https://github.com/daangn/onyx) is an open-source Chat UI tool designed for building LLM-based systems that can incorporate external knowledge via retrieval. It supports:

- **Multi-stage RAG pipelines**
- **Flexible embedding support**
- **Data loaders and chunking for various formats**
- **Built-in support for vector databases like FAISS, Qdrant, and Weaviate**

Unlike complex frameworks that require weeks of setup, Onyx is minimalist and modular. You can start small and scale later.

## Why Use RAG for Trading?

RAG combines the reasoning ability of large language models (LLMs) with precise retrieval from curated knowledge sources. This hybrid setup is ideal for trading, where:

- **Context is critical** – Historical events, policy shifts, and macro conditions must be considered.
- **Domain expertise matters** – Pre-trained models lack specialized knowledge in niche trading strategies.
- **Speed is essential** – A RAG system can instantly answer questions like “What’s the impact of rising bond yields on tech stocks?” using your own materials.

## How to Use Onyx

### Hardware Requirements

Here are the minimum hardware requirements for running Onyx:

- 4vCPUs
- 16GB RAM
- 50GB SSD
- GPU (optional, but recommended for faster inference)

### Installation

Using Docker is the easiest way to get started. Here’s a quick command to set up Onyx:

**Installation:**

```bash
git clone https://github.com/onyx-dot-app/onyx.git
cd onyx/deployment/docker_compose
cp env.prod.template .env
# You may need to change the AUTH_TYPE in .env file to basic or other authentication method you like 
# if you don't want to use Google OAuth2.0 which requires a client_id and client_secret
# AUTH_TYPE=disabled
cp env.nginx.template .env.nginx
ln -s docker-compose.prod.yml docker-compose.yml
```

**Usage:**
```bash
cd onyx/deployment/docker_compose
docker compose up
## if you are plan to use Onyx constantly, you can run `docker compose up -d`, so the server will run in the background
```

To start the App: [http://localhost](http://localhost)

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/03/building-a-retrieval-augmented-generation-system-using-open-source-tools/onyx2.png)
![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/03/building-a-retrieval-augmented-generation-system-using-open-source-tools/onyx3.png)
![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/03/building-a-retrieval-augmented-generation-system-using-open-source-tools/onyx4.png)
![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/03/building-a-retrieval-augmented-generation-system-using-open-source-tools/onyx5.png)

## Step-by-Step: Creating the RAG with Onyx

### 1. Preparing the Knowledge Base

If you prefer you could separate your documents into following categories:

- **Macroeconomic theory**
- **Technical analysis**
- **Historical trades**
- **Research papers**
- **Earnings playbooks**
- **Risk models**

They can be in PDF, DOCX, or Markdown format. Onyx’s data loaders made it easy to convert and chunk these documents into retrievable vectors.

To me, for the sake of simplicity, I kept everything in a single folder called `trading-materials`.

### 2. Choosing an Embedding Model

The default embbedding model in Onyx is **NomicAI’s nomic-embed-text-v1**, which is fast and effective for most use cases. However, you can also use other cloud-based models from Cohere, OpenAI, Google, and Voyage.

{/* | Provider  | Model                          | Description                                                  | Price per 1M Tokens |
|-----------|--------------------------------|--------------------------------------------------------------|---------------------|
| Cohere    | embed-english-v3.0             | Good performance for English-language tasks                  | $0.10               |
| Cohere    | embed-english-light-v3.0       | Lightweight, faster and more efficient for simpler tasks     | $0.10               |
| OpenAI    | text-embedding-3-large         | Large model with best performance                            | $0.13               |
| OpenAI    | text-embedding-3-small         | Efficient model with good performance and lower cost         | $0.02               |
| Google    | text-embedding-005             | Most recent embedding model                                  | $0.025              |
| Google    | textembedding-gecko@003        | Gecko model, powerful and efficient                          | $0.025              |
| Voyage    | voyage-large-2-instruct        | High-performance model with instruction fine-tuning          | $0.12               |
| Voyage    | voyage-light-2-instruct        | Lightweight model with balanced performance and efficiency   | $0.12               | */}

<Table data={
{
  headers: ["Provider", "Model", "Description"],
  rows: [
    ["Cohere", "embed-english-v3.0", "Good performance for English-language tasks"],
    ["Cohere", "embed-english-light-v3.0", "Lightweight, faster and more efficient for simpler tasks"],
    ["OpenAI", "text-embedding-3-large", "Large model with best performance"],
    ["OpenAI", "text-embedding-3-small", "Efficient model with good performance and lower cost"],
    ["Google", "text-embedding-005", "A cost-effective embedding model"],
    ["Google", "textembedding-gecko@003", "Gecko model, powerful and efficient"],
    ["Voyage", "voyage-large-2-instruct", "High-performance model with instruction fine-tuning"],
    ["Voyage", "voyage-light-2-instruct", "Lightweight model with balanced performance and efficiency"]
  ]
}
} />

From the account icon in the top right corner, you can go to the `Admin Panel` and from the panel on the left select `Configuration` > `Search Settings` to configure the `Search Settings`. 

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/embedding-model.png)

To change the model, just clicke on `Upate Search Settings` and select the model you want to use and follow the instruction.

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/embedding-model2.png)

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/embedding-model3.png)

### 3. Creating a File Connector

In the `Admin Panel`, click on the `Connectors` > `Add Connector`, then choose `File` connector. 

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/file-connector.png)


From there, you can drag and drop the `trading-materials` folder or whatever folder you want to use.  This allows Onyx to automatically index new documents as they are added.

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/existing-connectors.png)

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/existing-connectors2.png)


### 4. Setting Up Document Sets

In the `Admin Panel` > `Document Management` > `Document Sets` from the page click on `New Document Set`.

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/docment-sets.png)

From there, you enter the name of the document set (`Trading Materials`) and the description (optional) and pick the `trading materials` file connector you just create.

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/docment-set.png)

### 5. Creating a Custom Assistants

In the `Admin Panel` > `Custom Assistants` > `Assistants`, from the page click on  `New Assistant`.

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/assistants.png)

From there, you enter the name of the assistant and the description, and turn on the `Knowledge` option pick the `Trading Materials` document set you just create.

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/new-assistant.png)

For the Instrcutions, you might want to use something like:

```markdown
You are an AI trading assistant designed to help users make informed trading decisions. Your primary tasks include analyzing market data, identifying trends, answering questions about financial instruments, providing risk assessments, and suggesting potential trading opportunities.

Always communicate clearly, concisely, and objectively. Avoid speculation and state when data is uncertain or incomplete. Use up-to-date financial reasoning, basic technical and fundamental analysis, and explain your suggestions in plain language.

When asked about specific assets, consider recent price movements, volume, relevant news, and broader market context. Tailor your answers to suit short-term traders or long-term investors based on user cues.

Do not provide financial advice. But also offer insights, highlight risks, and support decision-making by presenting pros and cons. Use markdown formatting for clarity, including bullet points, tables, and simple charts or examples when helpful.
```

### 6. Configuring Gen AI API 

If you haven't done that already, you can go to the `Admin Panel` > `Configuration` > `LLM` and select the LLM Provider you want to use.

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/llm-provider.png)

I configured Onyx to use GPT-4o as the reasoning engine.

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/llm-provider-openai.png)

If you haven't have an OpenAI account, you can sign up for one [here](https://platform.openai.com/signup). After signing up, you can create an API key by going to the [API keys page](https://platform.openai.com/api-keys).

## Ask the Assistant

Once you have everything set up, you can start asking your assistant questions. If you haven't exited `Admin Panel`, click on th `Exit Admin` in the top left corner, then click on `Assistants` > `Explore Assistants`. You can find the assistant you just created and click on `Start Chat` to start asking questions.

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/start-chat.png)

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/chat.png)


## Example Queries I Use

Once the system was live, I began using it for tasks like:

- **"How did I trade around rate hikes in 2018?"**

    In 2018, trading strategies around rate hikes typically involved adjusting positions based on the anticipated impact of interest rate changes on various financial instruments. Traders often employed strategies such as:

    * Bull Put Spread: This strategy involves a long position in an out-of-the-money (OTM) put option and a short position in another OTM put option with a higher strike price. It is a net credit trade with a bullish outlook, profiting if the stock price rises 

    *Bear Call Spread: This strategy consists of a long position in an OTM call option and a short position in another OTM call option with a lower strike price. It is a net credit trade with a bearish outlook, suitable for profiting from a decline in stock prices 

    * Long Iron Condor: This is a combination of a bull put spread and a bear call spread, involving long and short positions in OTM put and call options. It is a net credit trade with a neutral outlook, aiming to profit from low volatility 

    These strategies were used to manage risk and capitalize on expected market movements due to rate hikes. The choice of strategy depended on the trader's outlook on market direction and volatility.


- **"Help me develop a psar based strategy but it could also use other indicators for buy/ sell signals"**

### Developing a Trading Strategy Using the Parabolic SAR (PSAR)

To develop a trading strategy using the Parabolic SAR (PSAR) indicator, you can enhance it by incorporating additional indicators to generate more reliable buy and sell signals. Here's a structured approach:

#### Core Strategy: PSAR

**PSAR Basics**:  
The Parabolic SAR is a trend-following indicator that provides potential reversal points. It is plotted as dots above or below the price chart.  
- When the dots are below the price, it indicates a **bullish** trend.  
- When they are above, it indicates a **bearish** trend.


#### Enhancing with Additional Indicators

##### **Moving Averages**

- **Simple Moving Average (SMA)**:  
  Use a short-term SMA (e.g., 20-day) to confirm the trend direction.  
  - A buy signal is stronger if the price is above the SMA.  
  - A sell signal is stronger if the price is below the SMA.

- **Exponential Moving Average (EMA)**:  
  Consider using a 50-day EMA for a smoother trend confirmation.

##### **Momentum Indicators**

- **MACD (Moving Average Convergence Divergence)**:  
  Look for MACD line crossovers to confirm PSAR signals.  
  - A **bullish crossover** (MACD line crossing above the signal line) can confirm a PSAR buy signal.  
  - A **bearish crossover** can confirm a sell signal.

- **RSI (Relative Strength Index)**:  
  Use RSI to identify overbought or oversold conditions.  
  - An RSI **below 30** can strengthen a PSAR buy signal.  
  - An RSI **above 70** can strengthen a sell signal.

##### **Volume Indicators**

- **On-Balance Volume (OBV)**:  
  Use OBV to confirm the strength of a trend.  
  - **Rising OBV** can support a PSAR buy signal.  
  - **Falling OBV** can support a sell signal.

#### Strategy Implementation

##### **Buy Signal**

- PSAR dots switch from **above to below** the price.  
- Price is **above** the 20-day SMA and 50-day EMA.  
- MACD line **crosses above** the signal line.  
- RSI is **above 30**, indicating the end of an oversold condition.  
- OBV is **rising**, indicating increasing buying pressure.

##### **Sell Signal**

- PSAR dots switch from **below to above** the price.  
- Price is **below** the 20-day SMA and 50-day EMA.  
- MACD line **crosses below** the signal line.  
- RSI is **below 70**, indicating the end of an overbought condition.  
- OBV is **falling**, indicating increasing selling pressure.


#### Risk Management

- **Stop-Loss**:  
  Set a stop-loss slightly **below** the recent swing low for buy positions and **above** the recent swing high for sell positions.

- **Position Sizing**:  
  Use a fixed percentage of your capital for each trade to manage risk effectively.


#### Backtesting and Optimization

- **Backtest**:  
  Test the strategy on historical data to evaluate its performance. Adjust parameters like the period of moving averages and the sensitivity of the PSAR to optimize results.

- **Review and Adjust**:  
  Continuously review the strategy's performance and make necessary adjustments based on market conditions.


This strategy combines the trend-following nature of the PSAR with confirmation from moving averages, momentum, and volume indicators to enhance the reliability of trading signals.

## What about the Cloud-based LLMs?
I also tried using OpenAI and Claude to answer the same question about develping the PSAR strategy. Here’s how they performed:
### OpenAI

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/pasr-openai.png)


![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/pasr-openai2.png)

### Claude

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/pasr-claude.png)

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/05/finding-your-edge-build-a-rag-system-with-onyx-for-better-trading-strategies/pasr-claude2.png)

By comparing the responses, it's evident that both OpenAI and Claude provided solid, reasonable answers—likely due to being trained on similar materials. However, their responses tend to lack depth, concreteness, and specificity when faced with more detailed or domain-specific questions—something a custom knowledge base could address more effectively.

## Final Thoughts

As shown above, even without a custom knowledge base, cloud-based LLM services like OpenAI and Claude can provide reasonable answers. However, the real strength of a Retrieval-Augmented Generation (RAG) system lies in its ability to incorporate your own knowledge and experience. By building a high-quality, domain-specific knowledge base—unique to you—you can dramatically enhance the usefulness and accuracy of your AI assistant.

Tools like Onyx, RagFlow, or LobeChat offer traders a powerful and flexible way to transform years of accumulated trading insights into an active, consultative resource. If you're a trader with a trove of notes, recordings, or charts, creating a RAG system could be one of the most impactful ways to integrate AI into your workflow.

Helping traders find edge is just one application that A RAG system could be used for. You could also use it to build a research assistant that could help you for study analysis, or even a engineering assistant that could help you write and debug code, generate documentation, or explore technical specifications across large sets of engineering data. Onyx is one of the many open source tools that could help you to build a RAG system that could be used for various applications. Here is [an article](/blog/2025/03/14-building-a-retrieval-augmented-generation-system-using-open-source-tools/) discussing a list of these open source tools that you could use to build a RAG system.

If you'd like help setting up your own Onyx-based RAG system for trading or research, feel free to reach out.

<ContactFooter />

ff7cac9acd915aa3203d07e03f3946d0b273d38056581f7a773850ec26212eb4

In the fast-paced world of trading, information is power—but only if you can access and synthesize it effectively. Over the years, I’ve collected an extensive library of trading materials: research reports, technical guides, strategy notes, economic models, and annotated charts. The challenge has always been how to organize this knowledge and use it dynamically. Onyx, a LLM Chat UI that can be used for building Retrieval-Augmented Generation (RAG) systems

In this article, I’ll walk through how I used Onyx to turn a trading knowledge base into a responsive AI assistant.

Finding Your Edge: Build a RAG System with Onyx for Better Trading Strategies


# Take Some Words Seriously: Adding Edge to Your Trading


Not all market crashes start with a chart. Some begin with a single word — whispered by a central bank, printed in a report, or trending on Twitter. In a world ruled by data and sentiment, the right (or wrong) word can trigger waves of volatility.

Smart traders don’t just watch price action — they read the narrative. Headlines, statements, earnings calls, and even social media posts hold hidden clues about what’s about to move. In this piece, we explore how paying close attention to certain market-moving words can add a real edge to your trading.

Anybody remembers this?

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2025/04/take-some-words-seriously-adding-edge-to-your-trading/2025-05-08-15-29-01.png)

(Sources: [Walter Bloomberg, X(Twitter)](https://x.com/DeItaone))

In the market, anything can happen.

## What Makes a Word ‘Market-Breaking’?

Some words carry outsized weight. They don’t just describe the market — they move it. Here's what gives them power:

- **Tied to systemic risk** or macroeconomic shifts
- **Widely interpreted by news-trading algorithms** and market participants
- **Frequently surface in headline-level data**: Bloomberg, Reuters, Fed statements, Twitter

Words like "default" or "rate hike" don’t exist in a vacuum — they set off a chain reaction in asset repricing, sentiment shifts, and volatility spikes.


## Categories of Market-Breaking Words

## 🏦 Central Bank Triggers
These signal shifts in monetary policy — often leading to broad market repricing.
- "Rate Hike"
- "Taper"
- "Pause"
- "Pivot"
- "Liquidity Trap"

## 🧨 Geopolitical Flashpoints
These can disrupt supply chains, oil prices, and global confidence.
- "Invasion"
- "Missile Strike"
- "Nuclear"
- "Sanctions"
- "Embargo"

## 💰 Economic Doom Indicators
These words create fear about broad economic health and growth.
- "Recession"
- "Default"
- "Crisis"
- "Contagion"
- "Debt Ceiling"

## 🐻 Market Panic Phrases
Instant triggers for fear-driven moves, often accompanied by high volatility.
- "Crash"
- "Bear Market"
- "Selloff"
- "Circuit Breaker"
- "Flash Crash"

## 🧮 Corporate Bombshells
Company-specific shocks that ripple through indices or sectors.
- "Bankruptcy"
- "Fraud"
- "Restatement"
- "Layoffs"
- "Guidance Cut"

## 🕵️‍♂️ Regulatory Triggers
Often the start of longer-term negative sentiment, especially in sectors.
- "SEC Probe"
- "Ban"
- "Investigation"
- "Fine"
- "Lawsuit"


## How These Words Trigger Reactions

- **News-reading trading bots** pick up certain keywords and act within milliseconds
- **Retail traders** respond emotionally through social media sentiment
- **Institutional desks** quickly re-evaluate exposure and hedge risk

The result? Spikes in volume, volatility, and often, trend reversals or accelerations.


## Historic Examples

- **"Subprime"** — sparked the 2008 global financial crisis
- **"Taper"** — triggered the 2013 bond market tantrum
- **"Pandemic"** — caused a global crash in early 2020
- **"GameStop"** — ignited the 2021 meme stock mania
- **"Rate Hike"** — repeated shocks in 2022 during Fed tightening cycles
- **"Liberation Day"** - the term used by Trump during the US-led trade wars in 2025. As tariffs and retaliations spread across global economies, markets reacted violently. The phrase came to symbolize a shift toward protectionism and nationalistic rhetoric, culminating in sharp sell-offs across global markets.

These were more than just events. They were language moments that changed how the market perceived the future.


## How to Build a Word Alert System

To make language part of your trading strategy, set up alerts or scripts to detect high-impact terms in real-time sources:

**Sources to Monitor:**
- Fed minutes & speeches
- Central bank statements
- Earnings calls and transcripts
- Real-time Twitter and financial news APIs
- SEC filings

**Tools You Can Use:**
- NLP libraries (spaCy, NLTK)
- Large language models (LLMs)
- News APIs (NewsAPI, Alpha Vantage, Benzinga)
- RSS + keyword matchers
- Google Alerts + Zapier/IFTTT


## Final Thoughts

In the noise of the markets, it’s easy to overlook a single word. But often, that’s where the signal hides. The smartest traders don’t just read the charts — they read the room.

One word can change the market — and if you're not listening, you're not trading with the full picture.

So next time you see a headline with “unexpected,” “default,” or “recession,” don’t just scroll past. Take it seriously. It might just be the edge you’ve been missing.



09cface00e30ed751ae9f9de0aae628ac673b3b044644bfac8d5ea1eb71c8986

In the world of finance, words can be as powerful as numbers. This article explores the vocabulary that can trigger market reactions and how traders can use this to their advantage.

In this article, we explore how certain words can trigger market reactions and how traders can leverage this knowledge for better decision-making.

Take Some Words Seriously: Adding Edge to Your Trading


# Words That Move Markets: The Most Dangerous Vocabulary in Finance


Markets don’t always collapse because of numbers. Sometimes, it’s a single word — quietly dropped in a press release, a news headline, or a central banker’s speech — that sends shockwaves through global finance.

Words like **“default,” “invasion,” “rate hike,”** or **“fraud”** don’t just carry meaning — they carry weight. They trigger selloffs, spark algorithmic trades, and fuel investor panic. In a world driven by speed and sentiment, language itself can become a catalyst for volatility.

This article explores a comprehensive list of words that have the power to break markets — the flashpoints in our financial vocabulary that every trader, analyst, and AI model should be watching.

## The Power of Words in Finance

## 🟡 Commodities

### 🪙 Gold (XAU/USD)
- war, conflict, inflation, crisis, risk-off, safe haven, geopolitical tension, central banks buying, flight to safety

### 🛢️ Crude Oil (WTI, Brent)
- OPEC, supply cut, Middle East, Iran, Saudi, demand destruction, recession, strategic reserves, inventory build/draw, war

### 🥈 Silver (XAG/USD)
- industrial demand, inflation hedge, green energy, solar, risk sentiment, safe haven, monetary policy

### 🌾 Wheat / Corn / Soybeans
- drought, El Niño, Ukraine, Black Sea, harvest, supply chain, food crisis, weather, planting season

### 🔌 Natural Gas
- winter, cold snap, storage levels, LNG, Russia, Europe energy crisis, heating demand

---

## 🟦 Currencies (Forex)

### 💵 US Dollar (DXY, USD pairs)
- interest rates, Fed, CPI, PCE, NFP, GDP, risk-off, global reserve, strong dollar, dollar milkshake

### 🇪🇺 Euro (EUR/USD)
- ECB, Eurozone inflation, Germany, trade surplus, energy crisis, Draghi/Lagarde

### 🇯🇵 Yen (USD/JPY)
- BoJ, yield curve control, carry trade, safe haven, intervention, deflation

### 🇨🇳 Yuan (USD/CNY)
- China slowdown, PBoC, trade war, tariffs, capital control, stimulus, Evergrande

### 🇨🇭 Swiss Franc (USD/CHF)
- SNB, safe haven, Euro correlation, negative rates

### 🇬🇧 Pound Sterling (GBP/USD)
- BoE, Brexit, UK GDP, inflation, fiscal policy, gilts, housing market

---

## 📈 Equities / Indexes

### 🇺🇸 S&P 500 / Nasdaq / Dow Jones
- tech earnings, inflation, interest rates, Fed policy, consumer spending, GDP, unemployment, recession, soft landing

### 🇩🇪 DAX
- German manufacturing, energy prices, ECB policy, exports, euro

### 🇯🇵 Nikkei 225
- BoJ policy, yen strength, tech exports, Japan GDP

### 🇨🇳 Hang Seng / CSI 300
- China real estate, regulation, CCP, stimulus, tech crackdown

### 🟩 Tech Stocks (e.g. NVDA, AAPL)
- AI, innovation, chip shortage, cloud, interest rates, earnings growth

### 🟥 Banking Stocks
- interest margin, yield curve, credit crisis, default, liquidity, stress test

---

## 🟧 Bonds / Fixed Income

### 🏦 US Treasuries (10Y, 2Y)
- inflation, rate hike, Fed pivot, recession fears, safe haven, yield curve inversion

### 🏠 Mortgage-Backed Securities (MBS)
- housing market, delinquency, mortgage rates, Fed balance sheet

---

## 🟨 Cryptocurrencies

### ₿ Bitcoin
- inflation hedge, fiat collapse, risk-on, ETF approval, halving, decentralization, regulation, adoption

### 🪙 Ethereum
- smart contracts, DeFi, staking, upgrade (e.g., Merge), NFT boom

### 🧾 Stablecoins
- regulation, peg risk, reserves, SEC, CBDC

---

## 🟩 Other Assets / Themes

### 🏠 Real Estate
- housing bubble, interest rates, mortgage rates, affordability, supply, zoning, default

### 🧭 Volatility (VIX)
- fear, uncertainty, crash, market panic, event risk

### 🪙 Precious Metals ETF (e.g., GLD, SLV)
- hedge, safe haven, ETF inflows, dollar weakness


## Final Thoughts

In today’s markets, words move faster than fundamentals. A single headline, a sudden phrase, or a carefully chosen word from a policymaker can shift sentiment and trigger massive market reactions within seconds.

By understanding the vocabulary that shakes confidence or fuels momentum, traders and analysts can stay one step ahead. Whether you're building sentiment-driven bots or simply paying closer attention to news flow, knowing these words isn't just useful — it's essential.

Stay sharp, stay curious — and always read between the lines.



379164b92997ea014c21fdfadca17501d63c4b7e0d9528b7ee249a6ec14a8507

In this article, we delve into the words that can cause market volatility, from commodities to currencies to equities. Understanding this vocabulary is essential for traders and analysts.

In the world of finance, words can be as powerful as numbers. This article explores the vocabulary that can trigger market reactions.

Words That Move Markets: The Most Dangerous Vocabulary in Finance


# The Hidden Vocabulary of the Market


In financial markets, not all signals come from charts or price action — some of the most powerful insights are embedded in the **language of the news cycle**. Words like "inflation," "conflict," or "stimulus" can send ripples across commodities, currencies, and equity markets.

Each asset class, from gold to tech stocks to foreign exchange, tends to react to specific themes and phrases. Understanding these linguistic correlations forms the foundation for a new kind of edge — one based not on technical indicators, but on **sentiment, context, and narrative**.

This is the hidden vocabulary of the market — the subtle, often recurring language that signals opportunity or risk. With modern tools like **natural language processing (NLP)** and **AI-driven sentiment analysis**, traders and developers can now harness this data in real time to enhance decision-making, trigger trades, or power predictive models.



## Understanding Word-Asset Correlation

Every trading instrument reacts to a unique set of words or themes. These word clusters tend to reflect the economic, geopolitical, or sector-specific dynamics that most influence the asset.

For example, the word "inflation" might correlate with gold prices, while "rate hike" could be more relevant for tech stocks. By mapping these correlations, traders can create a **keyword library** that serves as a powerful tool for sentiment analysis.

### Example Correlations

<Table data={{
    "headers": ["Asset", "Common Correlated Words / Phrases"],
    "rows": [
        ["Gold", "war, conflict, inflation, crisis, safe haven"],
        ["Crude Oil", "recession, OPEC, Middle East, Iran, supply cut, inventory build"],
        ["Tech Stocks", "rate hike, earnings miss, chip shortage, AI, innovation"],
        ["US Dollar", "interest rate, Fed, strong dollar, CPI, dollar milkshake"],
        ["Bitcoin", "regulation, halving, ETF approval, fiat collapse, risk-on"],
        ["Real Estate", "housing crash, mortgage rates, affordability, bubble"]
    ]
    }
} />

{/* | Asset              | Common Correlated Words / Phrases                                     |
|--------------------|----------------------------------------------------------------------|
| Gold               | war, conflict, inflation, crisis, safe haven                         |
| Crude Oil          | recession, OPEC, Middle East, Iran, supply cut, inventory build      |
| Tech Stocks        | rate hike, earnings miss, chip shortage, AI, innovation              |
| US Dollar          | interest rate, Fed, strong dollar, CPI, dollar milkshake             |
| Bitcoin            | regulation, halving, ETF approval, fiat collapse, risk-on            |
| Real Estate        | housing crash, mortgage rates, affordability, bubble                 | */}

These correlations are not always perfect, but they represent **patterns of influence** that emerge from repeated market behavior over time.

You can find a comprehensive list of words that move markets in [**this post**](/blog/2025/04/25-words-that-move-markets-the-most-dangerous-vocabulary-in-finance/).


## Use Cases

### 🧠 1. Sentiment-Based Trading Systems

By feeding news and social media data into NLP engines, you can design systems that identify shifts in sentiment tied to specific assets. When the volume or intensity of certain keywords spikes, these systems can signal or trigger trades.

Example:
- **Keyword:** "Middle East conflict"
- **Reaction:** Long gold and oil, short airline stocks

This approach allows you to trade the **emotional layer** of the market — the part that responds to fear, euphoria, or uncertainty.



### 🤖 2. Trading Bots & Automated Strategies

Traders can build bots that scan headlines, Twitter posts, and economic reports in real time. Using keyword libraries mapped to asset behavior, bots can:
- Trigger entries or exits based on phrase detection
- Score the sentiment of each headline
- Prioritize signals by sentiment intensity and source credibility

Example workflow:
1. Detects phrase: *“Fed signals pause in rate hikes”*
2. Bot scores sentiment: Positive for equities, negative for USD
3. Executes: Buy tech ETFs, short DXY


### 📊 3. Risk Management & Early Warnings

Keyword-based monitoring isn't just for trade entries — it can be used to **manage portfolio risk** and issue alerts when dangerous sentiment builds.

Consider a bond portfolio. If keywords like “default,” “downgrade,” or “liquidity crunch” suddenly spike in financial media, a system can alert the manager to de-risk or hedge positions.

This approach brings a new level of **precautionary intelligence**, especially useful during uncertain macro environments.


### 🔎 4. Backtesting Word Impact on Price

Historical market data paired with archived news allows traders to **backtest how certain words affected asset prices**. By building keyword timelines and comparing them to price moves, you can find statistically significant patterns.

Questions you might answer:
- How did BTC price react 24–48 hours after the word “ETF approval” trended?
- Did “rate pause” consistently result in S&P 500 rallies?

This can validate the usefulness of your keyword library and refine its weighting in your models.


### 🧠 5. Enhanced Human Decision-Making

For discretionary traders, this keyword framework acts as a **mental filter**. Instead of trying to digest all news, you focus on the language that directly impacts your portfolio.

Example:
- A currency trader focuses on words like "intervention," "FX reserves," or "carry trade"
- A commodity trader tracks "harvest failure," "OPEC decision," and "pipeline disruption"

The result is better clarity, faster reaction time, and improved consistency.


## Implementing a Keyword Strategy

Building a keyword-sensitive trading framework can be broken down into several layers:

1. **Keyword Mapping**  
   Create a dictionary of correlated words tied to each asset or asset class.

2. **Data Ingestion**  
   Pull in data from news APIs, RSS feeds, Twitter, financial blogs, or Reddit.

3. **NLP Processing**  
   Use tools like spaCy, Hugging Face Transformers, or OpenAI to:
   - Extract keywords and themes
   - Score sentiment (positive/negative/neutral)
   - Detect sudden changes in word frequency

4. **Signal Logic**  
   Assign weights to different words. Use thresholds or scoring systems to generate alerts or trades.

5. **Backtesting**  
   Validate your keyword signals using historical data to filter out noise and overfitting.


## Final Thoughts

Markets are narratives in motion — and every narrative is built with language. By listening carefully to **the hidden vocabulary** behind every asset, traders can uncover early signals, better manage risk, and even teach their bots to understand the tone of the times.

In an era dominated by noise, having a linguistic edge might be one of the most underappreciated forms of **alpha**.

## Want to Build Your Own?

If you're a trader, quant, or developer interested in building a keyword-aware trading system, here's where to start:
- Explore free APIs like [NewsAPI](https://newsapi.org), [Reddit API](https://www.reddit.com/dev/api/), or [Twitter/X APIs](https://developer.twitter.com)
- Use NLP libraries like [spaCy](https://spacy.io), [TextBlob](https://textblob.readthedocs.io), or [OpenAI GPT APIs](https://platform.openai.com/)
- Backtest with tools like [Backtrader](https://www.backtrader.com/), [QuantConnect](https://www.quantconnect.com/), or a custom Python stack with pandas and yfinance


*Let the market speak — and more importantly, learn to understand what it’s really saying.*


## Promt for the Image
> A glowing robot hand could be reaching towards the word- "Inflation", indicating the bot’s control over the market movement. The entire image should evoke a sense of power, control, and high-tech disruption in financial markets. The overall style should be modern and futuristic, with high contrast, vibrant colors, and a digital, tech-heavy aesthetic.

44cae56c4a54d0119233c95cfc579eff377d4537a5bc9c40eaf390826e7a354b

In this post, we explore the hidden vocabulary of the market and how it can be used to gain an edge in trading. We discuss the importance of understanding the language of the market and how it can impact trading decisions.

In this post, we explore the hidden vocabulary of the market and how it can be used to gain an edge in trading.

The Hidden Vocabulary of the Market


# What Is Your Thesis for Shorting


## Introduction  
Lots of new traders are thrilled by the potential in shorting and are particularly interested in shorting assets and markets. Sometimes they simply short for the sake of it. For traders who have a strong short bias, they do enjoy watching the movie "The Big Short" where the market crash was predicted by a group of hedge funds and Michael Burry was admired for being one of the few who made a huge profit in betting against the housing market. At that time, he had a valid thesis for shorting, but not in 2023 when he tweeted "Sell" and the market has been up over 40% since. So here's a question for you, my friend, when you're shorting: **What is your thesis?**.

## Why Do We Short?  
### Core Reasons:  

1. **Capitalizing on Overvaluation:**  
   - When assets or markets deviate significantly from their intrinsic value, short selling captures the opportunity for correction.  

2. **Navigating Downturns:**  
   - Provides a mechanism to profit or protect portfolios during bear markets or sector-specific declines.  

3. **Hedging and Diversification:**  
   - Allows traders to offset risks in their long portfolios by betting on declines in correlated or overexposed assets.  

### What to Short?
Short selling isn’t merely about betting against an asset; it’s about crafting a thesis grounded in research, logic, and market dynamics. Whether addressing structural flaws in a market or capitalizing on cyclical downturns, shorting provides traders with tools to profit from or hedge against declines. 

Below are the common theses for what traders might want to short:

- **Oil:** 
    - Structural Weaknesses: Global push towards renewable energy threatens oil's long-term demand.
    - Economic Sensitivity: Oil demand is closely tied to economic cycles.
    - Supply and Demand: The US has become the world's biggest producer with the goal of driving down inflation and providing affordable energy.
    
- **Housing Market:** 
    - Bubble Dynamics: Persistent price increases fueled by speculation, low-interest rates, and loose lending can create unsustainable housing bubbles.
    - Macroeconomic Pressures: Rising interest rates increase mortgage costs, cooling demand and leading to price corrections.
    - Demographic Shifts: Aging populations and changes in work preferences (e.g., remote work reducing demand for urban housing) can alter long-term housing demand. 

- **Foreign Currencies:**
    - Structural Weaknesses: Overvaluations in foreign currencies often signal good shorting opportunities.
    - Macroeconomic Pressures: Overheated economies, excessive inflation, fiscal pressures, and geopolitical tensions can lead to adverse currency conditions.
    - Economic Sensitivity: Foreign currencies are often sensitive to economic shocks and market conditions, making them a suitable asset for shorting.

- **Cryptocurrencies:** 
    - Pump and Dump: Artificially inflated prices, psychological manipulation to trigger FOMO.
    - Sentiment Analysis: High levels of hype or overly optimistic projections often precede downturns.

- **Stocks:**
    - Financial Fundamentals: Overvalued, over-leveraged, weak earnings growth, high leverage, and unsustainable valuations often signal good shorting opportunities.
    - Market Dynamics: Rapid shifts in investor sentiment make stocks prone to price spikes, often defying fundamental analysis.
    - Sentiment Analysis: High levels of hype or overly optimistic projections often precede downturns.

- **Overhyped Pharmaceuticals:** 
    - Research Fundamentals: Highly touted, unproven therapies, likely-to-fail or failed clinical trials, unrealistic expectations of future clinical trials, fraudulent marketing, and lack of transparency.
    
- **Markets / Companies Impacted by Negative Economic Events / News:**
    - Panic Selling: Market panic with strong fears of recession or market crash.
    - Market Downturn: Economic downturns, geopolitical tensions, or geopolitical tensions.
    - Market Sentiment: Scandalous events, unhappy investors, damaged reputations, or negative media coverage.
    - Regulatory Changes: Regulatory changes, tax changes, or changes in government policies.


### Specific Cases:  

#### Shorting Oil  
1. **Structural Weaknesses:**  
   - Global push towards renewable energy threatens oil's long-term demand.  
   - Increasing regulatory scrutiny, carbon taxes, and the rise of electric vehicles contribute to systemic pressures on oil markets.  

2. **Economic Sensitivity:**  
   - Oil demand is closely tied to economic cycles. A global recession or slowdown in industrial production can significantly reduce consumption.  

3. **Geopolitical Factors:**  
   - Oversupply driven by OPEC dynamics or major producers failing to align production cuts with demand trends can create price collapses.  
   - Example: The 2020 oil crash when demand plummeted due to COVID-19 lockdowns.  

#### Shorting the Housing Market  
1. **Bubble Dynamics:**  
   - Persistent price increases fueled by speculation, low-interest rates, and loose lending can create unsustainable housing bubbles.  
   - Examples: The subprime mortgage crisis of 2008 highlighted the fragility of over-leveraged housing markets.  

2. **Macroeconomic Pressures:**  
   - Rising interest rates increase mortgage costs, cooling demand and leading to price corrections.  
   - Stagnant wage growth further exacerbates affordability issues, shrinking the buyer pool.  

3. **Demographic Shifts:**  
   - Aging populations and changes in work preferences (e.g., remote work reducing demand for urban housing) can alter long-term housing demand.  

## Why Avoid Long-Term Shorting Some Assets
Market always has its ups and downs, and you can always make a profit by shorting an asset when it is in a downturn. Long-term shorting some assets can be a bad idea and you can actually make more profit in going long than shorting. So why shorting? Here are a couple of assets that can be considered in this category:   

### US Market Indices
1. **Economic Growth:**  
   - For the market such as the US market, there is always strong economic growth with rising GDP and advancement of technologies.

2. **Popularity:**
   - The S&P 500, Dow Jones, and Nasdaq Composite are among the most popular market indices attracting investors and traders around the world.

### Bitcoin  
1. **Scarcity and Adoption:**  
   - Bitcoin’s limited supply combined with growing adoption as a store of value and payment system reinforces its long-term strength.  

2. **Hedge Against Monetary Instability:**  
   - During periods of high inflation or fiat currency debasement, Bitcoin attracts capital as a decentralized and inflation-resistant asset.  

3. **Branding and Reputation:**
   - Bitcoin is a well-known asset and has created a strong reputation around the global market. People trust it as a safe haven and a store of value.

4. **Money Without Borders:**  
   - Bitcoin is a digital currency that can be used globally, making it a popular choice for international transactions. 

### Tesla  
1. **Innovative Growth:**  
   - Tesla dominates the EV market with a significant technological and branding edge. Its forward-thinking strategy spans beyond vehicles to energy storage and software innovation.  

2. **Investor Loyalty:**  
   - Tesla benefits from a fervent retail investor base, making its stock prone to sharp upward moves that can disrupt short positions.  

Even given the above theses, it can't be said those assets are not shortable and during the market turmoils and uncertainties, it is up to each individual's trading skills and trading strategies to make the most of it. For examples, although there weren't many times that markets crashed, but when the markets tanked, they tanked hard.

## Building a Robust Shorting Thesis  
### Key Steps for Crafting a Winning Strategy:  

1. **Understand Macro and Micro Trends:**  
   - Evaluate the broader economic environment and specific industry trends impacting the asset.  

2. **Examine Financial Fundamentals:**  
   - Weak earnings growth, high leverage, and unsustainable valuations often signal good shorting opportunities.  

3. **Identify Catalysts for Decline:**  
   - Look for events that could trigger a significant drop, such as earnings misses, regulatory shifts, or external economic shocks.  

4. **Incorporate Sentiment Analysis:**  
   - High levels of hype or overly optimistic projections often precede downturns. Use sentiment indicators to gauge market exuberance.  

5. **Define Exit Strategies:**  
   - Set clear entry, exit, and stop-loss levels to manage risk effectively.  

## Shorting Thesis and Shorting Strategy  
1. **Thesis:**  
   - Define the specific asset, market, or industry you're shorting.

2. **Shorting Strategy:**  
   - Craft a clear and effective shorting strategy that includes entry, exit, and stop-loss levels.

3. **Risk Management:**
   - Set clear entry, exit, and stop-loss levels to manage risk effectively.

4. **Execution Strategy:**  
   - Develop a disciplined and consistent execution strategy to reduce risk and optimize profits.

5. **Monitoring and Adaptation:**
   - Continuously monitor market conditions and adapt your strategy as market dynamics change.

In order to increase your chances of success, it is important to have a clear and well-defined shorting strategy that aligns with your thesis and risk management objectives. Even with a solid thesis, there’s no guarantee the market will move in your favor; success hinges on timing and aligning with the moment when the tides actually turn. For examples, it took two years for the housing market to crash after Michael Burry began buying credit default swaps on subprime mortgages. As the old saying goes, "The market can stay irrational longer than you can stay solvent".

## Conclusion  
Shorting is as much an art as it is a science. By focusing on assets like oil and housing markets, where structural and cyclical weaknesses are evident, traders can develop a strong thesis for shorting. However, long-term bets against transformative assets like Bitcoin and Tesla often defy logic due to their innovation, strong fundamentals, and unpredictable sentiment.

The key to mastering shorting lies in rigorous analysis, disciplined execution, and the humility to adapt as markets evolve.  


c984e2a42f0cbb0f1772490c4f573ba56df9fe5cf58d6020b7d71f6e91df53b2

This article explores the fundamental aspects of short selling in trading, emphasizing the importance of having a well-defined thesis before taking short positions. It discusses the core reasons for shorting, including capitalizing on overvaluation, navigating market downturns, and portfolio hedging.

This article explores the fundamental aspects of short selling in trading, emphasizing the importance of having a well-defined thesis before taking short positions.

What Is Your Thesis for Shorting?


## Introduction
MetaTrader doesn't have strong support for algorithmic trading if you want to develop a trading bot that doesn't heavily rely on this particular trading platform. Typically, people doing automated trading on MetaTrader platforms have to code it completely in MQL, and a terminal must be run actively in the foreground.

To reduce the dependency solely on the MQL programming language or to increase the flexibility of a trading system (bot) that can be coded in a different language or run on different operating systems using any preferred technical analysis libraries, some MetaTrader libraries using a message queue (MQ) service, such as ZeroMQ, have been developed. In this way, the part of the code that is coupled with MetaTrader platforms acts only as a relay (proxy agent).

Some people want to completely overcome the limitations of what MetaTrader can provide, so they hack the underlying communication protocol between the MetaTrader terminal and the server and develop libraries in a programming language of their choice, such as C# or C++. With such MetaTrader libraries, an active terminal running in the foreground is no longer necessary, and the trading bot can run headless on Windows, MacOS, or Linux. However, there is a catch with this method: since such a hack is not officially supported by the trading platform developer, it may need constant updates due to requirements for latest terminal version or may simply stop working due to a change in the underlying protocol.

In this article, we will try to develop a trading gateway (or proxy agent) that is intended to reduce the dependency on MetaTrader platform, but also maintain the stability and reliability of such a program in the long term. When the gateway is running or live trading is in action, we don't want the unpredicted interrupt the whole workflow: our trading system (bot) could stop receving market data updates all of a sudden, or it could not place order or receive order status information anymore without an update to the trading program.

## The Design of the Trading System (Bot)

In order to achieve the following goals:
- You do not have to use MQL and do extra compilations, and needs as little operations as possible on MetaTrader, other than leaving it running in the foreground
- The gateway should be stable and reliable. The gateway should continue working even with that the updates of MetaTrader is required constantly.

Fortunately, MetaQuote, the developer of MetaTrader, has provided an official Python package for integration since MetaTrader 5. With this package, you can create a trading bot entirely coded in Python, or you can build a trading gateway that acts as a bridge between the trading platform and an existing trading system (bot) developed in any language (e.g., NodeJS, C++, C#, Java). In the latter case, you can leverage the ecosystem of each language and enjoy the freedom from limitations in both programming language and operating system.

In this article we will focus on developing such a trading gateway using the MetaTrader 5 Python package for the itegration. To ensure seamless communication within the trading system, a complete setup should consist of the following main components:
- a message queue server for messaging and message relaying (price, orders, etc.) 
- trading gateway(s) for getting price updates and managing orders
- indicator updater after receiving new price data  
- a trading decision maker (strategy) for generating buy or sell signals

A gateway is a small but essential part of the entire traiding system. It obtains the latest price updates and manages the placement of new orders or the closure of existing orders when buy or sell signals are triggered.

The following figure illustrates the workflow of a complete trading system:

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/mt5-python-gateway/Trading System Flowchart.png)

## The Implementation of Core Funtinalities of the MetaTrader Gateway

In this section we will discuss the core functionalities of a trading gateway. We will expect at least the following for the MetaTrader Gateway:
- pull the price updates
- submit an order when a signal is triggered
- close the order when it is time to 

A complete implementation of a proper gateway could be complicated, more functionalities might be also needed, for examples, sush as acquisition of account's balance, current P&L or historial data. All depend on what is required for the trading system. Being able to get the account balance could be benificial but is not an absolute requirement.

The actual implementation of this gateway could be straight-forward, as we can just call the coresponding functions provided by the API. However, how to use those functions to achieve the goals of auto trading may be not as that simple as you may think. The following sections will discuss the each aspect of the functionalities in details.

### Initialzation of MetaTrader Terminal
As discussed previously, the MetaTrader python library requires MetaTrader terminal running in the foreground, so it can communicate with the terminal in order to retrieve the price updates and place orders.

There are two parts in this process:
- start the terminal if it has not been running already
- loging into your account

#### Start The terminal
```python
    path = "C:\\Program Files\\MetaTrader 5\\terminal64.exe"
    timeout = 1000000
    portable = False 
    if not mt5.initialize(path=path, portable=portable, timeout=timeout):
        print("initialization failed, error code =", mt5.last_error())
        quit()
```
To start a terminal, the library needs to know where the terminal executable (terminal64.exe) is located, and that is the variable "path" for. In some cases the library might have difficulties in building communciation with the terminal, "timeout" is the time limit in millionseconds when it will stop trying.  

Also, different terminal executables provided by the brokers are actually all the same just with different brandings and a different installation path. You can copy and paste the terminal program to a different directory and run it from there. In this case, you can specify the "portable" mode to "True", so the data from the terminal will be installed in that directory.

#### Account Access
Once the terminal is up and running. You will need to authenciate yourself with your broker's MetaTrader trading server.
```python
    account = 888888
    password = "Your Password Here"
    server = "MetaQuotes-Demo"
    timeout = 60000
    authorized = mt5.login(account, password=password, server=server, timeout=timeout)
    if authorized:
        print("Login successful")
    else
        print("Failed to connect to the trade account with error code =", mt5.last_error())
```

### Getting Price Updates
One major limitation of this package is that you cannot get the price updates through "pumping", which is a machanism that you can get notification whenever new data arrives. In order to get the latest price data, you have to make query to the terminal to see if there is any new data. A chart window for the symbol you are watching should be opened as well, as this active window would make sure the terminal to receive the price updates from the server. 

So we can set a timer, or create a infinit loop inside a child thread and then wake up after being asleep for a certain amount of time. In the example below, we will use a timer to get latest bitcoid price:
```python
last_updated = None
timezone = pytz.timezone("Etc/UTC")
def check_updates():
    # print ("checking for updates") 
    # for different brokers, different tickers for BITCOIN could be used
    # such as 'BITCOIN' or 'BTCUSD'
    time_from = None
    time_to = datetime.now(timezone)
    if last_updated is None:
        time_from = datetime.now(timezone)
        # this is the first time we get the price we will try get all ticks in the last two mintues
        time_from = time_from  - timedelta(minutes=2)
    else:
        time_from = last_updated
        time_from = time_from + timedelta(seconds=1)

    ticks = mt5.copy_ticks_range('BTCUSD', time_from, time_to, mt5.COPY_TICKS_ALL)
    if ticks is None or len(ticks) == 0:
        # print("No ticks received from ", utc_from, "to", server_now, "for ", symbol)
        return
    
    # print("Ticks received:", len(ticks))
    time = None
    for tick in ticks:
        # print the time of the tick
        # named time, bid, ask, last and flags 
        # convert ecpoch to datetime
        time = datetime.fromtimestamp(tick[0])
        # conver time to native with timezone
        time = time.astimezone(timezone)

        # quote = quote()
        # quote.bid = tick[1]
        # quote.ask = tick[2]
        # quote.time = time
        # quote.last = tick[3]
        # quote.volume = tick[4]
        #
        # DO WHATEVER YOU WANT WITH THE DATA
        # For example we can send the quote data to the trading system
        # where indicators kept updated and strategies look for signals 
        # to send quotes to the trading system you can use a message queue system
        # such as ZeroMQ, RabbitMQ, Kafka, etc.
        # or you can use a simple REST API to send the data to the trading system
        # or you can my implementation of a message queue server ('TYO-MQ')
    last_updated = time
    set_timer()

def set_timer():
    # Will the bitcoin price again in 5 seconds
    # If you want to do high-frequency trading obviously this is not good for you
    threading.Timer(5.0, check_updates).start()  

# Don't forget to start the timer
set_timer()
```

### Placing a New order
Once we get a signal from the strategy we can place an order.

```python
# ORDER_TYPE_BUY
# Market Buy order
# ORDER_TYPE_SELL
# Market Sell order
# ORDER_TYPE_BUY_LIMIT
# Buy Limit pending order
# ORDER_TYPE_SELL_LIMIT
# Sell Limit pending order
# ORDER_TYPE_BUY_STOP
# Buy Stop pending order
# ORDER_TYPE_SELL_STOP
# Sell Stop pending order
# ORDER_TYPE_BUY_STOP_LIMIT
# Upon reaching the order price, a pending Buy Limit order is placed at the StopLimit price
# ORDER_TYPE_SELL_STOP_LIMIT
# Upon reaching the order price, a pending Sell Limit order is placed at the StopLimit price
# ORDER_TYPE_CLOSE_BY
# 
# depends on what the signal indicates for buying or sell
# the order type can be one of the above
# if we want buy from the market we can use: mt5.ORDER_TYPE_BUY
# if we want to sell to the market we can use: mt5.ORDER_TYPE_SELL
# if we want to place a buy limit order we can use: mt5.ORDER_TYPE_BUY_LIMIT

def place_order(symbol, market, type, price, volume, stop_loss, take_profit, comment, stop_price, expire_time):
    # Order to close a position by an opposite one
    trade_request = {
        "action": mt5.TRADE_ACTION_DEAL,
        "symbol": symbol,
        "volume": float(volum),
        "type": type,
        "price": price,
        "sl": float(stop_loss),
        "tp": float(take_profit),
        "comment": comment, # a comment for the order, max 31 characters
        "type_time": mt5.ORDER_TIME_GTC,
        "type_filling": mt5.ORDER_FILLING_IOC, # mt5.ORDER_FILLING_RETURN
        "deviation": 20,
        "magic": 0,
        # "expiration": datetime.now() + timedelta(minutes=5), # 5 minutes from now
    }
    trade_result = mt5.order_send(trade_request)
    # you will get a None if the trade_request is not constructed properly
    # for example, the volume is not a float
    # or the price is not a float
    # or the stop_loss is not a float
    # or the take_profit is not a float
    # or the stop_price is not a float
    # or the expire_time is not a datetime object
    # or the comment is not a string or longer than 31 characters
    if trade_result.retcode != mt5.TRADE_RETCODE_DONE:
        print("Error placing order: ", symbol, ", type: ", type, ", volume: ", volume, ", price: ", price, ", stop_loss: ", stop_loss, ", take_profit: ", take_profit, ", comment: ", comment, ", stop_price: ", stop_price, ", expire_time: ", expire_time)
        return None

    print("Order placed: ", trade_result.order, ", for ", symbol, ", type: ", type, ", volume: ", volume, ", price: ", price, ", stop_loss: ", stop_loss, ", take_profit: ", take_profit, ", comment: ", comment, ", stop_price: ", stop_price, ", expire_time: ", expire_time)

    return trade_result.order
```

If the order is successfully placed, the "place_order" function will return a ticket for the trade.

### Closing an order
Once we have an order in place, we may expect two outcomes:
- The signal is valid, and the price moves in the same direction as the signal indicates. In this case, we could exit: 1) if it reaches the price target; 2) when the trend reverses.

- The signal is not valid, and the price moves in the opposite direction than we expect. In this case, we should exit if the price goes beyond the stop-loss.

The stop-loss can be pre-calculated by the trading strategy or a pre-set threshold if the loss reach x percentage of the total value.

To close an existing order, what we have do is just to create a reverse order and include the ticket number as well. So if the order is a long position, we will create a sell order.

```python
def reverse_order_type(type):
    # if the order is a buy order, we will create a sell order
    # adjust the function based on your need, you may want to create a limit order instead of a market order
    if type == mt5.ORDER_TYPE_BUY:
        return mt5.ORDER_TYPE_SELL
    else
        return mt5.ORDER_TYPE_BUY

def close_order(symbol, type, price, volume, ticket, comment):
    trade_request = {
        "action": mt5.TRADE_ACTION_DEAL,
        "symbol": symbol,
        "volume": float(volume),
        "type": reverse_order_type(type),
        "type_time": mt5.ORDER_TIME_GTC,
        "type_filling": mt5.ORDER_FILLING_IOC, # mt5.ORDER_FILLING_RETURN,
        "deviation": 20,
        "magic": 0,
        "position": int(ticket),
    }
    trade_result = mt5.order_send(trade_request)

    # if the trade_request is not constructed properly, the trade_result will be None
    # for example, the ticket is not valid (e.g. it is a string not an integer)
    # and vloume is not a float
    if trade_result is None or trade_result.retcode != mt5.TRADE_RETCODE_DONE:
        print ("Failed to close order: ", ticket)
        return False
    return True
```

Please note there are at least three types of filling for opening or closing order: 1) 'ORDER_FILLING_IOC' means that after whatever maximum availabe volume get filled in the market, the remaining volume will be cancelled; 2) 'ORDER_FILLING_FOK' means an order has to be executed as a whole with the specified volume, so it is basically "deal or no deal"; 3) 'ORDER_FILLING_RETURN' can do other types can't do, for example, if the order is partially executed and the rest of the order won't be cancelled. However, not all brokers support this type of filling. It is understandable that the broker may not need the 'ORDER_FILLING_RETURN' filling type because for forex, metals, and other instruments, the volume is usually not a problem and for the share CFD products, basically it is you against the broker so as long as within the maximum volume you can trade, it can be garanteed that your order will be filled when the order price matches. 


### Conclusion

In this article, we have discussed the core functionalities of a trading gateway using the MetaTrader's python library. The core idea is that it is intended to reduce the dependency on MetaTrader platform, but also maintain the stability and reliability of such a program (the gateway) in the long term. We have discussed the initialzation of the MetaTrader terminal, getting price updates, placing a new order, and closing an existing order. After the implementation of these core functionalities, the gateway can be used as a bridge between the MetaTrader 5 trading platform and an existing trading system (bot) developed in any language (e.g., NodeJS, C++, C#, Java) once a messaging mechanism is in place.

### Further Reading

We will continue this topic in part II. In next part we will present a complete implementation of such a gateway for trading bitcoin with a messaging server for the communciation between the trading system and the gateway. Setting up a development environment for futher implementation of building a complete trading system will be also discussed.

## Contact

<ContactFooter />

{/* For further information, I can be reach via:

- X (formally twitter): [Eric Tang](https://x.com/_e_tang)
- email: [eric@tyo.com.au](mailto:eric@tyo.com.au) */}

3c973ca0827cf0b78701e312d02e2d77dec4b900451c18fda0fe7311de026ffc

In this article, we will try to develope a trading gateway (or proxy agent) that is intended to reduce the dependency on MetaTrader platform, but also maintain the stability and reliability of such a program in the long term. When the gateway is running or live trading is in action, we don't want the unpredicted interrupt the whole workflow: our trading system (bot) could stop receving quote updates all of a sudden, or it could not place order or receive order status information anymore without an update to the trading program.

A tutorial about implementing a trading gateway with MetaTrader 5 Python library. 

Building a Trading Gateway (Bot) with MetaTrader 5 Python Library - Part 1


## Introduction

cTrader is the second most popular trading platform after MetaTrader. It is a powerful trading platform that is designed for traders of all levels. It is a platform that is easy to use and has a lot of features that can help you trade better. In this post, we will explore the options of trading with cTrader as your trading platform.

One major advantage of cTrader over MetaTrader is that it provides official libraries for algorithmic trading. This means that you can easily create your own trading robots and indicators using the cTrader API. This is a huge advantage for traders who want to automate their trading strategies.

Unfortunately, compared to MetaTrader, cTrader has fewer brokers that support it. Also, even when a broker supports cTrader, the available symbols (tickers) for trading are far fewer than what are offered on MetaTrader. Nevertherless, cTrader is still a great platform for traders who want to trade manually or use algorithmic trading.

Below is a screenshot of the cTrader platform:

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/trade-with-ctrader/2024-05-16-15-36-09.png)

## Broker Support

One of the most important things to consider when choosing a trading platform is the broker support. cTrader has fewer brokers that support it compared to MetaTrader. However, there are still a number of reputable brokers that offer cTrader as a trading platform. Some of the most popular brokers that support cTrader include:
- [Pepperstone](https://pepperstone.com)
- [IC Markets](https://icmarkets.com)
- [FPMarkets](https://fpmarkets.com)
- [Go Markets](https://gomarkets.com)
- [BlackBull Markets](https://blackbull.com)
- [FxPro](https://fxpro.com) (FIX API not available or blocked at least by default). 

Personally, I won't choose any other brokers other than those above. They are reputable and have been in the industry for a long time. They also offer a wide range of trading instruments and have good customer support.

### Available Symbols

One of the major disadvatanges about cTrader is that there are fewer symbols available for trading compared to MetaTrader. And basically, stock trading on cTrader is only availabe for a couple of brokers. Anyway, there are still a number of symbols available for trading on cTrader. As FxPro dosn't allow connection via FIX API, the tradable symbols information cannot be collected through the FIX API gateway software. So with FxPro's data excluded, below are the lists of the statistics for the available symbols in different markets, but not all categories, such as bond, EFT and futures, are listed.

#### Total Number of Symbols


{/* | Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|-----------|
| 346         |  123       |  1000     |  77       | 2101     |  */}


<table>
    <tr>
        <td>Pepperstone</td>
        <td>IC Markets</td>
        <td>FPMarkets</td>
        <td>Go Markets</td>
        <td>BlackBull</td>
    </tr>
    <tr>
        <td>346</td>
        <td>123</td>
        <td>1000</td>
        <td>77</td>
        <td>2101</td>
    </tr>
</table>


Please note all the data are collected via FIX API. Some symbols may not be available for trading due to the broker's restrictions.

#### Forex Pairs


{/* | Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|-----------|
| 75         | 62          |   65      |  45       |  55 | */}


<table>
    <tr>
        <td>Pepperstone</td>
        <td>IC Markets</td>
        <td>FPMarkets</td>
        <td>Go Markets</td>
        <td>BlackBull</td>
    </tr>
    <tr>
        <td>75</td>
        <td>62</td>
        <td>65</td>
        <td>45</td>
        <td>55</td>
    </tr>
</table>



#### Commodities

{/* | Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|-----------|
|             |            |   7       |           |         | */}

<table>
    <tr>
        <td>Pepperstone</td>
        <td>IC Markets</td>
        <td>FPMarkets</td>
        <td>Go Markets</td>
        <td>BlackBull</td>
    </tr>
    <tr>
        <td></td>
        <td></td>
        <td>7</td>
        <td></td>
        <td></td>
    </tr>
</table>

Gasoline is considered as a commodity in this category.

#### Metals


{/* | Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|-----------|
| 13          |  11        |   13      |       2   |        7  | */}

<table>
    <tr>
        <td>Pepperstone</td>
        <td>IC Markets</td>
        <td>FPMarkets</td>
        <td>Go Markets</td>
        <td>BlackBull</td>
    </tr>
    <tr>
        <td>13</td>
        <td>11</td>
        <td>13</td>
        <td>2</td>
        <td>7</td>
    </tr>
</table>



#### Indices

{/* | Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|-----------|
| 20         |   26     |   16    |   13   |   16   | */}

<table>
    <tr>
        <td>Pepperstone</td>
        <td>IC Markets</td>
        <td>FPMarkets</td>
        <td>Go Markets</td>
        <td>BlackBull</td>
    </tr>
    <tr>
        <td>20</td>
        <td>26</td>
        <td>16</td>
        <td>13</td>
        <td>16</td>
    </tr>
</table>



#### Energies

{/* | Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|-----------|
|        |  5    |   5    |   2   |   2   | */}

<table>
    <tr>
        <td>Pepperstone</td>
        <td>IC Markets</td>
        <td>FPMarkets</td>
        <td>Go Markets</td>
        <td>BlackBull</td>
    </tr>
    <tr>
        <td></td>
        <td>5</td>
        <td>5</td>
        <td>2</td>
        <td>2</td>
    </tr>
</table>

In this category, oil and gas are included.

#### Cryptocurrencies


{/* | Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|-----------|
| 10       |  4    |   6    |  7    |  26 | */}

<table>
    <tr>
        <td>Pepperstone</td>
        <td>IC Markets</td>
        <td>FPMarkets</td>
        <td>Go Markets</td>
        <td>BlackBull</td>
    </tr>
    <tr>
        <td>10</td>
        <td>4</td>
        <td>6</td>
        <td>7</td>
        <td>26</td>
    </tr>
</table>

{/* #### Futures
| Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|
| 42       |      |       |      | 30 |*/}

#### Stocks


{/* | Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|-----------|
| nyse        |            |  nyse     |           |  nyse       |
| nasdaq      |            |  nasdaq   |           |  nasdaq     |
|             |            |  ams      |           |  ams       |
|             |            |  mad      |           |         |
|             |            |  etr      |            |  etr    |
|             |            |           |           |   bme      | */}

<table>
    <tr>
        <td>Pepperstone</td>
        <td>IC Markets</td>
        <td>FPMarkets</td>
        <td>Go Markets</td>
        <td>BlackBull</td>
    </tr>
    <tr>
        <td>nyse</td>
        <td></td>
        <td>nyse</td>
        <td></td>
        <td>nyse</td>
    </tr>
    <tr>
        <td>nasdaq</td>
        <td></td>
        <td>nasdaq</td>
        <td></td>
        <td>nasdaq</td>
    </tr>
    <tr>
        <td></td>
        <td></td>
        <td>ams</td>
        <td></td>
        <td>ams</td>
    </tr>
    <tr>
        <td></td>
        <td></td>
        <td>mad</td>
        <td></td>
        <td></td>
    </tr>
    <tr>
        <td></td>
        <td></td>
        <td>etr</td>
        <td></td>
        <td>etr</td>
    </tr>
    <tr>
        <td></td>
        <td></td>
        <td></td>
        <td></td>
        <td>bme</td>
    </tr>
        <tr>
        <td></td>
        <td></td>
        <td>hkg</td>
        <td></td>
        <td></td>
    </tr>
</table>


##### NYSE Stock Market


{/* | Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|-----------|
| 55          |            |   12       |          |   980      | */}

<table>
    <tr>
        <td>Pepperstone</td>
        <td>IC Markets</td>
        <td>FPMarkets</td>
        <td>Go Markets</td>
        <td>BlackBull</td>
    </tr>
    <tr>
        <td>55</td>
        <td></td>
        <td>12</td>
        <td></td>
        <td>980</td>
    </tr>
</table>

##### NASDAQ Stock Market


{/* | Pepperstone | IC Markets | FPMarkets | Go Markets | BlackBull |
|-------------|------------|-----------|-----------|-----------|
|  40         |            |   7      |           |   745      | */}
<table>
    <tr>
        <td>Pepperstone</td>
        <td>IC Markets</td>
        <td>FPMarkets</td>
        <td>Go Markets</td>
        <td>BlackBull</td>
    </tr>
    <tr>
        <td>40</td>
        <td></td>
        <td>7</td>
        <td></td>
        <td>745</td>
    </tr>
</table>

Not all categories, such as bond, EFT and futures, are listed.

Please note the above statistics are based on the data collected in May 2024. The number of symbols may change over time, and the numbers were compiled with the best effort and may not be exactly accurate. It only serves as a reference for what roughly you can trade with cTrader with these brokers. 

Basically if simply based on the number of symbols, FPMarkets, Pepperstone and BlackBull are good choices. Particularly, BlackBull has the crasy number of symbols that you can trade, and they actually more symbols on cTrader than on MetaTrader. If those brokers don't suit you , all other brokers are good too, just that they have fewer symbols available for trading, and oftenly you won't need to trade as that many as tickers anyway. It is surely hoped that the number of symbols available particularly for the stock markets (i.e. ASX) for trading on cTrader will increase in the future.

## algorithmic Trading Support

There are two options when it comes to algorithmic trading with cTrader. 

The first option is to use the cTrader's a complete package of automate trading libraries. This is a great option for traders who are new to algorithmic trading and want to create simple trading robots. However, there is a bit of learning curve to use the cTrader's libraries. Also, the large part of your trading software may not be resued for other trading platforms.

The second options is to use a good FIX API library such as QuickFIX. In this case, you don't have to reply on what cTrader's maker can provide. What you have to do is to talk to cTrader's trading server or any FIX API enabled server which is provided by the broker. This is a great option for traders who are experienced in algorithmic trading and want to create complex trading robots. FIX API is a standard protocol that is used by many brokers and trading platforms. This means that you can easily reuse your trading software for other trading platforms.

If you are intereste in the second option, you can have a look at another relevant article: [Trading with FIX API in C#](/blog/fix-api-gateway-in-csharp/). It mainly talks about how to use FIX API in C# but the idea is the same and it follows the same FIX protocol which is programming language neutral. Also, if you are interested building a general gateway for algorithmic trading, you can have a look at this post [Building an Efficient Gateway: Connecting Your Trading System to Brokers](/blog/introduction-about-trading-gateway/).

FxPro provides cTrader but it doesn't allow connection via its FIX API. If you are trying to connect to cTrader via FIX API with FxPro's settings, you may get the "RET_ACCOUNT_DISABLED" error. So, if you are interested in algorithmic trading, you may want to consider other brokers that support cTrader and FIX API.

## Conclusion
Luckily, there are still a few reputable brokers that support cTrader as a trading platform. If you are looking for a powerful and easy-to-use trading platform, cTrader is definitely worth considering. Whether you are a beginner or an experienced trader, cTrader has a lot of features that can help you trade better. If you are interested in algorithmic trading, cTrader is a great platform to use as it provides official libraries for creating trading robots and indicators. Overall, cTrader is a great platform for traders of all levels.

## Credits

Photo by <a href="https://unsplash.com/@isaacmsmith?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Isaac Smith</a> on <a href="https://unsplash.com/photos/pen-on-paper-6EnTPvPPL6I?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>
  

4642c5e00ba8e28e593bc002f2b8e10fb96fee90554bd5ff78f1a0e9b8bc4d6d

cTrader is the second popular trading platform after MetaTrader. It is a powerful trading platform that is designed for traders of all levels. It is a platform that is easy to use and has a lot of features that can help you trade better. In this post, we will explore the options of trading with cTrader as your trading platform.

Explore the options of trading with cTrader as your trading platform

Tag: Trading