CH NEO-ZÜRICH EDITION
WEATHER · CLEAR 14°C
BLEND OF THE DAY · 07/ROGUE
EST. 2027
THE AEC CYBER MORNING NEWS

PAZ Kaffi

DESIGN · DEMOLITION · CAFFEINE · DISPATCH
EDITION 0906 · 6 September 2026
BROADCAST 04:42 CET
2,400 BROADSHEETS PRINTED
READ TIME · 47 MIN
Your backtest is lying to you — build the 40-line Python bot that proves it
AI
FRAME · 06:55
06-09-2026

Your backtest is lying to you — build the 40-line Python bot that proves it

A Sunday pandas tutorial on the three ways a backtest fools you — lookahead, survivorship, overfitting — and the one-line fix every builder's model needs.

Start with the confession, because it is the whole piece. In AutoScientist-Quant (arXiv 2608.28632, v1 first posted 5 August 2026, v2 revised 1 September 2026), Zongqian Li, Yaoyiran Li, Nigel Collier, Eugene Ie and co-authors wrote that they had to “fix two lookahead problems” in an evaluation pipeline they had reused from prior work — and then keep “the feedback window disjoint from the held out test window.” Read that twice. Results had already been published on top of that pipeline. The exact bug this Sunday build teaches you to catch in forty lines of your own Python was sitting, unnoticed, in the shared plumbing of a frontier quant lab. That is the licence for a beginner tutorial that isn’t condescending: the bug does not care how senior you are.

The map for the field comes from a companion survey — Fengrui Hua, Hengyi Yang, Jia Li, Jian Guo and colleagues (eight authors in all), arXiv 2608.31041v1, submitted 31 August 2026. They decompose agentic quantitative trading into five named stages: factor mining, signal discovery, portfolio construction, order execution, and risk management. Their honest finding is that everyone crowds the same stage: systems “remain concentrated on signal discovery, while complete integration with portfolio construction, execution, and risk control is still uncommon.” We are going to be honest in the same direction. This tutorial builds a toy of stage two (signal discovery) and a sliver of stage five (risk as validation). Stages three and four — where real money is actually won or lost — we leave alone.

The survey’s sharpest sentence is the one to pin above your desk: “strong model or forecasting capability does not reliably translate into trading performance under live market conditions.” A better predictor is not a better system. Any engineer who has watched a gorgeous simulation meet a real building knows exactly that sinking feeling.

←TODAY: In 2026 a quant lab had to fix two lookahead bugs in a pipeline others had already published on. →3012: The offices that survive to Zurich-3012 are the ones that held out a window and never peeked. Fulcrum: A backtest and an energy model fail the same way — both grade the exam using the answer key.

The three lies, with their real names

Lookahead (peeking). Your signal at time t is computed with information that did not exist at t — most often by earning the return of the very bar whose close produced the signal. A moving-average crossover computed on today’s close cannot be traded at today’s close; you trade tomorrow. One misaligned row turns a losing rule into a winner.

Survivorship. The dataset holds the names still listed today. Everything delisted, bankrupt, or acquired was quietly removed — so the universe you test on is defined by the outcome you are trying to predict.

Overfitting the report window. You tune parameters (20 and 50, say) on a stretch of history, then report on that same stretch. With enough combinations, a random rule looks brilliant on any fixed window. The remedy, named plainly in AutoScientist-Quant, is to keep the feedback window disjoint from the held-out test window: tune on one, report on the other, and do not look at the second until the end.

The Tool: pandas — created by Wes McKinney in 2008, now maintained by the open-source pandas team. No trading library, no broker, no API key. The whole lesson lives in how pandas aligns two Series by their index, and one wrong .shift() is the entire bug. If you can run pandas, you can see the lie with your own eyes in an afternoon.

Setup:

python -m venv .venv
.venv\Scripts\activate            # Windows PowerShell
pip install pandas numpy
python -c "import pandas, numpy; print(pandas.__version__)"

First steps:

  1. Make a price series with a synthetic random walk — no real tickers, no data vendor. This is deliberate: a random walk has no signal, so any “profit” you see is the bug talking, not skill. It also sidesteps survivorship entirely, because you never touched a list of survivors.
  2. Compute the 20/50 moving-average crossover as your signal (1 when fast is above slow, else 0).
  3. Score it two ways: earning the next bar’s return (honest) versus the same bar’s return (cheating). Print both sums.
  4. Watch the cheating line look like a strategy and the honest line drift toward zero — or below it, after you subtract a realistic cost per trade. That gap is the payoff.

Atelier: Every builder already runs backtests and calls them something else. The energy model calibrated against the same year it is then asked to predict — that is training on the test window. The cost estimate built from the practice’s completed projects — the tenders you lost are not in that dataset, and neither are the jobs so bad nobody archived them; that is survivorship, exactly. The programme assembled from historical trade durations on sites that finished, then nudged after the fact with knowledge of how the job actually went — that is lookahead. Monday move: take the practice’s last ten projects, tune the cost estimate on seven, freeze the number, and do not open the other three until it is locked. Nobody enjoys that afternoon. It is the only one that tells you anything.

Hack: Shift the forward return by one bar before you multiply — that single .shift(-1) is the line between an honest loop and a peek. Here px is a pandas Series of daily closes; the whole lesson is the difference between the last two lines.

sig = (px.rolling(20).mean() > px.rolling(50).mean()).astype(int)
ret_fwd = px.pct_change().shift(-1)            # return earned from t to t+1
honest = (sig * ret_fwd).dropna()             # decide at t, earn t+1
cheating = (sig * px.pct_change()).dropna()   # earns the bar that made the signal

Why it works: px.pct_change() at row t is the return from t−1 to t — the bar whose close produced the signal. .shift(-1) slides the tt+1 return onto row t, so multiplying by sig means “the decision made at t earns the next bar.” Multiply sig by the unshifted pct_change() and you have paid yourself for information you did not have. The two sums are usually not close, and the gap is the article.

Say it plainly, because it is the guardrail and the moral: the honest version of this toy loses money. That is the expected result and the entire point. A tutorial that ends on a profitable backtest has taught you the wrong lesson. This is validation practice, not investment advice — paper and synthetic data only, no tickers, no strategy, no real capital. A profitable backtest just means you haven’t found the bug yet.

If you want to see the self-hosted end of this world, a Show HN from 27 July 2026 (“AlgoDeploy — Python algo trading you run yourself, wired to Alpaca and IBKR”) exists for running Python algo code yourself. Treat it as a signpost, not an endorsement — it drew two points and no comments, and carries no weight here.

Move: Clone nothing. Open one .py file, paste the four Hack lines around a random walk, and run it before your coffee is cold. Then go find the nearest spreadsheet in your office that grades itself on its own answer key — and split it.

Learn-it:

FILED FROM
CO-SIGNERS
PAZ Academy
CONFIDENCE
HIGH
REPRINTS
© PAZ - PARAMETRIC ACADEMY ZURICH · ALL RIGHTS RESERVED

SOURCE ·

PAZ Kaffi · multidisciplinary editorial, led by PAZ Academy

⚑ REPORT AN ERROR · SUBMIT A CORRECTION
◂ BACK TO FRONT PAGE · PAZ KAFFI

© 2026 PAZ Academy.