Lessons › AI tools for traders › Backtesting an idea with AI-written code

World 8 · AI tools for traders · lesson 12 · level 4

Backtesting an idea with AI-written code

AI can write test code fast, but you must check that the code only uses information that was known at the time.

In one line

A backtest that never loses almost always has a bug. Where is it hiding?

Explained simply

A backtest is like replaying last season's football matches to test a strategy. If your notes accidentally include the final scores before each kick-off, your strategy will look perfect. Look-ahead bias is exactly that: the test peeks at information it could not have had.

The lesson

A backtest runs a rule over historical data to see how it would have performed. AI assistants can draft the code, but they often make errors such as deciding a trade at the open using that day's closing price, a mistake called look-ahead bias, or leaving out costs such as fees and slippage (getting a worse price than planned). Other common bugs are impossible position sizes, gaps in the data, and testing only companies that still exist today. Checking a few trades by hand against the chart is the fastest way to catch these bugs.

A worked example

Illustrative example over four days, open to close: day 1 went 100 to 104, day 2 went 104 to 103, day 3 went 103 to 106, and day 4 went 106 to 105. The AI's code buys at the open whenever that same day closes higher, so it only 'takes' days 1 and 3 and wins 4 + 3 = 7 points with no losses. But nobody knows the close at the open. The fixed code buys at the open only if yesterday closed higher, so it trades days 2 and 4: (103 - 104) + (105 - 106) = -2 points. With costs of 0.5 point per trade, that becomes -2 - 2 x 0.5 = -3 points.

The same idea at four levels

1. Beginner. A backtest tests a trading rule on old price data.
2. Foundation. AI can write the test code quickly, but the code can hide bugs that make results look too good.
3. Intermediate. Look-ahead bias means the code uses information from the future, such as today's close to decide a trade at today's open.
4. Advanced. Read what each line does, add realistic costs and slippage, and hand-check a few trades on the chart, because a perfectly smooth equity curve is a warning sign, not proof.
5. Expert. Also check for survivorship bias from testing only today's surviving stocks, for missing or wrongly adjusted data, and for sizes you could not really trade, such as fractions of a contract.

Mistakes to avoid

• Believing a backtest because its equity curve looks smooth.
• Running AI-written code without reading or testing what it does.
• Leaving out fees and slippage, which can turn a small edge negative.

Goal of this lesson: Use AI to draft simple backtest code, and check it for common errors before trusting the results.