A Tsinghua team had ten large models trade China's stock market for about 16 months. Once style exposure was stripped out, only Claude Opus 4.7 kept a positive excess return, at 0.2%; the rest were negative.
The author argues that AI's promise of productivity gains is still far off where stock picking is concerned. A Tsinghua team let several models trade Chinese stocks to test whether they could pick winners from public information alone.
The backtest trap: the models may have 'peeked' at the future
Large models are trained on nearly all public information within their training window. If the backtest period overlaps with that window, the model effectively knows the future, and its results look good for the wrong reason. The researchers stripped stocks of identifying information and controlled for training-window effects and other contaminants.
The headline numbers look strong. Qwen 3.6 Plus returned 85.3% against 36.9% for the CSI 300, and five of the ten models beat the benchmark.
Once accidental style and factor exposures were removed, the picture changed sharply. Over the 16 months, Claude Opus 4.7's excess return, or alpha, was only 0.2%, while the rest were negative, with Gemini, DeepSeek, GLM and Kimi falling furthest.
In the end, these models seem to earn more from style and luck than from any real stock-picking skill.
Why it matters
A backtest that looks brilliant may only reflect what the model already absorbed in training. Any AI fund sold on historical returns deserves the same question, and the number to compare is alpha after style and luck are removed, not the flattering total-return line.
Curated from high-quality sources, with concise summaries and key takeaways.
