Researchers in South Korea and the US found that in open-weight models such as Qwen3 and Llama 4, adjusting the activation of a single neuron changes the model's recommendations for individual stocks and entire portfolios, with no change to the prompt or the parameters.

The researchers identified one neuron in each of several open-weight models, including Qwen3, Llama 4, DeepSeek R1, Gemma 4 and Mistral Small, and found that tweaking its activation changed what the model recommended for individual stocks and whole portfolios, with not a word of the prompt altered and no parameters updated.
That is more unsettling than plain model bias: with a closed-weight model you cannot touch the activation, and you do not know which way it leans; with an open-weight one you can, and the lean you get is the one whoever tuned it wanted.
An adviser using these models may hand out advice that has nothing to do with the investor's own preferences.



Curated from high-quality sources, with concise summaries and key takeaways.