Study says Mistral AI models struggle to filter Russian disinformation

In short: An Estonian study found Mistral and other open-source AI models were less reliable at spotting and resisting Russian disinformation than many commercial AI systems.

What's going on

Researchers at the state-backed Institute of the Estonian Language tested 60 generative AI models (tools that write answers in plain text, like a very fast autocomplete). They checked how well each model could handle Russian propaganda and misleading claims.

In the results, the most advanced Mistral model ranked 47th out of 60. The study said all four Mistral versions it tested scored under 40 percent at identifying sources the researchers labeled as “malicious” Russian propaganda.

The researchers compared Mistral with other systems, including Anthropic’s Claude, Grok, and some Chinese models. They reported that Claude and several others did better at refusing false claims or correcting them.

The institute used 75 questions in English, Russian, and Estonian. It also tested whether models could be tricked by “malicious questions”, meaning prompts designed to push the AI into repeating propaganda (like leading a witness in court).

The study looked at 14 common propaganda themes, including claims about Nato expansion and statements suggesting Russians, Ukrainians, and Belarusians are not separate peoples.

Mistral said it takes disinformation seriously and keeps improving its protections. It also said the study tested its “raw models”, meaning versions before customers add extra controls and filters.

What to watch

This matters because governments and schools are starting to use AI tools for research and fact-checking. Estonia, for example, has contracts with OpenAI and Google for use in its school system, and the study may affect how public groups choose between open-source models (more like a kit you can modify) and commercial tools (more like a sealed appliance).

Source: Financial Times