Report says US nearly boarded Chinese ship over false AI intel

In short: A CNN report, cited by Ars Technica, says the US military nearly intercepted and boarded a Chinese ship after an AI-assisted intelligence report turned out to be wrong.

What happened

CNN reported that a US Special Operations Command analyst produced an intelligence report claiming a Chinese ship was moving parts linked to a nuclear weapons program through the Middle East. CNN cited four sources familiar with the episode.

Based on that report, the US military reportedly began preparing to intercept and board the ship, with air support. The plan changed after officials discovered that a chatbot used while creating the report had misidentified what the ship was carrying.

CNN said the chatbot had combined information from public sources with classified “signals intelligence” (secretly collected communications or electronic signals). The final report was described as “entirely false,” and one source told CNN the incident “almost started a war.”

Why it matters

This episode is a high-stakes example of an AI “hallucination,” which is when a chatbot produces incorrect details that can sound confident, like a calculator giving an answer with the wrong numbers. It also comes as the US Department of Defense expands its use of generative AI tools, including a government version of Google’s Gemini and an option to use Grok through its GenAI.mil platform, according to Ars Technica. The more these tools are used in official workflows, the more important it becomes to double-check results before they drive real-world decisions.

Source: Arstechnica