2025 AI failures show weak controls, not just bad technology

FORMAT B - Trend

In short: A string of AI problems in 2025 showed that many failures came from poor planning, weak security, and lack of human checks.

What's going on

Several high profile AI incidents in 2025 highlighted how normal organizational mistakes can turn AI into a real risk. One example was a privacy breach tied to McDonald’s McHire hiring platform, where reports said 64 million job applications were exposed because the system used a simple default password, “123456”, and did not require multi factor authentication (a second login step, like a text code).

Other incidents involved people trusting AI results too much. Facial recognition tools were linked to wrongful arrests when an “algorithmic match” (a computer’s best guess) was treated like proof, without enough double checking. Separately, deepfake videos (realistic fake videos) were used to impersonate Canadian Prime Minister Mark Carney in investment scams, with seniors mentioned as key targets.

Security teams also warned that criminals can use general purpose AI systems to speed up harmful work. One reported case involved Anthropic’s model, Claude, being used to automate parts of cyber espionage. Another repeating issue was “hallucinations”, where AI tools confidently make things up, like fake legal citations or broken code, which can be dangerous if a company relies on the output.

What to watch

In 2026, more organizations are expected to move from small experiments to stricter rules and clearer ownership. That includes better data checks, audit trails (records of what happened), “kill switches” (a way to quickly turn a system off), and human review for higher risk uses. The basic lesson is simple, AI needs the same kind of controls you would expect for any system that handles money, personal data, or safety.

Source: Financial Times