Researchers warn llms.txt files can make AI agents install unowned code

In short: Some company websites posted AI-readable documentation that included install commands for software packages nobody owned, and AI coding agents sometimes ran them.

What's going on

Researchers reported that more than 100 websites had documentation files called llms.txt and llms-full.txt that referenced software packages or web domains that were not registered. These files are meant to help AI systems read a site, similar to robots.txt, which tells search engines what to scan.

A team at a stealth startup in Israel scanned 6,214 domains tied to large companies, including defense contractors and Fortune 500 firms. They found 8,265 llms.txt and llms-full.txt files in total. Out of those, 120 files, each on a different site, contained 227 commands that pointed to unclaimed package names, like “pip install …” or “npm install …” (pip and npm are common app stores for code).

To test the risk, the researchers registered some of the unclaimed names and hosted harmless “beacon” code that only called back to their server, like a doorbell that rings when someone enters. They say they received these call-backs from a few dozen companies. The process information suggested AI coding agents, including Claude, OpenAI’s Codex, and Nous Research’s Hermes, were involved. Anthropic, OpenAI, and Nous Research did not respond to Ars Technica by publication time.

The researchers also found at least one case that pointed to real malware. A command on clerk.com used “npx,” which can download and run a package right away. The researchers said someone had claimed the previously unowned package slot and used it for malware. Clerk said it has resolved the issue.

What to watch

The key risk is simple: if an AI agent is allowed to run commands, it may treat a web page as an instruction list. Companies may need to review these AI-focused files and make sure any package or link they mention is actually owned and safe.

Source: Arstechnica