Anthropic seeks legal advice on how AI agents should negotiate

In short: Anthropic is consulting lawyers about how AI agents should negotiate deals for people.

What's going on

Anthropic, an AI company based in San Francisco, is in talks with legal groups about a new problem, how software “agents” should negotiate when they act for a person. An AI agent is a tool you direct to do a task, then it takes steps on its own, like a personal assistant that can make calls and follow up without you.

People familiar with the company’s thinking said Anthropic wants to learn how lawyers negotiate for clients while still following fair rules. The idea is that, in the future, many agents could talk to other agents and bargain over things like product prices, customer service disputes, or even salary offers.

Anthropic has been testing this in research. In one experiment, employees used Claude-powered agents to buy and sell books in a virtual trading space. The study found that agents told to be “ruthless” did slightly better, while more cooperative agents sometimes accepted worse outcomes so everyone still got a book.

Outside Anthropic, researchers have also studied agent negotiations. Harvard Business School professor Michelle Vaccaro ran more than 180,000 agent negotiations and found tactics that can weaken the other side’s ability to represent a user, including “prompt injection” (tricking an AI with carefully worded text to reveal private information or break rules).

What to watch

Rules for AI systems that buy, sell, or sign agreements for people are still unclear. A key concern is fairness, since people who can afford stronger agents could gain an advantage in negotiations, much like hiring a top lawyer or agent. Anthropic declined to comment.

Source: Financial Times