AT&T shifts more AI work to open models to cut costs

In short: AT&T is sending more everyday AI tasks to open models to lower costs, while still using paid models from companies like OpenAI and Anthropic for tougher work.

What's going on

AT&T says open models now handle about 25% of its overall AI usage. Over time, it expects that share could rise to 70% to 80%.

The company processes roughly 45 billion AI tokens per day. Tokens are small chunks of text that AI reads and writes (think of them like the “word pieces” on an itemized bill). If more of that daily volume is handled by cheaper models, the total bill can drop a lot.

AT&T says it uses an internal routing layer that sends each request to the cheapest model that can do the job. It considers cost, speed, and quality, similar to how a delivery app might choose between a bike courier and a car depending on distance and urgency.

In some applications, AT&T says shifting work from closed, proprietary models to open models has cut costs by 80% to 90%. It has used open models for tasks like summarizing text, basic analysis, and telecom-specific workflows, while keeping premium models for more complex jobs like generating computer code.

There is an important nuance. “Cheap, freely available” can be misleading. Open-weight or open-source models are often free to download or run, but they still require computers, data center capacity, and staff to operate them.

What to watch

This does not mean AT&T has stopped using OpenAI or Anthropic. The clearer takeaway is that big companies are trying to limit how fast their spending on paid AI services grows by moving routine work to open models, and saving the expensive options for the hardest tasks.

Source: NYTimes