ZML releases free ZML/LLMD to run AI models faster on many chips

In short: ZML has launched a free product called ZML/LLMD that helps run AI models faster across many different AI chips.

What happened

ZML, a Paris-based AI startup, released ZML/LLMD, which it calls an “inference server.” Inference is the step where an AI model answers your prompt, like when a chatbot writes a reply.

The company says ZML/LLMD can run a range of open source large language models on many kinds of chips. These include chips from Nvidia and AMD, plus Google TPUs, Apple Metal, and Intel Arc. The goal is to make it easier for companies to use the chips they already have, instead of being tied to only one vendor.

ZML founder Steeve Morin told TechCrunch that software barriers often stop people from mixing and matching chips. He argues that improving inference speed and efficiency matters more and more as AI features show up in everyday products.

ZML/LLMD is not open source, unlike an earlier ZML project released in 2024. Still, it is launching as free to use, with ZML saying it wants to learn how people use it before deciding how to charge.

ZML has raised $20 million from venture firms, and it has backers that include well-known tech founders. The startup is also publicly endorsed by Yann LeCun, a Turing Award winner.

Why it matters

Running AI costs money and electricity. If tools like ZML/LLMD let businesses use cheaper or lower-power chips, it could lower the cost of AI services. Think of it like an adapter that lets the same appliance plug into different outlets (so you are not forced to buy one brand of outlet).

Source: TechCrunch AI