Vals raises $40 million to grow independent AI model testing

In short: Vals, a startup that tests how well AI models perform, raised $40 million in Series A funding led by Andreessen Horowitz.

What happened

Vals is an AI testing company founded in 2024. It says many common AI “benchmarks” are no longer reliable. Benchmarks are standardized tests for AI models, like giving different students the same exam to compare scores.

According to Vals, some benchmarks have become easier to “game.” If test questions are public, companies can train their AI on those exact questions, which is like memorizing the answer sheet instead of learning the subject.

Vals says it keeps its test materials private and focuses on real work tasks. It evaluates whether an AI model can complete complex jobs in areas like law, finance, and coding. It also looks for potential downsides, including work related to mental health, cybersecurity, biosecurity, and even how models handle rules like the Geneva Convention.

The company makes money by charging other companies to evaluate their models. Vals co-founder Rayan Krishnan compared this to paying to take the SAT, where the test helps show what you are good at and what needs work.

Vals also says its revenue is eight times higher than last year. Its team grew from 8 people to 25, and it plans to hire 10 to 15 more. The company recently launched a program to provide evaluations to US federal agencies.

Why it matters

As AI tools show up in more products and workplaces, it becomes harder for non-experts to know which claims to trust. More independent, harder-to-game testing could make it clearer whether an AI system is actually safe and useful for the job it is sold for.

Source: TechCrunch AI