Hugging Face policy chief warns AI benchmarking and embedded evaluators are not settled

In short: Hugging Face’s chief policy officer said today’s AI scoring tests and the push for “embedded evaluators” are still messy and hard to standardize.

What's going on

AI companies often use benchmarks, meaning standardized tests that score AI systems, to claim their product is better than a rival. Irene Solaiman, Hugging Face’s chief policy officer, said at TechCrunch Disrupt that these tests are still “an evolving science,” and the industry is not yet good at comparing results across different tests.

A key issue is reproducibility. That means someone else can repeat the same test and get the same result, like following a recipe and getting the same cake. Solaiman said reproducibility depends partly on access to the model itself, and she pointed to “open weights,” which is when a company shares the main settings of an AI model so others can inspect and run it.

Solaiman also questioned how well “embedded evaluators” will work. This idea puts third-party reviewers inside AI labs to check safety, instead of relying on government rules. She said there is “hunger” for this approach, but the industry lacks clear standards for who is qualified to evaluate which tasks, and how long those evaluations stay useful.

Another complication is that evaluations are shifting from testing just the model to testing the whole setup around it. Solaiman said tools used to access a model, sometimes called a harness (like the dashboard and controls in a car), can change performance. A weaker model with better tools could outperform a stronger model with worse tools.

What to watch

If AI companies want the public, customers, and regulators to take benchmark scores seriously, they may need shared rules on how tests are run, what gets measured, and who can independently check the work. Expect more debate over access to models, and whether embedded evaluators can be truly neutral.

Source: TechCrunch AI