Financial Times essay urges universities to teach AI testing skills

In short: Some educators are calling for universities to teach students how to test AI systems, not just use or ban them.

What's going on

A new opinion essay in the Financial Times argues that universities should make sure students learn how to evaluate AI tools. The writer, Andrew Hall, teaches at Stanford Graduate School of Business.

He focuses on “evals,” short for evaluations. An eval is a repeatable test that checks how an AI model behaves in a specific situation, like a driving test checks whether someone can handle real road rules and surprises.

Hall says many schools are restricting or banning AI in class because of cheating concerns. He argues that this can leave students unprepared for jobs where AI tools will be common.

The essay also points to an idea described by Microsoft CEO Satya Nadella. He describes a “loop” where an organization combines its own expertise with AI and then regularly checks whether the AI still meets its needs. In practice, that means constantly testing the tool, improving how it is used, and being able to switch from one AI model to another without losing quality.

Hall describes building a “dictatorship eval,” which tested whether leading AI models would help a would-be autocrat. He says Anthropic later included his eval in its “system cards,” which are documents that summarize a model’s risks and safety measures.

What to watch

If more universities teach eval-building, more graduates may enter the workforce able to spot when AI is wrong, unsafe, or biased. Hall argues this could widen a divide between people who can measure and control AI tools, and people who can only accept whatever the tools output.

Source: Financial Times