Study finds AI models disagree on which jobs are exposed to AI

In short: A new study finds that different AI models give very different estimates of which jobs could be affected by today’s AI tools.

What's going on

Many studies about AI and jobs start with a basic step, they score each occupation for how “exposed” it is to AI. “Exposed” here means how much the tasks in that job could be done faster with current, everyday AI tools (like using a writing assistant to draft text).

The Financial Times reports that one widely used set of exposure scores came from a 2024 study by researchers at OpenAI. In that work, GPT-4, an AI model, judged whether AI could handle thousands of work tasks.

A new paper by Northwestern University researcher Michelle Yin tested how consistent those judgments are. She asked four different AI models, including GPT-4 plus newer models from OpenAI, Anthropic (Claude), and Google (Gemini), to score 705 US occupations using the same method.

Yin found big disagreements. Depending on the model, the estimated share of at-risk jobs ranged from under 15% (Gemini) to 50% (Claude). For some white-collar jobs, the differences were extreme. Economists were rated about 10% exposed by GPT-4 in the original OpenAI study, just over 50% by GPT-5, and 80% by Claude.

What to watch

These gaps matter because they can change the conclusion of follow-on research. Using one set of scores suggests AI has had a weak negative effect on employment, while using Gemini’s scores flips that to a weak positive effect. Yin suggests a simple fix, researchers should run analyses using several models, like getting multiple second opinions before making a big decision.

Source: Financial Times