Researchers say OpenAI math proofs do not meet field guidelines yet

In short: OpenAI released hundreds of claimed solutions to hard math problems, and researchers say the release does not fully follow new community guidelines for trust and review.

What's going on

OpenAI published 719 manuscripts that it says contain solutions, or “proofs,” for difficult open math problems. A proof is the step by step reasoning that shows a result is true, like showing your work on a test.

OpenAI said it consulted a group called the Advisory Group on Mathematics and Artificial Intelligence (AGMAI), hosted by Princeton University’s Institute for Advanced Study. AGMAI recently published guidelines for companies that use AI to work on advanced math.

TechCrunch reports that OpenAI’s release appears to miss key parts of those guidelines. AGMAI’s first request was to stop testing open research problems on proprietary models, meaning systems outsiders cannot fully inspect. OpenAI’s release says it is evaluating its proprietary models using open math problems.

The guidelines also stress the need for human understanding. Only 10 of the 719 manuscripts included the model’s “chain of thought,” meaning the detailed internal reasoning the model used to reach an answer.

Another issue is “formalizing” proofs, which means translating them into a strict computer-checked format. OpenAI used Lean, a programming language used to check proofs like a spellchecker for logic. A separate paper from researchers at the University of Cambridge and King’s College London reported discrepancies between the plain English explanation and the Lean code in an OpenAI-linked solution related to the Navier-Stokes equations.

What to watch

Mathematicians like Terence Tao and Harvard professor Melanie Wood argue that AI results still need the same careful peer review and follow-up work humans do. The next question is whether OpenAI and other labs will share more supporting details, fund human review, and make it easier to match each written proof to its computer-checked version.

Source: TechCrunch AI