OpenAI's maths solutions fail to meet standards set by experts

Summarised from 2 outlets · Updated 9 Oct, 20:25 · Archive

OpenAI released hundreds of claimed mathematical proofs this week but fell short of guidelines set by an advisory group of mathematicians.

OpenAI released nearly 400 AI-generated mathematical results spanning more than 700 manuscripts across multiple disciplines this week, including combinatorics, geometry, number theory and theoretical computer science. The lab said it had consulted the Advisory Group on Mathematics and Artificial Intelligence (AGMAI), hosted by the Institute for Advanced Study, to establish guidelines for solving mathematical problems responsibly.

However, TechCrunch reports that OpenAI fell short of those standards in several ways. AGMAI's first recommendation was for frontier labs to stop testing advanced mathematical problems on proprietary models, yet OpenAI's release explicitly states it evaluated proprietary models using open research problems. The advisory group emphasised the need for human understanding of mathematical results, an area where OpenAI particularly fell short.

More than three dozen mathematicians told The Verge they found the scale of the release 'overwhelming' and 'unprecedented'. Most said simply digesting the collection's roughly 40-page table of contents and abstracts took the better part of an hour. Researchers agreed that understanding what OpenAI had released could take years, with many expressing anxiety about what the company would release next before the mathematical community could properly assess the current batch.

How it is being reported

• ‘Pure insanity’: Mathematicians will need years to make sense of OpenAI’s latest drop"Staggering." "Overwhelming." "Unprecedented." "Surreal." "Pure insanity." Those were among the descriptions more than three dozen mathematicians reached for in conversations with The Verge as they tried to make sense of the flood of mathematical results OpenAI abruptly dropped on the field this week. Amid the awe, excitement, and uncertainty over the sheer scale of the […]The Verge · 9 Oct, 20:09
• OpenAI’s math solutions aren’t meeting the field’s standards yetOpenAI's flood of proofs deviated from the guidelines set by a group of mathematical researchers consulted by the frontier lab.TechCrunch · 8 Oct, 19:10

In this story: OpenAI · Advisory Group on Mathematics and Artificial Intelligence · Institute for Advanced Study · University of Connecticut

This summary was written by AI from the headlines and standfirsts above, and states as fact only what at least two outlets report. How we use AI · Report a problem