Models
OpenAI publishes hundreds of math results from a model it has not yet released
OpenAI has released solutions to open math problems produced by an internal model. So far there is no sign that mathematicians outside the company have checked them, and several criticize how they are presented.
The essentials
1 confirmed fact · 3 according to the source · 3 open questions
- OpenAI has published results on open math problems from an internal model it has not released, with proofs in Lean on GitHub.1
- According to the sourceAccording to The Verge, the batch comprises 722 manuscripts in 372 families of results; the advisory group AGMAI speaks of "hundreds" of open questions.3
- According to the sourceOpenAI says the model, which has been training since August 28, has solved more than 100 long-standing open problems.2
- According to the sourceAccording to WIRED, several mathematicians criticize companies for publishing through blog posts and announcements rather than scientific papers.2
Why it matters
If the results hold up, they would be a notable advance for AI in mathematics. But a proof only counts once other mathematicians review it, and that requires papers they can read and debate. That part of the community is complaining about the format suggests the debate is no longer only about what AI does, but about how companies describe what it does2.
The details
OpenAI has published new results on open math problems obtained by an internal model, and has shared on GitHub the proofs formalized in Lean (a language that lets a program check proofs) and details of the research1. According to The Verge, the batch contains 722 manuscripts grouped into 372 families of results. The independent advisory group AGMAI, made up of top mathematicians, says it includes solutions to "hundreds" of open questions3.
WIRED had reported in advance that OpenAI planned to dump hundreds of results on GitHub on Tuesday, according to people familiar with the plans. OpenAI spokesperson Lindsay McCallum told WIRED that the model began training on August 28 and has already solved more than 100 open problems. She added that the company is working on publishing the next results responsibly, relying on the advice and public recommendations of an advisory group from the Institute for Advanced Study, and that it had not set a date2.
According to WIRED, several mathematicians criticize OpenAI and Anthropic for publishing results through blog posts and announcements instead of scientific papers, which are easier to verify. Bryna Kra, of Northwestern University, says that in August the group of mathematicians asked OpenAI to publish papers and not just blog posts, and that, "apparently," that advice was "ignored"2.
What we don't know
- Whether mathematicians outside OpenAI have already checked the results.
- Whether OpenAI's figure (more than 100 problems) and AGMAI's ("hundreds" of open questions) measure the same thing.
- When the model will be released to the public, and whether it will ever be.
Sources
- 1Sharing AI progress in mathematics
- 2OpenAI Is Pissing Off a Bunch of Mathematicians—Again
- 3OpenAI drops another batch of mathematical breakthroughs
Was this story useful? Yes Not really