OpenAI has released 722 mathematical papers covering proofs and disproofs of a wide range of problems, a scale that has astonished mathematicians while also prompting caution over how the results can be assessed and incorporated into established mathematical knowledge.
The company has not identified the artificial intelligence system responsible for the work, describing it only as an “internal frontier model”. The release follows a rapid increase in the ability of AI systems to tackle difficult mathematics. In 2019, AI models struggled to achieve a passing grade on GCSE mathematics papers. Last month, OpenAI solved a problem related to the Navier-Stokes equations of fluid dynamics, described as one of the most difficult and enduring problems in mathematics.
Francis Johnson of University College London, who spent 25 years working on Wall’s D(2) problem, said the problem was among those addressed in OpenAI’s release. Johnson produced two books on the subject before abandoning his own work on it. “I worked on this problem for 25 years. I produced two books on it. I, personally, gave up,” he said. “I’m surprised that [AI] has done it quite so quickly, but I’m not surprised that it’s done it.”
Johnson said his recent experience working with AI models had left him “astonished with the sophistication and clarity of the analysis that they gave”. But he also described the implications for mathematicians as profound. “It puts us all in a very strange position,” he said. “Let’s face it, the genie is out of the bottle now. We’re going to have to live with it. Human beings are supposed to be adaptable, so we’re going to have to adapt. I think for the moment, we just stand back and be astonished.”
Kevin Buzzard of Imperial College London urged greater caution in assessing the release. Of the 30 papers relevant to his field of number theory, he said only seven appeared impressive to him, while only one had been formally verified in Lean, a computer-based system used to establish mathematical results with a high degree of certainty.
“Unfortunately, acceptance of these results by the community will take time, and journalists are going to have to wait while the mathematicians do their job,” Buzzard said. The remaining six results, he added, would require either an expert to examine and verify the work or a Lean formalisation.
Buzzard nevertheless described the broader rise in AI mathematics as astonishing. If most of the new results prove to be correct, he said, they could provide an indication of what the new normal in mathematics might look like. “It has been a long time since there were humans who were experts in all of mathematics, but now we seem to have machines with this property,” he said.
Ben Allanach of the University of Cambridge said AI companies were hiring mathematicians and investing substantial resources in research, with mathematical discoveries becoming “an intellectual trophy, an achievable intellectual trophy” for technology companies. Yet he described the scale and presentation of OpenAI’s release as “confusing and disrupting”, questioning whether machine-generated results were being sufficiently integrated into human mathematical knowledge.
The papers were unusually published on GitHub, a platform more commonly associated with computer code. The release also comes amid criticism of how AI-generated mathematical research is presented. Terence Tao of the University of California, Los Angeles, has warned that discoveries being “dumped” on the mathematical community could harm the field.
An independent Advisory Group on Mathematics and Artificial Intelligence has since been established to advise AI companies on best practice. Its recommendations include identifying models and prompts, providing estimated computational costs, formalising proofs where possible, and disclosing unsuccessful attempts and how problems were selected.
Lindsay McCallum Rémy of OpenAI said the company was exploring alternatives that would meet the group’s guidelines and was committed to improving future releases through better citations, mathematical exposition and presentation. “We want to give the mathematical community time and space to assess this work,” she said.
