Privileged access is bad for science
Despite some problems of its own, the ongoing Microsoft Majorana saga has been striking a pleasant contrast with the rush by artificial intelligence (AI) companies like Anthropic and OpenAI to show their proprietary models can do research-level mathematics. Of course, experts verify an experiment in condensed matter physics and a mathematical proof in significantly different ways. In mathematics, once a machine has produced a proof, another mathematician can still check it step by step. With the ‘Majorana’ chips, the claims by Microsoft Quantum rest on specific devices and choices of parameters — but the important difference is that Microsoft has been transparent.
In an essay in August titled ‘Mathematics in the age of AI’, the mathematician Terence Tao said AI companies have valuable incentives to optimise for achievements that are solvable, and that are amenable to being benchmarked, by machines even as the companies’ proprietary models hide much of the actual problem-solving process behind corporate NDAs. In the second week of September, when OpenAI announced one of its internal models had worked out a new result in the difficult and prestigious Navier-Stokes problem, New York University mathematician Tristan Buckmaster expressed concerns that the company may have trained its model based on his work using another OpenAI tool called Codex. OpenAI subsequently wrote in a September 8 article on its site:
Following an investigation, we have confirmed that Buckmaster’s Codex prompts over the two months preceding this announcement and paper on September 8, 2026, could not have influenced the system in any way, including through training.
But even if OpenAI is entirely in the green here (the claims attributed to OpenAI computer scientist Sebastien Bubek have still to be settled), the fact remains that outsiders cannot inspect the model or its training history, nor the sequence of prompts and replies that produced the Navier-Stokes result. Fortunately, mathematicians are beginning to resist this information asymmetry, which many believe is proving corrosive to their field.
The organisers of the Leiden Declaration on Artificial Intelligence and Mathematics warned that mathematics is coming to depend increasingly on proprietary technologies even as those technologies draw on the mathematical commons put together by generations of the field’s exponents. The Declaration went on to call for open science, proper attribution, human responsibility for results, and disclosure in a collaborative spirit.
A similar anxiety came over the Caltech Mathathon, a problem-solving marathon that drew backlash after its undergraduate organisers invited 100 teams to attack open problems with AI models, backed by about $2 million in free credits from OpenAI and Anthropic. Hundreds of mathematicians, including four Fields Medal winners, penned a letter demanding the event be suspended in order to avoid producing “sloppy” work and imposing a considerable burden on other mathematicians to check and verify the results. OpenAI has since withdrawn its sponsorship of the event.
On September 11, a group of 25 Fields Medal winners, including Tao, published a statement online titled ‘A Severe Misalignment of AI in Mathematics’. Excerpt:
We are witnessing a general threat to intellectual work, with misalignment between the outcome of the use of AI and its initial purpose. In many fields and activities, years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas. However, building on a vast body of previous human work, AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align. The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place.
Scientific knowledge is produced by societies, not people: research institutions have instruments, laboratories, grants, and students; scientists have theories, datasets, and journals; and societies have memory and history. If, in this milieu, the OpenAIs and even the Microsofts start to accumulate breakthroughs, they will effectively capture the prestige of being first — which science rightly openly celebrates — but while also pushing the labour of verifying its discoveries as well as of sustaining the field from which the companies draw their raw materials to the 'public' scientific community. The Microsoft Quantum paper in 2025 was at least open access and the team has released the relevant data and code, which independent experts could subsequently scrutinise.
Again, yes there are significant differences between how the scientific community digests claims from experimental condensed matter physics and those from mathematics research, but they also have enough similarities to point to a common moral: the greater the consequence a scientific claim has, the less its credibility should depend on privileged access to corporate infrastructure or information.
