Home Business OpenAI Is Pissing Off a Bunch of Mathematicians—Again

OpenAI Is Pissing Off a Bunch of Mathematicians—Again

0
4

In August, OpenAI convened around 40 mathematicians to discuss what to do if AI outpaces human capabilities in the field. The company hinted that its powerful models had solved hundreds of longstanding math problems, say people who were in attendance, but company representatives assured attendees it would not release the solutions all at once—an assurance OpenAI spokesperson Lindsay McCallum says the company is “not aware of”—and, worried by how the community might react, sought advice on how to publish the findings.

Attendees responded with “a mixture of excitement and dread,” but the meeting was a promising first step, recalls Northwestern University mathematician Bryna Kra. The group, she tells WIRED, asked the firm not to just publish them in a blog or tweet, like they had done with 10 problems earlier that month. Instead, it would be important for OpenAI to publish papers explaining the work so that mathematicians could absorb, digest, and use the results, according to Kra. “Apparently, that input was ignored,” she says.

OpenAI has told people it plans to dump hundreds of the results it referenced in the August meeting on GitHub on Tuesday, people familiar with the plans tell WIRED. The release would be the latest of tens of thousands of mathematical solutions generated by AI this year, as frontier models become increasingly capable.

“On August 28, we began training a new internal model. In addition to resolving the Navier—Stokes Millennium Prize problem⁠, this model has now resolved more than 100 long-standing open problems across most areas of mathematics,” says McCallum. “We are working to responsibly release the next math results from our model, drawing on advice and public recommendations from the Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study to inform how we release these results. We have not set a release time.”

But for several leading mathematicians, the impending output is also a sign that the company has learned little from the controversies over previous releases.

In conversations with WIRED, the academics say they feel the field has become a playground for OpenAI and its rival Anthropic to show off their models as both prepare for blockbuster initial public offerings. In their rush to outdo each other, mathematicians say, traditional scientific processes for releasing and attributing results have been cast aside.

The most striking case so far came in September. OpenAI deployed thousands of agents to solve a legendary million-dollar Millennium Prize problem after hearing “rumors” that others were closing in on solutions. Tristan Buckmaster, a mathematician and professor at New York University, accused the company of front-running the work he had done toward solving an element of the problem in a personal collaboration with Anthropic employee Levent Alpöge. The pair hadn’t published their work, but had been using OpenAI’s tools to help them.

During an attempt to negotiate credit, Buckmaster claims, OpenAI researcher Sébastien Bubeck seemed to imply that Alpöge should be excluded from any paper, since that would make things complicated. (In response to a request for comment, Bubeck pointed to a previous public statement in which he denied asking for Alpöge to not be listed as an author.) Buckmaster said he refused. In a call, Bubeck claimed that there is some worry about what Anthropic is doing, according to meeting notes seen by WIRED. Levent must be talking to his leadership right now letting them know fucking OpenAI can get a Millennium Prize problem, he said, per the notes. What’s to stop Anthropic from giving all their compute to get a Millennium Prize problem?

According to the notes, Bubeck also seemed to suggest that Buckmaster would be ruining his career when Buckmaster threatened to tell the press about his belief that OpenAI had stolen his work. If you don’t want me to be nice, then I don’t have to be nice, Bubeck said, according to the meeting notes.

Nestor Guillen, a visiting math professor at NYU, tells WIRED that “there’s a perception of mobster behavior” from the AI companies among mathematicians.

“We disagree with that characterization,” says McCallum.

“I feel a lot of angst, and I see this more and more in my colleagues in mathematics, not over AI, but over the AI companies,” Guillen says. “I think a lot of the angst is about the accumulation of power in one place.”

OpenAI assembled an advisory group in mid September to help determine how the company assesses and communicates new results, upholds academic and professional standards, and builds tools that support mathematical research and learning. But the company doesn’t seem to have made much progress, mathematicians say.

Many are frustrated that OpenAI and Anthropic continue to release results through blog posts, rather than scientific papers, which makes them harder to verify and often excludes prior work from other mathematicians. Piecemeal announcements aren’t limited to the labs: Alpöge himself announced he had disproved an 87-year-old conjecture through a tweet sent shortly after the World Cup final.

“Math by tweet and math by press release to me is not the way to nurture the ecosystem that created the fertile ground that they have trained on,” says Kra.

Some have set up new tools this year in reaction to the increase in machine-assisted proofs; the idea is to help the community both use new results and sort through what needs to be further digested. The tools include Hexagon, a repository for primarily AI-generated material, and Palomar, a registry of machine-verified mathematics.

While OpenAI has been directly encouraged to use these tools, “they haven’t changed their behavior,” Kra notes. Nor does she feel the company’s actions are actions aligned with the Leiden declaration, a call by more than 4,000 mathematicians for AI companies to meet mathematicians’ standards that she helped write.

Several employees within OpenAI believe their technology has rendered math dead anyway, according to people who have spoken with them. (“We don’t believe the future of mathematics is set,” says McCallum. “We’re working with the math community to navigate the future collaboratively.”) While the scenario where AI capabilities outpace those of human researchers was framed as hypothetical during the August meeting, attendees who spoke to WIRED interpreted it as a warning about what was to come. The company has briefed mathematicians about its capabilities and breakthroughs in the same way “police might want to notify family before reporting a death in a car accident,” the person noted.

Bubeck appears to believe that his firm’s technology will swiftly end most mathematicians’ careers, according to a person familiar with his thinking. Bubeck tells WIRED he believes more capable AI could help mathematicians tackle more ambitious questions, better connect their work to solve real-world challenges, and make mathematics more accessible. “I see this as an opportunity to expand what mathematicians can do and the impact their work can have,” he says.

Mathematicians know they need to adapt, especially to equip younger generations, but do not believe that their field is dead. “It changes how we’re going to operate, but I think it’s a moment that we can think bigger,” says Kra. She’s happy to have a new powerful tool at her disposal to solve problems–she just wants companies’ disclosures to be better so that she can trust and build on the results.

“It’s a scary time,” she says, “but it’s also really a deeply exciting time.”

Disclaimer : This story is auto aggregated by a computer programme and has not been created or edited by DOWNTHENEWS. Publisher: wired.com