Did OpenAI Cheat on Its Big Math Test?

How clever is a mannequin that memorizes the solutions earlier than an examination? That’s the query dealing with OpenAI after it unveiled o3 in December, and touted its mannequin’s spectacular benchmarks. On the time, some pundits hailed it as being nearly as highly effective as AGI, the extent at which synthetic intelligence is able to attaining the identical efficiency as a human on any activity required by the person.

However cash modifications every little thing—even math checks, apparently.

OpenAI’s victory lap over its o3 mannequin’s gorgeous 25.2% rating on FrontierMath, a difficult mathematical benchmark developed by Epoch AI, hit a snag when it turned out the corporate wasn’t simply acing the take a look at—OpenAI helped write it, too.

“We gratefully acknowledge OpenAI for his or her assist in creating the benchmark,” Epoch AI wrote in an up to date footnote on the FrontierMath whitepaper—and this was sufficient to lift some purple flags amongst fans.

Worse, OpenAI had not solely funded FrontierMath’s improvement but additionally had entry to its issues and options to make use of because it noticed match. Epoch AI later revealed that OpenAI employed the corporate to offer 300 math issues, in addition to their options.

“As is typical of commissioned work, OpenAI retains possession of those questions and has entry to the issues and options,” Epoch mentioned Thursday.

Neither OpenAI nor Epoch replied to a request for remark from Decrypt. Epoch has nevertheless mentioned that OpenAI signed a contract prematurely indicating it might not use the questions and solutions in its database to coach its o3 mannequin.

The Data first broke the story.

Whereas an OpenAI spokesperson maintains OpenAI did not straight practice o3 on the benchmark, and the issues have been “strongly held out” (that means OpenAI didn’t have entry to among the issues), specialists observe that entry to the take a look at supplies might nonetheless permit efficiency optimization by means of iterative changes.

Tamay Besiroglu, affiliate director at Epoch AI, mentioned that OpenAI had initially demanded that its monetary relationship with Epoch not be revealed.

“We have been restricted from disclosing the partnership till across the time o3 launched, and in hindsight we must always have negotiated more durable for the power to be clear to the benchmark contributors as quickly as attainable,” he wrote in a put up. “Our contract particularly prevented us from disclosing details about the funding supply and the truth that OpenAI has knowledge entry to a lot, however not the entire dataset.”

Tamay mentioned that OpenAI mentioned it wouldn’t use Epoch AI’s issues and options—however didn’t signal any authorized contract to be sure that can be enforced. “We acknowledge that OpenAI does have entry to a big fraction of FrontierMath issues and options,” he wrote. “Nonetheless, now we have a verbal settlement that these supplies won’t be utilized in mannequin coaching.”

Fishy because it sounds, Elliot Glazer, Epoch AI’s lead mathematician, mentioned he believes OpenAI was true to its phrase: “My private opinion is that OAI’s rating is legit (i.e., they did not practice on the dataset), and that they haven’t any incentive to lie about inside benchmarking performances,” he posted on Reddit.

The researcher additionally took to Twitter to handle the scenario, sharing a hyperlink to a web based debate concerning the difficulty within the on-line discussion board Less Wrong.

Not the primary, not the final

The controversy extends past OpenAI, pointing to systemic points in how the AI business validates progress. A current investigation by AI researcher Louis Hunt revealed that different high performing fashions together with Mistral 7b, Google’s Gemma, Microsoft’s Phi-3, Meta’s Llama-3 and Alibaba’s Qwen 2.5 have been in a position to reproduce verbatim 6,882 pages of the MMLU and GSM8K benchmarks.

MMLU is an artificial benchmark, identical to FrontierMath, that was created to measure how good fashions are at multitasking. GSM8K is a set of math issues used to benchmark how proficient LLMs are at math.

That makes it inconceivable to correctly assess how highly effective or correct their fashions actually are. It’s like giving a scholar with a photographic reminiscence a listing of the issues and options that will likely be on their subsequent examination; did they purpose their strategy to an answer, or just spit again the memorized reply? Since these checks are meant to show that AI fashions are able to reasoning, you possibly can see what the fuss is about.

“It is truly A VERY BIG ISSUE,” RemBrain founder Vasily Morzhakov warned. “The fashions are examined of their instruction variations on MMLU and GSM8K checks. However the truth that base fashions can regenerate checks—it means these checks are already in pre-training.”

Going ahead, Epoch mentioned it plans to implement a “maintain out set” of fifty randomly chosen issues that will likely be withheld from OpenAI to make sure real testing capabilities.

However the problem of making actually unbiased evaluations stays important. Laptop scientist Dirk Roeckmann argued that perfect testing would require “a impartial sandbox which isn’t straightforward to comprehend,” including that even then, there is a threat of “leaking of take a look at knowledge by adversarial people.”

Edited by Andrew Hayward