As AI systems grow more powerful and complex, the methods used to measure their performance are beginning to buckle under the pressure. According to a recent deep dive by MIT Technology Review, researchers and developers are increasingly questioning whether current AI benchmarks—once a staple of evaluating innovation—are still up to the task.
One major flashpoint is the SWE-Bench, a benchmark launched in late 2024 to assess an AI’s ability to solve real-world coding problems. Built from over 2,000 Python programming issues pulled from GitHub repositories, SWE-Bench rapidly became a key metric for major tech firms. OpenAI, Anthropic, and Google routinely cite it in model releases, and fierce competition to top the leaderboard has led some developers to optimize specifically for the benchmark, rather than for broader programming competence.
“It looks nice and shiny at first glance,” says John Yang, a researcher from Princeton University who helped create SWE-Bench. “But then you try to run it on a different language and the whole thing just kind of falls apart.” Yang warns that models may be learning to ace the test, not to code robustly in diverse environments—a trend he calls “gilded” performance.
The issue, researchers argue, isn’t necessarily bad faith but a systemic flaw. Many benchmarks—once trusted tools of evaluation—are being gamed, misinterpreted, or applied far beyond their original design. As MIT Technology Review notes, this has led to what OpenAI cofounder Andrej Karpathy recently described as an “evaluation crisis.”
The limitations of benchmarks like SWE-Bench echo broader concerns in the field. Other well-known tools, including WebArena and Chatbot Arena, have faced criticism for design flaws and manipulation. Meanwhile, even classic benchmarks like ImageNet, once the gold standard for AI image recognition, are losing credibility in real-world performance tests.
In response, a new wave of researchers is advocating for a radically different approach: drawing inspiration from the social sciences. Key to this movement is the concept of validity—how well a test actually measures what it claims to. “Taking validity seriously means asking folks in academia, industry, or wherever to show that their system does what they say it does,” says Abigail Jacobs, a professor at the University of Michigan and a leading voice in the validity movement.
Stanford PhD student Anka Reuel has gone a step further by launching BetterBench, a public scoring system for AI benchmarks based on transparency, reproducibility, and validity. Unsurprisingly, benchmarks long trusted by the industry, like the MMLU (Massive Multitask Language Understanding), scored poorly. Meanwhile, older, more narrowly defined tools like the Arcade Learning Environment fared better.
Reuel and others are also collaborating with Hugging Face, EleutherAI, and the University of Edinburgh on new research to design more valid and task-specific benchmarks. “You need to have a structural breakdown of the capabilities,” she explains. “What are the actual skills you care about, and how do you operationalize them into something we can measure?”
These reforms may take time to influence industry practice. Despite widespread acknowledgment of the problem, recent model releases from OpenAI, Google, and Meta continue to rely heavily on traditional benchmarks like MMLU. As MIT Technology Review points out, the industry’s obsession with “general intelligence” remains a dominant force—making it hard for nuanced, task-specific evaluation frameworks to gain traction.
Still, there are signs of a shift. A joint paper from researchers at Google, Microsoft, and Anthropic in March proposed a framework for improving AI evaluations, placing validity at the forefront. And a February position paper by Microsoft and academic researchers called for benchmarking practices to adopt social science methodologies for measuring complex traits like reasoning and ideology.
“Evaluating GenAI systems is a social science measurement challenge,” the paper argues. If widely adopted, such a perspective could fundamentally change how AI capabilities are validated and understood.
For now, experts are walking a tightrope. “It’s too easy to throw the system out,” says Irene Solaiman, head of global policy at Hugging Face. “But evaluations are really helpful in understanding our models, even with these limitations.”
