Amsterdam’s latest venture into using artificial intelligence to reform its welfare system has become a cautionary tale in the global conversation around “responsible AI.” As detailed in a recent MIT Technology Review investigative report, the city’s pilot program—dubbed Smart Check—sought to use predictive algorithms to detect welfare fraud in a fair, effective, and unbiased manner. Despite good intentions and procedural diligence, the project ultimately failed to meet its lofty ideals.
The report, published in collaboration with Lighthouse Reports and Dutch newspaper Trouw, reveals how Amsterdam’s Smart Check was developed by following many of the best practices outlined in responsible-AI frameworks. City officials consulted independent experts, performed bias assessments, implemented technical safeguards, and even engaged with affected communities. Yet, the algorithm—and the system that produced it—still fell short.
Anke van der Vliet, a longtime welfare advocate and member of Amsterdam’s 15-person Participation Council, was direct in her criticism of the project. “We didn’t want the program,” she told MIT Technology Review’s American correspondent. Despite some of the council’s concerns being addressed—such as the removal of potentially discriminatory variables like age—the city ignored its core recommendation: to halt the development altogether.
Welfare advocates expressed concerns that the algorithm would perpetuate distrust and further marginalize the city’s 35,000 welfare recipients. “Fraud is found in only 3% of applications,” van der Vliet noted, arguing that deploying AI to scrutinize everyone could cause more harm than good. Many beneficiaries already face what they describe as dehumanizing treatment, constantly needing to prove their eligibility amid rising living costs and an overburdened system.
City officials were not blind to these criticisms. Harry Bodaar, a senior policy advisor focused on welfare fraud enforcement, admitted to systemic shortcomings, describing the welfare system as being “held together by rubber bands and staples.” Despite these admissions, proponents of Smart Check hoped technology might patch the cracks—only to find the ethical fissures were deeper than code could solve.
Suresh Venkatasubramanian, a former White House tech advisor and co-author of the now-rescinded AI Bill of Rights, praised the MIT Technology Review story for highlighting the core issue: that the very premise of these algorithms must be questioned. “We need participation early on from communities,” he said, “but it also matters what officials do with the feedback.”
The U.S. political climate added a sharp contrast to the Amsterdam experiment. Just weeks before the journalist’s trip, the Trump administration rescinded Biden’s executive order on AI safety. Soon after, House Republicans passed a 10-year moratorium on state-level AI regulation—though it has yet to clear the Senate. As the article points out, ethical oversight in AI appears to be fading in the U.S. just as Europe grapples with its own accountability dilemmas.
Digital rights advocate Hans De Zwart offered a provocative alternative: “Why doesn’t the municipality build an algorithm that searches for people who do not apply for social assistance but are entitled to it?” His comment underscores a recurring theme in the investigation—that sometimes the problem is not how we apply AI, but what we choose to solve with it.
Despite its failure, Smart Check offers valuable lessons for cities and countries considering AI-driven governance. As Venkatasubramanian emphasized, even with “good intentions” and robust design processes, systems can still be “fundamentally flawed” if they neglect the complex social and ethical contexts in which they operate.
