Why Embedding AI Watchdogs Is Harder Than It Sounds

As governments hesitate to write hard rules for AI, the industry has offered an alternative: let independent experts sit inside the labs and check the models before they ship. Irene Solaiman, chief policy officer at Hugging Face, thinks that idea is sound in principle and a lot messier in practice.

Speaking at TechCrunch Disrupt, Solaiman argued that embedded evaluators cannot do their job until the field agrees on some basic questions, starting with who is even qualified to be one. "The qualifications that determine who is an effective evaluator across what tasks need a lot more standardization," she said, in comments reported by TechCrunch.

The context is a recent push by AI companies to present third-party evaluation as a substitute for government oversight, a pitch that gained urgency after a string of safety incidents. Anthropic recently brought in Accenture as its first embedded evaluator, a choice that drew criticism from people who questioned whether a consulting firm has the expertise to probe a frontier model. It follows the company's earlier move to embed the research group METR inside its building.

Benchmarks that do not agree with each other

Part of the difficulty is that measuring an AI model is still a young and unsettled craft. Solaiman described benchmarking as "an evolving science, where we don't know how to meaningfully compare them to each other." A test can look authoritative and still tell you very little once you try to line it up against another one.

Access matters too. An evaluator who can only poke at a model from the outside sees less than one who can inspect its workings, which is why Solaiman tied credible results to openness. "Do you have the level of access that you need to the model itself," she asked, adding that "this is where open weights come in in a big way."

The harder frontier is agents, the systems that chain a model together with tools and memory to act on their own. Evaluating those, Solaiman said, is "really hard and really expensive." She noted that a weaker model wrapped in a better harness can outperform a stronger one, which means testing the model alone misses most of what determines how the whole system behaves. Yet there are far fewer evaluations for harnesses than for models.

None of this makes embedded evaluation worthless. It does puncture the hope that it offers a quick, clean answer to the regulation question. Solaiman's point is that an evaluator is only as trustworthy as the standards, the access and the methods behind the verdict, and right now those are still being invented. A watchdog without an agreed test is just another opinion in the room.