“AI companies can and should voluntarily work together to set standards.”
— Dario Amodei, We Must Pace the Frontier
AI does not make standards obsolete. But it challenges the assumptions behind traditional standardization. When systems are probabilistic, data-dependent, and tied to proprietary models, the question is what we standardize, how we verify compliance, and whether independent organizations can realistically participate.
Amodei’s letter concerns pacing frontier AI, not technical standardization in the conventional sense. But its emphasis on common safety standards, independent evaluators, and verifiability raises a related question: how can outsiders assess systems whose models, data, and infrastructure remain private?
In my last blog, I argued that AI capabilities emerge from combinations of hardware, software, data, and human choices. That same characteristic complicates standardization.
When the specification is no longer enough
Standards are part of the invisible infrastructure of modern technology. They allow devices to communicate, content to move between systems, and different companies to build compatible products.
The traditional model is relatively easy to understand. A specification is agreed, companies implement it, compliance is tested, and compatible implementations work together.
AI makes this less straightforward. Even where a system is reproducible under controlled conditions, behavior may vary across models, data, prompts, hardware, and deployment environments. Systems designed for the same task may meet it in different ways.
With a conventional video codec, conforming implementations should decode the same content consistently. Once learned components are introduced, differences in model execution or numerical precision may affect the output.
So what should a standard require: identical results, an acceptable range of behavior, a defined evaluation process, or safeguards for unexpected outcomes?
The answer depends on the use case. Variation may be acceptable when recommending a film. It is different when AI contributes to a safety-critical decision. “Usually correct” cannot be the standard for everything.
From specifications to frameworks
Traditional technical standards will not disappear. Formats, interfaces, protocols, and infrastructure still require precise specifications. Common interfaces may become more important if independent evaluators need reliable access to models, tools, and evidence across providers.
But AI may require another layer. Frameworks, best practices, and policy-led interoperability can define acceptable behavior, responsibilities, and safeguards without prescribing how every system is built. Testing and certification can assess whether those expectations are met.
A traditional standard asks whether an implementation conforms to a specification. For some AI systems, the better question may be whether it behaves acceptably for its intended purpose.
Amodei’s proposal makes the connection practical. Safety commitments matter only if they can be verified. That may require agreed evaluations, comparable evidence, access protocols, and trusted third parties able to examine both models and the processes around them.
These approaches may be called “soft” standards, but their commercial impact may be anything but soft. A framework adopted by regulators, procurement bodies, or major platforms can become a condition of doing business.
Can an open standard depend on closed technology?
Many technical standards assume that different organizations can independently implement a common specification. What happens when meaningful participation depends on access to a particular model, dataset, computing environment, or platform?
A technology does not need to pass through a standards body to become a standard in practice. If enough businesses build around a model, API, or platform, it can become a de facto standard.
The question then changes from “Can I implement the standard?” to “Can I access what the standard depends on, and on what terms?”
This matters because some of the organizations shaping AI also control much of the infrastructure required to participate: models, computing capacity, platforms, and data.
That is not necessarily negative. Integrated platforms can accelerate innovation. But this differs from an open technical standard that any sufficiently capable organization can implement.
A process may be formally open because anyone can read its documents. But is it meaningfully open if only a few organizations have the resources to influence it or verify compliance?
Who gets to shape the standard?
Standards bodies must decide where fixed specifications remain appropriate and where frameworks or policy-led approaches work better. Policymakers must encourage interoperability without locking markets into particular platforms. Businesses must consider whether today’s technical choices allow others to participate tomorrow.
AI does not make standards less important. The greater the consequences of AI decisions, the stronger the need for shared rules and credible verification.
The difficult part will be ensuring that those rules do not simply reflect the interests of the organizations already in the strongest position. A standard can be open on paper and still be closed in practice.
Can our current approach to standardization evolve to accommodate AI, or will the industry end up building a different framework around it?
