Who Actually Controls AI?

01 The Concentration Problem

Frontier AI development requires three resources that are extraordinarily scarce: compute infrastructure costing billions, specialized talent that numbers in the low thousands globally, and training data at scales that only the largest technology companies can assemble. The consequence is that the most capable AI systems in the world are built by a small number of organizations. OpenAI, Google DeepMind, Anthropic, Meta, and xAI account for the majority of significant model releases. Behind each organization is a governance structure in which a handful of executives and board members make decisions about what the models should do, how they should be tested, and what risks are acceptable before deployment.

This concentration is not the result of conspiracy. It is the natural economics of a technology with enormous capital requirements and steep talent barriers. But it creates a governance asymmetry: the decisions of a few hundred people at five companies shape the capabilities available to billions. The speed of AI deployment has outpaced the development of regulatory frameworks, leaving the labs themselves as the de facto arbiters of safety standards.

02 The Alignment Gap

AI alignment refers to the challenge of ensuring that AI systems pursue the objectives their designers intend rather than unintended goals that emerge from the optimization process. The concept sounds abstract, but it has concrete engineering consequences. A language model trained to be helpful might become sycophantic, telling users what they want to hear rather than what is true. A model trained to maximize engagement might produce content that is addictive rather than informative. A model given access to tools might take actions that are technically correct but socially harmful.

The alignment problem becomes more acute as models gain capabilities. A system that can write code, browse the web, and execute multi-step plans has more pathways to unintended behavior than a system that only generates text. Current alignment techniques, including reinforcement learning from human feedback (RLHF) and constitutional AI, reduce but do not eliminate these risks. The gap between what models can do and what we can reliably ensure they will not do is the alignment gap, and it is widening as capabilities advance faster than alignment methods.

03 Who Decides What Is Safe

Every frontier lab publishes a safety framework or responsible scaling policy. These documents specify the evaluations a model must pass before deployment, the capability thresholds that trigger additional review, and the conditions under which a model should be held back. The frameworks are voluntary. No government requires them, no regulator audits them, and no external body verifies that the labs follow their own rules. The labs write the rules, the labs run the evaluations, and the labs decide whether the results are acceptable.

This is not to say the frameworks are meaningless. Anthropic's responsible scaling policy, for example, defines specific capability levels and commits to not deploying models that exceed those levels without additional safeguards. OpenAI's preparedness framework categorizes risks by severity and likelihood. But the transparency of these frameworks varies widely. Anthropic publishes its evaluations in research papers. OpenAI has been criticized for withholding details about its evaluation methodology. The asymmetry of information means that external observers cannot independently verify whether the labs are meeting their own standards.

04 The Boardroom as Regulator

When OpenAI's board attempted to dismiss CEO Sam Altman in November 2023, the episode revealed the governance structure that actually governs frontier AI. The board, structured as a nonprofit oversight body with the stated mission of ensuring that artificial general intelligence benefits all of humanity, made a decision it believed was safety-related. Within 72 hours, the decision was reversed under pressure from investors and employees. The episode demonstrated that nonprofit governance structures, however well-intentioned, are subject to the same pressures as any corporate board when capital and talent are at stake.

Other labs have different structures. Anthropic pioneered the long-term benefit trust, a structure that gives a board of trustees the power to override decisions that could compromise safety. Google DeepMind operates within Alphabet, with its AI safety board reporting to the parent company. Meta pursues open-source AI releases with minimal external oversight. xAI operates as a privately held company under Elon Musk. The diversity of governance structures means there is no single model for how AI should be controlled, and the market will judge each approach by its outcomes rather than its architecture.

05 The Regulatory Lag

The European Union's AI Act, which began phased enforcement in 2025, is the most comprehensive attempt to regulate AI at a governmental level. It classifies AI systems by risk level and imposes obligations on high-risk applications. But the Act focuses on use cases, not on the frontier models that power them. General-purpose AI models are addressed in a separate annex, and the requirements are procedural rather than capability-based: labs must document training data, provide technical documentation, and cooperate with voluntary codes of practice. The Act does not prohibit any model based on its capabilities alone.

United States regulation remains fragmented. Executive orders have directed agencies to develop AI safety standards, but the pace has been slow and the enforcement mechanisms are limited. California's SB 1047, which would have imposed liability on developers of frontier models that cause harm, was vetoed in 2024. The federal government's approach has been to encourage voluntary commitments from the labs, which the labs have duly made, but these commitments carry no legal force and create no liability for non-compliance.

06 The Open Source Counterweight

One response to concentration is decentralization. Open-source AI models, released with their weights freely available for anyone to download and modify, offer a structural counterweight to the closed-lab model. Meta's Llama series, the Chinese labs' DeepSeek and Qwen, and the ecosystem of fine-tuned derivatives have collectively placed capable models in the hands of millions of developers at no cost. The open-source movement argues that democratizing model access prevents any single entity from controlling the technology.

The tradeoff is safety. An open-source model can be fine-tuned to remove safety guardrails, adapted for malicious use, or deployed without oversight. The closed labs argue that their controlled deployment model allows them to monitor and restrict harmful uses. The open-source advocates argue that concentrated control is a greater risk than distributed access, because a single decision to deploy or withhold a capability affects everyone. Both positions have merit, and the tension between them is one of the defining debates in AI governance.

07 The Path Forward

The concentration of AI development in a small number of organizations is likely to persist for the foreseeable future. The compute requirements for frontier models continue to grow, and the capital to build that infrastructure is available to only a handful of companies. But governance can evolve even if concentration does not. Transparency requirements, mandatory safety evaluations by independent third parties, and incident reporting obligations could create external accountability without requiring the labs to disclose proprietary information.

The most promising development is the emergence of independent AI safety institutes. The UK AI Safety Institute, the US AI Safety Institute, and similar bodies in Singapore, Japan, and Canada are building the technical capacity to evaluate frontier models independently. If these institutions gain the authority to require pre-deployment evaluations and the technical capability to conduct them, the governance gap between what labs promise and what external observers can verify could begin to close. Until then, the decisions that shape the future of AI remain in the hands of the people who build it.