AI Auditors May Get Desks Inside the Labs

1 Oversight moved indoors

Proposals for embedded evaluators would place independent AI-safety auditors inside frontier labs with standing access to systems, staff, and training runs — a change from the episodic audits outside firms conduct today. This is an oversight-model design under discussion, not adopted policy. The question it raises is whether putting auditors inside the building strengthens supervision or absorbs it.

2 How the arrangement would work

As proposed, an embedded evaluator would maintain an ongoing presence rather than arriving for scheduled reviews. Standing access would cover pre-release model versions, internal evaluation results, and the safety team's own findings, with the auditor observing development in real time instead of reconstructing it afterward.

3 Incident reporting duties

The proposal pairs access with a reporting obligation: auditors aware of serious safety incidents or capability surprises would be expected to escalate, either to the lab's leadership, to a government body, or to the public. The design question is the trigger. Escalation thresholds written too loosely would make auditors a notification pipeline; thresholds written too tightly would miss the incidents that matter. Neither threshold has been settled anywhere.

4 The independence problem

Auditor independence, in its classic definition, requires freedom from financial interest in the audited party. Embedded evaluation strains that principle by design: the auditor occupies the lab's premises, depends on the lab's infrastructure for access, and may be paid by the arrangement it supervises. Proposals typically answer with structural firewalls — separate reporting lines, protections against removal, contractual access guarantees — each of which would have to be tested in practice.

5 Practical limits of inside supervision

Three limits recur in critiques. Scale: frontier development involves thousands of people and training runs; a small resident team cannot watch everything, so coverage would be negotiated, not total. Information: auditors see what the lab shows plus what they can find, and insider status does not guarantee insider knowledge. Capture: familiarity compounds — the risk that the resident auditor gradually adopts the lab's framing of its own risks.

6 What would make it credible

Credibility would not come from presence alone. A workable embedded model would need externally set evaluation standards, publication rights the lab cannot veto, a funding stream the lab does not control, and rotation of personnel to blunt capture. Absent those features, embedded evaluation would function as an extended tour — better informed than a snapshot audit, but not independent in any strict sense.

7 The central question, answered

Can supervision survive being stationed at the supervisee? The proposal's wager is that ongoing access outweighs the capture risk. The honest answer is that the design has not been tested: embedded evaluation exists only as a proposal, its incident-reporting triggers are unspecified, and its independence safeguards would face exactly the pressures they are meant to resist. The desks are not yet real; the trade-off is.