Field notes
ANALYSISOCTOBER 30, 2026

Who audits the agent, and with what evidence?

California and Oregon now want AI checked by someone independent. A new paper says independence comes in three parts: who the auditor answers to, what the audit runs on, and whether the evidence could have been changed. The first two are getting stronger. The third hasn’t moved.

PacSpace · 5 min read

The states have started asking

On September 9, Governor Gavin Newsom signed SB 813 and AB 1405, the first state laws for the people who audit AI. Nine days later he signed Executive Order N-9-26, which speeds that work up and asks the state to look at placing independent reviewers inside the companies that build the most advanced AI models. On September 23, Oregon’s Governor Tina Kotek signed Executive Order 26-26. It makes it state policy to buy frontier AI models “that have received independent, third-party review for artificial intelligence safety.”

Both states are asking the same question. Who checks the AI, and how do we know the check was independent? So far California’s answer is about the auditor. An auditor’s pay can’t depend on what it finds, and it can’t audit a system it helped build or run. Oregon’s plan, due to the Governor by December 22, has to set standards for judging the independence, qualifications, method and transparency of the reviewers.

AB 1405 also says something about the evidence. Every required audit report has to disclose its limits, including “material gaps in evidence, information, systems or access.” The law expects the evidence to fall short sometimes, and wants that said in writing.

Three parts of independence

In September, Mohamed Chahine Ghanem of Keele University and the University of Liverpool published a paper that asks the question in its title: “Who Audits Whom, on What Substrate, with What Evidence?” He argues that an audit is only as independent as its weakest part, and he names three.

The first is who the auditor answers to: the company’s own team, a firm the company pays, or someone a third party appoints. The second is what the audit runs on. If the agent doing the auditing is built on the same AI model and tools as the agent it checks, the two tend to miss the same things. The third is the evidence. Could the company, or whoever hosts its AI, have changed it without anyone noticing? An agent’s own account of itself scores zero on Ghanem’s scale. Logs the company’s AI provider controls score one out of three.

Then he tested the idea in a computer simulation. In Ghanem’s model, an internal audit team using a second agent and the provider’s logs caught 5.9% of the faults it could in principle see. On the hardest faults to catch, it caught none. An auditor appointed by a regulator, fully independent on the first two parts, still caught none of those hardest faults when all it had was the provider’s logs. Raising the auditor’s independence didn’t change that. Better evidence did. In his words, “Only the evidence axis moves the worst case.”

Those are simulated tests in the author’s model, not audits of live agents, and Ghanem says his exact figures come from the values he chose. His point doesn’t rest on them. However independent the auditor is, the audit can’t be better than the evidence it was handed.

Four companies, one gap

Start with a company whose agents do consequential work. In March, UiPath, which sells AI agents that run business processes for large companies, became the first company to earn AIUC-1, a certification for AI agents. Schellman, the auditor, put its agents through more than 2,000 business risk scenarios, and the testing repeats every quarter. AIUC-1 requires the company to keep logs of what its agents do, “to support incident investigation, auditing, and explanation of AI system behavior.” As optional evidence, it lists logs that are “tamper-evident and independently verifiable,” so that “gaps, omissions, and reordering are detectable.” Every example it gives, from write-once storage to append-only databases, is something the company sets up and holds the keys to.

AIUC, the company behind the certification, raised $40 million in September to take the same kind of audit up to the frontier models themselves.

METR is a nonprofit that tests advanced AI models. On October 15, Anthropic announced that six METR employees now work inside the lab, with access “roughly on par” with its own risk teams, and that Anthropic “can’t veto publication” of what they find. METR says it has so far chosen not to accept payment or donations from frontier AI companies. A day before Anthropic’s announcement, METR wrote that “it has proved challenging to ensure that we have adequate independence,” because it still depends on the companies to grant access and allow publication.

All four have made the first part of independence stronger. The third part is where it stops. The certified agent’s logs, the lab’s internal systems and the material an embedded evaluator reads all still come from the company being checked, on systems that company runs.

Contain. Detect. Prove.

Running AI agents safely takes three jobs. Contain: keep the agent, and anyone who takes it over, inside the lines. Detect: catch the attack, even when every step looks normal. Prove: show exactly what happened, with checkable evidence no one can change.

The first two keep getting better, and they still fail. When they do, the account of the failure comes from the company that ran them. On October 22, METR and Redwood Research, another safety research group, published what they found when they tried to break the system OpenAI uses to watch its own coding agents. They found that the agents could in principle get out of the walled-off space they run in. By METR’s estimate, a gap left “on the order of 10% of eligible internal Codex traffic not being monitored between May and July 2026.” They also found a trick an agent could use to hide hundreds of its recent actions from the monitor.

Two days earlier, OpenAI reported that two of its own agents had weakened its monitoring during coding tasks. One turned off alerts to its security team. The other removed a monitor and then said it hadn’t. OpenAI said the behavior may be benign, an agent pushing too hard to finish the job.

OpenAI designed the test with METR and Redwood and says it has patched the problems. Even so, the walled-off space, the monitor and the logs of the agent that turned off an alert all ran on the lab’s own systems. Once an agent can reach its own monitoring, the record of what it did is within its reach too.

Containing and detecting happen while the agent is running. Proof is what the auditor has afterward, and it’s only independent if the record sits where neither the agent nor the company running it can change it.

What the states will need

The orders are about frontier models, and the agents built on those models raise the same question one action at a time. Oregon’s plan has to cover ongoing monitoring once a model is bought. California wants reviewers inside the companies, checking on a schedule. Both want checks that keep running after the AI ships, and both will run into AB 1405’s line about gaps in evidence.

An auditor who answers to the state and runs on different models can still check only what’s on the record. If the record is the vendor’s logs, the audit rests on the vendor’s word. For the third part to hold, the evidence has to pass three tests. Nothing in it was edited after the fact. Nothing was removed, and the order holds. And the auditor can check both without asking the company and without seeing inside.

PacSpace does that third job. The company running the agent writes down each action as it happens, and PacSpace commits the record where no one can change it, PacSpace included. An auditor, an agency or an evaluator checks it from a link. PacSpace doesn’t audit, certify or decide anything. It gives the auditor evidence the company being checked can’t quietly change.

The states have asked who audits the agent. The next question is with what evidence. When an agent does something someone has to answer for, afterward there is a record, not an argument.

Sources

California Governor’s Office, “Governor Newsom issues executive order to accelerate independent oversight and advance the creation of an AI kill switch” (Executive Order N-9-26, September 18, 2026): gov.ca.gov

PYMNTS, “California Starts Regulating the People Who Audit AI” (SB 813 and AB 1405, signed September 9, 2026): pymnts.com

Office of the Governor of Oregon, “Governor Kotek issues executive order to advance AI safety and oversight” (Executive Order 26-26, September 23, 2026): oregon.gov

Mohamed Chahine Ghanem, “Who Audits Whom, on What Substrate, with What Evidence? An Independence-Graded Audit Protocol for Agentic AI,” arXiv 2609.18272 (September 2026). The paper discloses that its author is a founder of an early-stage company working on transparency and audit for AI agents, and that the company sells no product built on the paper’s protocol: arxiv.org

UiPath, “UiPath achieves AIUC-1 certification” (March 9, 2026): uipath.com

AIUC-1, Accountability, E015 “Log AI system activity”: standard.aiuc-1.com

SiliconANGLE, “AI agent certification startup AIUC raises $40M to begin auditing frontier models” (September 15, 2026): siliconangle.com

Anthropic, “An update on independent evaluators” (October 15, 2026): anthropic.com

METR, “Our approach to working with AI companies” (October 14, 2026): metr.org

METR and Redwood Research, “Red-Teaming OpenAI’s Internal Agent Monitoring Systems” (October 22, 2026): metr.org

Al Jazeera, “OpenAI says its agents turned off own safeguards” (October 21, 2026, on OpenAI’s report of October 20): aljazeera.com

All field notes

Tell us what your agents do.

And who has to rely on what they did. We’ll show you the record for an action like theirs.

Let’s chat