The burden of proof should be on the AI

No engineering manager signs off a software release by reading every line of code. They sign off on evidence: test results, a demo, a clear account of what changed and why. The manager lays down the criteria, but the team carries the burden of convincing them. However, software developers managing agents still operate on a fundamentally broken system: we expect them to carry the burden of verifying the code. To do so, they receive raw, unverified code and are expected to understand it and validate it. The same is true from junior developers to senior management: the burden of proof is on the human. We get raw AI artefacts, no context, and a burden of proof that is bound to reach a breaking point.

Let’s face it: in the AI coding era, every engineer is now in a manager's position to some degree, even at entry level. They are managing an increasing number of agents, and are expected to validate all the work they produce. As we go up the seniority ladder, we are still trying to keep up with the sheer amount of stuff AI produces. Proofreading, correcting, ensuring accuracy.

In physical AI, every code change goes out to live production sites. In warehouse logistics, a single change can move a customer's pick rates, throughput or uptime. Customers care about that, and about safety, as much as any regulator does.

And so, engineering leaders are under two pressures right now. The first is productivity. Every organisation wants to become AI-native, and that means shipping more with the same or smaller teams. The second is accountability. Under the EU AI Act, the Cyber Resilience Act and the updated Product Liability Directive, it doesn't matter who or what wrote the code. What matters is what it does. The same is true for customers.

This is where the two pressures meet. Code moves robots through warehouses where people work. A bad change is not a broken web page. It can mean damage, injury, legal exposure or a customer's operation falling behind.

The problem, then, is how do we get the benefits and speed of AI without reducing quality and safety?

Most organisations are approaching this AI-native problem by adding AI tools to existing ways of working. That gets you speed, but it does not ensure safety and accountability. This may be OK for cloud-native infrastructure, but in physical AI, it has real consequences.

So far, the standard answer has been to keep a human in the loop. However, I don't think that holds up under productivity targets. AI produces code faster than people can read it, so reviewers have two options: they can hold the line and become the bottleneck, in which case productivity pressure will eventually push them out. Or they can approve what they're given, and become yes-men with their name on the sign-off. In both cases there is an appearance of accountability without the real thing. I don’t think I’m alone in this: some of the research community is now arguing that mandatory line-by-line human review is no longer a stable endpoint (Monperrus, 2026).

The burden of proof should be on the AI, not on the person signing off.

I think we should do the exact opposite. Instead of the developer proving the AI wrong, the AI should prove itself right, the same way a good team does. Instead of raw output, it should bring evidence: what it was trying to achieve, how it was tested, and what the robot will actually do. That doesn't change who is accountable, but it changes who does the work of convincing and gathering evidence.

So how do we do it?

The feedback loop depends on seniority

Everyone works with AI, but at different levels. The more junior you are, the tighter the loop. The more senior, the larger the unit you're checking.

VPStrategy

Broken todayNo cohesive way to grasp the complexity of what the organisation is building.

Make the AI earn itResearch agents map competitor gaps, cluster tickets into initiatives, and produce decks and docs the team can act on.

DirectorInitiatives

Broken todayYou can inspect the code or architecture, but nothing proves it solves the business problem.

Make the AI earn itRequirements pulled from Salesforce and Aha! become objects the AI argues against, traced to code and evidence. KPIs tracked and evidenced.

ManagerArchitecture

Broken todayStatus boards show which tickets closed, not whether the pieces still fit together.

Make the AI earn itArchitecture reviews where the AI presents the design, its trade-offs and the components it touches, linked to each senior engineer's evidence.

SeniorSystems

Broken todayIsolated PRs across many repos, no view of how they interact without hours of work.

Make the AI earn itA graph of every repo, plain-language summaries, and expert agents per component that answer with evidence.

JuniorCode blocks

Broken todayComplex code blocks and paradigms. Permission prompts for arcane atomic commands: read every line, or reject.

Make the AI earn itPlain-language intent first, backed by tests, a sim run and a visual of what the robot will do. Effectively a first-level manager of AI agents, but still closest to the code and responsible for its correctness.

The altitude ladder: the review unit grows with seniority, but the rule stays the same.

What “earned” evidence looks like

For a senior engineer, the change arrives as one system view rather than a pile of PRs. Click a component to question its expert agent.

Select a component.

Evidence completeEvidence thinUnvalidatedInterface changed

Illustrative mock-up of a generic robot software stack. Components, changes and figures are invented examples.

Visualisation is the evidence

One thing research taught me is how powerful visualisation is. It's often the fastest way to find what the numbers don't show. We used it extensively in research, but it used to be expensive to build. With AI, it's cheap. There is no reason not to make AI produce crisp, rich visuals that you can use as evidence and share among team members. Every level should get evidence it can see: robot trajectories for a junior, system graphs for a senior, traceability for a director, a strategic view for a VP.

So we’ve got the view. How does accountability work?

Validation follows the organisation

Responsibility for checking AI output should follow the hierarchy. What comes back up is not detail but confidence.

Responsibility is delegated down; confidence rolls up. The weakest link sets the rating, and the AI keeps the ledger.

The VP collects the strategy and evidence and delegates the sanity check of each area to directors. Directors break their area into an architecture, which managers check. Senior engineers validate the technical side, such as connectors, APIs and principles, and own behaviours and safety. Junior engineers own the code itself: errors, crashes and failure handling. At every level, the AI agents are specialised to the level of detail each role needs.

Each sign-off links to its evidence. The weakest part sets the overall rating. Issues only escalate when they cross a boundary. The AI keeps a record of who validated what, when, and against which evidence, which gives you an audit trail for safety and liability.

Summary

AI changes who writes the code. It doesn't change who is accountable for it.

In physical AI, accountability is not a cost on top of AI-native productivity. It is what we deliver.

We shouldn't ask engineers to trust AI. The AI should have to earn it.