Ambition Under Regulation

Ambition Under Regulation

mallyanitin 27 Oct 2026 3:38

Regulated teams assume they must be the least ambitious with agents. They are holding the best inputs anyone has.

Kauṭilya, writing the Arthaśāstra more than two thousand years ago, had no illusions about officials and money. He catalogued, in the text’s own systematic way, dozens of methods by which treasury officers could embezzle: delaying entries, recording receipts as smaller than they were, booking expenses that never happened, and so on.

He also offered a famous comparison. Just as you cannot tell whether a fish swimming in water is drinking it, you cannot easily tell whether an official handling money is taking some.

His answer was not to trust officials more, or to stop using them. It was mechanism. Written records in prescribed formats. Accounts submitted on fixed dates. Independent checks by separate officers. Rewards for informers and penalties for discrepancies. The state ran on people it could not fully trust, by building the instruments to verify them.

That is the most useful framing I know for agentic coding in regulated industries.

The fish in the water

I spent many years building software for medical devices. The instinct when agents arrive is caution bordering on refusal. “We can’t let AI write that. Think of the audit.”

I understand the instinct. But it gets the logic backwards.

The agent is Kauṭilya’s official: capable, fast and impossible to fully observe from the inside. The answer is the same as his. Not trust, not prohibition, but instruments. And regulated teams already have more of those instruments than anyone else.

Compliance artefacts are oracles in waiting

For twenty years, medical-device and payments teams have written down, in structured form, what must never happen and how they would prove it has not. We call this compliance overhead. For an agent, it is the richest specification available.

Consider what an ISO 14971 hazard analysis actually is. (ISO 14971 is the international standard for risk management of medical devices.) It is a list of things that must never happen, each with a cause, a severity, a probability and a control. That is a list of invariants with their oracles attached. An IEC 62304 traceability matrix (IEC 62304 governs how medical-device software is developed and maintained) links every requirement to its design and its tests. That is a machine-checkable map of what the oracle must cover. A central bank’s control framework is a set of rules waiting to be written as policy code.

In plain terms: SOUP (software of unknown provenance) is third-party code not built under your own regulated process; an SBOM (software bill of materials) is an inventory of every component in a build; PCI DSS is the payment card industry’s data-security standard; OPA (Open Policy Agent) is an engine that evaluates rules written in its language, Rego, so a policy can be tested and versioned like code; a fault-injection test causes a failure on purpose to prove the system copes.

I had earlier said altitude requires an oracle for the invariant. Regulated teams have spent two decades writing those oracles in prose. The work now is to turn the prose into code.

Traceability as a by-product

The biggest change I expect is in timing. Today, traceability is often reconstructed in a painful phase before release. With agents, it can be maintained continuously as a side effect of every change.

The agent that changes a function also updates the trace from requirement to test. A CI check verifies the trace is complete. The audit trail records which agent made the change, which checks passed and which human approved it. By release day, the design history file, the regulated record of how a device was designed and verified, is already written, because it was written one commit at a time.

Maryādā: ambition within bounds

Rāma is called maryādā puruṣottama, the ideal person who lives within bounds. He crosses an ocean and fights a war. But he does it within limits he accepts and keeps.

That is the right posture for regulated agentic work. Be ambitious about what agents do, and absolutely clear about the bounds. Here is a small example of a bound written as code, an Open Policy Agent rule evaluated on every pull request:

package agent.pullrequest
import rego.v1
deny contains msg if {
    input.author.type == "agent"
    some f in input.changed_files
    startswith(f, "ledger/")
    count(input.approvals.human) < 2
    msg := sprintf("%s changes the ledger: two human approvals required", [f])
}

The agent can work on the ledger. It cannot merge to it alone.

In my Bounded Autonomy posts I went deeper into the mechanisms: policy as code, workload identity with SPIFFE (an open standard that gives each service a verifiable cryptographic identity), durable execution (work that survives crashes and resumes where it stopped) and tamper-evident audit trails (logs that reveal any later alteration). They are what make maryādā enforceable rather than aspirational.

What stays human

Some things should not move to agents, however good the oracle:

• Risk acceptance. Deciding that a residual risk is acceptable is a judgement with a name attached.
• Clinical and financial judgement. Whether a behaviour is safe for a patient or fair to a customer.
• Release sign-off. The decision to ship belongs to an accountable person.

The agent can prepare all the evidence for each. It should not make the call.

For the Weekend

Pick one regulatory artefact your team already maintains. Choose five entries. For each, write the mechanical check that would detect a violation.

Example. Suppose the artefact is the hazard log for a dose-calculation module in an infusion-planning application.

Step 1: make the hazard log machine-readable. Most teams keep it in a spreadsheet or a Word table. Move it to a versioned file in the repository, under a protected path:

# risk/hazards.yaml  (protected: changes need the risk owner's approval)
- id: H-07
  hazard: Dose calculated in mg when the order is in mcg (1000x error)
  severity: catastrophic
  controls:
    - RC-07a: unit-aware types make mg and mcg impossible to mix
    - RC-07b: output unit always equals the ordered unit
  verified_by:
    - tests/test_dose_units.py::test_mg_and_mcg_cannot_be_added
    - tests/test_dose_units.py::test_output_unit_matches_order
- id: H-12
  hazard: Occlusion alarm suppressed for more than 120 seconds
  severity: serious
  controls:
    - RC-12a: suppression timer is capped and cannot be extended
  verified_by:
    - tests/integration/test_alarm_suppression.py::test_suppression_capped_at_120s

Step 2: turn each control into code. RC-07a as a type:

from dataclasses import dataclass
from decimal import Decimal
@dataclass(frozen=True)
class Mg:
    value: Decimal
    def __add__(self, other: "Mg") -> "Mg":
        if not isinstance(other, Mg):
            raise TypeError(f"cannot add {type(other).__name__} to Mg")
        return Mg(self.value + other.value)
@dataclass(frozen=True)
class Mcg:
    value: Decimal
    def to_mg(self) -> Mg:
        return Mg(self.value / Decimal(1000))

RC-07b as a property test:

from hypothesis import given, strategies as st
UNITS = st.sampled_from(["mg", "mcg"])
AMOUNTS = st.decimals(min_value="0.001", max_value="5000", places=3)
@given(ordered_unit=UNITS, amount=AMOUNTS, weight_kg=st.integers(1, 200))
def test_output_unit_matches_order(ordered_unit, amount, weight_kg):
    order = DoseOrder(amount=amount, unit=ordered_unit, per_kg=True)
    result = calculate_dose(order, patient_weight_kg=weight_kg)
    assert result.unit == ordered_unit

Step 3: make traceability a CI gate. This script fails the build if any hazard’s verifying test is missing, failed or skipped. It reads pytest’s JUnit XML report:

# ci/check_traceability.py
import sys, xml.etree.ElementTree as ET
import yaml
hazards = yaml.safe_load(open("risk/hazards.yaml"))
passed = {
    tc.get("name")
    for tc in ET.parse("reports/junit.xml").iter("testcase")
    if not any(child.tag in ("failure", "error", "skipped") for child in tc)
}
broken = [(h["id"], t) for h in hazards for t in h["verified_by"]
          if t.split("::")[-1] not in passed]
for hazard_id, test in broken:
    print(f"{hazard_id}: verifying test not passing: {test}")
sys.exit(1 if broken else 0)

Step 4: tell the agent how to live within the bounds. This goes in the project’s CLAUDE.md:

This repository implements a regulated medical device under IEC 62304. Every change to src/dose/ must keep ci/check_traceability.py passing. You may add tests and update verified_by entries in a proposed diff, but you may not edit risk/hazards.yaml directly; describe the change in your pull request and the risk owner will apply it. Never add, remove or reword a hazard or control. If a change you are making could affect a hazard, name the hazard ID in the pull request title.

The hazard log has become an oracle the agent cannot edit and must keep green, and the design history file, the regulated record of how the device was designed and verified, grows one commit at a time.

You will find that the compliance team has been writing your agent’s oracle for years. Nobody told them.