AI governance operating model

The problem this quarter

Most Australian agencies and enterprises now have an “AI policy.” Far fewer have an AI governance operating model — the running machinery that decides which use cases get scrutiny, who signs off, and how risk stays visible after deployment.

The gap matters more this quarter than last. The DTA’s Policy for the responsible use of AI in government (v2.0, effective 15 December 2025) has already passed its 6-month tranche: non-corporate Commonwealth agencies were due to publish a strategic position on AI adoption and an AI transparency statement by 15 June 2026. The 12-month tranche (~December 2026) is the one that bites — an AI use-case register, an accountable owner for every use case, an impact assessment before deployment, and incident handling. A policy PDF cannot produce any of those. Only an operating model can.

The symptom is familiar. A team spins up a chatbot or a triage model. Nobody classified it. No gate caught it. There’s no owner of record and no monitoring after go-live. The organisation cannot answer “how many AI use cases do we run, and which are high-risk?” — which is precisely the question the register exists to answer. Risk is invisible, and invisible risk is unmanaged risk.

The opposite failure is just as common: a heavyweight review board that treats a meeting-notes summariser the same as a model that decides someone’s payment. Teams route around it. Shadow AI grows. You get the bureaucracy without the assurance.

The fix is one principle: govern in proportion to risk. Low-risk use cases move fast; elevated-risk ones get real scrutiny; and you can prove which is which.

The method

An AI governance operating model has six components. None of them are a PDF.

(a) Risk classification at intake. Every use case is classified the moment it appears — before build, before procurement. Classification is the routing logic for everything downstream: it decides how much governance a use case attracts. This is the step Kettera’s risk classifier exists to standardise, so two teams reach the same tier for the same kind of system. Classify on what the AI does to people and decisions — autonomy, consequence, who’s affected, whether it touches personal information or a government decision — not on which model or vendor sits underneath.

(b) Right-sized decision gates. A small review board and three gates:

• Intake gate — classify the use case, confirm it’s permitted, assign an accountable owner. Minimal-risk use cases clear here on a fast-track.
• Pre-build gate — for elevated-risk use cases only: confirm the impact assessment, data sources, human-oversight design and guardrails before money is spent building.
• Pre-deployment gate — confirm testing results, monitoring is wired, incident path exists, and the use case is entered in the register before it touches production.

The discipline is scaling. The board should see fewer use cases over time, not more — because the criteria, not the committee, do the filtering.

(c) Guardrails and meaningful human oversight. Proportionate technical and procedural controls — input/output constraints, logging, and a human who can actually understand and override the system, not a rubber-stamp “human in the loop.” Oversight is only meaningful if the person has the time, information and authority to say no.

(d) Incident handling. A defined path for when an AI system behaves wrongly — who’s notified, how it’s contained, how it’s recorded. The DTA’s 12-month tranche names incident handling explicitly; build it before you need it.

(e) Assurance. Governance is not a one-off sign-off. Elevated-risk use cases need testing before deployment, monitoring in production (for drift, performance, and unexpected behaviour), and periodic review on a set cadence. A model approved in March can be wrong by September without anyone changing a line of code.

(f) Mapping to the canon and the standards. The components above are what you do. The Australian canon tells you the expected practices, and the international standards tell you how to structure them:

Layer	Source	Role in your model
AU expectation	AI6 (NAIC/DISR, six essential practices)	Primary AU government guidance; evolves the VAISS
AU controls	VAISS (ten guardrails)	The detailed control catalogue beneath AI6
Govt obligation	DTA Policy v2.0	Register, owner, impact assessment, incident handling
Assurance scaffold	NIST AI RMF (Govern/Map/Measure/Manage)	Structures classification through assurance
Management system	ISO/IEC 42001:2023	Certifiable AIMS; harmonises with ISO 27001/9001

You don’t pick one. AI6 and the VAISS set the AU expectation; NIST AI RMF gives the lifecycle scaffold; ISO 42001 turns the whole thing into a certifiable management system that procurement can ask for.

A worked example

Meridian Water Authority is a fictional mid-sized state-owned utility. It has roughly a dozen AI use cases in flight — a customer-service assistant, a meeting summariser, a leak-detection model, a hardship-assessment tool that helps prioritise payment-relief cases, and a scatter of team-built experiments nobody’s tracking.

Meridian stands up a lightweight AI review board: four people (CISO delegate, privacy lead, a senior architect, a service-line representative), meeting fortnightly, with authority to approve, hold, or reject. Crucially, the board doesn’t see everything.

The fast-track. At intake, every use case is run through the classifier. Anything that lands minimal-risk — the meeting summariser, internal drafting aids, the search assistant — clears the intake gate automatically against a published checklist, with an owner assigned and a register entry created. No board meeting. Median time from request to “approved to build”: two days.

The scrutiny path. Two use cases classify as elevated-risk: the hardship-assessment tool (it influences a decision affecting vulnerable customers and touches personal information) and the leak-detection model (operational-safety consequence). These go through all three gates. The hardship tool gets an impact assessment, a documented human-decision step (an officer makes the call; the model only ranks), and a monitoring plan for disparate outcomes across customer cohorts. It’s held once at pre-build — the oversight design was too thin — then approved.

The result. Within a quarter Meridian has a complete use-case register, an accountable owner against each entry, impact assessments on the two that need them, and an incident path. Ten of twelve use cases never consumed a minute of board time. The board spent its attention where the risk was — which is the entire point.

The template

Copy and adapt. The gate criteria define what each gate checks; the checklist runs at intake.

Gate criteria

Gate	What it checks	Applies to
Intake	Use case classified; permitted (not prohibited); accountable owner named; register entry created; transparency obligations flagged	All use cases
Pre-build	Impact assessment complete; data sources and lawful basis confirmed; human-oversight design documented; guardrails specified; success/failure metrics defined	Elevated-risk only
Pre-deployment	Testing results reviewed (incl. failure modes); monitoring and drift detection wired; incident path live; transparency statement/notice ready; register entry finalised	Elevated-risk only (minimal-risk confirms register entry)

Use-case review checklist (intake)

• What does it do? One-line description of the decision or output, and who relies on it.
• Risk tier assigned via the classifier — minimal / limited / elevated.
• People affected — staff, customers, the public; any vulnerable cohort?
• Decision impact — does it inform or make a decision about a person, payment, or service?
• Personal information — does it ingest or infer it? Is automated decision-making transparency in scope?
• Accountable owner named (a person, not a team).
• Permitted use — confirmed not prohibited under policy.
• Human oversight — who can understand and override it, and do they have the authority?
• Register entry created.
• Route — fast-track (minimal) or escalate to pre-build gate (elevated).

A use case that can’t answer the first six lines isn’t ready to be governed — it’s ready to be paused.

What changes when the regulation moves

This model is built to absorb change at the classification and gate layer, not be rebuilt each time an obligation shifts.

• DTA, ~December 2026 (12-month tranche). The register, accountable owner per use case, pre-deployment impact assessment, and incident handling move from good practice to obligation for non-corporate Commonwealth agencies. Legacy use cases must be brought in by 30 April 2027. If your intake gate already creates register entries and assigns owners, this is a reporting exercise, not a scramble — but the legacy backfill is the work most agencies underestimate.
• Privacy Act ADM transparency (APP 1), commences 10 December 2026. Automated decision-making that significantly affects individuals will require transparency. Your checklist already flags personal-information and decision-impact use cases — those are the ones that need a notice. Make “ADM in scope?” a standing intake question now.
• EU AI Act. If you serve EU users, note the timing has moved: under the Digital Omnibus (agreed 6 May 2026, not yet formally adopted), Annex III high-risk obligations are delayed to 2 December 2027 and Annex I to 2 August 2028. The reprieve is real but soft — adopt the risk-tiered posture now; the obligations land regardless.
• National AI Plan (2 December 2025). No standalone AI Act; mandatory guardrails are shelved. Australia governs AI through existing technology-neutral laws and sector regulators. That makes your internal operating model the primary control surface — there is no single statute to wait for.
• ISO/IEC 42001 certification. Increasingly named in procurement. If your model already maps to NIST AI RMF and the VAISS, the gap to a certifiable AIMS is structural, not foundational — worth scoping before a tender forces the timeline.

The throughline: when regulation moves, you adjust the criteria inside the gates, not the gates themselves. That’s what an operating model buys you over a policy PDF.

Where this connects

Risk classification is the front door of this model — start with the risk classifier to tier a use case before it hits a gate. Benchmark how far your governance machinery actually runs with the maturity assessment (Governance axis). The obligations and standards referenced here are tracked in the regulatory canon. And see how governance sits alongside the other domains in the model.