Skip to content

Guide

The SME Guide to AI Governance

4 min read

In short

AI governance for a small or mid-sized organisation rests on three things: an inventory of the AI actually in use including undeclared tools, a risk tier for each system based on the harm it could cause, and an acceptable use standard staff will follow. None of it requires a dedicated compliance function.

Why "wait and see" stopped being an option

The argument for waiting used to be reasonable: the regulation was unsettled, the tooling was immature, and the risk of moving early looked higher than the risk of moving late. That has inverted, and not primarily because of regulation. It inverted because AI arrived inside organisations without being adopted. Nobody approved it and it is running anyway.

For an organisation without a compliance function, the practical question is not how to build a governance programme. It is how to get to a defensible position with the people you already have.

Part 1: the shadow AI problem

Your staff are already using AI. If it has not been sanctioned, they are using it on personal accounts and personal devices, which means company data is leaving through a route you cannot see, log or revoke.

The instinct is to ban it. Banning does not work and makes the position worse: usage moves further out of sight, onto devices you have no visibility of at all, and you lose the only signal you had.

What works is sanctioning with guardrails. Provide a tool that is genuinely good enough, with an enterprise agreement covering data use, and most usage moves to it within weeks. Shadow AI is overwhelmingly people trying to do their jobs faster, not people trying to evade a policy.

Finding it takes four routes used together, because each one misses different things:

  • SSO and identity logs show which AI services staff have authenticated to.
  • Card and expense data surfaces individual subscriptions, but misses free tiers.
  • Network or egress data shows traffic to AI endpoints, but misses personal devices.
  • Structured conversations with team leads surface what the data cannot.

Part 2: tier by harm, not by technology

The single most useful decision is refusing to treat all AI the same. A model that reorders internal search results and a model that screens job applicants warrant completely different treatment, and applying uniform rigour to both wastes effort on one while under-protecting the other.

A workable three-tier model, which also maps onto how the EU AI Act thinks about risk:

Prohibited. Practices banned outright under the EU AI Act since 2 February 2025, including social scoring and certain biometric categorisation. If a use case is here, the answer is no, and the conversation is about finding a different approach.

High risk. AI used in decisions that materially affect people - hiring, credit, insurance pricing, access to essential services, education, and safety components in regulated products. This tier gets a full impact assessment before deployment, documented human oversight, and a named owner. Under the EU AI Act, Annex III high-risk obligations apply from 2 December 2027, deferred from August 2026 by Regulation (EU) 2026/1744.

Limited and minimal risk. Coding assistants, drafting support, internal summarisation, customer service chatbots. This tier needs an acceptable use standard and, where a person is interacting with an AI system or seeing synthetic content, transparency - which has been an EU AI Act obligation since 2 August 2026.

Part 3: the acceptable use standard

The policy that works is short, specific and explains itself. A rule without a reason gets ignored the first time it is inconvenient.

Permitted, and worth saying explicitly, because a policy that only lists prohibitions reads as hostile: drafting and editing, summarising material that is already internal, refactoring and reviewing code, generating test data, research and exploration.

Not permitted without approval: entering customer personal data, financial or commercially sensitive material, or unreleased code into a service without an enterprise agreement covering data use. Using AI output in a decision affecting a person without a human reviewing it. Publishing AI-generated material externally without disclosure where disclosure is required.

Always required: a human accountable for any output that reaches a customer, and disclosure where a person is interacting with an AI system.

Part 4: what "governed" actually means

An organisation is in a defensible position when it can answer four questions without a project:

  1. What AI is running? A maintained inventory, with an owner per system.
  2. Who approved it? A named person, and a record of the decision.
  3. What harm could it do? An assessment proportionate to the tier, completed before deployment.
  4. Where did the model come from? Provenance, licensing and change notification for each provider.

None of those require a compliance team. They require someone to own it, an hour a week, and the discipline to write decisions down at the point they are made rather than reconstructing them later.

Where this leads

If certification becomes the destination, this work is not wasted - it is roughly the first third of an ISO 42001 programme. If it does not, you still have the four answers, which is what a customer's security questionnaire and your own legal team are going to ask for either way.