Governance and the AI Act — security, evidence and responsibilities

Governance rarely fails for lack of intent. It fails for lack of evidence. A company usually has a rough idea which AI systems are running, who introduced them and what they are used for — but cannot demonstrate any of it when a customer, an auditor or a regulator asks. This page covers four tasks that close that gap: knowing your inventory, classifying risk, naming responsibilities, and keeping evidence that exists when it is needed rather than being reconstructed afterwards.

This is an assessment from implementation practice, not legal advice. Where the legal classification of a specific system is at stake, that needs a lawyer.

Know the inventory first

Almost every governance project starts with an uncomfortable discovery: more AI-assisted systems are running than the list shows. Bought-in software gained an assistant feature, a team procured a service through its own budget rather than IT, a script has been calling a model API for months. Without a reliable inventory, every subsequent assessment describes a fragment:

• Bought-in as well as built in-house. An assistant feature inside standard software raises the same questions as your own model — the difference is who has to supply the answers, not whether they are needed.
• Purpose rather than product name. What a system actually decides or prepares in daily use is the relevant fact. The product name says little about that, and it changes with the next release.
• The kinds of data flowing in. Personal data, trade secrets, customer data held under third-party contracts: the data type determines which obligations apply at all.
• Who introduced it and who operates it now. In practice these are often two different parties, and the second frequently does not know what the first promised.

This inventory is not a one-off document. It is only useful if there is a fixed trigger for updating it — at every procurement and every significant release, for instance.

Classify risk instead of treating everything alike

The AI Act is built on risk tiers: a small number of practices are prohibited, a limited group of applications counts as high-risk and carries substantial obligations, a further set is mainly subject to transparency requirements, and a large share of everyday business use falls into none of these categories. That gradation is the real reduction in workload — provided it is actually used:

• Classify per application, not per vendor. The same model can be uncritical in one application and high-risk in another. What matters is the purpose of use, not the technology underneath.
• Establish your own role. Whether a company provides a system or merely deploys it shifts the obligations considerably. Passing on a bought-in system under your own name, or modifying it substantially, can move you into a different role than procurement assumed.
• Do not underestimate transparency duties. Ensuring people can tell when they are interacting with an AI system, or looking at synthetic content, affects many applications that would otherwise count as uncritical.
• Record the reasoning, not just the verdict. A classification whose justification nobody remembers is worthless at the next review. The sentence explaining why a system is *not* high-risk matters as much as the classification itself.

The Act's obligations phase in over several years. Practically, that means the classification should be in place so effort lands where it belongs instead of being spread evenly across everything.

Name responsibilities before an incident forces it

The most common governance gap is not a missing policy but a policy with no named person behind it. As long as ownership stays implicit, nobody picks it up:

• Approval. Who decides a system may go into operation — and on what basis? An approval without a documented basis is a signature, not a control.
• Operation. Who watches the system in live use and notices when its behaviour changes, for example after the vendor swaps the underlying model?
• Shutdown. Who may stop a system without first working through an approval chain? During an incident this is the most important authority, and in normal operation the least often settled.
• Answering questions. Who responds to a customer, a regulator or a works council — and does that person have access to the records they need?

Four named roles are little effort. Their absence only becomes visible during the first incident, and it is expensive then.

Keep evidence that exists when it is needed

The obligations nearly all reduce to one question: can you demonstrate what you claim? Evidence assembled on demand is weakest at precisely the moment it is required:

• Record decisions when they are made. Why this system, why this classification, why this level of autonomy — written down when the decision was taken, not reconstructed after someone asked.
• Logs that support reconstruction. For systems with scope to act, it has to be possible to establish later what the system did and on what basis.
• Treat changes as events. A vendor swaps a model, a prompt is adjusted, a new data source is added: each of these can affect the classification and belongs on the record with a date.
• Make human oversight demonstrable. "A person checks it" only becomes a control when it is visible who checked what, when, and what happens when they reject the output.

Cybersecurity belongs in the same evidence frame. An AI system widens the attack surface by accepting input from untrusted sources, by holding tools it may call, and by the data it can emit. Access rights, logging and possible data-egress paths need documenting for these systems exactly as for any other production system.

Where this goes wrong in practice

The typical pattern is a governance project that starts with the document instead of the inventory. A policy is written, approved and filed — describing a state nobody verified. Six months later the first customer enquiry reveals that several production systems do not appear in the policy at all, that nobody approved two of them, and that the person who introduced the third has left. The effort the policy was meant to avoid arrives in full — only now under time pressure, with a customer waiting.

The second recurring error is uniform treatment: every system is pushed through the same heavyweight procedure because no classification exists. The result is a process teams route around, because it is disproportionate for the uncritical case — and so it stops working where it would actually have mattered.

Next step

If you want to know which systems in your organisation are affected at all, how a specific use case classifies, or which evidence is missing for an upcoming customer or audit request, get in touch. A short description of the systems in use and the specific trigger is enough to start from.