The company’s board approves the deployment of an AI assistant for customer service. Two months later, it turns out the model sometimes answers questions about prices with data from before the price list update, no one can say which decisions were made based on AI recommendations, and the legal department doesn’t know if the system requires registration as a high-risk system under the AI Act. This isn’t an exceptional scenario. It’s the default outcome of AI deployment without governance.
Below, I describe how governance works in practice: what policies need to be defined, what roles should be established, what technical mechanisms to implement, and where the AI Act sets hard boundaries.
What is AI governance and why is its importance growing
#AI governance is a set of rules, procedures, and technical mechanisms that allow an organization to manage AI systems throughout their lifecycle: from concept through deployment, operation, to decommissioning. The concept covers three dimensions:
- Policy — what the AI system can do, what data it can process, what decisions it can make autonomously, and which require human confirmation.
- Organization — who owns each AI system, who is responsible for its operation, who can shut it down.
- Technical — guardrails, audit logs, alerts, regression tests, incident procedures.
In 2026, governance gained a regulatory dimension. The AI Act entered the stage of application for general-purpose systems and high-risk systems. RODO imposes on data controllers the obligation to assess risk (DPIA) before deploying a system processing personal data. The lack of governance is no longer a matter of organizational culture: it has become a potential basis for penalties from UODO or the AI Act supervisory authority.
For companies deploying AI in Poland, the practical context is also relevant: most organizations don’t employ dozens of AI specialists. Governance must be proportionate to the scale and maintainable by a team that also handles other tasks.
AI systems map: the starting point for any governance
#Governance can’t function if the organization doesn’t know what AI systems it operates. The first step is inventory.
The inventory should cover:
- In-house systems (built internally or by a technology partner).
- SaaS systems using AI (e.g., CRM with scoring functionality, recruitment platform with CV screening).
- Models embedded in existing tools (e.g., "suggested response" feature in help desk).
For each system, determine: what data it processes (including whether it processes personal data), what decisions it supports or makes autonomously, and what the potential impact of an error is on the user or organization.
The last point determines the risk classification. The AI Act in Annex III lists high-risk system categories: employment (scoring, CV selection), access to services (credit scoring, insurance scoring), education, critical infrastructure, public safety. A system outside these categories may still require a DPIA under RODO if it processes personal data on a large scale or processes sensitive data.
The result of the inventory is a register of AI systems: a living document updated with every new deployment or architecture change.
AI policies: what needs to be documented
#An AI policy is an operational document, not a marketing declaration. A good policy contains specific rules, not general aspirations.
Key areas that the policy should cover:
Permissible use cases. Which processes can be supported by AI, which require additional approval, and which are excluded (e.g., autonomous credit decisions without human-gate, processing health data by a model in an external cloud without data-residency in the EU).
Data processing principles. Whether and what personal data is sent to the model. Whether data is masked before being sent to the LLM. Where the model is hosted (self-hosting or external API) and how this affects the legal basis for processing.
Human-in-the-loop. What decisions the model can make autonomously and which require human confirmation. For high-risk systems, the AI Act requires human oversight as a condition for operation.
Incident management. How to respond to model errors, hallucinations in operational decisions, or data protection breaches by the AI system. Who decides to shut down the system. How much time the organization has to report to UODO (RODO: 72 hours from detecting a personal data breach).
Review and update. The AI policy should be reviewed at least once a year or after any significant change in the system architecture.
Roles and responsibilities: who is accountable for what
#Governance without assigned responsibilities is a wish list. Each AI system should have clearly defined roles.
| Role | Scope of Responsibility | Minimum Form |
|---|---|---|
| AI Owner (system owner) | responsible for system decisions: deployment, changes, shutdown; is the contact point for audits | specific person, not "IT department" |
| Data Steward | responsible for data used in the system: quality, legal basis, retention, removal of sensitive data | may combine with IODO/DPO role |
| AI Engineer | responsible for implementing guardrails, audit logs, regression tests, model updates | technical team |
| Human Reviewer | responsible for verifying decisions requiring human-gate; escalates anomalies | operational role, e.g., in customer service or HR |
| AI Auditor (internal or external) | conducts compliance reviews, tests red-team, verifies logs | may be external for smaller organizations |
For high-risk systems, the AI Act requires appointing a person responsible for system oversight (Art. 26 et seq.). In practice, this is the AI Owner role with a formal scope of authority to intervene in the system’s operation.
Small organizations often combine several roles in one person. This is acceptable, provided the scopes are clearly documented and do not create conflicts of interest (e.g., the AI Engineer should not be the sole AI Auditor of their own systems).
Technical mechanisms: how governance works in code
#Policies and roles are useless if they aren’t reflected in the technical architecture. Governance at the code level consists of four categories of mechanisms.
Guardrails. Barriers at the model’s input and output: blocking prompts attempting to extract out-of-scope data, filtering output for personal data, blocking responses outside the system’s scope. Guardrails are the first line of policy enforcement in real time.
PII masking. Personal data is identified and anonymized or pseudonymized before being sent to the model. This is both a RODO requirement (data minimization) and a safeguard against leaks via the model API. The LLM router (llm-router) is the right place to implement this layer.
Human-gate. Actions with irreversible consequences (sending, payment, data deletion, credit decision) require human confirmation before execution. Implemented via HMAC token: the agent generates a proposed action with a token, the human approves, the token is verified before execution. We describe this pattern in detail in the article on the role of humans in the AI loop.
Observability and audit logs. Every model call should generate a log containing: session identifier (without PII), tools used (in the case of an agent), response time, flag indicating whether the response required escalation. Logs are the basis for AI Act audits and post-incident analysis. Implementation details in the article on monitoring AI agent quality.
Risk assessment and DPIA: when is it mandatory
#DPIA (Data Protection Impact Assessment) is mandatory when the processing of personal data "may result in a high risk" to individuals. In the context of AI systems, UODO (and EROD) indicate the obligation for DPIA particularly for:
- large-scale profiling systems,
- automated decision-making with legal or similarly significant effects,
- processing sensitive data (health, biometrics, political opinions) by AI,
- employee monitoring using AI.
DPIA has three elements: a description of the system and processing purpose; assessment of necessity and proportionality; risk assessment and mitigation measures.
For AI systems, the third element is key: the risk assessment should cover both risks to personal data (leak, unauthorized access) and decision risks (model error, bias, hallucination). Risk mitigation measures are precisely the mechanisms described in the previous section: guardrails, PII masking, human-gate, observability.
DPIA is not a one-time document. It should be updated with every significant system change: new base model, expansion of processed data scope, infrastructure provider change.
AI system lifecycle: governance from concept to decommissioning
#Governance doesn’t start at deployment. Each lifecycle stage has its own requirements.
| Stage | Key Governance Actions |
|---|---|
| Concept and assessment | risk classification (AI systems register), preliminary DPIA assessment, verification of legal basis for data processing |
| Design and build | architecture documentation (input data, model, agent tools, output data), implementation of guardrails and PII masking, definition of human-gate for irreversible actions |
| Testing | functional tests, red-team tests (injection attempts, data leaks), bias tests (whether the model behaves differently for different user groups), DPIA review |
| Deployment | pilot with limited scope (shadow mode), monitoring of quality metrics (faithfulness, hallucination rate, escalations), decision to expand after proving safety |
| Operation | regular reviews (quarterly or after incidents), regression tests after model changes, DPIA updates for system changes |
| Decommissioning | removal of personal data from indexes and fine-tunings, archiving of audit logs in accordance with retention, documentation of decommissioning reason |
The decommissioning stage is often overlooked, but it’s important both for RODO (the right to be forgotten requires removing data from the model, not just the database) and security (a decommissioned model shouldn’t remain accessible via forgotten API endpoints).
Try it live
#Describe one of your AI systems or a planned deployment, and the model will indicate which governance elements are key for it and where to start (playground: PII masked, zero retention):
FAQ
#Do small companies also need to implement AI governance?
#Yes, though proportionate to scale. The AI Act and RODO don’t have an organizational size threshold: obligations arise from the nature of the system and the type of data processed, not the number of employees. A small company deploying a chatbot processing customer data must assess the legal basis for processing and implement technical measures (guardrails, PII masking). The difference lies in formalization: a large organization needs a formal AI systems register and an AI committee; a small one can manage this through a system owner and a simple checklist. The starting point is the readiness assessment tool.
When does an AI system require a DPIA?
#DPIA is mandatory when processing may result in a high risk to individuals. In practice for AI systems: when the system profiles individuals, when it makes or supports decisions with significant consequences (employment, credit, insurance), when it processes sensitive data, or when it monitors employees. In case of doubt, UODO recommends conducting a DPIA proactively, as the cost of preparation is significantly lower than the cost of a breach. We detail DPIA requirements in the article on AI Act and RODO 2026.
How often should the AI policy be reviewed?
#At least once a year, plus whenever there’s a significant system change: new base model, expanded data processing scope, infrastructure provider change, security incident. The review doesn’t need to be a comprehensive audit every time: a checklist verifying whether the policy still reflects the system’s technical reality and current regulatory requirements is sufficient. The review rhythm should be documented in the AI policy as a formal element.
What does it mean for an AI system to require human-gate?
#Human-gate is a mechanism requiring human confirmation before executing actions with irreversible or high-risk consequences. Examples: decision to reject a candidate in recruitment, sending a commercial offer, data deletion, executing a transfer, changing system configuration by an agent. The AI Act requires human-oversight for high-risk systems as a condition for operation. In technical practice, human-gate is implemented via HMAC token: the agent proposes an action with a signed token, the human verifies and approves, the token is checked before execution. We describe this pattern in the article on multi-step AI agents.
How to start building AI governance in an organization with no procedures yet?
#Start with three steps. First: inventory existing AI systems (in-house and in SaaS tools) with a risk assessment for each. Second: assign an owner for each system (a specific person, not a "department") and document basic rules: what the system can process, what decisions require human approval. Third: implement minimum technical measures: guardrails, PII masking, audit logs. This takes a few weeks for a simple system and provides a foundation for building more formal structures. The AI deployment plan step-by-step and agent blueprint help structure this process.