AI Act: high-risk systems in practice (HR, finance, scoring)

A company builds a recruitment tool that ranks candidates based on CVs. Or a credit pipeline that pre-assesses applications. Or a scoring system for B2B customers that prioritizes leads. Each of these tools does exactly what the AI Act scrutinizes most closely: it affects an individual’s situation in a way that can bring them benefit or harm.

In 2026, this is not theory. From August 2026, regulators have enforcement mandates, and fines can reach 3% of global turnover for violations in high-risk systems. Below, we describe which systems fall under this regime, what obligations arise from it, and how to design compliance so it doesn’t block business value.

AI Act Annex III explicitly lists high-risk areas. For Polish companies, four are relevant:

The boundary between "limited risk" and "high risk" hinges on one question: does the system evaluate or classify a person in a way that affects a specific decision about them? A chatbot answering questions from a knowledge base is limited risk. The same chatbot, which assesses a customer’s creditworthiness during a conversation and adjusts the offer based on that, enters high risk.

When a system qualifies as high risk, the AI Act imposes six concrete obligations. These are not recommendations.

1. Technical documentation. Before the system is deployed, the provider must prepare documentation describing: the system’s purpose and scope, data used for training or calibration, quality and accuracy metrics, known limitations and edge cases, logging paths. The documentation must be available to supervisory authorities upon request.

2. Human oversight. Every decision significant to a person must be confirmed by a human or be technically reversible. The pattern: AI recommends, a human approves or rejects, the system logs both actions. Human-gate is not optional.

3. Transparency toward the affected person. The person impacted by the system has the right to know that the decision was AI-assisted and to request an explanation of the logic. This requirement must be implemented technically, not just mentioned in terms of service.

4. Risk management. Continuous cycle: risk identification before deployment, monitoring post-deployment, documentation updates when the model or data changes. At least annually; more frequently for significant changes.

5. System registration. Providers of high-risk systems must register their products in the EU AI database. For internally used systems (deployers), registration lies with the system provider, but the deployer must verify that the system is registered.

6. DPIA for personal data processing. If the system processes personal data (which it almost always does), GDPR applies in parallel. A data protection impact assessment is required when the risk is high. In practice: candidate scoring, credit scoring, and customer profiling always require a DPIA.

A recruitment system is a classic example of an AI Act high-risk system. The key workflow looks like this: the model reads CVs, extracts features, classifies the candidate, and ranks applications. Each of these steps is permitted. The problem arises when the ranking result is applied without verification by a recruiter.

Three technical requirements that must be met before such a system goes into production:

• Classifier must operate on anonymized features. Name (indicating gender and ethnicity), age, and address must be removed before being passed to the evaluation model. Not by policy, but by PII masking at the code level.
• Every recommendation must have an explanation log. Which features determined this candidate’s ranking? Without this log, you cannot demonstrate the absence of discrimination or answer a candidate’s question about the decision’s basis.
• Human-gate before communication with the candidate. No automatic rejection response is sent without recruiter approval. AI proposes, a human approves.

More about the architecture of a recruitment pipeline and training data is discussed in the article AI in HR and recruitment.

Credit scoring and insurance risk assessment are areas where the AI Act imposes the strictest obligations. The reason is simple: a decision to grant or deny credit can radically change a person’s life situation.

Specific requirements for financial systems:

• Explainability of every decision. A customer denied credit with the help of an AI system has the right to an explanation of the main factors behind the decision. SHAP values, LIME, or another explainability mechanism must be implemented in production, not just in an analytical notebook.
• Testability for discrimination. The scoring model must be regularly tested on control datasets to ensure approval rates do not systematically differ between protected groups. A difference that cannot be explained by financial variables is a regulatory problem.
• Human oversight on borderline decisions. Applications with scores within the uncertainty range should be sent for verification by an analyst, not automatically rejected by the algorithm.
• Data isolation and data residency. Customers’ financial data should not leave the institution’s controlled infrastructure without encryption and data processing agreements. Self-hosting the inference layer for the most sensitive data is the standard, not a luxury.

In Poland, the Financial Supervision Authority (KNF) issued guidelines in 2025 on AI in financial services, which supplement the AI Act with sector-specific requirements. Every scoring system for institutions regulated by the KNF must be audited for compliance with both sets of requirements.

Not every lead scoring is high risk. Classifying B2B leads by fit to ICP (industry, company size, website activity) is limited risk, as long as the result does not determine access to a service.

The problem arises when scoring starts to affect pricing, offer availability, or how a customer is treated. Then, depending on the area, the system may fall under the high-risk regime or at least require a DPIA.

A practical rule to apply when designing the system:

Lead scoring based on anonymous activity (website, UTM, company type) is safe. Scoring that combines financial data, social media data, and behavioral history of a specific person falls under regulated territory.

Compliance with the AI Act for high-risk systems is not a layer added at the end of the project. Six elements we design from the first commit:

Log every step. Every model call, every recommendation generated, every approval or rejection by a human goes into a log with a timestamp, session identifier, and the identifier of the affected person. The log is immutable and stored for at least 5 years for financial systems, at least for the duration of complaint processing for recruitment.

Human-gate as a technical block. Human-gate is not a message saying "remember to check." It is a technical impossibility to send the result to the end user without confirmation by an authorized person. We implement it as a condition in the pipeline, not as a request in the documentation.

Guardrails with a rejection log. Every query the system rejects due to guardrails (out-of-scope, attempt to force a decision beyond the system’s competencies) is logged. The rejection log serves as proof that the system operates as designed.

Explanation module. Every decision affecting a person must be technically reproducible: what the model saw, which features were significant, what the result was. Structured output with a mandatory reasoning field facilitates building this module.

Discrimination tests before production. Before deployment, we run the classifier on a test set with a controlled distribution of protected features and measure the difference in positive rates. Results go into the technical documentation.

Model change management. Changing the base model, data, or decision threshold requires going through the validation cycle again and updating the documentation. We do not make silent model swaps in production.

More about monitoring AI system quality post-deployment is discussed in a dedicated article.

A Data Protection Impact Assessment (DPIA) is required by GDPR when processing may pose a high risk. In practice, every AI Act high-risk system that processes personal data requires a DPIA. Which is almost always.

Minimum DPIA content for an AI system:

• Description of the processing purpose and input data
• Necessity assessment: can the goal be achieved with less data?
• Risk identification: unauthorized access, algorithmic discrimination, incorrect decision
• Mitigation measures: encryption, PII masking, data isolation, human-gate, logging
• Review after 12 months or with significant system changes

A DPIA is a working document, not a formality. Written thoroughly, it becomes the best technical documentation for the project and shortens response time to supervisory inquiries from weeks to hours. The DPIA methodology is described in more detail in the article AI Act and GDPR 2026.

Describe your system (HR, scoring, finance) and the scope of processed data, and the model will preliminarily assess which AI Act obligations may apply—as a starting point for a conversation with a lawyer, not a replacement (playground: PII masked, zero retention):

Not always. Scoring based on company data (industry, size, activity) without affecting service access is usually limited risk. High risk arises when scoring concerns individuals in areas listed in the AI Act—employment, finance, insurance, public services. If scoring affects decisions about an entrepreneur as an individual (e.g., sole proprietorship, microloan scoring), AI Act obligations may apply. Always verify with a lawyer before production deployment.

Through logs and testing. The log must contain for each recommendation: which features were significant and with what weight. Pre-deployment test: run the classifier on a dataset with a controlled distribution of protected features (gender, age, ethnicity indicated by name) and measure whether positive recommendation rates are comparable between groups. A difference of more than a few percentage points requires model or data correction. Test results go into the technical documentation as evidence, not just a declaration.

The AI Act distinguishes between provider and deployer but does not exempt small entities from high-risk system requirements. Micro-enterprises and small companies benefit from a simplified technical documentation format, but substantive obligations remain: human oversight, logging, transparency, DPIA when applicable. The system’s scale and resources affect implementation, not whether the requirements apply.

When the change affects decision logic or input data. Swapping the base model (new LLM version, different embedding), changing training data, adjusting the decision threshold beyond a set margin, adding new input features—each of these changes requires retesting for discrimination, updating documentation, and an entry in the change registry. Changing the system prompt without altering decision logic usually does not require full revalidation but requires an entry in the change log and a smoke test.

No. Guardrails block unauthorized queries and limit the system’s response scope—this is a necessary security layer. But the AI Act requires that a human have a real ability to verify and reject every decision significant to the affected person. Guardrails are not a human decision—they are a filter before the decision. Both layers must be present and logged separately.