






















Last month, the draft guidelines were published defining when an AI system qualifies as high-risk under the EU AI Act. We asked Carlo Salizzo and David Kirton of Dentons Ireland for their legal assessment.
On 19 May, the European Commission published draft guidelines defining when an AI system qualifies as ‘high-risk’ under the AI Act, triggering a substantial set of legal obligations. The guidelines are open for consultation until 23 June 2026.
We asked tech partners at Dentons Ireland, Carlo Salizzo and David Kirton, to take a deep dive into these guidelines and answer some of the key questions that organisations and start-ups are asking.
The draft guidelines are a long-awaited publication from the European Commission designed to give more clarity to businesses across the EU. Many of the most burdensome obligations under the AI Act are only triggered in respect of AI systems that are designated high-risk, so whether or not a given AI system will fall into that category is a very significant question for businesses as they prepare for compliance with the legislation.
The guidance is designed to be a supporting layer on top of the legislation itself, and specifically the triggers for the very detailed requirements that apply in respect of the supply and the use of AI systems that are considered to “pose a significant risk of harm to health and safety, or an adverse impact on fundamental rights”.
While the guidance sets out the European Commission’s approach, it is not binding on the courts and will ultimately be subject to the decisions of the Court of Justice of the EU.
Realistically, every business in Europe using or selling AI is impacted by the EU AI Act. With agreement in principle between the EU institutions on a delay to the application of the rules on the two categories of high-risk AI systems until 2 December 2027 and 2 August 2028, the immediate priority for every organisation is to map out what AI is already being used for, and checking which of those use-cases is considered ‘high risk’.
It’s also important to have a process in place for evaluating newly proposed use-cases against the AI Act rules as they arise. These draft guidelines, once final, will be the critical document used to answer those questions. If a use falls into the ‘high risk’ category, the organisation has a range of specific obligations to comply with when it is carrying out the corresponding activities.
As in the Act itself, the line is structurally clear even if there may be overlaps and grey areas in practice. There are two distinct sections of the guidance, one applicable to AI systems that are themselves, or are used as safety components in, specific regulated products (under Article 6(1) of the AI Act) – such as medical devices or certain industrial machinery – and the other applicable to AI systems which are being used for purposes that are considered ‘high risk’ (under Article 6(2) of the AI Act). The section of the guidance dealing with the latter category runs to some 148 pages and is by far the most detailed of the three parts.
If previous guidance is anything to go by, this document should be a strong indication as to where the final guidance will land, but it is absolutely not final. Businesses should use this document to inform the steps they take toward compliance, but on the understanding that the content will change between now and the date the requirements come into force.
The European Commission has clearly drafted these guidelines with a focus on promoting competitiveness and innovation. One example given as falling outside the high risk category is a chatbot provided by a university for admission information, partly because it does not provide personalised recommendations that could influence an admissions decision. Given the impact that such a chatbot could have on a young person’s decisions around their education, this is the kind of AI system that the Commission could well have regarded as high-risk on a more conservative view.
Of course, it is entirely possible that such a chatbot could be designed to engage with a prospective student and give personalised advice which materially influences them toward a particular course – which is where product-specific risk and compliance reviews will need to come in. The Commission’s approach overall appears to be more pro-innovation than might have been expected, and there may well be pushback from submitters as to the breadth of some of the examples falling outside the high-risk category in the Commission’s eyes.
The examples in this document seem to have been considered carefully. Naturally, there is not an example to match every possible use of AI, and the bulk of the guidance is dedicated to providing explanations and examples for ‘high risk’ use cases under Article 6(2). However, it compares favourably to guidance given in the data protection context, which due to the breadth of the regulation in that context will often hedge its examples with “Depending on the circumstances…” or similar.
The use case examples are drawn from a broad array of industries and businesses, and in most cases will be relevant by analogy across industries. For example, the scenario of a logistics company using an AI system to assign shifts, rest periods and on-call windows is considered high risk, and this may well translate to a similar application within another industry, such as healthcare or manufacturing.
The requirements for high-risk uses of AI systems operate on a granular level. It is likely that a large organisation will be using a number of different AI systems across different functions, often with very different use cases. Each one needs to be assessed separately, so organisations of all sizes need to know what AI systems they are using, and also what purposes those AI systems are being used for.
A compliance programme generally involves some form of governance committee or steering group – with experts drawn from areas such as IT, information security, legal, privacy, operations, HR and elsewhere – reviewing each proposed new use case to identify the potential issues in each of their areas of responsibility. Once a use case has been approved, there will be a range of conditions and compliance requirements that must be put in place both under the AI Act itself as well as other laws such as data protection, employment or sector-specific rules. Businesses in regulated sectors like finance will be well acquainted with the challenges that overlapping frameworks can present.
The obligations for providers and deployers are very distinct, with the more onerous requirements falling on the provider of an AI system (as they are ultimately the one putting the product on the market and therefore have more control over how that system is designed and trained). The guidelines themselves are focused on classification of risk rather than role, but they are very significant in terms of the restrictions that providers are likely to place on deployers as to the purposes for which AI systems are permitted to be used under the applicable terms of service.
These take on extra significance under the AI Act itself, as the deployer is required to operate an AI system within the boundaries set by the provider. If they market the product under their own trade mark, make a substantial modification, or modify the intended purpose of a non-high risk system such that it becomes high-risk (potentially by removing guardrails to redirect its purpose to a high-risk one), they are treated as the provider of such modified system.
Start-ups are known for squeezing every bit of value out of their tools, so they need to be careful to avoid straying into high-risk areas that might put them in breach of their contractual and legal obligations, and impose upon them all the obligations of being a provider.
For those kinds of organisations, the most likely boundary issues often arise less from customer-facing AI and more from internal workforce tools. For example, AI used to support recruitment, allocate shifts or tasks, monitor staff activity, evaluate performance, or influence promotion or termination decisions may fall within one of the AI Act’s core high-risk employment categories. By contrast, a tool used only for generic stock forecasting, customer-service drafting, or marketing content generation may be outside the high-risk regime, although other AI Act and data protection obligations may still apply.
A non-technical business owner should not try to answer the question at the level of the technology alone. The key questions are: what is the tool actually being used for, whose decisions does it influence, does it assess or profile individuals, and is it being used in a sensitive area? If the business has started to map its AI tools and has a proportionate governance and review process for new use cases, it is on the right track.
Business owners should use the period before the high-risk rules apply to take advice, put in place policies, train staff and make sure that high-risk uses are escalated before deployment. That work should sit alongside compliance with the parts of the AI Act already in force, including AI literacy. Where there is any doubt it is often best to take a cautious approach and prepare for compliance from the outset, rather than running the risk of needing to reverse-engineer a compliance programme after something has gone wrong.
Both. The platform is likely to be a provider and the customer a deployer in that scenario, but it’s possible that by putting its own brand on the product or making changes to it that the deployer becomes a provider in its own right. In any case, while the obligations on the provider are heavier and more product-specific, the deployer still needs to closely track which high-risk use cases its AI systems are being utilised for and comply with a range of obligations for each one.
Risks are significant – including the potential for fines once the Irish and EU regulatory enforcement model is implemented in full. This is another situation where a deployer needs to be very careful that it does not inadvertently trigger obligations as a provider, due to a well-meaning project modifying an AI tool in a way which crosses the line into ‘high risk’.
For providers, it is very important to not only state clearly what the intended purposes of an AI system are, but also to implement additional protections to prevent it from being misused. Otherwise, the guidance indicates that there will be a presumption that it may be used for feasible and reasonably foreseeable high-risk purposes. This is a very significant part of the guidance, and we can expect to see submissions as to how much effort the provider needs to put into warding deployers away from unsupported uses.
For most businesses, the most likely area where high-risk AI will be used (whether intentionally or inadvertently) is using AI in the context of employee relations, performance reviews or other employee-management tasks. Staff with responsibility for those functions should be extra vigilant that their use of AI remains within appropriate boundaries.
Having a human sign-off is regularly cited as a shortcut to avoiding compliance with AI rules. While a ‘human in the loop’ is a sensible (and required) way to manage the risk posed by high-risk AI systems, it is not a ‘silver bullet’ to avoid regulation under the AI Act or indeed other legislation such as the GDPR. The draft guidelines confirm this, noting that human involvement cannot change the purpose and area in which a system is intended to be used. Rather, oversight by a human with appropriate competence and resources is one of several conditions for compliance.
That said, if a system is genuinely designed to perform a ‘narrow procedural task’ or is preparatory in nature, or is designed to improve a previously completed activity, then it will fall outside the scope of the high risk requirements.
There is a narrow exemption for AI that does not materially influence decision-making. In practice, which types of business use cases are most likely to qualify – and which will find that argument hard to sustain?
An AI system may fall within an Annex III area but escape high-risk classification if it performs a narrow procedural task, improves a completed human activity, detects decision-pattern deviations with proper human review, or merely performs a preparatory task.
This so-called ‘filter’ is designed to account for AI being used within the scope of high-risk activities, but in such a limited way that, in practice, the risk posed by that system is low.
This filter only applies in respect of the Article 6(2)/Annex III purposes rather than already-regulated products in Annex I, so if an AI system is a regulated product (eg a medical device) or a safety component for one, this potential ‘get out’ is not available.
It also cannot be used when the AI system performs profiling of individuals. This might involve performing sorting or formatting functions in relation to data rather than classifying it or profiling the people it relates to. Administrative tasks like scanning handwritten documents onto a filing system and converting legacy files into a new format are also likely to be captured by the filter. There are helpful examples given at section 2.7 onward in the draft guidance.
On the other hand, if an AI system is being used to make assessments or recommendations to a human who is expected to ‘sign off’ on them, or who is too busy or inexperienced to critically exercise their own judgement, this is unlikely to be caught by the filter and will pass through to be considered a high-risk use case if it meets the criteria.
While there are challenges in each aspect of compliance with new rules, putting in place risk management processes is likely to be the most complex obligation. The reason for that is because it is not a paint-by-numbers exercise: the identification of risks and the implementation of appropriate mitigation measures is a challenge very familiar to regulated businesses today.
Start-ups and small businesses which find themselves regulated as providers of AI systems will need to ensure that not only do they have their own internal measures in place, but that their products and documentation are designed in such a way that their customers are also confident in their own compliance position.
For all businesses using AI, the immediate practical steps are to take advice on setting up an appropriate governance process and structure for reviewing existing and future AI systems and use cases to make sure that they are used in compliance with the AI Act and other applicable laws like the GDPR.
Under the AI Act, there are three relevant effective dates (based on the amendments in the provisionally agreed AI Omnibus):
For AI systems that are themselves, or that are safety components of, certain specified regulated products, the high risk rules apply from 2 August 2028.
For AI systems that otherwise fall into the high-risk categories within Annex III of the AI Act (including HR functions like recruitment, performance evaluation or workplace monitoring), the high risk rules apply from 2 December 2027.
For systems that are already on the market or in service as of 2 August 2026, there is an extension of time and the high-risk rules will only apply if and when the AI system is subject to significant changes in its design, unless intended for use by public authorities.
Otherwise, certain specific large-scale IT systems must comply by 31 December 2030, high-risk AI systems intended for use by public authorities must comply by 2 August 2030, and general-purpose AI models placed on the market before 2 August 2025 must comply by 2 August 2027.
First and foremost, those companies need to ensure that they are up to date on the other compliance obligations under the legislation, such as appointing an EU Authorised Representative where required.
The guidance itself applies equally to non-EU businesses, as it regulates the products as they arrive on the market: if a business is using an AI system or its outputs on the EU market, then the legislation applies. Those providers should be prepared to engage with European customers about whether their products are intended for use in high-risk areas, as part of their overall projects to produce compliant product documentation.
Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。