|
GENERAL FACTS
Objectives of the new rules
The proposed regulation has three main objectives:
-establish rules for the placing on the market, putting into service and use of high-risk AI systems
-establish harmonised transparency rules for AI systems interacting with natural persons and AI systems used to generate or manipulate image, audio or video content
-create the conditions for the uptake of AI compatible with EU law and values (fundamental rights etc.)
AI systems at stake
The AI systems concerned by the proposed regulation will be defined by the Commission via delegated acts. The regulation does not apply to AI systems used for military purposes.
Definition of AI
The proposed regulation provides the following definition of AI:
‘artificial intelligence system or AI system’ means software that is developed with one or more of the approaches and techniques listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing real or virtual environments. AI systems are designed to operate with varying levels of autonomy. An AI system can be used as a component of a product, also when not embedded therein, or on a stand-alone basis and its outputs may serve to partially or fully automate certain activities, including the provision of a service, the management of a process, the making of a decision or the taking of an action.
PROHIBITED AI PRACTICES
Title II of the proposed regulation is fully dedicated to prohibited practices. Among such prohibited practices, it is worth mentioning:
-manipulation of behaviours, opinions or decisions to the person’s detriment.
-AI systems used for indiscriminate surveillance applied in a generalised manner to all natural persons without differentiation.
-AI systems used for social scoring of natural persons, including online.
However, the text also explains that such practices can still take place if they are authorised by law and carried out by public authorities to safeguard public security. Appropriate safeguards shall be put in place.
HIGH-RISK AI SYSTEMS
What is a high-risk AI system?
High-risk AI systems are listed in Annex II (page 68) of the proposed regulation. The Annex distinguishes two types of high-risk AI systems:
-those that are subject to third party conformity assessment: a) systems used in biometric identification in public spaces and b) systems used as safety components in essential public infrastructure networks.
-those that are subject to self-assessment of conformity: these are more numerous, they include recruitment systems, asylum and visa systems, justice systems etc.
The Commission can update this list of high-risk AI systems via delegated acts. The Commission gives details on the general rules that lead to a system being classified as high-risk (Article 6). An AI system is high-risk if it generates a high level of risk of harm. Such harm can take the shape of injury or death, damage to property, systemic adverse impact on society, significant disruptions to the provision of essential services, adverse impact on fundamental rights (incl. privacy) etc.
An EU database on high-risk AI systems is established and publicly available. Data is entered into the database by AI providers themselves.
Requirements for high-risk AI systems
Compliance with requirements shall be assessed before the placement of high-risk AI systems on the market or their putting into service via conformity assessment procedures.
The requirements are the following ones:
-high quality of the datasets (no biases that could lead to discrimination). In addition, high-risk AI systems shall not be tested on data sets that have already been used in full or in part for the training of the same high-risk AI systems. Personal data can be used for the purpose of ensuring bias monitoring, detection and correction, subject to technical limitations of the re-used, use of state-of-the-art security and privacy-preserving measures, such as pseudonymisation and encryption.
-documentation and record keeping: the outputs of high-risk systems can be verified and traceback. The technical documentation shall demonstrate that the conformity of the high-risk system with the requirements has been assessed.
-transparency and provision of information to users: operation of high-risk system shall be designed so that users understand and control how the AI system produces its output. Information must be provided to the user, including the level of accuracy, robustness and security against which the AI system has been tested and validated.
-human oversight: high risk system shall be designed in a way that they can be overseen by natural persons through appropriate technical and/or organisational measures. The system shall be designed in a way that make it possible for the responsible person to 1) safely and instantly turn it off and 2) disregard, correct, override or reverse the output of a high-risk AI system.
-robustness, accuracy and security: high-risk AI systems shall meet a high-level of security. They shall be resilient vis-Ã -vis attempts to alter their use or performance by malicious third parties intending to exploit system vulnerabilities. Technical solutions shall be appropriate to the circumstances and the risks. They may include measures to prevent and control for data poisoning, adversarial examples, or model flaws.
Remote biometric identification systems in public spaces
In addition, remote biometric identification systems get an entire title in the legislation (Title V). The use of remote biometric identification systems in publicly accessible spaces is subject to an authorisation by national authorities.
Such a use shall only be authorised in limited number of cases, including counterterrorism, or with a valid EU declaration of conformity. National authorities shall inform the European Data Protection Board and the European AI Board of its decision.
Obligations of providers, importers and distributors of high-risk AI systems
The regulation applies to both providers established inside and outside the EU.
Providers of high-risk systems shall ensure that these systems comply with the legislation. They shall put in place a quality management system that ensures compliance with the regulation. Once compliance has been demonstrated through the relevant conformity procedure, providers shall draw up an EU declaration of conformity and affix the CE marking.
Providers shall also put in place a post-market monitoring system. They shall report to competent authorities any serious incidents or any malfunctioning of a high-risk AI system.
Importers shall place on the EU market only high-risk AI systems that comply with the requirements. They shall ensure that the appropriate conformity assessment procedure has been carried out by the provider. Likewise, distributors of high-risk AI systems shall ensure that the systems they make available on the market are CE marked.
Obligations of users of high-risk AI systems
Users are also subject to obligations. They shall follow the instructions and monitor the operation of the AI systems.
Notified bodies
Notified bodies shall be entitled to control the conformity of the high-risk AI systems. The regulation lays down a series of requirements for notified bodies and their subcontractors (article 22). Notified bodies shall submit an application for notification to the national competent authority of the Member State in which it is established.
Member States may only notify conformity assessment bodies that comply with the requirements. However, the Commission is still entitled to investigate cases where it doubts the competence of a notified body.
Harmonised standards
There is a presumption of conformity for high-risk AI systems which are in conformity with harmonised standards. Providers who apply harmonised standards may opt to carry out a conformity assessment by themselves (instead of a third-party conformity assessment).
Common specifications
Where no harmonised standards exist of where relevant harmonised standards are not sufficient or where there is a need to address specific safety or fundamental rights concerns, the Commission may adopt common specifications via implementing acts. High-risk AI systems which are in conformity with the common specifications shall be presumed in conformity with the legislation.
Substantial modification of a high-risk AI system
For all AI systems, a provider shall undergo a new conformity assessment of the high-risk AI system whenever they operate a substantial modification of the system, regardless of whether the modified high-risk AI system is intended to be further distributed or continues to be used by the current user.
Duration of the certificates
Certificates shall be valid for period not exceeding five years. The validity of the period may be extended (for five years maximum) based on a re-assessment.
TRANSPARENCY OBLIGATIONS FOR SPECIFIC AI SYSTEMS
The Commission drew up further transparency obligations for three types of AI systems (not necessarily high-risk):
-AI systems intended to interact with natural persons: users must be notified that they interact with an AI system;
-emotion recognition system or categorisation system: users must be notified they are exposed to such a system;
-systems that generate or manipulate image, audio or video content that resembles existing persons, objects, places or other entities: they shall disclose to users the content that has been artificially created or manipulated.
ENFORCEMENT
Market surveillance
If the market surveillance authority finds that the AI system does not comply with the requirements laid down in the regulation, it shall require the relevant economic operator to bring the system into compliance, to withdraw the system or to recall it.
Placing on the market prohibited AI or non-cooperation with notifies bodies can lead to a fine up to 20 000 000 euros or, in the case of an undertaking, up to 4% of the total worldwide annual turnover, whichever is higher.
Transition period
AI products already placed on the market before the application of the regulation will benefit from a transition period to comply with the text. The duration of the transition period remains to be determined.
FOSTERING INNOVATION
The regulation contains some provisions that aim at fostering innovation. These provisions echo Eurosmart’s call for an AI Competence Centre. One of the measures is the AI regulatory sandbox. National competent authorities from one or more Member States and/or the European Data protection Supervisor may establish AI regulatory sandboxing schemes. The objective is to develop and test innovative AI systems under strict regulatory oversight before those systems are placed on the market. SMEs and start-ups shall have priority access to the AI regulatory sandboxes.
The text also refers to the Digital Hubs and Testing Experimentation Facilities that will be created by Digital Europe (EU funding programme). They shall support the implementation of this regulation.
GOVERNANCE
European AI Board
The text creates a European AI Board composed of one representative per national authority, the European Data Protection Supervisor and a representative of the European Commission. The Board will supervise the consistent application of the regulation across the EU. It will also contribute and participate in the development of AI related harmonised standards and common specifications.
The Board shall exchange with stakeholders on a regular basis.
Expert group
The regulation also establishes an expert group to provide technical and scientific advice to the Board. The expert group shall consist of independent experts appointed for a renewable three-year term by the Commission.
If you have any questions on these issues, please contact Camille Dornier - Policy Manager: camille.dornier@eurosmart.com
|
