|
Definition of AI
The definition of AI has been shortened compared to the draft. An AI system is now defined as:
“software that is developed with one or more of the techniques and approaches listed in Annex I [machine learning, logic and knowledge-based approaches, statistical approaches] and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environment they interact with”.
Banned AI practices, incl. facial recognition in public spaces
The proposed AI Regulation prohibits a few AI practices, such as:
-behavioural manipulation to the detriment of the user
-social scoring by public authorities or on their behalf
-use of real time remote biometric identification in publicly accessible spaces for the purpose of law enforcement
A publicly accessible space is a physical place accessible to the public, regardless of whether certain conditions for access may apply.
However, when it comes to facial recognition for law enforcement, the proposed text lays down a few exceptions where this is still possible: searching for a missing child, imminent terrorist threat, finding a criminal or a suspect. In these cases, facial recognition must be authorised by a judicial or other independent body. There needs to be limits in time, geographic reach and the data bases search. The authorisation is prior to the use of the system, unless duly justified by a situation of urgency.
Requirements for high-risk AI systems
Some AI systems are considered high-risk in the proposed Regulation. Safety components or products already subject to third party conformity assessment -pursuant to certain EU laws- are always considered high-risk. Annex II of the proposed text contains the list of concerned EU legislations. It includes machinery products, planes, rail systems etc.
In addition, Annex III contains a list of areas that also fall into the category of high-risk AI systems. This includes:
-biometric identification and categorisation of natural persons
-management and operation of critical infrastructures
-education and vocational training
-employment, workers management and access to self-employment
-access to and enjoyment of essential private services and public services and benefits
-law enforcement
-migration, asylum and border control management
-administration of justice and democratic processes
Those high-risk AI systems are subject to requirements: accuracy, robustness, cybersecurity, quality of the data sets etc. As part of their transparency obligations, AI providers shall also specify the level of accuracy, robustness, and cybersecurity against which the high-risk AI system has been tested and validated.
Additionally, AI providers must establish and maintain a risk management system. The risk management measure shall take into account the state of the art.
Conformity assessment for high-risk AI systems
High-risk AI systems shall undergo a conformity assessment before they are placed on the market. If the system complies with the requirements, the providers shall draw up an EU declaration of conformity and affix the CE marking.
Conformity assessment bodies shall submit an application for notification to the relevant national authorities. Conformity assessment bodies established under the law of a third country with which the EU has concluded an agreement may be authorised to carry out the activities of notified bodies.
High-risk AI systems which are in conformity with harmonised standards shall be presumed to be in conformity with the Regulation. The Commission may also adopt common specifications.
Link to the Cybersecurity Act
High-risk AI systems that have been certified via a European cybersecurity certification scheme shall be presumed to be in compliance with the cybersecurity requirements set by the Regulation – in so far as the cybersecurity certificate or statement of conformity covers those requirements.
Post-market monitoring system
Providers of high-risk AI systems shall establish and document a post-market monitoring system. They shall report to competent authorities any serious incident or any malfunctioning.
Repetition of the conformity assessment
If the high-risk AI system is substantially modified, it shall undergo a new conformity assessment procedure. This does not apply to high-risk AI systems that continue to learn after being placed on the market if the changes have been pre-determined by the provider.
Penalties for non-compliance
Providers are subject to fines up to 30 000 000 euros or, if the offender is a company, up to 6% of its total worldwide turnover, whichever is the higher, if:
-the provider places on the market a prohibited AI system
-the provider does not comply with the requirements on data and data governance
For non-compliance with other requirements, the penalties are up to 20 000 000 euros or 4% of the turnover.
Transparency obligations for specific types of AI
The Commission drew up further transparency obligations for three types of AI systems (not necessarily high-risk):
-AI systems intended to interact with natural persons: users must be notified that they interact with an AI system;
-emotion recognition system or categorisation system: users must be notified they are exposed to such a system;
-systems that generate or manipulate image, audio or video content that resembles existing persons, objects, places or other entities: they shall disclose to users the content that has been artificially created or manipulated.
Fostering innovation
The regulation contains some provisions that aim at fostering innovation. These provisions echo Eurosmart’s call for an AI Competence Centre. One of the measures is the AI regulatory sandbox. National competent authorities from one or more Member States or the European Data protection Supervisor may establish AI regulatory sandboxing schemes. The objective is to develop and test innovative AI systems for a limited time before those systems are placed on the market. SMEs and start-ups shall have priority access to the AI regulatory sandboxes.
The proposed text also authorises the further processing of personal data for developing certain AI systems in the public interest (public security, public safety, public health, protection of the environment). Such development shall take place within an AI regulatory sandbox.
European AI Board
The text creates a European AI Board composed of one representative per national authority and the European Data Protection Supervisor. The Board shall be chaired by the European Commission. The Board will supervise the consistent application of the regulation across the EU. It will also contribute and participate in the development of AI related harmonised standards and common specifications.
Next steps
The European Parliament and the Council will examine the text and come to their own positions on the proposal.
If you have any questions on these issues, please contact Camille Dornier - Policy Manager: camille.dornier@eurosmart.com
|
