AI: latest developments in the European Commission

On 17 July, the European Commission released two insightful documents on Artificial Intelligence:

-the results from the public consultation on its White Paper on AI;

-the final version of the (self) assessment list for trustworthy AI.

Please find below the links to both publications and briefings on these documents.

Summary report on the AI public consultation
Final assessment list for trustworthy AI

 

Key extracts from the report on the public consultation

In total, 1215 contributions were received, of which 352 were on behalf of a company or business organisations / associations, 406 from citizens (92% EU citizens), 152 on behalf of academic / research institutions, and 73 from public authorities. Civil society voices were represented by 160 respondents (among which 9 consumer’s organisations, 129 non-governmental organisations and 22 trade unions). 72 respondents contributed as “others”.

Of the 352 business and industry representatives, 222 were companies and business representatives, 41.5% of which were micro, small and medium-sized enterprises. The rest were business associations. Overall, 84% of business and industry replies came from the EU-27.

 

Mandatory requirements

42% of respondents request the introduction of a new regulatory framework on AI, another 33% think that the current legislation needs to be modified in order to address the gaps identified. Only 3% think that current legislation is fully sufficient.

Respondents seemed to agree with all the mandatory requirements proposed by the White Paper with high percentages ranging from 83% to 91% for each requirement. Clear liability and safety rules (91%), information on the nature and purpose of an AI system (89%), robustness, and accuracy of AI systems (89%). Human oversight (85%), quality of training datasets (84%) and the keeping of records and data (83%).

Concerning the scope of this new possible legislation, opinions are less straightforward. While 42.5% agreed that the introduction of new compulsory requirements should only be limited to high-risk AI applications, another 30.6% doubt such limitation. It is interesting to note that respondents from industry and business were more likely to agree with limiting new compulsory requirements to high-risk applications with a percentage of 54.6%.

However, several respondents do not appear to have a clear opinion regarding what high-risk means: although 59% of respondents support the definition of high-risk provided by the White Paper, only 449 out of 1215 (37% of consultation participants) responded to this question.

 

Biometric identification

Respondents had doubts on the public use of [biometric identification systems] with 28% of them supporting a general ban of this technology in public spaces, while another 29.2% required a specific EU guideline or legislation before such systems may be used in public spaces.

Finally, 6.2% of respondents did not think that any further guidelines or regulations are needed.

 

Enforcement and voluntary labelling

To make sure that AI is trustworthy, secure and in respect of European values, the White Paper suggests a series of conformity assessment mechanisms for high-risk applications. Of those mechanisms, 62% of respondents supported a combination of ex-post and ex-ante market surveillance systems. 28% support external conformity assessment of high-risk applications. 21% of respondents support ex-ante self-assessment.

Voluntary labelling systems could be used for AI applications that are not considered of high-risk. The 50.5% of respondents find it useful or very useful, while another 34% do not agree with it. 15.5% of respondents declared that they do not have an opinion on the matter.

 

Safety and liability implications of AI, IoT and robotics

60.7% of respondents supported a revision of the existing Product Liability directive to cover particular risks engendered by certain AI applications.

Among the particular AI related risks to be covered, respondents prioritised cyber risks with 78% and personal security risks with 77%. Mental health risks follow with 48% of respondents flagging them, and risks related to the loss of connectivity, flagged by 40% of respondents.

Moreover, 70% of participants supported that the safety legislative framework should consider a risk assessment procedure for products subject to important changes during their lifetime.

Robustness in the final assessment list for trustworthy AI

 

Background: Ethics guidelines for trustworthy AI

The release of the new assessment list for trustworthy AI follows the publication of the Ethics guidelines for trustworthy AI by the High-Level Expert Group in April 2019. In these guidelines, the Expert Group had drafted a first version of an assessment list to concretely implement the principles. This list contains questions that AI designers and developers, data scientists, procurement officers, and other key stakeholders, should ask themselves when designing or assessing an AI system. Over 350 stakeholders took part in a piloting phase to evaluate the relevance of this self-assessment list. The final version of the assessment list was built on the conclusions drawn from this piloting phase.

 

Resilience against attacks: enriched version of the list

When it comes to resilience against attacks, the initial list drafted by the High-Level Expert Group was further enriched with references to the Cybersecurity Act, security certification and penetration testing. These references did not appear in the first version of the list. The final list for AI robustness is the following one:

-Could the AI system have adversarial, critical or damaging effects (e.g. to human or societal safety) in case of risks or threats such as design or technical faults, defects, outages, attacks, misuse, inappropriate or malicious use?

-Is the AI system certified for cybersecurity (e.g. the certification scheme created by the Cybersecurity Act in Europe) or is it compliant with specific security standards?

-How exposed is the AI system to cyber-attacks?

-Did you assess potential forms of attacks to which the AI system could be vulnerable?

-Did you consider different types of vulnerabilities and potential entry points for attacks such as:

-Data poisoning (i.e. manipulation of training data);

-Model evasion (i.e. classifying the data according to the attacker's will);

-Model inversion (i.e. infer the model parameters)

-Did you put measures in place to ensure the integrity, robustness and overall security of the AI system against potential attacks over its lifecycle?

-Did you red-team/pentest the system?

-Did you inform end-users of the duration of security coverage and updates?

-What length is the expected timeframe within which you provide security updates for the AI system?

 

If you have any questions on these issues, do not hesitate to contact Camille Dornier, Policy Manager: camille.dornier@eurosmart.com

Eurosmart
Rue de la Science 14B - 1040 Brussels BELGIUM
Privacy Policy - EU transparency register #21856815315-64
Twitter LinkedIn
Modify your subscription    |    View online