Trustworthy AI – AI Policy Lab

by: Virginia Dignum and Maja Fjaestad

The Artificial Intelligence (AI) Act envisages the establishment of a scientific panel of independent experts to advise on, and assist the AI Office and national market surveillance authorities with, implementing and enforcing the AI Act. The Commission is currently seeking public for input on implementing regulation establishing a scientific panel of independent experts. Here is our response to this request

The AI Policy Lab welcomes the opportunity to provide feedback on the European Commission’s draft regulation establishing a scientific panel of independent experts in artificial intelligence. This initiative is crucial in ensuring robust, transparent, and impartial oversight of AI systems, aligning with EU objectives to foster innovation while safeguarding fundamental rights. We commend the Commission’s focus on multidisciplinary expertise, diversity, and transparency in panel operations. However, to enhance effectiveness, we offer recommendations to streamline procedural workflows and strengthen data security protocols, ensuring the panel’s structure fully supports its mission in this rapidly evolving field.

Strong Points

We commend key features of the proposal that strengthen the panel’s credibility, flexibility, and proactive oversight:

Transparency and Conflict of Interest: Requirements for experts to make declarations of interest and to act in the public interest enhance the panel’s credibility and independence
Flexible Structure for Task Allocation: The document enables adaptability by allowing specific members to serve as rapporteurs for individual tasks, ensuring expertise aligns with task requirements?
Qualified Alerts for AI Risks: The ability of the scientific panel to issue qualified alerts to the AI Office is an innovative mechanism for highlighting potential AI risks.

Recommendations

Several areas for improvement could enhance efficiency, security, and impartiality in the panel’s operations. In particular, we suggest to address the following issues:

Complex Bureaucracy: The involvement of multiple administrative bodies, such as the AI Office, Joint Research Centre, and the Commission, could introduce delays and administrative bottlenecks in the panel’s operations. Streamlined procedural workflows and clarified responsibilities for each entity could enhance the panel’s responsiveness and effectiveness in providing timely guidance.
Strengthening Data Security and Confidentiality Measures: Although confidentiality is mentioned, the document could benefit from more detailed procedures on data handling to further mitigate risk related to data security?. Adding explicit guidelines for the secure storage, sharing, and destruction of sensitive information would strengthen the protocol for data handling, especially in cases involving sensitive AI data
Enhancing Panel Independence through Conflict of Interest Protocols: While the requirement for declarations of interest is a positive step, more rigorous conflict of interest safeguards – such as independent audits or periodic reviews – could reinforce the panel’s impartiality, especially given the rapidly evolving nature of AI and potential industry influences.
Streamline Procedural Steps: Simplifying interactions between the AI Office, Joint Research Centre, and the Commission could enhance efficiency without compromising oversight.
Equitable selection criteria: Equality is crucial to guarantee diverse input, to have democratic legitimacy, and to avoid bias. Article 3, par 5, Selection criteria and composition of the scientific panel would therefore benefit from a clearer formalation: Instead of “The Commission shall aim to ensure gender balance” a better formulation could be “the commission shall ensure gender balance”.
Multidisciplinary relevance: The importance of humanities and social sciences expertise should be emphasized. For instance, in Article 3, paragraph 3, by removing “scientific or technical expertise” would broaden the focus, avoiding an unnecessary bias toward natural sciences and valuing multidisciplinary insights on rights, equality, and ethics in AI.

Proposals for consideration

Additional measures could increase the panel’s adaptability, responsiveness, and independence in handling evolving AI challenges, as follows:

To maintain the panel’s relevance across rapidly evolving AI fields, we propose supplementing the core panel with a flexible pool of specialized experts. These “on-call” experts would offer guidance on niche areas like ethical AI, quantum AI, or specific sectoral applications, allowing the panel to draw on targeted expertise without permanently expanding its membership.
Recognizing the potential risks posed by high-stakes AI applications, we recommend a dedicated Rapid Response Protocol within the panel. This would enable the panel to perform expedited assessments of AI models flagged as potentially harmful, particularly those impacting public safety or fundamental rights, ensuring that urgent cases receive timely and focused attention.
To safeguard the panel’s impartiality, we suggest enhanced conflict of interest protocols, including independent audits or periodic reviews of expert affiliations and potential biases. This would reinforce trust in the panel’s independence, especially important given AI’s sensitive and influential role across industries.
To promote transparency and public trust, the panel could introduce an AI Accountability Dashboard that provides the public with non-sensitive summaries of decisions, recommendations, and qualified alerts issued by the panel. This dashboard could track metrics like panel activity levels, time-to-decision for urgent alerts, and diversity statistics, thus allowing stakeholders to observe the panel’s impact on AI governance.

Roberta Calegari (Bologna University), Virginia Dignum (Umeå University)

Published on 24 June 2024

PDF

Cite

What is Responsible AI?

Currently, for most practical uses, Artificial Intelligence (AI) is first and foremost a technology that can automatise tasks and decision making processes. However, considering its societal impact and need for human contribution, AI is much more than a technique but can best be understood as a socio-technical ecosystem, recognising the interaction between people and technology, and how complex infrastructures affect and are affected by society and by human behaviour. As such, AI involves the structures of power, participation, and access to technology that determine who can influence which decisions or actions are automated, which data, knowledge, and resources are used for learning, and how interactions between decision-makers and those impacted are defined and maintained.

The main focus of Responsible AI is ensuring that AI systems are developed, deployed, and used in a manner that is ethically sound, respects human rights, and considers societal implications. This encompasses not just ethical and legal considerations, but also the socio technical aspects that ensure that accountability for the development and use of the AI system is guaranteed. Responsible AI practices often involve processes and guidelines that organisations follow during the design, development, and deployment stages of AI systems. This could include impact assessments, reviews, and monitoring of AI systems in real-world applications.

Trustworthy AI emphasises the reliability, safety, and robustness of AI systems, as well as their ethical implications. The goal is to ensure that users and stakeholders can have confidence in AI systems’ decisions and behaviours. This might involve ensuring an AI system functions correctly under various conditions, is robust against adversarial attacks, and can explain its decisions in understandable terms. Trustworthiness often requires technical solutions, such as robustness testing, adversarial training, and explainability methods, in addition to governance and ethical guidelines.

Generally, Responsible AI practices encompass Trustworthy AI requirements. A responsible, ethical, and trustworthy approach to AI will ensure transparency about how adaptation is done, responsibility for the level of automation on which the system is able to reason, and accountability for the results and the principles that guide its interactions with others, most importantly with people. In addition, and above all, a responsible approach to AI makes clear that AI systems are artefacts manufactured by people for some purpose, and that those which make these have the power to decide on the use of AI.

In this sense, AI ethics is not, as some may claim, a way to assign responsibility to machines for their actions and decisions, thereby absolving people and organizations of their own responsibility. On the contrary, ethical AI imposes greater responsibility and accountability on the people and organizations involved: for the decisions and actions of the AI applications, and for their own decision to use AI in a given context.

Guidelines, principles and strategies to ensure trust and responsibility in AI refer to the socio-technical ecosystem in which AI is developed and used. It is not the AI artefact or application that needs to be ethical, trustworthy, or responsible. Rather, it is the people, organisations and institutions involved that can and should take responsibility and act in consideration of an ethical framework such that the overall system can be trusted by users and society.

In a nutshell, we can recap the main definitions as follows:
“Responsible AI” refers to the concept of developing and deploying AI systems in a way that aligns with ethical principles, societal values, and legal requirements. Overall, responsible AI seeks to foster the development and adoption of AI technologies in a way that promotes ethical values, respects human rights, and contributes to the well-being of individuals and communities.

“Trustworthy AI,” on the other hand, refers to the concept of developing and deploying AI systems that are reliable, ethical, lawful, and transparent, thereby earning the trust of users, stakeholders, and society at large. By embodying these principles and characteristics, trustworthy AI inspires confidence and trust among users, stakeholders, and society, facilitating the responsible adoption and utilization of AI technologies for the benefit of society.

So in a way, trustworthy AI is the enabler for responsible AI. While the former is more focused on the technical aspects to build systems reliable, transparent, accountable, and ethical, thereby earning the trust of users, stakeholders, and society, the latter emphasizes the ethical and moral dimensions of AI development and deployment, aiming to promote ethical behavior, respect for human rights, and the well-being of individuals and communities.

Keywords (comma separated):
Socio-technical ecosystem, Ethical principles, Trustworthy AI

How to cite this article:

Calegari, R., & Dignum, V. (2024). Defining Responsible AI. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/KWEU5144

ON EU’s plans for a scientific panel of independent experts

Defining Responsible AI

Leave a Reply Cancel reply