Bridging the Gap Between AI Research and Policy

The future of Artificial Intelligence (AI) lies not just in its technical advancements but in its responsible governance, underpinned by human-centered principles and policies. As such, AI policy research is an urgently needed area of focus, not just AI research, not just policy research, but a deliberate intersection of the two. This realization was at the core of the recent AI Policy Summit, a collaborative platform bringing together researchers from around the world and co-organised by MILA and the AI Policy Lab that I have the previlege to direct.  This was not just an event but a pivotal step toward shaping the trajectory of AI policy and governance. As AI technologies increasingly permeate every aspect of society, their potential to drive progress must be balanced with safeguards to ensure they align with human-centered values. This balance cannot be achieved by technical or legislative approaches alone; it demands the collaborative efforts of researchers, policymakers, and civil society.

The AI Policy Summit provided a unique platform for representatives from independent research organizations, spanning academia and civil society from diverse national contexts to engage in an open, informal environment that enable deep and heated exchange of ideas. A panel discussion with policymakers from multiple countries added depth and diversity to the discussions. Their contributions underscored the varying challenges and opportunities faced across different governance frameworks. Policymakers from Sweden, Tanzania, Canada, the Netherlands, and Portugal shared insights into their regional experiences with AI regulation, highlighting both shared objectives—such as transparency and accountability—and the unique cultural and legislative nuances that influence AI governance.

I was also especially encouraged by Marietje Schaake’s keynote, which highlighted the critical role of researchers in engaging with policymakers through building lasting relationships, providing actionable insights like policy briefs, and actively contributing to both the creation and implementation of legislation, all while acknowledging the challenges both sides face.

During the two days, exchanges between the participants emphasized the critical need for localized approaches to AI policy that are informed by global best practices. The summit fostered an environment where academic and civil society researchers could present evidence-based findings while gaining a firsthand understanding of the practical realities policymakers face. This interaction not only enriched the dialogue but also set the foundation for future collaborations aimed at shaping inclusive, effective, and context-sensitive AI policies.

Why AI Policy Research Matters

The development and governance of Artificial Intelligence (AI) are complex, interconnected challenges that demand a dedicated focus on AI policy research, a field distinct yet integrative of AI technology research and policy governance. This emerging discipline addresses gaps that neither AI research nor policy alone can resolve, ensuring that governance frameworks are not only informed by cutting-edge science but also aligned with societal needs and values. While AI research focuses on advancing technology and policy research on governance frameworks, neither can address the multifaceted impacts of AI in isolation:  

  1. AI advancements without Governance: Left unchecked, rapid AI innovation can deepen societal inequalities, exacerbate environmental damage, and consolidate power among a few, undermining public trust and equitable access?.
  2. Policy without AI research: Policies uninformed by empirical evidence or understanding of AI’s dynamic landscape risk becoming outdated, excessively restrictive, or misaligned with technological realities, stifling innovation and public benefits.

AI policy research as foundation for Responsible AI

Responsible AI begins well before algorithms are written or systems deployed: it starts with the fundamental questions: What problems are we solving? For whom? And with what consequences? What are the most suitable solutions? Is it AI? Addressing these questions requires a nuanced interplay between policy and research. The summit highlighted the growing need for this alignment to ensure that AI technologies foster societal progress, uphold human rights, and contribute to global sustainability goals.

At its heart, AI policy research navigates complex trade-offs. Fostering innovation while mitigating societal inequities requires a framework that ensures AI benefits are equitably distributed, particularly to those most vulnerable to its disruptions. AI policy research creates a vital bridge between these domains by focusing on actionable, evidence-based governance. It emphasizes transparency, accountability, and sustainability while ensuring equitable outcomes. By addressing issues such as inclusivity, environmental trade-offs, and regulatory foresight, AI policy research supports:

  • Proactive governance: Anticipating the implications of AI advancements demands foresight-driven policies that anticipate potential risks and societal impacts before they arise. By proactively identifying challenges—such as biases, security vulnerabilities, or unintended social consequences—governance frameworks can mitigate harm and establish safeguards that evolve alongside technological innovation.
  • Cross-sector collaboration: Effective AI policy requires a united effort from academia, industry, and government. Collaborative frameworks enable the sharing of expertise, aligning research insights with regulatory needs and industrial priorities. This synergy fosters the creation of policies that are both practical and evidence-based, ensuring comprehensive oversight and adaptability.
  • Responsible innovation: Encouraging the use of AI only when its benefits outweigh costs and align with ethical standards?. That is, AI should be deployed only when its advantages clearly outweigh associated costs and risks. Responsible innovation emphasizes ethical design, sustainability, and equitable access, ensuring that AI systems contribute to societal well-being without exacerbating inequalities or environmental harm.

The AI Policy Summit’s Contribution

The recent AI Policy Summit brought together global policymakers, academic researchers, and civil society actors to highlight this integrative approach. Discussions focused on immediate and long-term goals, such as fostering global accountability standards, developing foresight mechanisms, and crafting practical tools for inclusive governance. By emphasizing a shared roadmap and cross-sectorial expertise, the summit illuminated how AI policy research can drive actionable solutions for the responsible development of AI technologies?. This collective effort underscores the urgency of AI policy research as a means to guide innovation and governance toward equitable, sustainable outcomes. It is a field poised not only to mitigate the risks of AI but to maximize its potential as a force for societal good. Building on insights from the summit, several ideas were proposed to solidify the role of AI policy research, including:

  • Establish Visiting AI Policy Fellowships: These programs at different research institutes connect researchers with policymakers, fostering mutual understanding and collaboration?.
  • Launch an AI Policy Research Network: A global platform to share best practices, insights, and resources for evidence-based policymaking.
  • Develop AI Policy Briefs: Translating research findings into actionable insights tailored for policymakers is essential for informed decision-making.
  • Focus on Education and Capacity Building: Initiatives like student exchanges and Erasmus programs can cultivate a new generation of leaders at the intersection of AI and governance?).

A Shared Responsibility

AI policy research is not just a necessity, it is an opportunity to ensure that AI serves humanity rather than shaping societies in ways that exacerbate inequities or environmental harm. By combining the rigor of scientific inquiry with the pragmatism of governance, this field provides a pathway to align AI innovation with ethical, human-centered values.

The AI Policy Summit marked the beginning of a critical journey, one that bridges the gap between technological innovation and governance to ensure AI serves humanity responsibly. This initiative is more than a conference or a network; it is a call to action for researchers, policymakers, and civil society to collaborate in shaping an equitable and sustainable AI future.

Looking ahead, the true measure of its success will be our ability to foster lasting impact. This includes creating actionable frameworks, building trust through transparency and accountability, and policy instruments that ensure that the benefits of AI are accessible to all. As AI continues to evolve, our collective efforts must remain grounded in shared principles of fairness, sustainability, and human-centered development.

The challenges are immense, but so too is our collective potential. By uniting diverse perspectives and expertise, we can navigate the complexities of AI with integrity and purpose. Together, we have the opportunity not only to mitigate risks but to redefine AI as a tool for societal good—one that reflects the values and aspirations of all. The journey is just beginning, but the urgency is clear. I welcome you all to join us to #InformAIpolicy, a joint commitment to building a future where AI contributes to societal progress, respects the planet, and ensures equity for all.

ON EU’s plans for a scientific panel of independent experts

by: Virginia Dignum and Maja Fjaestad

The Artificial Intelligence (AI) Act envisages the establishment of a scientific panel of independent experts to advise on, and assist the AI Office and national market surveillance authorities with, implementing and enforcing the AI Act. The Commission is currently seeking public for input on implementing regulation establishing a scientific panel of independent experts. Here is our response to this request

The AI Policy Lab welcomes the opportunity to provide feedback on the European Commission’s draft regulation establishing a scientific panel of independent experts in artificial intelligence. This initiative is crucial in ensuring robust, transparent, and impartial oversight of AI systems, aligning with EU objectives to foster innovation while safeguarding fundamental rights. We commend the Commission’s focus on multidisciplinary expertise, diversity, and transparency in panel operations. However, to enhance effectiveness, we offer recommendations to streamline procedural workflows and strengthen data security protocols, ensuring the panel’s structure fully supports its mission in this rapidly evolving field.

Strong Points

We commend key features of the proposal that strengthen the panel’s credibility, flexibility, and proactive oversight:

  • Transparency and Conflict of Interest: Requirements for experts to make declarations of interest and to act in the public interest enhance the panel’s credibility and independence
  • Flexible Structure for Task Allocation: The document enables adaptability by allowing specific members to serve as rapporteurs for individual tasks, ensuring expertise aligns with task requirements?
  • Qualified Alerts for AI Risks: The ability of the scientific panel to issue qualified alerts to the AI Office is an innovative mechanism for highlighting potential AI risks.

Recommendations

Several areas for improvement could enhance efficiency, security, and impartiality in the panel’s operations. In particular, we suggest to address the following issues:

  • Complex Bureaucracy: The involvement of multiple administrative bodies, such as the AI Office, Joint Research Centre, and the Commission, could introduce delays and administrative bottlenecks in the panel’s operations. Streamlined procedural workflows and clarified responsibilities for each entity could enhance the panel’s responsiveness and effectiveness in providing timely guidance.
  • Strengthening Data Security and Confidentiality Measures: Although confidentiality is mentioned, the document could benefit from more detailed procedures on data handling to further mitigate risk related to data security?. Adding explicit guidelines for the secure storage, sharing, and destruction of sensitive information would strengthen the protocol for data handling, especially in cases involving sensitive AI data
  • Enhancing Panel Independence through Conflict of Interest Protocols: While the requirement for declarations of interest is a positive step, more rigorous conflict of interest safeguards – such as independent audits or periodic reviews – could reinforce the panel’s impartiality, especially given the rapidly evolving nature of AI and potential industry influences.
  • Streamline Procedural Steps: Simplifying interactions between the AI Office, Joint Research Centre, and the Commission could enhance efficiency without compromising oversight.
  • Equitable selection criteria: Equality is crucial to guarantee diverse input, to have democratic legitimacy, and to avoid bias. Article 3, par 5, Selection criteria and composition of the scientific panel would therefore benefit from a clearer formalation: Instead of “The Commission shall aim to ensure gender balance” a better formulation could be “the commission shall ensure gender balance”. 
  • Multidisciplinary relevance: The importance of humanities and social sciences expertise should be emphasized. For instance, in Article 3, paragraph 3, by removing “scientific or technical expertise” would broaden the focus, avoiding an unnecessary bias toward natural sciences and valuing multidisciplinary insights on rights, equality, and ethics in AI.

Proposals for consideration

Additional measures could increase the panel’s adaptability, responsiveness, and independence in handling evolving AI challenges, as follows:

  • To maintain the panel’s relevance across rapidly evolving AI fields, we propose supplementing the core panel with a flexible pool of specialized experts. These “on-call” experts would offer guidance on niche areas like ethical AI, quantum AI, or specific sectoral applications, allowing the panel to draw on targeted expertise without permanently expanding its membership.
  • Recognizing the potential risks posed by high-stakes AI applications, we recommend a dedicated Rapid Response Protocol within the panel. This would enable the panel to perform expedited assessments of AI models flagged as potentially harmful, particularly those impacting public safety or fundamental rights, ensuring that urgent cases receive timely and focused attention.
  • To safeguard the panel’s impartiality, we suggest enhanced conflict of interest protocols, including independent audits or periodic reviews of expert affiliations and potential biases. This would reinforce trust in the panel’s independence, especially important given AI’s sensitive and influential role across industries.
  • To promote transparency and public trust, the panel could introduce an AI Accountability Dashboard that provides the public with non-sensitive summaries of decisions, recommendations, and qualified alerts issued by the panel. This dashboard could track metrics like panel activity levels, time-to-decision for urgent alerts, and diversity statistics, thus allowing stakeholders to observe the panel’s impact on AI governance.

How Europe is Shaping AI for Human Rights

A Comparative Analysis of the EU AI Act and the Council of Europe Framework Convention

The “Council of Europe Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law” (CETS 225), approved in May and open for signatures today, 5 September 2024, creates a legal framework with a strong focus on safeguarding human rights, democracy, and the rule of law in AI development and usage. The convention emphasizes key principles such as transparency, accountability, risk management, and special protection for vulnerable groups, aligning in many ways with the European Union AI Act, which is more detailed in its categorization of AI systems by risk levels and introduces specific regulatory mechanisms for high-risk AI applications.

Based on our analysis, it can be said that the EU AI Act excels in its market-centric approach, providing clear regulatory guidelines that ensure a safe, innovation-friendly environment for businesses while protecting consumer rights. Its risk-based framework is well-defined, allowing for differentiated oversight based on the risk posed by AI applications, particularly in high-risk sectors like healthcare and transportation. This precision fosters compliance and encourages AI development within clear ethical boundaries. On the other hand, the Council of Europe Framework Convention, broader in scope, with its primary strength being a robust focus on human rights, democracy, and the rule of law. It emphasizes transparency, accountability, and inclusivity across all sectors, going beyond economic concerns to ensure that AI systems respect fundamental rights. The convention’s commitment to protecting vulnerable groups and fostering international cooperation for global AI governance is another key strength, ensuring AI development aligns with global human rights standards. In short, I describe the similarities and differences between the two approaches as follows:

Similarities

  1. Human Rights Focus: Both the EU AI Act and the Council of Europe Framework Convention emphasize the importance of safeguarding human rights in the development, deployment, and use of AI systems. This includes ensuring that AI systems do not infringe upon fundamental rights such as privacy, freedom of expression, and non-discrimination.
  2. Risk-Based Approach: Both frameworks adopt a risk-based approach to AI regulation. They require measures to be scaled according to the potential risks that AI systems pose to human rights, democracy, and the rule of law. This involves stricter oversight and requirements for high-risk AI systems.
  3. Transparency and Accountability: Transparency is a key principle in both documents, mandating that AI systems be designed and operated in a way that is understandable and explainable. They also emphasize accountability, requiring entities deploying AI systems to take responsibility for their impacts.
  4. Non-Discrimination: Both the EU AI Act and the Convention address the need to prevent and mitigate discrimination that might arise from the use of AI systems, particularly in vulnerable groups, including women, minorities, and people with disabilities.
  5. International Cooperation: Both frameworks recognize the importance of international cooperation in AI governance, aiming to create a harmonized approach across jurisdictions to address the global nature of AI technology.

Differences

  1. Legal Scope and Binding Nature: The EU AI Act is a regulatory framework specific to the European Union and its member states, establishing binding legal obligations for entities operating within the EU. In contrast, the Council of Europe Convention is a treaty that countries can choose to ratify, and it applies to a broader range of countries, not limited to the EU. As a treaty, the Council of Europe Convention provides a broad framework, leaving it to member states to determine how to implement its provisions into their national laws. This gives countries flexibility in adapting the convention to their legal systems while still adhering to its overarching principles.
  2. Focus on Democracy and Rule of Law: The Council of Europe Convention places a stronger emphasis on protecting democratic processes and the rule of law. While the EU AI Act also addresses these issues, it is more focused on market regulation and the safe integration of AI into the internal market.
  3. Implementation Mechanisms: The EU AI Act includes specific enforcement mechanisms, such as fines for non-compliance, and assigns responsibilities to existing national authorities for implementation. The Council of Europe Convention, on the other hand, establishes a Conference of the Parties to oversee implementation and foster cooperation among the signatories. While the Council of Europe document includes explicit exemptions for national security and defense, these are explicitly excluded from coverage in the EU AI Act.
  4. Definitions and Scope: The scope of what constitutes an AI system and the range of activities covered differ slightly. For instance, the Council of Europe Convention includes a broad definition of AI systems and explicitly covers their entire lifecycle, from development to decommissioning. The EU AI Act also refers to the AI lifecycle but is more focused on categorizing AI systems by risk levels.
  5. Public Consultation and Participation: The Convention explicitly requires public consultation and multistakeholder involvement in discussions about AI governance, which is less emphasized in the EU AI Act, where the focus is more on regulatory compliance by businesses and public sector entities. As such, The Council of Europe convention places a notably more emphasis on promoting digital literacy and skills across all populations, which is less prominent in the EU AI Act. Moreover, the Council of Europe convention explicitly calls for measures to address the rights of specific vulnerable groups, such as children and persons with disabilities, which is less explicitly stated in the EU AI Act.
  6. Remedies and Oversight: The Council of Europe convention explicitly calls for accessible remedies for violations of human rights caused by AI systems, which is detailed in Chapter IV. While the EU AI Act also emphasizes accountability, the approach to remedies might differ in terms of implementation mechanisms.

Both the EU AI Act and the Council of Europe Framework Convention provide strong foundations for regulating AI, but they leave certain gaps. One major shortcoming is their lack of specificity on how to adapt to rapid technological advancements.  AI evolves quickly, and both frameworks focus heavily on supporting current innovation, which, while beneficial in the short term, may undermine public trust and hinder broader adoption in the future if societal concerns and risks are not adequately addressed. Additionally, while they emphasize international cooperation, neither framework offers a clear path for integrating their approaches into a broader, global AI governance system. This lack of alignment could result in fragmented regulations across countries, making it harder to establish consistent ethical standards worldwide.  Another critical omission is the ethical use of AI in military and national security contexts. Both frameworks largely sidestep this issue, leaving a significant gap in ensuring that AI applications in these areas respect human rights and ethical principles. Lastly, while both stress accountability and oversight, there are challenges in implementing clear and practical enforcement mechanisms, particularly for cross-border AI applications and private actors outside direct government control. Addressing these issues would enhance the comprehensiveness and effectiveness of both frameworks in governing AI responsibly.

Multilevel oversight of AI systems in line with the AI Act

Diana M. Popa (Delft University of Technology)
PDF
Cite

Abstract

The AI “oversight – control” nexus is a matter of debate in both scientific literature and policy papers, given the complex and disruptive nature of the technology and the intricate legislative systems regulating the deployment of AI systems. Most approaches give precedence to the need for a multilayered governance model, while in the same time taking into account the lifecycle stages of the AI product. Oversight over the ecosystem in which the AI is deployed (the governance layer) should not be confused with the process level human oversight. At process level, human oversight is needed in the case of “high-risk systems”, and AI literacy is a precondition for this oversight to be relative. Effective oversight requires many resources at both ecosystem and organisational levels. Additionally, oversight of AI systems is also highly regulated in democratic societies and therefore can be seen as a reflection of the importance that democratic values have in the way disruptive technologies are deployed within society.   

The need for control and human oversight of AI systems is acknowledged and set down in both legislation texts and research and policy papers, but operationalisation of both concepts is either broad or diverse and in practice address different levels of applicability in relation to the given AI system. Control and oversight of or over the functioning of the AI system itself should not be equalled to how the legislative framework address measures of control and oversight of the ecosystem in which the AI systems are deployed. The multitude of initiatives aiming to establish generally accepted definitions and determine the practical limitations of the “meaningful human oversight” – control nexus (Verdiesen, et al. 2021) underline the difficulty of determining  the control capacity over the AI system. Efforts for identifying the correct balance of human oversight and control are meant to counter the “black box” effect that the disruptive nature of the technology brings with itself, especially when used as a decision support tool with a high impact on the individuals.

The Artificial Intelligence Act (AIA) itself mentions both control and human oversight as risk mitigation measures, using terms such as “relevant” / “appropriate”/ “meaningful” human oversight. What the specific implementation measures are remain at a general level (Article 14 primarily) while in the same time, the scientific literature addresses the fact that there is ambiguity in the way human oversight is operationalized and implemented, even more so when it comes to “meaningful human oversight”. This is not necessarily a drawback of the legislation, given the fact that the AI Act was just approved by the European Parliament and also considering that, like in the case of the GDPR, the AI Act wants to be an overarching legislation, leaving room for personalized application at Member State level and remaining broad enough so that the AIA itself needs not reviewing or updating every year, taking into consideration also the alert rhythm of technological development in the field.

From the policy and research approaches, different initiatives have addressed the operationalization of human oversight and the “control – oversight” relationship, either as a nexus or as opposed measures, with approaches depending on the applications field of the concepts. Both control and oversight are defined and embedded within the given broader ecosystem in which the AI model is implemented, and most approaches adhere to a three layer ecosystem: the governance layer (the supra-national and/or national ecosystem in which the AI applications are developed), the socio-technical or organisational layer (based on internal regulations and sector rules) and the technical or process layer (also addressing the product safety regulations) (Verdiesen et al., 2021; Adams et al., 2024; Novelli et al.,2024). Oversight takes place at European level, within dedicated structures (still in development), at national level, with data protection authorities, at organizational level and at process level. Multilayer oversight overlaps (even if one might argue not perfectly) with a multistakeholder governance system of AI systems implementation within a certain regional or national context. Oversight is implemented both vertically and horizontally: the former within national supervisory authorities, expanding their structures to take on formal oversight and control roles with sanctioning capabilities and the latter within the organisations deploying the AI system themselves.

From a component perspective, building blocks for an effective oversight system include good functioning of the democratic rule of law principles, sound legislative frameworks accompanied by binding powers (both for the pre and post deployment phase), appropriate financial and human resources in all three layers and technical capabilities (technological maturity). In line with democratic principles, an oversight system should address the principles of transparency, accountability and responsibility regarding the way the AI systems are used in public context and incorporate human rights standards. Binding involvement of oversight bodies (in the sense of consultation before the implementation of a tool) and binding powers post factum are necessary for oversight to be effective (Wetzling, 2024), the latter with supervisory/ investigative and sanctioning powers.

Not addressing here prohibited practices that are equally regulated by the legislation, in the case of high-risk or impactful AI systems, the risk approach managed through measures and degrees of oversight is also influenced by the type of deployer, the sector it activates in and the process in question. While in the case of internal processes, it is a reflection of the risk appetite of an organisation, in the case of processes that have an impact on individuals outside the organisation, such as is the case with AI used by public authorities in the execution of the public administration act, oversight is tighter regulated from the governance level, since it should also be in line with the social values of the system it operates in. Although actual alignment with public values is not required by the legislation, it is a practice in line with democratic values that increases transparency of the governing act and public decision making and is a reflection of the governing style of a certain nation state, such as is the case of the Netherlands.

The ecosystem division is also often overlapped with the stages of the product’s lifecycle, given that AI is a product deployed on the unique market and also has to comply with product safety regulations, approach which identifies key points during the life of the AI system when human control or oversight is needed, also in relation to the inherent risks of that certain stage. Therefore, within each layer, different control and oversight measures are put in place, at different moments in the AI product’s lifecycle: before deployment, during deployment and after deployment.

A well-structured oversight model therefore includes both ex-ante and ex-post elements. Expert assessment in the form of expert bodies should be included in both stages, with pre-deployment expert advice going in the design phase of the system and in the high level regulating frameworks and with ex-post oversight in the form of expert assessment and democratic scrutiny (Oetheimer, 2024). These recommendations are also valid in the case of regulatory AI frameworks.

From this temporal perspective, control is implemented at three different points in time: ex-ante or pro-active control, on-going or simultaneous control, and ex-post control. Ex-ante or pro-active controls are (or should be) implemented by way of the AIA or by the means of the AI national strategy or organisational policies and practices through: 

  • bans on prohibited (high risks) systems (AIA);
  • enforcement through market surveillance and control (AI Act);
  • (quick) scans/ risk analysis for selection of trustworthy suppliers and safe acquisitions (NCTV, 2024) and identification of supply chain risks (Bluebird & Hawk BV, et. al, 2024);
  • DPIAs and FRIAs, (AI Act, National AI strategy, GDPR, Organisational policies);
  • attribution of supervisory roles and compartmentalisation;
  • establishment of internal ethics committees addressing issues such as data ethics compliance or data “pedigree”.

Ongoing or simultaneous control:

  • supervisory authorities at national and international levels with sanctioning powers;
  • Meaningful human oversight within the process (organisational policies);
  • Process- related activity of ethics committees for deviating cases.

Equally relevant, for oversight to be effective, practical and technical capabilities giving external parties (either supervisory authorities of the broader public) relevant insight into log frames to evaluate how the data processing took place, details on the way the training data was processed and altered at every step should also be made available. These are forms of ex-post control, giving external parties the possibility to get insight into the system, either through information made public by the deployer beforehand or through standard access requests. A different form of ex-post control would be the requirement to make the AI models FAIR (Findable, Accessible, Interoperable and Reusable) and have metadata or a more extensive documentation package made available, either openly or upon request. This is partially addressed by the requirement to have high-risk AI systems registered in the European data base for High Risk systems that is under development (Article 71 of the AIA).

Article 14 of the AIA specifically addresses human oversight measures at the process level, placing responsibilities on both developer and deployer: on the developer during the design phase (therefore an ex-ante measure) for the identified risks, and for the deployer for the attribution of the actual human oversight during the process to human resources with the appropriate AI literacy. For high-risk systems, technical provisions include measures similar to kill switches (Article 14, point 4, e.) and oversight measures on supervision of two separate natural persons with appropriate training, competence and authority. Thus, for human oversight to be meaningful, AI literacy is a precondition (Recital 20, Articles 14, 26, 91). Adequate human and financial resources at eco-system and socio-technical levels are also paramount for a functioning oversight system. This is where fragmentation at European level is foreseen to remain high, For example, the Dutch Data Protection Authority (AP), as the dedicated supervisory authority for the use of AI systems that have an impact on personal data, has had an increased budget of 3.6 million euro for 2026 and 2027 (AP, 2024) to address the new tasks under its coordination streaming from the upcoming of the AIA. Yet, despite the budget increase, the Dutch AP is still challenged by finding the necessary numbers of trained specialists for the exercise of the given tasks. While the oversight and governance systems involve multiple entities, data protection authorities, as the responsible oversight entities of the AI also have the role of curbing the data appetite of public authorities. At the implementation level, human oversight remains both challenging and fragmented, influenced by level of digital maturity within a certain society, financial capabilities of each deployer (hyper-scalers having the higher ground), the availability of a trained workforce or lack thereof, the difficulty to keep up with technological and legislative developments, efforts by central government to harmonize implementation of EU legislation in a top bottom approach within national context  and uniformise/change already established daily practices.

The fact that AI is a disruptive technology is also reflected by the intense debates and analyses in legislative and policy-making circles in order to set it into a unified, comprehensive and EU level actionable text. Oversight of AI systems is highly regulated in democratic societies and while democratic states spend years and a great deal of financial resources to harmonize these legislative frameworks, less democratic states or ones that rather focus on market  competition principles seem more inclined to invest in and focus on the development and implementation of the technology self, focusing on capitalizing on the benefits without the restrictions of ethical and robust legislative frameworks, and as such gaining larger market shares in the world competition stage. 

Addressing the “why” of the need for human oversight includes the fact that AI is a disruptive technology, the level of trust in technology in general (at social level for example – for example the World Values Survey) and AI technology in particular, and the approach within a certain society towards rules and regulations. As Adams (2024) suggests, AI governance does not translate into responsible AI. Oversight structures are a form of foresight and a practice of a healthy ecosystem, in which decision makers, weather at supra-national, national or sectorial and even company levels implement checks and balances in a proactive manner.  Foresight is limited though to the overview of the known variables of the given environment at a certain moment and unforeseen uses remain a reality and very much in the control and responsibility of the deployer. Taking the discussion on when human oversight is needed further, it comes down to the cases and moments of high risk situations. Despite the fact that automated instruments for decision making have been hailed for the benefits they bring regarding time saving with repetitive tasks, it is in “life or death” situations where they failed and when ultimate responsibility was passed to the human. Identifying such “life or death” situations should be addressed in the design phase of the AI system, despite the limitations or impossibility of mapping out all possible real life situations in a design/ test phase or lab setting.  And while the AIA does address ex-ante or pre-deployment controls, lists prohibited AI practices, and identify highly regulated sectors and exceptions, in practice, specific sector level knowledge and also case by case analysis are needed in order to identify what qualifies as such and to identify the correct moment and forms of human oversight.

Empirical case studies involving the use of responsible AI in line with the AIA within different contexts are much awaited. Paradoxically, despite extensive regulatory systems, what gives rise to the need for case by case approaches is the uniqueness stemming from the intersection of legislations (for an overview see Annex 1 of the AI Act: list of Union Harmonization legislation), technological regulations, ethics, organizational settings and target groups affected by the AI. Harmonizing legislation attempts such as those in the case of products for the single market are underway. However, if the harmonization process will not work, it is foreseeable that the EU commission will come with its own standards, with the consideration that technical safety is easier to implement and standardize, whereas human oversight is more difficult both to regulate and ethical implications even more difficult to evaluate, reflecting also political choices. As practice evolves, it is expected that in the next years the scientific literature will abound with these case studies. It will be interesting to see how early adaptors, independently of their size, will set trends, how fast good practices will be absorbed and, as was the case with the GDPR, how case law will evolve. These will also serve as measures for measuring the effectiveness of the human oversight and control nexus, currently a challenge that will take the form of consistent practices in the coming years.  

References

Adams, R., Adeleke, F., Florido, A., de Magalhaes Santos, L.G., Grossman, N., Junck, L., Stone, K. (2024) Global Index on Responsible AI 2024  (1st Edition). South Africa: Global Center on AI Governance.

Autoriteit Persoonsgegevens. (2024). Directie Coordinatie Algoritmes Werkagenda coördinerend algoritmetoezicht in 2024. Accessed Mey 2024 at: https://www.autoriteitpersoonsgegevens.nl/documenten/werkagenda-coordinerend-algoritmetoezicht-2024

Bluebird & Hawk BV., De Nederlandse Vereniging van Banken, ICT Group, Nederlandse Spoorwegen en Technolution;  Nationaal Coördinator Terrorismebestrijding en Veiligheid (NCTV), Nationaal Cyber Security Centrum (NCSC); Algemene Inlichtingen en Veiligheidsdienst (AIVD); CIO Rijk (2024). Cybercheck: ook jij hebt supply chain risico’s. Available at: https://www.ncsc.nl/documenten/publicaties/2024/april/18/cybercheck-ook-jij-hebt-supply-chain-risicos

Oetheimer, M. (2024) Presentation during the European Data Protection Summit. Rethinking data in a democratic society. Session 3: Zooming out onto democracy and the rule of law. How to build a functioning democratic oversight. Available at:  https://20years.edps.europa.eu/en/summit/media

Nationaal Coordinator Terrorismebestrijding en Veiligheid (2024). Quick Scan nationale veiligheid bij inkoop en aanbesteding. Accessed June 2024 at: https://www.nctv.nl/onderwerpen/economische-veiligheid/documenten/publicaties/2024/02/01/quick-scan-nationale-veiligheid-bij-inkoop-en-aanbesteding

Novelli, Claudio and Hacker, Philipp and Morley, Jessica and Trondal, Jarle and Floridi, Luciano, A Robust Governance for the AI Act: AI Office, AI Board, Scientific Panel, and National Authorities (May 5, 2024). Available at SSRN: https://ssrn.com/abstract=4817755 or http://dx.doi.org/10.2139/ssrn.4817755

Verdiesen, I.; Santoni de Sio, F., Dignum, V. (2021).  Accountability and Control Over Autonomous Weapon Systems: A Framework for Comprehensive Human Oversight. Minds and Machines (2021) 31:137–163. https://doi.org/10.1007/s11023-020-09532-9

Wetzling, T. (2024) Presentation during the European Data Protection Summit. Rethinking data in a democratic society. Session 3: Zooming out onto democracy and the rule of law. How to build a functioning democratic oversight. Available at:  https://20years.edps.europa.eu/en/summit/media

Keywords: , , , , , ,

How to cite this article:

Popa, D. M. (2024). Multilevel oversight of AI systems in line with the AI Act. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/XEFY3802

A reappraisal of the AI Act in light of the qualitative requirements of the Law

Inês Neves (Faculty of Law, University of Porto / CIJ - Centre for Interdisciplinary Research on Justice)

Published on 29 June 2024
PDF
Cite

As the Artificial Intelligence Act (‘AI Act’ or ‘Regulation’) approaches its entry into force, it is tempting to conclude that the “job is (well) done”. However, a closer examination of the final text (signed on 13 June 2024) reveals a more nuanced reality. In addition to being implemented gradually over time and extending until 2026 (Article 113), the Regulation forms an integral part of the “New Legislative Framework” (‘NLF’) (Recitals 9, 46, 64, 83, 84). As such, it operates similarly to a ‘framework’ law, setting out a minimum set of essential requirements and high-level obligations applicable to a defined group of AI systems (those deemed high risk and specific risk).
As a result, before the Regulation is implemented in an administrative and jurisdictional manner, a process of normative creation must be initiated. In short, the actual law-making process – whereby the essential requirements of the AI Act will be specified and concretised through standards and other instruments, specifying the broad and open-texture wording of the AI Act through technical or quality specifications – is yet to commence (along with the governance structure and paraphernalia).

Nonetheless, it would be erroneous to assume that no progress has been made thus far or that nothing existed before the AI Act. There are ethical guidelines, standards that have already been published or are currently under development, and of course, there is all the European Union (‘EU’) and national legislation that is not affected by the AI Act (in terms of consumer rights, product safety, labour law, privacy and data protection, competition law, digital services, financial services, among others). Indeed, the existing legislation even raises doubts about the usefulness of the AI Act, and the EU legislator is not agnostic about the expectations and challenges of misalignment and coherence (see, among others, Recitals 46, 64, 158, and Articles 8(2), 72(4)).
While it is accurate to reflect that the AI Act is not intended to reinvent the wheel, the existing framework is deemed insufficient to provide a comprehensive framework of essential requirements specifically applicable to AI systems (high risk and specific risk). With this in mind, the AI Act follows the logic of product safety legislation, appearing as a hybrid between the need to promote trade and reduce barriers to innovation and the need to guarantee the protection of fundamental rights in the face of the specific risks of AI.

The AI Act thus exhibits a shared genetic identity with other acts of the NLF. However, it can be neither summarised nor equivalated to product safety legislation. On the one hand, this is due to the evolution and dynamism of AI systems, which make them very different from products typically associated with this type of legislation (toys, radio equipment, medical devices, machinery, among others). On the other hand, in addition to concerns for safety and health, the Regulation expressly includes fundamental rights among its objectives.
While the connection with fundamental rights does not extend to the point of turning the AI Act into an instrument that enshrines or guarantees new or specific fundamental rights per se, this link requires us to question whether greater demands on the applicable legal frameworks shall be ensured.

As is well known, the principles of legal certainty and legality demand that the law prescribing duties and obligations fulfil specific requirements of accessibility, foreseeability and precision. These requirements ensure that those obliged to (binding) legal frameworks understand what is expected of them in order to comply with such demands. Of course, these qualitative or substantive requirements do not go so far as to demand absolute certainty, which is incompatible with the recurrence of indeterminate concepts and vague terms (and the pace of technology). Furthermore, they do not negate the potential need for legal counsel or case law to comprehend the obligations prescribed by the legal system comprehensively.
However, if this is the case, there will be limits to the openness of the law, mainly when it is associated with burdens and requirements that restrict fundamental rights. This is the case of the freedom to conduct a business (Article 16 of the Charter of Fundamental Rights of the European Union), which is limited by the requirements and obligations applicable to the systems covered by the AI Act. Alternatively, the Regulation provides for sanctions that may be manifestly harmful to public interests, such as innovation and the very realisation of fundamental rights (whose realisation, at least in the current era, may depend on AI solutions).

A review of the AI Act’s recitals reveals a clear awareness of the necessity for legal certainty (see, for example, Recitals 3, 12, 83, 84, 97, 139 and 177) and the importance of defining its terms and requirements in a way that is consistent with the broader regulatory framework. However, in contrast to the limited references to this general principle of EU law, searching for the qualifier “appropriate” yields 230 results, which clearly demonstrates the broad and generic way in which the requirements for operators and systems are laid down and designed in the AI Act.
The necessity to reconsider the limitations of openness and the high-level approach adopted by the AI Act is highlighted by the fact that the AIA challenges the traditional remit of the requirements of legal certainty and the quality of the law, essentially on two levels.
Firstly, by referring to guidelines, codes of conduct, standards and common specifications, the question arises as to whether this connection between basic-fundamental law and implementation, not just by the European legislator but by other bodies, including private ones, will comply with the requirements of legitimacy and substance imposed on law in the material sense.
Secondly, insofar as the AI Act does not appear in a regulatory vacuum but instead leaves relevant European legislation (equally imposing obligations) untouched, it is essential to question whether the Regulation’s entry into force will have an impact on the predictability of what is expected of operators (in the face of perhaps contradictory expectations arising from different legal acts). It is believed that potential misalignments may be solved through a revision of the legislation in force, to identify conflicts and, if necessary, ensure that the combination of sectoral (old) and transversal (new) obligations and requirements does not result in greater entropy than advantage.

In this piece, we focus on the first area of concern.
Indeed, the Regulation leaves the implementation of a significant portion of requirements to the European Commission through a variety of means, including guidelines (Article 96), delegated acts (Article 97), and implementing acts (Articles 41(1) and 50(7)), among which codes of practice, where the AI Office acquires prominence (Articles 50(7) and 56).
The Commission is but one of many relevant players, however. Among other actors (including at the national level), the European Artificial Intelligence Board (‘AI Board’) will have the task of issuing recommendations and written opinions on issues relating to the implementation of the AI Act, aimed at ensuring its consistent and effective application (Article 66(e)). Furthermore, the role of the Advisory Forum (Article 67) and the Scientific Panel of Independent Experts (Article 68) must not be neglected.

In this regulatory landscape, harmonised standards must be recognised as primus inter pares (Article 40), as they will provide detailed technical specifications on how to comply with and meet the public interest objectives and the high-level requirements in the AI Act. The shortcomings of this co-regulation or delegation become evident when one considers that the acts in question are not those of EU institutions, bodies, offices, or agencies, and thus are not subject to judicial scrutiny (Article 263 TFEU). Additionally, fundamental rights are not a subject that can be technically specified. Finally, the fact that the institutions responsible for law-making are private entities and do not offer adequate guarantees of inclusion, representativeness and transparency creates the risk of regulatory capture and reinforces competitive foreclosure (which is already a consequence of the exclusion of AI systems already placed on the market or put into service (Article 111(2)).
In addition to standardisation, the Regulation also provides for common specifications (Article 41) in the event of i) non-acceptance of the standardisation request by any of the European standardisation organisations; ii) insufficient coverage by the harmonised standard of fundamental rights concerns, or iii) failure by the standard to meet the requirements of the request (promptly).
In both cases, voluntary compliance with harmonised standards or common specifications is associated with a presumption of conformity, particularly relevant for high-risk AI systems and general-purpose AI models. Although this presumption contributes to legal certainty, it merely guarantees a reversal of the burden of proof. The presumption is rebuttable, and the degree of legal certainty depends on the material content of the standards and specifications that will be adopted.
The preceding analysis leads to the conclusion that, in terms of meeting the requirements of accessibility and foreseeability, the AI Act does not represent an exemplar piece of legislation. The model adopted here is collaborative, with the intervention of various other players, and where the law and technology are inextricably linked. This does not imply that it is inherently inadmissible.
In fact, this way of co-regulation may be the only viable means of ensuring that the law aligns with reality. However, if this is indeed the case, the requirements for legitimising the processes for drawing up the intervening acts and actors should ensure high(er) levels of inclusion, participation, representativeness, and expertise. About the adoption of standards, it is crucial to bear in mind that the representativeness of smaller players and civil society may be impeded by factual inequalities (resources and expertise), even when abstract equality is provided for.

The interconnection between the AI Act and the fundamental rights renders these demands particularly urgent. Indeed, in this regard, the EU Regulation can be seen as both a virtue and a vice. On the one hand, its concern with fundamental rights is a distinctive feature, setting it apart from other acts within the NLF. On the other hand, however, this approach presents a tremendous challenge, as it is not aligned with the complexity and non-technical nature of fundamental rights, which are best understood as positions of value that neither the vagueness of the AIA nor the loopholes of standardisation fully consider.


In our view, there is hope in the subsidiary role of common specifications (Article 41(1)(a)(iii)). These are tailored to address the shortcomings of harmonized standards with regard to fundamental rights concerns and should be drawn up by the European Commission (an EU institution) in consultation with the consultative forum (Article 67). According to the provisions of the AI Act, this forum should represent a balanced selection of stakeholders, including industry, start-ups, SMEs, civil society and academia. Furthermore, it should be balanced about commercial and non-commercial interests, and within the category of commercial interests, with regard to SMEs and other businesses. In addition to the European Union Agency for Cybersecurity (‘ENISA’), the European Committee for Standardisation (‘CEN’), the European Committee for Electrotechnical Standardisation (‘CENELEC’), and the European Telecommunications Standards Institute (‘ETSI’), the Fundamental Rights Agency is also a permanent member of the advisory forum.

In our view, common specifications provide an additional and important safeguard. Firstly, they ensure that gaps in protecting fundamental rights are filled. Secondly, their adoption addresses the lack of representation and inclusiveness in the standardisation process, including regarding the participation of business and civil society. Thirdly, implementing acts (just as delegated acts) are subject to judicial review by the Court of Justice of the EU, as a result of which there is no lack of control.

As we can see, the Artificial Intelligence Act is not without flaws. It introduces particular challenges to the basic principles of the rule of law and the substantive requirements applicable to legal acts. This is not only because of the regulatory model it adopts but also because of the (lack of) legitimacy and democratic deficit of the players and procedures participating in the normative creation entailed by the AI Act.

It is acknowledged that the law and legislative procedures are not impervious to technological advancement and that they must, therefore, be capable of adapting frameworks and embracing new procedures and instruments (of a technical nature if needed). The key is to ensure that, in this process of openness, the law is not conflated with technology, but that technology is shaped to align with the requirements and fundamental principles of the (rule of) law.

Keywords: , , , , ,

How to cite this article

Neves, I. (2024). A reappraisal of the AI Act in light of the qualitative requirements of the law. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/TJZR2589

Welcome to our first fellows!

During spring the first call for AI Policy Fellows was launched. This first call was open for researchers working at all faculties at Umeå University. Many strong submissions were made and after a review process, we are now happy to announce the appointment of the inaugural fellows of the AI Policy Lab. The fellows will join the lab in August and during their time with us the fellows will lead research projects that investigates and address critical intersections of artificial intelligence and societal impact. The selected fellows are:

Elin Kvist

Elin Kvist will address the profound implications of data-driven and algorithmic systems on work environments and workers’ rights. Kvist aims to explore how AI influences workers’ abilities to mobilize and organize, crucially examining the landscape of political attitudes and union preparedness. This initiative seeks to empower workers amidst evolving AI-driven work dynamics.

Lars Norqvist project focuses on AI’s transformative role in leadership practices within educational settings. His interdisciplinary approach aims to unravel the complexities of AI’s impact on leadership autonomy, accountability, and decision-making processes. By employing qualitative methods, Norqvist seeks to provide actionable insights that can inform policy at various governance levels, aiming to bridge theoretical constructs with practical applications in educational leadership.

Lars Norqvist

Henry Lopez Vega research undertakes the challenge of shaping AI policies in healthcare delivery across the region of Västerbotten. Collaborating with regional healthcare stakeholders, Lopez Vega’s research aims to evaluate how AI regulations can enhance service delivery and ecosystem emergence in healthcare. This project will analyze policy implications and foster innovation through the adoption of AI technologies like chatbots and healthcare platforms.

The AI Policy Lab remains committed to advancing cutting-edge research that addresses critical societal challenges posed by AI technologies. Each fellow’s work aligns closely with the lab’s vision to inform policy and practice, ensuring AI’s responsible integration into society.

Opening AI Policy Lab

Invitation

On 18 June 2024, from 13.00-17.oo, the AI Policy Lab will officially open!

You are welcome to join us at MIT-huset A400 (see location in mazemap) for an afternoon filled with presentations, informal conversations and the opportunity to explore the premises.

The program is as follows:

13.00 – Opening by Virginia Dignum, director of the AI Policy Lab

13.15 – Bertram Malle, Brown University: Trust in and Trustworthiness of Artificial Agents

Abstract:

If artificial agents should deserve human trust, they must be trust-worthy.  But what makes a system worthy of trust?  I will introduce a model of human trust in other agents (whether persons, institutions, or machines) that specifies five dimensions of trustworthiness: competence, reliability, integrity, transparency, and benevolence.  I will show that ordinary people think of trust as expectations that the other agent has those attributes. I will then explore how an artificial agent might meet those expectations.  

14.00 – Ericka Johnson and Saghi Hajisharif, Linköping University: Bias and representation in synthetic data

Abstract:

Bias is an issue in the real world and for the AIs learning from real world data. But if we know a dataset is biased, one could hoped that making a curated synthetic data would be a way of eliminating that bias. However, while there is quite a bit of work being done to produce more just datasets through synthetic data, at the same time, work we are doing is demonstrating the challenges of caring for intersectional representation when generating synthetic data. This presentation of our findings ends with a question to the audience about how synthetic data should be labelled and regulated.

14.45 – Q&A

15.15 – Snacks and open house

17.00 – End

We look forward to welcoming you at the opening of the AI Policy Lab!

Morning program

In the morning, we are holding presentations from the MMW Project “AI: destroyer or enabler of democracy”

9.30 – Welcome and fika 

10.00 – 12- 00 – Short presentations (10+minutes each, followed by panel Q&A (topics tbc)

·     Privacy and Self-determination (Kalle Grill and Björn Lundgren) 

·     Automated decision-making in the public sector (Andreas Ojehag) 

·     Political studies of automated governing (Malin Rönnblom) 

·     Democracy and self-determination in a participatory design process in the public sector of a virtual coach for behaviour change (Helena Lindgren) 

·     Global AI governance (Virginia Dignum) 

·     Tool/method for exploring enactment of self-determination (Luis Gustavo Ludesher) 

12.00 – Lunch 

LAUNCHING AIPEX – AI policy Exchange Forum

We are thrilled to introduce the AI Policy Exchange Forum (AIPEX), an open online platform dedicated to fostering academic discussion on AI policy and governance. AIPEX bridges the gap between an academic journal and a blog, providing rapid, citable, and lightly-reviewed publications to stimulate global debates on emerging AI issues.

We look forward to your contributions and engagement in meaningful discussions.

Learn more and submit your contributions here: aipolicy.se/aipex.

2023 – The year of AI governance

The year 2023 marked a turning point for Artificial Intelligence (AI) governance. As AI technologies rapidly evolve, their profound impact on society and the economy is leading to increasing needs towards a coordinated approach to governance, a challenge that can only be addressed globally. 2023 has been a testament to that need, with major strides taken internationally to address the promises and perils of AI. In this article, I provide a timeline of the main events that shaped policies around AI during the past year, providing a comprehensive record, chronicling pivotal events and developments that have shaped the global discourse on AI regulation and its ethical implications.

Not only global events such as the groundbreaking approval of the EU’s AI Act to the creation of the UN’s Advisory Body on AI, but also various nations and organizations are stepping up their own AI policies and calls for action for a more responsible and inclusive AI were heard from different directions. The timeline below illustrates how the world is understanding the need for regulation of the potential risks and opportunities presented by AI. It’s a global challenge, one that demands collective effort and vigilance. Each entry in this timeline represents a significant moment in the journey towards responsible AI governance. They reflect a world in rapid transition, grappling with the complexities of AI and striving to harness its potential responsibly and ethically.

Looking forward, we can expect further recognition of the global impact of AI. Rapid advancements in AI technology, as evidenced by the release of e.g. GPT-4 and Gemini, have instigated a sense of urgency among policymakers and industry leaders to establish a regulatory framework.

However, in my opinion this does not imply that a uniform, global approach to AI regulation is needed, or even desirable. It is important to understand and accept that different regions, countries and sectors have unique backgrounds, cultures and needs. Nevertheless, increased urgency towards collaboration across the globe is needed. In order to support shared understanding, build bridges and ensure that benefits and opportunities are accessible to all and inclusive of differences. The variety of regulations and guidelines proposed or enacted worldwide indicates that different regions tailor their AI governance strategies to their unique cultural, ethical, and socio-economic contexts, but initiatives like the G7 Hiroshima AI Process and UNESCO’s urgent call for AI rules, or UN’s efforts to define a global governance approach, highlight the importance of international dialogue for effective AI policies.

Finally, regulation and governance must be implemented and understood not as something that hinders innovation, but much more as a stepping stone encouraging AI innovation. Governance measures not only spur AI developments towards responsible and beneficial directions, but also open up new research and innovation opportunities to define, develop and implement methods, tools and frameworks to support governance.

As AI continues to evolve, so will the challenges and opportunities it presents. This requires governance frameworks to be adaptable and forward-thinking. Ensuring that AI governance is inclusive, considering the needs and voices of diverse populations, will be crucial for equitable and sustainable AI development. This requires an ongoing, open and transparent dialogue among governments, industries, academia, and civil society to navigate the complex landscape of AI governance effectively. Such efforts will undoubtedly shape the trajectory of AI development and its societal integration, emphasizing the need for thoughtful, inclusive, and dynamic governance strategies.

An annotated timeline of AI (policy) events in 2023

January

  • 26: In the US, the National Institute of Standards and Technology (NIST) releases AI Risk Management Framework. This framework, developed in collaboration with the private and public sectors, is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems.

March

April

  • 11: China issues strict regulations on AI systems. These measures, which draft was open to public comments until 10 May, aim to govern generative AI service provision in China.
  • 13: US Senate leader Chuck Schumer announces plans to legislate AI. The proposal focuses on building a flexible and resilient AI policy framework across the federal government that can adapt as the technology continues to advance. It aims to foster innovation and the continued US leadership in the development of AI, while enhancing security, accountability, and transparency.
  • 17: EU legislators call for an emergency global summit. In a reaction to FLI’s ‘pause letter’, a group of members of the EU parliament urged world leaders to hold a summit to find ways to control the development of advanced artificial intelligence (AI) systems such as ChatGPT. Comment: Such a Summit has, as yet not materialised, having been superseded by the UK’s Safety Summit in October.
  • 26-28: The World Economic Forum (WEF) holds a summit on Responsible AI that resulted on the Presidium Recommendations to guide technical experts and policy-makers on the responsible development and governance of generative AI systems.

May

  • 4: US’s White House calls emergency meeting of leading AI CEOs during which the president stressed the need to mitigate both the current and potential risks AI poses to individuals, society, and national security, in order to realize the benefits that might come from advances in AI.
  • 12: Brazil’s government proposes an AI Bill. This proposal, that follows a risk-based approach similar to the EU’s AI Act, aims to create rules for the operation of AI systems in Brazil, establishes rights for people affected by their operation, and provides for penalties for violations, as well as information regarding the supervising body. Correction by Dora Kaufman : There isn’t yet a proposal for an AI bill, but the Brazilian Senate constituted a commission of Senators to analyze proposal PL 2338.
  • 16: OpenAI calls for governments to enact AI safety
  • 25: European leading AI researchers meet at the EU parliament to discuss the role of Europe on AI development and fundamental researcher, stressing the need for sovereignty
  • 30: G7 ‘Hiroshima AI process’ on global AI governance started aiming to aims to promote safe, secure, and trustworthy AI worldwide and provide voluntary guidance for actions by organizations developing the most advanced the rules for digital technologies,  including the most advanced foundation models and generative AI systems, to ensure that these are in line with “our shared democratic values”.
  • 30: The Center for AI Safety releases an open letter aiming at a less alarmistic and futuristic view than the FLI open letter, with the very short message: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

June

August

  • 15: China’s “Interim Measures” on Generative AI  enter into force. The measures, that reflect the feedback from different stakeholders, are issued by the Cyberspace Administration of China (CAC) together with six other central government regulators. The measures aim at regulating the provision of generative AI services, such as ChatGPT, to the public of mainland China and have been formulated in accordance with existing laws and regulations. The AI Measures aim to ensure that a healthy environment can be fostered within China that allows for the responsible use of generative artificial intelligence without causing undue harm to the national security, social and public interest, and the legitimate rights and interests of the citizens, including legal persons and organizations.

September

  • 27: Canada releases a voluntary code of conduct specific to generative AI. This proposal goes beyond risk mitigation, encouraging its signatories to promote and build a robust and responsible AI ecosystem in Canada. The code provides a set of measures that support upcoming regulation pursuant to AIDA, emphasizing developing and managing the operations of generative AI systems

October

November

  • 1: UK holds the AI Safety Summit aiming to set UK’s UK’s position as a world leader in AI safety. The summit, hailed as a diplomatic breakthrough after it produced an international declaration (the Bletchley Declaration on AI Safety)  to address risks with the technology, as well as a multilateral agreement to test advanced AI Models. The event was however heavily criticised by a.o. civil society organisations for mostly including tech execs and government officials.
  • 6: (entry proposed by Clara Lin Hawking): release of 01.AI’s open source large language model Yi-34B trained from scratch and finetuned for various chat usecases.
  • 13-15: WEF’s AI governance summit: This initiative, following WEF’s meeting in April, focused on responsible generative AI, bringing together influential regional voices and global stakeholders to harness the benefits of generative AI systems and technologies while ensuring equitable and sustainable global impacts.
  • 15: In the US, a bipartisan group of senators introduces the AIRIA Act.The AIRIA is the latest in the efforts of the US in establishing a safe and innovation friendly environment for the development and deployment of AI, with the dual aim of encouraging innovation while establishing a framework for accountability.

December

  • 6: Google launches Gemini: Gemini is Google’s latest multimodal (text, image and video) large language model, and their answer to compete with OpenAI’s GPT4. Gemini, they claim, is their ‘most flexible model yet — able to efficiently run on everything from data centers to mobile devices’.
  • 9: Europe’s trilogue agrees on the AI Act: A significant step on AI regulation, this world primeur, aims not only to enhance governance and effective enforcement of existing law on fundamental rights and safety, but also to promote investment and innovation in AI within the EU, and to facilitate the development of a single market for AI applications.
  • 21: The UN Advisory Body on AI releases draft recommendations: this interim report calls for anchoring AI in international law, human rights, and the Sustainable Development Goals. It also identifies AI governance critical functions and principles. In 2024, the UN AI Body will explore options for institutionalizing these functions, through a program of consultations with diverse stakeholders worldwide.

Our book “Social Simulation for a Crisis” is out now

Simulating for a crisis is far more than creating a simulation of a crisis situation. In order for a simulation to be useful during a crisis, it should be created within the space of a few days to allow decision makers to use it as quickly as possible. Furthermore, during a crisis the aim is not to optimize just one factor, but to balance various, interdependent aspects of life. In the COVID-19 crisis, decisions had to be made concerning e.g. whether to close schools and restaurants, and the (economic) consequences of a 3 or 4-week lock-down had to be considered. As such, rather than one simulation focusing on a very limited aspect, a framework allowing the simulation of several different scenarios focusing on different aspects of the crisis was required. Moreover, the results of the simulations needed to be easily understandable and explainable: if a simulation indicates that closing schools has no effect, this can only be used if the decision makers can explain why this is the case. This book describes how a simulation framework was created for the COVID-19 crisis, and demonstrates how it was used to simulate a wide range of scenarios that were relevant for decision makers at the time. It also discusses the usefulness of the approach, and explains the decisions that had to be made along the way as well as the trade-offs. Lastly, the book examines the lessons learned and the directions for the further development of social simulation frameworks to make them better suited to crisis situations, and to foster a more resilient society.

The book is available here: https://www.springer.com/gp/book/9783030763961

ASSOCC on national tv

On 23 April, the Swedish broadcaster TV4 run a piece on the work of ASSOCC in their evening news program showing how social simulation provides insights into the impact of corona pandemic policy interventions. You can watch it here: https://www.tv4play.se/program/nyheterna/nyheterna-1900-23-april-2021-s%C3%A4song-2021/13336311 (starts at 14:35)

ASSOCC session at SocSimFesT 2021

Join us on Tuesday 16th March (9:30 – 12:40 CET) for a session on Modelling for Crises in times of Crisis, as part of SocSimFesT 2021

All information and schedule: simassocc.org/assocc-at-socsimfest2021/

Session organizers: Amineh Ghorbani, Frank Dignum, Harko Verhagen, Fabian Lorig,  Paul Davidsson, Lois Vanhee (correspondence: a.ghorbani@tudelft.nl)