Fairness inside and out: A situated approach to algorithmic allocation in complex sociotechnical systems

Abstract

This article outlines a framework for modeling and simulating complex sociotechnical systems in which an allocation mechanism acts as the interface (and sometimes a barrier) between the public and institutions. Two examples contextualise algorithmic allocation challenges: the Amsterdam school choice and organ allocation for transplant patients. We highlight how algorithmic fairness does not guarantee systemic fairness, and propose a situated approach with institutional modeling and social simulations. With the increasing adoption of AI in decision making, a situated simulation approach provides an alternative to opaque institutional governance and decision making.

The mechanism, institutions, and people

Sociotechnical systems are formed of technical and social components; in present-day societies, most technology is used within a social environment. Mindful of algorithmic discrimination cases, researchers have sought ways to measure the fairness of technical systems. The impact of a technology on its surrounding environment, however, is not always clear, especially when complex social dynamics are at play. Designing a “fair” technical component is not a guarantee of bringing fairness to the system as a whole. Selbst et al. argue that this is due to a focus on solutions (i.e. “make the outcome fair”) instead of a focus on the process (i.e.”make the system fair-er”) [1]. We now need ways to understand the impact of a technical system on fairness, within the technology, and without it.

Resource allocation problems are routinely faced by public institutions, be it in the form of determining who gets access to benefits, or which infrastructure to renovate among a set of proposals. The decision making process forms in institutions, bound by laws of fairness and non-discrimination. However, institutional decisions change the dynamics of the system before they take action, while they are communicated, and after they take hold, altering the behaviour of the social system.

Social simulations can estimate the effect of policy changes by dynamically modeling the social and institutional aspects of sociotechnical systems [2]. The field of institutional modeling investigates the interplay of individual and institutional agents within complex sociotechnical systems [3].  Social and institutional simulations might thus guide the process of creating fairness throughout the system and not only within the mechanism. We use the framing of simulations to explore allocation mechanisms and their impact on complex sociotechnical systems.

Algorithmic allocation

We propose an approach to modeling sociotechnical systems under specific conditions: resource allocation with a registry (here called mechanism) facing the public in one direction and based on rules determined by one or multiple institutions. The communication between public and institution occurs through the mechanism, which is designed to optimize for a certain outcome. Value preferences inform the outcome, but are not always explicit, nor uniform across parties. The institution designs the rules of the allocation mechanism based on (i) its internal principles, and (ii) a fair outcome for the public. Without communication between public and institutions, the rules are decided based on inferred preference models, i.e., the institution designs the mechanism to please the assumed value preference of the people and reach a “fair” outcome (Fig. 1).

In this setting, which we contextualise in two examples below, the mechanism design affects the behaviour of agents independently of its inherent degree of algorithmic fairness. Specifically, we encountered two cases that pose what seem to be “already-solved” allocation problems. Upon further scrutiny, we found underlying, more complex dynamics.

Figure 1. Motivating setup: A sociotechnical system consisting of agents, a mechanism, and policy actors. The agents provide an input to the mechanism, and are affected by its output, while a policy actor sets the rules for the mechanism.

Figure 1. Motivating setup: A sociotechnical system consisting of agents, a mechanism, and policy actors. The agents provide an input to the mechanism, and are affected by its output, while a policy actor sets the rules for the mechanism.

As shown in the examples, the mismatch between the allocation rules and the preference model of the population influences the mechanism’s efficacy, prompting gamification for those who have the resources to invest in understanding and playing the mechanism’s rules. In the second example, the complexity of conflicting institutional values limits the mechanism’s fairness and efficiency.

This raises questions of prosociality, trust, governance and fairness outside algorithmic bounds. How do people respond to a mechanism that does not align with their preference model (and perhaps their values)? How can institutions consider the longitudinal effects of algorithmic allocation within the process of policymaking?

Amsterdam school choice

The Amsterdam school choice system is an especially useful example of a sociotechnical system where the deployment of algorithmic allocation led to unexpected changes in the behavior of the social system, causing continuous updates to the algorithm and subsequent degradation of trust in the system. It is also a good example of when a poorly designed “fair” mechanism leads to diminishing prosocial behavior.

In Amsterdam, there is an open-choice policy when moving from primary to secondary school [4]. Group 8 students (equivalent to 6th grade in the US) are allowed to pick any school within their education level (e.g., vocational, general secondary, pre-university, etc.) anywhere in the city, unrestricted by their zone of residence. Every year, students are asked to rank 8 to 12 schools (based on education level), and then a centralized matching algorithm automatically matches students to schools.

The system is communicated to be based on the famous Deferred Acceptance algorithm [5], a New York based school choice algorithm which won its author the Nobel prize in Economics in 2012 [6]. However, the algorithm is modified in several key places to fit the open-choice policy in Amsterdam.

Firstly, the Deferred Acceptance algorithm is a two-sided matching method, meaning that students and schools both have preferences over each other and are then matched accordingly. In Amsterdam, schools do not have a preference over students and therefore this preference is simulated with the means of a random lottery number. This choice was also grounded in fairness, as with a random lottery, every student has an equal chance of getting a good lottery number. Secondly, the Deferred Acceptance algorithm does not specify a fixed number of schools to rank. In Amsterdam, however, a policy named “placement guarantee” was introduced, where students are guaranteed a position if they rank a fixed number of schools. This guarantee is ensured by increasing the capacity of schools by a marginal amount in a second round of allocation, where students who listed the required number of schools in the first round and still did not get a placement are eligible, and are then allocated using the increased capacity in the second round. These changes unsurprisingly led to the matching algorithm performing very differently from its Nobel-prize-winning counterpart, creating problems of inefficiency and inequality.

Random lottery numbers led to an inefficient allocation of schools, increasing dissatisfaction among students and their parents [7]. Moreover, the “placement guarantee” policy led to strategic gaming from the parents, where they are incentivized to report schools they do not prefer just so they can ensure eligibility for the second round [8]. This further degraded the system’s efficiency, as well as its fairness, as parents from privileged backgrounds are better able to apply strategies than parents from marginalized communities.

The authorities responsible for the school choice system have repeatedly come under public scrutiny for continuous changes to the system without successful results [9]. This has led to a degradation of trust in the institution.

Underneath this allocation problem is a complex sociotechnical issue, one that requires:

  1. Making clear what values the system is actually trying to serve, and whether these align with what parents themselves care about.
  2. Understanding how the rules of the mechanism change social behavior, so that reported preferences are seen not just as choices, but also as responses to risk, incentives, and unequal access to information.
  3. Identifying problems that are easy to miss if we only look at fairness within the mechanism itself, such as distorted preferences, unequal strategic burden, and declining trust in the system.
  4. Looking beyond the matching rule alone, and instead supporting better communication, better alignment between stakeholders, and more situated policy design.

This requires a multidisciplinary sociotechnical approach, one in which a social simulation model can help make the problem and its tradeoffs easier to communicate and discuss. It also requires a systematic way of eliciting stakeholder values and using those discussions to work toward a shared understanding of what the system should aim to achieve.

Organ allocation systems

Fairness plays an important role in organ allocation systems. For patients on the waiting list to receive a transplant, waiting time is a strong determinant of short-term survival, and long-term quality of life. Multiple factors influence the structure of the waiting list and the distribution of donated organs, particularly those from deceased donors. Initially, biological compatibility between donor and recipient seems to provide a baseline allocation rule: the most compatible donor-recipient match (considering blood type and HLA presence) should be prioritised to ensure the best graft survival outcomes. However, the length of the waiting list and persistent scarcity of organs complicates the allocation problem.

Cold ischemia time restrictions (meaning how long an organ is deprived of blood flow before its quality deteriorates beyond utility for transplantation) brings into consideration logistics, donor and recipient location and a corollary of geographical factors such as regional allocation rules, coordination practices and transplant capacity. Allocation policies have been studied via simulations and algorithmic optimization for decades, with awareness of the complex interplay of medical, economic, political and legal factors.

The other component of the system is the prioritization of patients within the waiting list structure. The state of the patients is dynamic, with probabilities of becoming too ill to receive a transplant. The waiting list design attempts to accommodate for this by modeling the disease progression and setting thresholds for transplant eligibility.

In the U.S., Organ Procurement Organizations (OPO) and Transplant Centers’ performance is evaluated competitively based on how many organs were retrieved and how many successful transplants were performed [10], adding economic incentives to the mix.  This influences the local center’s decision to accept available organs or reject them in the hope of a better quality alternative.

Even seemingly aligned bioethical values can create conflicting interests, especially if multiple institutional actors are pushing for their preferred outcomes. Beside the responsibility to ensure “fairness” in allocation, the meaning of “do no harm” manifests differently for Transplant Centers (i.e. reject lower quality organ offers) and OPO (i.e. provide as many quality organs as possible), exacerbating inefficiencies. Concerning the value preferences of the patients themselves, little is known. Without a channel of communication to the algorithmic rule setters, and with conflicting interests among institutions, we risk optimizing for less relevant values.

Most importantly, the algorithm of allocation can be gamed by multilisting. Multilisting is the practice of assigning one patient to multiple waiting lists across Transplant Centers, which initially seemed to alleviate the length of the waiting list at the national scale in the U.S. However, this came at the cost of individual fairness (not everyone can be listed in multiple registries as not everyone can afford to quickly travel across states to receive a life-saving transplant) and collective fairness: while the national wait time decreased, regional variance in wait time can increase [11]. Multiple organ offers also seem to produce a utilitarian gain in efficiency by reducing organ wastage and time to transplant, but may do so at the cost of societal trust in the system [12].

Within this critical healthcare system, the allocation mechanisms endlessly attempt to reconcile fairness and efficiency. The value prioritization shifts from face-value fairness (e.g. centralised FIFO allocation) to utilitarian fairness (prioritizing patients with “the most to gain” from a transplant) or need-based fairness (prioritizing the most sick patients based on disease models with thresholds for exclusion), or introducing market logic to the system (monetary rewards to centers, multilisting, multioffer). Formalizing any value-rule into algorithmic allocation mechanisms creates, as in the example above, new ways for the system to be strategised upon, shifting the locus of inequality and obscuring its methods.

Decontextualised fairness as an attribute of technical (sub)systems has already been criticized [1]. The algorithms designed for organ allocation systems present a clear example of the Framing and Formalism abstraction traps, in which fairness evaluation stops at the technical part of the sociotechnical system and its formalization generates new, unaccounted behaviours in the system. Some literature acknowledges these limitations, suggesting we might expand our abstraction boundary, or even consider other processes beside algorithmic allocation [13,14]. In the absence of a consensus of what “fair” organ allocation is, we might want to shift our focus to a process approach. Social and institutional modelling provide the best toolset for this endeavour.

Our proposal

We propose a four-step approach for studying complex sociotechnical systems in which an allocation mechanism mediates between the public and institutions. The goal is to understand how to embed algorithmic allocation mechanisms in a wider system of stakeholder values, behavioral adaptation, and institutional feedback. As the Amsterdam school choice reminds us, the algorithmic allocation must be designed to accommodate a response to the mechanism itself. To this end, we consider social/institutional simulations as the most eligible, situated approach.

The four steps are as follows:

  1. Value elicitation: The first step is to determine which values the mechanism is meant to serve, and whether those values are actually shared by the stakeholders affected by it. Mechanisms are often designed around an inferred preference model: institutions assume what matters to the population and encode those assumptions into rules and objectives. But in high-stakes systems, these assumptions may diverge from the lived priorities of the people who interact with them. A situated approach therefore begins by making these values explicit and contextable.
    For example, in the Amsterdam school choice, policymakers may prioritize procedural fairness and avoiding unassigned students, while parents may care more about genuine access to preferred schools, reduced strategic burden, transparency, and trust. This gap matters for how the system performs in practice. Within organ allocation, value elicitation is needed to clarify which purpose the allocation system serves within the transplantation system as a whole, and what systemic fairness means to stakeholders. Although this might not result in a univocal “solution”, it lights the fire to improve procedural awareness. The value elicitation should start from Question 0, meaning without the assumption that an allocation mechanism is mandatory [15].
  2. Modeling and simulation: The second step is modeling and simulation. The simulation model is not the end goal, but a structured approach to facilitate and open communication around assumptions, tradeoffs, and possible interventions. If the response of the population to policy is modeled, strategic gamification can be accounted for in the simulated scenarios. In systems such as school choice and organ allocation, the mechanism does not act on fixed inputs. The inputs themselves change in response to the mechanism. Parents adapt their reported preferences strategically; OPOs and patients respond to waiting list structures, matching rules, and institutional incentives. Social simulation makes it possible to examine these interactions explicitly and to ask not only what outcomes a rule produces, but how that rule changes behavior across the wider system.
  3. Identifying bottlenecks and blind spots: The third step is to identify systemic bottlenecks and blind spots. Mechanisms are often evaluated in terms of fairness or efficiency within the allocation procedure itself, but this can obscure deeper problems elsewhere. A mechanism may be fair on paper while placing informational or strategic burdens unevenly across groups. It may also optimize one part of the process while ignoring upstream or downstream failures. A situated simulation framework helps uncover these blind spots by tracing feedback loops across the system: strategic adaptation, unequal ability to navigate rules or exploit advantages, changing interpretations of fairness, and degradation of trust over time. The modeling of heterogeneous agents and institutions allows the representation of multiple values and goals.
  4. Policy recommendations: The final step is policy recommendations. These recommendations should go beyond a technical redesign of the mechanism itself, as the system is sociotechnical; interventions may also need to address communication, participation, institutional coordination, trust, and policy stability. As the approach is situated within a community of stakeholders (institutions and population), the policy recommendations are tailored to local needs. This step can be expressed in policy briefs, appointment of committee and representatives to coordinate across stakeholder groups.

The approach is intended to be iterative. Fixed time intervals between iterations allow for policy to take action, and structure the process of altering it. Moreover, they facilitate participation of stakeholders that are not institutional, constructing the communication channel that was lacking in the initial setting (Fig. 1).

Conclusions

Algorithmic allocation mechanisms are a component of critical sociotechnical systems where institutions and populations interact. We argue that their fairness should be evaluated inside the algorithms and outside of them, encompassing their effect on the system they become a part of. With the increasing use of AI to automate allocation processes, aligning the values of stakeholders and opening communication channels between the population and institutions becomes our priority. A four-step situated approach with social and institutional simulations is presented. The approach shifts the focus from a fair outcome to a fairer process. While we do not claim that the approach is perfect, it provides a step forward in the integration of algorithmic allocation and fosters a procedural, situated view of fairness.

References

[1] Selbst, Andrew D., danah boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet Vertesi. 2019. “Fairness and Abstraction in Sociotechnical Systems.” In Proceedings of the Conference on Fairness, Accountability, and Transparency, 59–68. New York: Association for Computing Machinery. https://doi.org/10.1145/3287560.3287598.

[2]  Lorig, Fabian, Fabris, Bertilla, Tucker, Jason. 2025. “Hybrid-Human Policy Modeling: Enhancing Decision-Making Using Social Simulations” Frontiers in Artificial Intelligence and Applications. Doi: 10.3233/FAIA250675

[3] Ghorbani, Amineh. 2022. “Institutional Modelling: Adding Social Backbone to Agent-Based Models.” MethodsX 9: 101801. https://doi.org/10.1016/j.mex.2022.101801.

[4] Ruijs, Nienke, and Hessel Oosterbeek. “School choice in Amsterdam: Which schools are chosen when school choice is free?.” Education Finance and Policy 14, no. 1 (2019): 1-30.
https://doi.org/10.1162/edfp_a_00237.

[5] Abdulkadiroğlu, Atila, Parag A. Pathak, and Alvin E. Roth. 2005. “The New York City High School Match.” American Economic Review 95 (2): 364–67. https://doi.org/10.1257/000282805774670167.

[6] Nobel Prize Outreach. 2012. “The Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel 2012.” NobelPrize.org. https://www.nobelprize.org/prizes/economic-sciences/2012/summary/.

[7] Het Parool. 2026. “Weer Minder Achtstegroepers Kunnen Naar Eerste Voorkeursschool, Volgend Jaar Wordt Het Beter.” Parool.nl. https://www.parool.nl/amsterdam/weer-minder-achtstegroepers-kunnen-naar-eerste-voorkeursschool-volgend-jaar-wordt-het-beter~b0ec8fd5f/.

[8] Tasnim, Mayesha, Paul Verhagen, Tobias Blanke, Erman Acar, and Sennay Ghebreab. 2025. “Modeling Strategic Risk in School Choice: A Case for Transparent Design”. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society 8 (3):2470-79. https://doi.org/10.1609/aies.v8i3.36731

[9] Het Parool. 2026. “Loting voor Middelbare School Weer Terug naar Oude Systeem: 12 in Plaats van 9 Voorkeursscholen.” Parool.nl. https://www.parool.nl/amsterdam/loting-voor-middelbare-school-weer-terug-naar-oude-systeem-12-in-plaats-van-9-voorkeursscholen~b7972bda/.

[10] Washburn, Kirt. 2012. “Maximizing Donor Potential: Evolving Organ Procurement Organization Metrics and Optimizing Organ Distribution and Allocation in the United States.” Liver Transplantation 18 (suppl. 2): S1–S4. https://doi.org/10.1002/lt.23507.

[11] Harvey, C., and J. R. Thompson. 2016. “Exploring Advantages in the Waiting List for Organ Donations.” In 2016 Winter Simulation Conference (WSC), 2006–17. Washington, DC: IEEE. https://doi.org/10.1109/WSC.2016.7822245.

[12] Erazo, Ignacio, David Goldsman, Pinar Keskinocak, and Joel Sokol. 2022. “A Simulation-Optimization Framework to Improve the Organ Transplantation Offering System.” In 2022 Winter Simulation Conference (WSC), 1009–20. IEEE. https://doi.org/10.1109/WSC57314.2022.10015431.

[13] Thompson, David, Larry Waisanen, Robert Wolfe, Robert M. Merion, Keith McCullough, and Ann Rodgers. 2004. “Simulating the Allocation of Organs for Transplantation.” Health Care Management Science 7 (4): 331–38. https://doi.org/10.1007/s10729-004-7541-3.

[14] Feccia, Mariano, Arianna Freda, Maurizio Naldi, Gaia Nicosia, and Andrea Pacifici. 2026. “Modelling and Simulating the Organ Donation Process Using Bootstrap and Event-Driven Process Chain Representation.” Journal of Simulation 20 (2): 135–51. https://doi.org/10.1080/17477778.2025.2486709.

[15] Dahlgren Lindström, Adam, Dignum, Virginia, Ericson, Petter, Titareva, Tatjana and Tucker, Jason. 2025. “Responsible AI Self-assessment Workshop: Start with Question Zero”. https://aipolicylab.se/2025/09/05/responsible-ai-self-assessment-workshop-start-with-question-zero/ AI Policy Lab, Published September 5, 2025. Accessed April, 2026.

Rethinking the Digital Omnibus’ Impact on the EU AI Act: Simplification or Dilution? 

Introduction

The adoption of the European Union AI Act (hereafter AI Act) marks a milestone in the union’s ambition to shape trustworthy, human-centric artificial intelligence (AI). It reflects an effort to ground innovation in fundamental human rights – an approach that has positioned the EU as a global standard-setter in digital regulation. The proposed Digital Omnibus represents an effort to streamline and harmonise an increasingly complex regulatory landscape. 

However, as the Digital Omnibus Regulation Proposal directly affects the implementation of the AI Act and related data governance frameworks, it raises a number of critical cross-cutting concerns. In particular, certain simplification measures – introduced with the intention to reduce administrative burdens, facilitate compliance procedures, and improve regulatory coherence – may have unintended consequences on the consistency, traceability, and risk-sensitivity of the EU’s digital regulatory ecosystem. These include potential reductions in the consistency of regulatory application, limitations in the traceability of data and AI systems, and a weakening of the granularity required for effective risk assessment within the EU’s digital framework. Moreover, while these measures are designed to ease obligations for AI providers and developers, they may have the effect of shifting complexity downstream onto deployers and end-users. This could result in increased uncertainty for those responsible for the use of AI systems in practice, particularly in high-risk contexts where clear allocation of responsibilities and robust risk assessment remain essential. 

These interactions are especially relevant where data processing rules, access to large-scale datasets, and incident reporting mechanisms intersect with the criteria used to assess and classify high-risk AI systems. Ensuring that these instruments remain coherent in their application is therefore essential to preserving both the safeguards and the credibility of the EU’s risk-based approach. 

In this context, the AI Policy Lab at Umeå University seeks to contribute constructively to the ongoing discussion by highlighting specific areas of concern and putting forward targeted recommendations. The main objective is to ensure that the EU’s regulatory ecosystem remains both effective and future-proof. 

1. Amendments to the GDPR for AI training, Article 3 of the Digital Omnibus proposal 

The Digital Omnibus proposal introduces clarifications regarding the legal bases and conditions under which personal data may be processed for the development and training of AI systems. In particular, it elaborates on the use of legitimate interest as a legal basis and introduces specific derogations for the processing of special categories of data (i.e., sensitive data such as health, biometric, or political information). 

While these changes primarily concern data protection law, they have indirect but significant implications for the application of the AI Act – especially Article 6, which governs the classification of high-risk AI systems. This is due to the fact that the scope, nature, and volume of data used in training are key elements in assessing the risks associated with AI systems. 

Several aspects of the proposal may inadvertently weaken the link between data governance and AI risk classification: 

  • The introduction of derogations for the processing of so-called “residual” sensitive data during model training (§33) risks expanding the volume of sensitive data that can be used without prior scrutiny. This could reduce the ability of regulators to accurately assess whether an AI system should be classified as high-risk under Article 6 of the AI Act. 
  • The broadening of legal bases for data processing in the context of AI training (§§30-31) may weaken the connection between the actual risks posed by a system and its regulatory classification, particularly in the absence of coordinated interpretative guidelines between data protection authorities – such as the European Data Protection Board (henceforth EDPB) – and AI governance bodies – such as the AI Office. 
  • The simplification of transparency obligations, including information notices and Data Protection Impact Assessments (henceforth DPIAs) (§36, §40), may reduce the level of detail available to regulators. This, in turn, could hinder a proper assessment of whether a system meets the criteria for high-risk classification. 

Our Recommendations: 

  1. Introduce a specific notification requirement for AI models trained using the “residual sensitive data” exemption (§33) to preserve traceability of training datasets and support risk classification under the AI Act.    
  2. Provide a DPIA section dedicated to AI systems. This could help preventing the simplification of DPIA (single EU lists) from reducing the granularity necessary to assess AI risk.  

2. Prevent the merger of the Data Act, Digital Governance Act, and Open Data Act from creating “shortcuts” for high-risk AI systems 

The Digital Omnibus seeks to consolidate several existing legislative instruments – the Data Act, the Data Governance Act (henceforth DGA), and the Open Data Directive – into a single, more coherent framework governing access to and reuse of data, including data held by public authorities. This consolidation is intended to facilitate access to large datasets, including non-personal data, and to promote data sharing across sectors. While this can significantly support innovation and the development of AI systems, it may also have implications for how such systems are classified under the AI Act. Among these, it is important to highlight the following:  

  • Easier access to large volumes of data – including datasets that can be combined or enriched – risks enabling the development of AI systems whose purpose or context of use would place them within the scope of Article 6 of the AI Act (high-risk AI systems).  
  • There is a risk that the simplification of data access mechanisms could be interpreted, in practice, as a justification for lowering the perceived risk level of such systems. In other words, increased data availability risks inadvertently being used as an argument to downgrade regulatory scrutiny.  

Our Recommendations: 

  1. Clearly establish that simplified access to data does not affect the criteria for high-risk classification under the AI Act. The availability of data should not be considered a mitigating factor in the assessment of risk. 
  2. Introduce an ex ante assessment requirement for cases where public or publicly accessible data is reused for AI systems that are likely to operate in high-risk domains (such as employment, education, healthcare, or access to essential services). 
  3. Require public administrations, when authorising the reuse of data for AI development, to explicitly indicate whether the intended use is likely to fall within the scope of Article 6 of the AI Act. This would provide greater legal clarity for developers and strengthen regulatory consistency.  

3. Single-entry point for incident reporting (Article 6 and 9 of the Digital Omnibus proposal) 

The proposal to establish a single European entry point for incident reporting constitutes a significant step towards simplifying and harmonising reporting obligations across multiple regulatory frameworks, including NIS2 (cybersecurity), GDPR (data protection), DORA (financial sector resilience), eIDAS (digital identity), and the Critical Entities Resilience (CER) Directive. By centralising notifications through a platform managed at the EU level – specifically by the European Union Agency for Cybersecurity (ENISA) – the proposal aims to reduce administrative burdens for operators and improve the efficiency of information sharing across authorities. 

However, the centralisation of notifications on a single platform managed by ENISA raises some critical issues that deserve careful consideration (in order to ensure the effectiveness of the system and the protection of operators subject to reporting obligations):  

  • Expanding ENISA’s mandate to manage a unified reporting platform may lead to capacity constraints, given the anticipated volume of notifications. Any delays in processing or triaging reports could negatively affect incident response times and overall system resilience. 
  • The single-entry point is designed as a hub rather than a replacement for Member States’ national competent authorities. However, without seamless technical and procedural interoperability with existing national systems, there is a risk of duplication, inefficiencies, or increased administrative complexity.  
  • The proposal does not sufficiently clarify the allocation of responsibilities between operators, ENISA, and national authorities in cases of system malfunction, delays, or errors. This lack of clarity risks exposing operators to legal consequences for circumstances beyond their control.  
  • The centralisation of incident-related data at the EU level also raises questions regarding data governance and technological sovereignty. In the absence of clear guarantees on data localisation, secure infrastructure, and Member State oversight, there is a risk that sensitive operational information – potentially critical for national security – may be insufficiently protected or subject to dependencies on non-EU technological backbones. 

Our Recommendations: 

  1. Establish an independent annual audit mechanism to assess the functioning of the single-entry point. This audit should evaluate: (i) the system’s capacity to handle notification volumes, (ii) the timeliness and accuracy of information processing, (iii) the cybersecurity of the platform, and (iv) its level of interoperability with national systems. Such an audit would help to ensure transparency, reliability and continuous improvement of the system. 
  2. Introduce a mandatory fallback protocol to be activated in the event of technical unavailability. This should include: (i) alternative reporting channels, (ii) clear criteria for demonstrating compliance efforts by operators, and (iii) automatic suspension of notification deadlines during system outages. This would prevent operators from incurring violations due to circumstances beyond their control.  
  3. Clarify the liability framework by explicitly defining the division of responsibilities between ENISA, national authorities, and reporting entities. This should specify: (i) situations in which failures are attributable to the central system, (ii) the implications for operators’ legal obligations, and (iii) the safeguards available in cases of technical malfunction. Clear and predictable rules are essential to ensure legal certainty and the consistent application of reporting obligations across the European Union. 
  4. Introduce explicit requirements ensuring that the infrastructure supporting the single-entry point is based on secure, EU-controlled technological backbones, with clear provisions on data localisation, access control, and Member States’ oversight. This could include, for instance, reliance on trusted European cloud frameworks or “no non-EU backbone” requirements for particularly sensitive categories of incident data. Such safeguards would strengthen trust in AI systems and ensure alignment with broader EU objectives on digital sovereignty. 

Conclusion 

Overall, the Digital Omnibus proposal reflects a necessary and timely effort to streamline an increasingly complex regulatory framework and to facilitate its practical implementation across sectors. At the same time, the analysis above highlights the importance of maintaining a careful balance between simplification and regulatory integrity. Across the areas examined – namely AI training data governance, access to public and non-personal data, and incident reporting mechanisms – there is a common need to preserve traceability, ensure risk-sensitive oversight, and safeguard legal certainty, while also reinforcing the Union’s strategic autonomy. The recommendations above by the AI Policy Lab at Umeå University are intended to support this balance by addressing specific gaps and ambiguities without undermining the overall objectives of the proposal. In doing so, they aim to contribute to a coherent, robust, and future-proof EU regulatory framework for AI and the data economy. 

References  

Directive (EU) 2022/2555 of the European Parliament and of the Council of 14 December 2022 on measures for a high common level of cybersecurity across the Union (NIS2), OJ L 333, 27.12.2022, p. 80–152.  

Directive (EU) 2022/2557 of the European Parliament and of the Council of 14 December 2022 on the resilience of critical entities (CER), OJ L 333, 27.12.2022, p. 164–198. 

Proposal for a Regulation of the European Parliament and of the Council establishing a Digital Omnibus for the simplification of Union digital legislation, COM(2025) XXX final. 

Regulation (EU) No 910/2014 of the European Parliament and of the Council of 23 July 2014 on electronic identification and trust services (eIDAS), OJ L 257, 28.8.2014, p. 73–114 (as amended).  

Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 (General Data Protection Regulation – GDPR), OJ L 119, 4.5.2016, p. 1–88. 

Regulation (EU) 2022/2554 of the European Parliament and of the Council of 14 December 2022 on digital operational resilience for the financial sector (DORA), OJ L 333, 27.12.2022, p. 1–79. 

Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), OJ L, 12.7.2024. 

AI first to purpose first: Rethinking Europe’s AI strategy

Abstract

This paper examines the European Commission ‘AI First’ strategy, arguing that it places acceleration and economic competitiveness above democratic values, societal benefit, and human-centric innovation. While substantial investment in AI is welcome when it promotes sustainable, equitable, and responsible innovation, the authors warn that policy is shifting from governance to unchecked deployment, risking fragmentation, dependency, and misaligned priorities. Rather than asking how AI can be applied, the paper urges policymakers to ask why, advocating a “People First” approach grounded in societal needs, digital sovereignty, and responsible innovation. The authors argue that Europe’s AI leadership should be shaped not by speed, but by principled direction, inclusivity, and a commitment to long-term public value.

AI First

AI is increasingly being framed as a strategic imperative for economic growth, competitiveness and innovation. Yet, this purpose is often at odds with a more fundamental question: what should the purpose of AI be, and under which conditions does it genuinely add value to society? Following the recent launch of The European Commission’s (2025a,b) Apply AI Strategy and the ambitious InvestAI Programme, aimed at building pan-European AI “gigafactories”, (European Commission 2025c), heralded by the Commission’s President as a cornerstone for Europe’s AI competitiveness, the policy discourse has shifted from governance to acceleration. This rhetoric of Europe becoming the “Continent of AI’” however may signal a worrying departure from Europe’s longstanding commitment to human-centric and responsible innovation.

The timing and framing of the European Commission (2025a) “AI First” narrative appears to be closely aligned with the recommendations of the Draghi Report (European Commission 2025d), which emphasises digital investment and competitiveness as central to Europe’s economic renewal. While the substantial funding and incentives for AI research and innovation are welcome, the framing of “AI First” ignores a deeper set of concerns, including the limited evidence, if any, of substantial productivity and societal gains from AI use ((Estrada 2025; Wearden 2025). As such the shift to “AI First’” not only threatens to erode the foundations of Europe’s long-standing commitment to human-centric and rights-based innovation, leaving citizens, both in Europe and beyond, as the ultimate losers.

Full Steam Ahead, But What’s The Heading?

Despite Europe’s foundational focus on trustworthy and human-centric AI, recent Commission announcements, and public statements from its leadership, suggest a radical shift away from precaution, governance, and shared responsibility on AI, to a position of acceleration and competitiveness. AI is seen as a means to bolster economic growth through a highly ambitious industrial policy. What this perspective overlooks, both in Europe and globally, is a clear “people first’” perspective: recognition that technology must serve human and societal goals, not the other way around. The “AI First’” approach glosses over this vital point. While, lip service is paid to an assessment of the benefits and risks of the technology, these are framed as checks and balances, and fail to asks, for example, if a non-AI solution may be better or safer.

This acceleration approach also is in direct contravention of the explicit instructions of the EU Parliament (2024), which called for stronger precautionary measures, transparency, and accountability in the design and deployment of digital technologies to safeguard human rights, democratic oversight, and consumer protection within the EU single market. On the other hand, the EU has recently been on the receiving end of considerable criticism from key industry actors in Europe and beyond, who claim overregulation is killing competition, supposedly leaving industry vulnerable and driving skilled professionals to Silicon Valley (Haeck 2025). At the same time, concerns about a potential generative AI bubble burst have been raised by industry leaders and governments (Makortoff 2025), sowing fear of an economic collapse. The AI First policy can thus be understood as a response to mounting pressure to increase investment, reduce regulatory constraints, and accelerate AI deployment across society. In doing so, the European Commission has effectively adopted a full-steam-ahead approach to AI, yet without the coherence, governance frameworks, and people-centric orientation necessary to ensure that such acceleration aligns with Europe’s foundational values and long-term public interests. The European Commission must also recognise that framing AI development as a global race is both misguided and counterproductive, because such a narrative reduces a complex societal transformation to a contest of speed, rather than a question of direction, purpose, and public value. Moreover, Europe will not win any AI race. The US is too dominant in the currently popular massive, centralised approaches to AI, with the EU being too dependent on the US for the tech stack that allows the most pervasive forms of AI to function. AI leadership and digital sovereignty will not come from a fragmented approach where Europeans are told to see if and where AI can be wedged into sectors and society at large. Strategic leadership, a focus on key areas of innovation, how Europe’s limited resources can be used to maximise both economic growth and social good are key. An exploratory and human rights-driven alternative is more suitable and aligned with Europe’s values and aims than trying to keep pace with the US at any cost. An AI First policy will only further fragmentation, increase inefficiencies, undermine the EU’s competitive advantage and increase its dependency on non-European actors.

This is a pivotal moment to reflect not only on how we govern AI in the EU, but why we are developing and deploying it in the first place. Too often, we see technology placed before purpose, and innovation before inclusion. So, if not AI First, what is the right question? And how can poorly resourced actors, such as SMEs, civil society, universities, small EU countries and those in the global south with limited AI literacy make this assessment?

Not AI First, But AI Where It Is The Best Solution

Rather than presuming that AI, as claimed by the Commission’s President Ursula von den Leyen, will inevitably deliver “smarter, faster, and more affordable solutions’” (von der Leyen 2025), Europe must first determine where, and whether, AI genuinely serves societal needs.

That is, we must start with Question Zero: Why AI? (Lindstrom 2025). What problem are we trying to solve? Is AI truly the right or only solution for each case where it is being applied or promoted? Who benefits, and who bears the costs? By asking these simple questions, we quickly realise that sometimes, not always, AI is the answer. This approach offers a quick, low-cost way to assess AI’s relevance, especially useful for poorly resourced actors, who are often expected to adopt AI without sufficient AI literacy, resources, or support.

Europe As An AI Leader

AI is not inevitable, nor is its current trajectory predetermined. The EU has real choices to make. As such, the EU need to focus their efforts on actively navigating the correct path forward, rather than assuming that the choices have been made for them, and the only thing they can do is try to catch up. This ability to make choices is what digital sovereignty really means. Having the ability to decide over our futures. While the Commission’s suggestion of AI First may miss the mark, the EU retains the power to define when and how AI should be used, and, vitally, when it should not. By doing so, the EU can lead not through speed, but through purpose, setting a global example of responsible innovation that strengthens independence, upholds democratic values, and turns digital sovereignty into a shared regional strength.

References

Adam Dahlgren Lindström, Virginia Dignum, Petter Ericson, Tatjana Titareva, and Jason Tucker. 2025. Responsible AI Self-assessment Workshop: Start with Question Zero. https://aipolicylab.se/2025/09/05/responsible-ai-self-assessment-workshop-start-with-question-zero/ AI Policy Lab, Published September 5, 2025. Accessed October 9, 2025.

European Commission. 2025a. Apply AI Strategy. Shaping Europe’s Digital Future. https://digital-strategy.ec.europa.eu/en/policies/apply-ai Accessed October 9, 2025.

European Commission. 2025b. Communication from the commission to the European Parliament and the Council. Apply AI Strategy (Brussels, 8.10.2025, COM(2025) 723 final). https://ec.europa.eu/newsroom/dae/redirection/document/120429

European Commission. 2025c. EU launches InvestAI initiative to mobilise €200 billion of investment in artificial intelligence. https://digital-strategy.ec.europa.eu/en/news/eu-launches-investai-initiative-mobilise-eu200-billion-investment-artificial-intelligence Last update: 12 February 2025; Accessed October 9, 2025.

European Commission. 2025d. The future of European competitiveness. Part A: A competitiveness strategy for Europe. https://commission.europa.eu/document/download/97e481fd-2dc3-412d-be4c-f152a8232961_en?filename=ThefutureofEuropeancompetitiveness_AcompetitivenessstrategyforEurope.pdf European Parliament. 2024. Addictive design of online services and consumer protection in the EU single market (P9 TA(2023)0459). Official Journal of the European Union C/2024/4164 (2024). https://eur-lex.europa.eu/eli/C/2024/4164/oj/eng Accessed October 9, 2025.

Wearden, Graeme. 2025. Entry-level workers face AI ‘job-pocalypse’; US probes Tesla’s self-driving system – as it happened. The Guardian (2025). https://www.theguardian.com/business/live/2025/oct/09/water-customers-bill-hike-winter-blackouts-risk-falls-stock-markets-pound-ftse-business-live-news Retrieved October 9, 2025.

Makortoff, Kalyeena. 2025. Bank of England warns of growing risk that AI bubble could burst. The Guardian (2025). https://www.theguardian.com/business/2025/oct/08/bank-of-england-warns-of-growing-risk-that-ai-bubble-could-burst Accessed October 2025.

Haeck, Pieter. 2025. Dutch chips giant ASML executive Roger Dassen slams EU AI overregulation. https://www.politico.eu/article/dutch-chips-giant-asml-executive-roger-dassen-slams-eu-ai-overregulation/ Accessed October 9, 2025.

Estrada, Sergio. 2025. MIT Report: 95% of Generative AI Pilots at Companies Are Failing. Fortune (2025). https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/ Retrieved October 9, 2025.

Von der Leyen, Ursula. 2025. From now on it’s “AI First” — today the European Commission launches its new approach to artificial intelligence. LinkedIn post. Available at: https://www.linkedin.com/posts/ursula-von-der-leyen_from-now-on-its-ai-first-today-the-activity-7381720419516452864-2aUp.