Policy Gap: AI and the Determinants of Public Health

Siri Helle (Psychologist, author and speaker)


Published on 30 September 2025

There is growing interest in how artificial intelligence (AI) can be applied in public health – from individual-level interventions such as diagnosis, treatment, and patient follow-up in healthcare, to broader public health applications like health data analysis or pandemic response. Ongoing debates about regulation have already led to guidelines, including those from the WHO (2021, 2024).

An important but neglected policy area concerns the secondary effects of AI technologies on public health. Here is a clear regulatory gap. Previous technological revolutions such as electricity and the internet reshaped society and lifestyles, with downstream public health consequences – such as rising sedentary behavior and cardiovascular disease. Already today, we can identify several potential risks and opportunities linked to AI development that must be addressed if we are to safeguard population health in the future.

Key determinants of health likely to be affected include work, relationships, cognition, physical activity, and psychosocial stress. Below are some examples and potential policy responses.

Work

The labor market impacts of AI remain uncertain, but some groups – such as translators and illustrators – are already reporting falling demand due to generative AI (Society of Authors, 2024). Even with opportunities for retraining, job insecurity and layoffs are often perceived as personal crises, with heightened risks of substance use disorders, depression, cardiovascular disease, and suicide (Kim & von dem Knesebeck, 2015; Zellers et al., 2025). Policymakers must be prepared from a public health perspective, for example through preventive health communication and scalable stepped-care interventions that can be expanded as needs increase.

Relationships

Strong social relationships are among the most important protective factors for health and wellbeing (World Happiness Report, 2024). Their effect on mortality risk is comparable to that of well-known risk factors such as smoking and binge drinking (Holt-Lunstad et al., 2010).

While AI services may help alleviate loneliness or coach users toward better social skills, there is also a risk that they replace human relationships due to their convenience. Researchers such as Mahari and Pataranutaporn (2024) have called for regulation in this area. One proposal is to mandate that non-humanized chatbots be the default in vulnerable settings such as health and wellness apps, to reduce the risk of users anthropomorphizing and misusing the technology (De Freitas & Cohen, 2025).

Cognition

AI tools may enhance cognition by supporting personalized learning or compensating for bias. At the same time, emerging evidence suggests they might impair higher-order functions over time. Just as books and calculators shaped cognition through “cognitive offloading,” AI tools may lead to declines in problem-solving, planning, and decision-making – especially among younger generations growing up with them. Although research is still limited, small-scale studies point in this direction (Gerlich, 2025).

Such changes could have broad societal implications, including dependence on AI, loss of critical thinking, and increased vulnerability to manipulation. They also carry direct health consequences: cognitive functioning is closely linked to outcomes such as emotion regulation, longevity, and resilience against neurological diseases like Alzheimer’s (Lövdén et al., 2020).

Sedentary Behavior

AI-driven tools for both work and leisure risk reinforcing already high levels of sedentary time by shifting more tasks to screen-based, automated, and remote interactions. Prolonged sedentary behavior is associated with higher risks of all-cause mortality, cardiovascular disease, type 2 diabetes, and certain cancers, even after adjusting for leisure-time physical activity (Biswas et al., 2015).

Psychosocial Stress

Rapid social change, including AI adoption, can heighten uncertainty, worry, and job insecurity – all well-established psychosocial stressors linked to poor health outcomes, including cardiovascular disease, mental illness, and elevated mortality (Guidi et al., 2021). Strengthening digital self-efficacy can help buffer these effects (Zhao & Wu, 2025), highlighting the need to monitor and address psychosocial consequences alongside technical and clinical AI governance.

Catastrophic Risks

Alongside gradual effects, there is also a class of extreme health risks from AI systems, including catastrophic accidents or loss of human control. Though their probability is debated, the potential scale – up to and including human survival – makes them relevant to a comprehensive public health framework. As with rare but devastating hazards like nuclear accidents or novel pandemics, AI warrants systematic assessment and planning.

POLICY RECOMMENDATIONS

To ensure AI development produces the best possible outcomes for public health, it is not enough to regulate AI applications within healthcare alone. Public health must be integrated into all AI policies, alongside other overarching sustainability perspectives such as climate, equity, and human rights. Here are three proposals:

1. Integrate public health into AI regulation

Frameworks governing AI development and deployment should explicitly include public health provisions. For example, the EU AI Act (Article 5) prohibits AI systems designed to manipulate user behavior in ways that cause significant harm to self or others. The EU Digital Services Act (Article 34) requires very large online platforms to assess and mitigate systemic risks, including those affecting public health and mental wellbeing. Digital services with underage users must be safe and free from harmful content, regardless of their size.

As with food safety standards, technologies should meet minimum health requirements. Consumers have a right not to be exposed to foreseeable risks such as disrupted sleep, distorted body image, or social dysfunction, where such harms can be anticipated and prevented.

2. Build AI capacity within public health institutions

Knowledge of AI remains limited among many public health professionals and officials. Capacity must be strengthened through education, recruitment, and expert networks so that AI-related challenges can be managed at local, regional, national, and international levels.

Global advisory bodies such as the WHO could support governments in integrating public health perspectives into national AI strategies, beyond the medical applications currently emphasized.

3. Stimulate research on AI and public health

Research on the public health effects of AI remains scarce. Neither the International AI Safety Report (2025) nor the MIT AI Risk Repository currently list health risks as a category. Most existing studies focus narrowly on healthcare applications rather than upstream determinants of health.

We need systematic investigation into emerging effects as well as foresight analyses to anticipate future impacts. By mitigating risks and promoting health benefits, AI can be developed in ways that support rather than undermine public health.

This is an initial attempt to articulate the secondary public health dimensions of AI as a societal challenge. I welcome comments, suggestions, and ideas.

References

Bengio, Y., Mindermann, S., Privitera, D., et al. (2025). International AI Safety Report (Research Series No. DSIT 2025/001). UK Department for Science, Innovation & Technology. https://assets.publishing.service.gov.uk/media/679a0c48a77d250007d313ee/International_AI_Safety_Report_2025_accessible_f.pdf

Biswas, A., Oh, P. I., Faulkner, G. E. J., et al. (2015). Sedentary time and its association with risk for disease incidence, mortality, and hospitalization in adults: A systematic review and meta?analysis. Annals of Internal Medicine, 162(2), 123–132. https://doi.org/10.7326/M14-1651

De Freitas, J., & Cohen, A. (2025). Unregulated emotional risks of AI wellness apps. Nature Machine Intelligence, 7(6), 813–815. https://doi.org/10.1038/s42256-025-01051-5

Helliwell, J. F., Layard, R., Sachs, J. D., De Neve, J.-E., Aknin, L. B., & Wang, S. (Eds.). (2024). World Happiness Report 2024. Wellbeing Research Centre, University of Oxford. (World Happiness Report)

Holt-Lunstad, J., Smith, T. B., & Layton, J. B. (2010). Social relationships and mortality risk: A meta?analytic review. PLoS Medicine, 7(7), e1000316. https://doi.org/10.1371/journal.pmed.1000316

Gerlich, M. (2025). AI tools in society: Impacts on cognitive offloading and the future of critical thinking. Societies, 15(1), 6. https://doi.org/10.3390/soc15010006

Guidi, J., Lucente, M., Sonino, N., & Fava, G. A. (2021). Allostatic load and its impact on health: A systematic review. Psychotherapy and Psychosomatics, 90(1), 11–27. https://doi.org/10.1159/000510696

Kim, T. J., & von dem Knesebeck, O. (2015). Is an insecure job better for health than having no job at all? A systematic review of studies investigating the health-related risks of both job insecurity and unemployment. BMC Public Health, 15, 985. https://doi.org/10.1186/s12889-015-2313-1

Lövdén, M., et al. (2020). Education and cognitive functioning across the life span. Psychological Science in the Public Interest, 21(1), 6–41. https://doi.org/10.1177/1529100620920576

Mahari, P., & Pataranutaporn, P. (2024, August 5). We need to prepare for ‘addictive intelligence’. MIT Technology Review. https://www.technologyreview.com/2024/08/05/1095600/we-need-to-prepare-for-addictive-intelligence/

Slattery, P., Saeri, A. K., Grundy, E. A. C., et al. (2024). The AI Risk Repository: A comprehensive meta?review, database, and taxonomy of risks from artificial intelligence. arXiv. https://arxiv.org/abs/2408.12622 (arXiv)

Society of Authors. (2024, April 11). SOA survey reveals a third of translators and quarter of illustrators losing work to AI. Society of Authors. https://www.societyofauthors.org/2024/04/11/soa-survey-reveals-a-third-of-translators-and-quarter-of-illustrators-losing-work-to-ai/

Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market for Digital Services and amending Directive 2000/31/EC (Digital Services Act). (2022, October 27). Official Journal of the European Union, L 277, 1–102. http://data.europa.eu/eli/reg/2022/2065/oj

Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). (2024, July 12). Official Journal of the European Union, L 2024/1689. http://data.europa.eu/eli/reg/2024/1689/oj

World Health Organization. (2021). Ethical guidelines for the application of AI in public health (applications such as screening, treatment, pandemic response strategies). https://www.who.int/publications/i/item/9789240029200

World Health Organization. (2024, January 18). WHO releases AI ethics and governance guidance for large multi?modal models in health care and medical research. https://www.who.int/news/item/18-01-2024-who-releases-ai-ethics-and-governance-guidance-for-large-multi-modal-models

Zellers, S., Azzi, E., Latvala, A., Kaprio, J., & Maczulskij, T. (2025). Causally-informative analyses of the effect of job displacement on all-cause and specific-cause mortality from the 1990s Finnish recession until 2020: A population registry study of private sector employees. Social Science & Medicine, 370, 117867. https://doi.org/10.1016/j.socscimed.2025.117867

Zhao, X., & Wu, Y. (2025). Artificial intelligence job substitution risks, digital self?efficacy, and mental health among employees. Journal of Occupational and Environmental Medicine, 67(5), e302–e310. https://doi.org/10.1097/JOM.0000000000003335

How to cite this article:

Helle S. (2025). Policy Gap: AI and the Determinants of Public Health. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/HNTR5780

The Ecological and Ethical Cost of Scaling AI

Irum Younis Khan (Dept. of Management Sciences, COMSATS University Islamabad (CUI))


Published on 11 August 2025

1. The Material Demands of AI

Artificial intelligence has been presented to the world as a technology driving economic and social transformation, being efficient and minimal environmental impact. Yet the reality of high energy and natural resource consumption, to keep its data centers operational, often remains invisible in mainstream narrative. This continuous energy and freshwater demand positions AI as a material actor within the planet’s ecosystem with considerable ecological costs, marking a major shift in Anthropocene, where human technologies shape planetary systems (Creutzig, et al., 2022; Wu, et al., 2022). Although the CO2 emissions from data centers have gained much attention, the water consumption remains opaque due to significant lack of independent third-party auditing and assessments (US Department of Energy, 2024). The only information source available comes from the tech giants owning these data centers. With accelerated adoptability rate of AI, expansion and scaling of Hyperscale and Edge data centers requires massive quantities of fresh, potable water – both directly and indirectly – draining the local sources, that local communities rely on, for their livelihoods. Data centers, though used for diverse digital services, are increasingly being scaled to host AI workloads. The lack of transparency and clear policies in design choices, selecting geographical locations and inequities in stakeholder inclusion emerges as a critical issue.

The increased deployment of AI-based systems results in increased demand of its infrastructure, raising urgent concerns about planetary boundaries. Water, a core utility, is a finite and unevenly distributed natural resource and already under stress in many regions (UNESCO, 2021). AI, with its environmental cost, cannot be treated an immaterial. Overlooking AI’s water consumption in sustainability assessments is no longer an option. There is a pressing need of transparency in sourcing, usage and reporting frameworks across the AI value chain.

Data centers rely heavily on fresh water with each query costing a measurable amount of water, as mentioned recently by Sam Altman, CEO of Open AI. Fresh water is used directly for ‘cooling’ and maintaining optimal operational costs (McKinsey & Company, 2024). A single hyperscale datacenter, for example, can use up to 550,000 gallons of water per day, totaling to 200 million gallons (760 million liters) annually. This is enough quantity for approx. 8,000 households (5 person) for their basic needs, based on WHO’s per capita daily water requirement (WHO, 2020).

The situation gets further complicated due to water-energy nexus. The demands of vast AI computational resources need electricity. Globally, most of electricity is produced via thermal or nuclear power plants which places additional strain on freshwater reserves. Analyzing the nexus reveals that the indirect water consumption, needed for electricity generation, can match or even exceed the amount of water needed for cooling, compounding the overall water footprint of AI. Despite this scale, companies do not share the Water Usage Effectiveness (WUE) report consistently and transparently (IEA, 2024). The claims of replenishment of water consumption such as ‘water-positive’ pledges remain vague, often lacking critical information about when, where and how freshwater is drawn and what is actual ecological benefit gained (Microsoft, 2025). Such data is vital in understanding the offset occurring outside the watershed, where extraction happens and therefore fail to provide meaningful accountability.

2. Water-Stressed Geographies and Data Center Boom

The freshwater scarcity has affected billions of people in recent years. Yet the growing mismatch between data center geographical placement and water availability reflects a negligence towards this escalating crisis. A closer look at spatial convergence of water risk locations and number of data centers they are hosting, reveals the overlooked construction of digital futures on fragile water foundations (World Resources Institute, 2023; Data Center Map, 2024). Countries like Belgium, Spain, Chile and India are hosting large number of data centers despite high or extremely-high baseline (2023) and projected (2030) water stress level, as reported by Aqueduct Water Risk Atlas. India, for instance, hosts 265 datacenters while facing extreme water stress while major tech firms like Google, plan to set additional hyperscale data centers in the country. Spain hosts 161 data centers with presence of Microsoft, Google, Meta and Amazon Web Services and Belgium has 48 data centers supporting operations of significant and long-standing hyperscale data centers such as Google.

Between depletion of water resources and economic benefits, countries show a paradox between digital infrastructure growth and water scarcity. The local ecosystems and communities increasingly compete for resource availability, without any mechanism to govern or mitigate this competition (Lehuedé, 2024; Vinuesa, et al., 2020). Meanwhile the developing countries position themselves in AI revolution by actively welcoming foreign investment brought by data centers as part of their digital transformation goals. Nations like South Africa, Egypt, Angola, and Pakistan, with ongoing water scarcity challenges, seek investments in data centers by offering land and rebates. They seemingly ignore any consideration of ecological trade-off that comes along with economic prosperity. Egypt, for example, faces extremely-high projected water-stress and when combined with its rising number of AI data centers, it may have a short-term economic uplift but also deepen long-term water scarcity.

3. AI Growth and Ecological Fragility

With scaling of global AI models and increase in localization, demand for data centers will intensify. This expansions risks triggering a rebound effect (Hertwich, 2005). The efficiency gains in AI models will lead to increase in overall resource consumption and a need for more data centers. With AI capabilities getting more distributed and embedded in daily systems, the consumption of energy and water will be accelerating too. So, despite any improvements in AI model efficiency or cooling innovations, the need of accountability and transparency in its environmental impact cannot be denied. To keep up with the demand, the data center operators pursue locations with low operational cost and greater water accessibility. Such sites are often situated in resource-constrained regions. Meanwhile developing countries are ready to avail this opportunity for digital transformation and foreign investment. However, any international standard, guiding the integration of water risk in AI infrastructure policy is still missing, leaving the discussions to fragmented national strategy or at public private discretion.

The absence of regulatory safeguards, along with its environmental impacts, may intensify conflicts between data center operators, agricultural users, diverse industrial consumers and local populations who share freshwater sources. Such conflict may lead to disrupted food systems, local protests and inequitable water allocations, and in some cases, the corporate interests overriding public welfare. Such politicization may further destabilize already vulnerable communities.

The expansion of AI calls for accountability of its environmental footprint in sustainability and governance debates requiring independent audits, inclusive planning and enforceable global standards. In Anthropocene, responsible AI must align with planetary limits and social equity or otherwise risk heavy ecological strain and systematic injustice.

References

Creutzig, F., Acemoglu, D., Bai, X., Edwards, P., Hintz, M., Kaack, L., . . . Rejeski, D. (2022). Digitalization and the Anthropocene. Annual review of environment and resources, 47(1), 479-509.

Data Center Map. (2024). Search data centers by country. Retrieved from Data Center Map: https://www.datacentermap.com/

Hertwich, E. G. (2005). Consumption and the rebound effect: An industrial ecology perspective. Journal of industrial ecology, 9(1?2), 85-98.

IEA. (2024). Data Centres and Data Transmission Networks. Retrieved from International Energy Agency (IEA): https://www.iea.org/energy-system/buildings/data-centres-and-data-transmission-networks

Lehuedé, S. (2024). An elemental ethics for artificial intelligence: water as resistance within AI’s value chain. AI & SOCIETY, 1-14.

McKinsey & Company. (2024, September 17). McKinsey.com. Retrieved from How data centers and the energy sector can sate AI’s hunger for power: https://www.mckinsey.com/industries/private-capital/our-insights/how-data-centers-and-the-energy-sector-can-sate-ais-hunger-for-power

Microsoft. (2025). Microsoft.com. Retrieved from Environmental Sustainability Report 2025: https://www.microsoft.com/en-us/corporate-responsibility/sustainability/report/

US Department of Energy. (2024). Recommendations on Powering Artificial Intelligence and Data Center Infrastructure. Retrieved from US Department of Energy: https://www.energy.gov/sites/default/files/2024-08/PoweringAIandDataCenterInfrastructureRecommendationsJuly2024.pdf

Vinuesa, R., Azizpour, H., Leite, I., Balaam, M., Dignum, V., Domisch, S., . . . Fuso Nerini, F. (2020). The role of artificial intelligence in achieving the Sustainable Development Goals. Nature communications, 11(1), 233.

WHO. (2020). Domestic water quantity, service level and health (2nd Ed.). Retrieved from World Health Organization: https://iris.who.int/handle/10665/338044

World Resources Institute. (2023). World Resources Institute. Retrieved from Aqueduct Water Risk Atlas: https://www.wri.org/applications/aqueduct/water-risk-atlas

Wu, C., Raghavendra, R., Gupta, U., Acun, B., Ardalani, N., Maeng, K., . . . Gschwind, M. (2022). Sustainable ai: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems, 4, pp. 795-813.

Keywords (comma separated):

AI governance, water-energy nexus, Anthropocene, environmental sustainability, ecological costs of AI, data center impact

How to cite this article:

Khan Y. (2025). The Ecological and Ethical Cost of Scaling AI. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/LGGN1494

Tracing labour, power, and information in Artificial Intelligence Systems

Petter Ericson (AI Policy Lab, Department of Computing Science, Umeå University)


Published on 24 June 2025

1. Introduction

It is common for technology to be used to obscure the role of humans, and artificial intelligence (AI)is a field where this is even more true than for many others. From ghost workers and data scraping to algorithmic management and automated decision making, AI technologies are used to displace, appropriate, and hide human labour in various ways. Decisions are hidden inside technical systems, externalising them from individuals and organisations who could be held meaningfully accountable, which can make agency and power flow in new and often poorly understood ways. By having automated systems imitate humans, teleworkers can be seamlessly swapped in and out as needed with users none the wiser, making the systems appear significantly more capable than they actually are.

A useful abstraction for studying and exposing the workings of these systems is to consider how and where information is flowing through them. However, tools for such analyses tend to either be highly abstracted and focused on the broader sociotechnical systems where AI components are situated, or highly technical and focused on the details of specific software and hardware architectures, or on idealized and abstract models thereof. The proposed work will attempt to bridge this gap. On the one hand, it will take a more rigorous approach grounded in information theory to qualify and quantify the information that the humans involved share both through some specific technical system and through outside channels. On the other hand, it will take a wider view on the concrete workings of those technical systems, incorporating sociotechnical metadata into analyses of digital information flows.

In sum, we aim to use tools and methods from information theory and sociotechnical system modelling, together with formal graph models and complexity theory, to investigate and explain how diverse human labour and knowledge is discretized, datafied, and expressed throughout the development and deployment of different types and architectures of AI systems.

A complementary aim of this work is to build on and further develop existing research connecting information, computation, labour, and value, and how these concepts interact, specifically in the context of AI systems, making concrete contributions to interdisciplinary studies on AI and data work. A second major aim is to yield new insights into how to quantify and qualify information flows, through a focused study of sociotechnical systems involving AI components, where information and its flows in the digital realm can be directly studied, and comparisons can be made to both models and empirical realities of the social realm surrounding them.

Ultimately, we aim to investigate the following research questions:

* RQ1 What types of information flows can be identified within different AI system architectures, and how can these be formally categorized?
* RQ2 How do human actors contribute to and engage with information flows in and around AI systems, and how can these social interactions be systematically modeled?
* RQ3 How can we develop and validate models of information flows in AI-based sociotechnical systems that integrate both technical and human components? * RQ4 How do modeled information flows reflect or reinforce particular organizational or institutional power structures?

2. Related work

Though to the best of the authors knowledge there is very little research on precisely the present topic, its interdisciplinary nature means that there are a number of intersecting areas of active research. In particular, works that cover the intersection between AI and information theory, between information theory and labour, between labour and AI, and between any of these three areas and sociotechnical modelling are all relevant.

For the first intersection, Jeon and Roy have recently investigated the connections between Bayesian machine learning and Shannon information theory, drawing an equivalence between the cumulative errors during a learning process of an optimal machine learning algorithm, and the amount of information contained in the data. From a different angle, several works such as Tseng et al. [2] and [3] have looked specifically at large language models (LLMs), drawing on compression and entropy calculations to study the use, and training, respectively of LLMs as related to natural language texts.

For the second, Dantas [4] has drawn direct connections between information and both labor and value in an explicitly Marxist framework, distinguishing between not only use and exchange value, but also semiotic value, and deriving a specific notion of information work which will be of direct use in the proposed work. Dantas also draws a distinction between random and redundant information work, which is similar to the distinction between semantic and syntactic information work in [5] which is further nuanced into an explicit spectrum in [6].

The third intersection is itself a broad area, with many different aspects of relevance. Nguyen and Mateescu [7] gives a good overview of the current landscape in relation to Generative AI specifically, while Davis [8] provides a broader review of relevant issues, making an explicit (and useful) distinction between cases where AI use impacts labour demand (through automation) and those relating more to worker power (through surveillance, algorithmic management, and the like). Further, works such as Crawford [9], Miceli and Posada [10], Gray and Suri [11], Merchant [12], Sadowski [13], and Mejias and Couldry [14] are all relevant for further developing this work. The Data Workers Inquiry (https://data-workers.org) will be another important source of alternate perspectives on AI and labour.

In terms of studying AI sociotechnical systems, once again several active areas are of interest. In particular Wu et al. [15] has developed a framework for integrating various types of models of sociotechnical systems (STS) into a single meta-model. Several modelling languages for sociotechnical systems exist, including STS-ml [16], which was developed for cybersecurity applications, and the host of standards and notations related to Business Process Model and Notation (BPMN), such as Decision Model and Notation (DMN), which is particularly relevant for models integrating AI decision support systems. However, all of these abstractions and models tend to integrate assumptions that are not always helpful for the purposes of this work. A relevant example of how existing modelling framework scan be extended to cover new areas is [17], which adds properties and functionality to STS-ml in order to check sociotechnical systems for compliance with the EU General Data Protection Regulation (GDPR).

A relevant parallel effort, though not directly related to the work we propose here, is that of Gutierrez Lopez and Halford [18], who aim towards an extension of XAI principles that including the sociotechnical environment of the machine learning system.

3. Aims

The main contribution of this work will be to integrate previous work on sociotechnical systems modelling with several notions of information and labour, specifically in the context of artificial intelligence. An additional benefit of this work will be to lay a basis for a further analysis of agency and accountability: By studying the information flows and potential inputs and decisions from humans involved in an AI sociotechnical system, together with an analysis of the power relations among them, accountability and responsibility can be transparently and meaningfully assigned.

We hope to make meaningful contributions to the practical use of information theory and information flow, as well as yield actionable and concrete directions for further exploration of new AI sociotechnical architectures. As part of this, a major component will consist of qualitatively and quantitively analysing the information flows into and out of AI and ML systems, which will also give new and useful insights into the design of Hybrid AI systems in particular.

By creating concrete tools and methods for tracing information flows through both technical and social layers of AI systems, this work will attempt to offer not just theoretical insight, but practical value for those developing, regulating, or critically analyzing such systems. In a time when the societal consequences of AI are increasingly opaque yet consequential, this research will provide actionable models that can inform transparency standards, system audits, and future AI governance efforts.

4. Preliminary results

4.1. Categories of information

A foundational topic of this work includes clarifying and classifying different types of information. In particular, though at extreme small ranges reality can occasionally appear to be digital, for most practical purposes, it is continuous. In contrast, digital information, and computer and AI systems, while implemented on physical hardware, are conceptually and practically discrete. As such, while abstractions of the human and physical sections and relations of a sociotechnical system are going to inevitably be lossy, for the digital parts it is in principle possible to be both precise and concrete. This then, must be our first distinction between fundamentally different types of information: Abstracted notions and models of the real world, and concrete digital bits and bytes.

In terms of different theoretical notions of information, we further contrast the more mathematical definitions of Shannon [19] (’minimal code’) and Kolmogorov [20] (’minimal program’) with the Batesonian concept of ’a difference which makes a difference’ [21]. A fourth relevant concept is Corning’s’ control information’[22], which rather than connecting Shannon entropy/negentropy directly to the physical thermodynamic concepts with the same name, instead quantifies the amount of information contained in some signal or phenomenon by the amount of physical changes that it can effect. An example taken directly from Corning [22] is that of a car approaching a stoplight. If the driver does not notice or understand the traffic light, there is no control information being transferred by whatever light is shown. Only if the driver both sees the light, understands it, and is prepared to change the future trajectory of the car, is there any control information being sent out by the light switching to red. Broadly, we can thus consider two very different types of information flows: The almost entirely discrete and abstract digital information exchanges between and inside of software components, and the messy, socially situated, and necessarily contingent and abstracted information flows that can be modelled to exist between humans, technological artefacts, and their surrounding physical context. The main interest of this work lies precisely where these information flows intersect and interact.

4.2. Human-computer information interactions

With a minimal distinction of information flows as above, consider the interactions between a human and a computer system: the transfer of information from human to computer will necessarily abstract some concrete intention of the human into a concrete digital signal, but likewise a (digital) computer output will take on a specific meaning to the human which depends on their prior knowledge and the context in which the output is given. We can depict these shifts as in figure 1.

4.3. Analysis example

As an illustrative example of the type of analysis that we aim to make more concrete, detailed, and empirically grounded, consider the case of an article being written about a sports event. It is, at this point, plausible that such an article could be written by a large language model (LLM) given an appropriate prompt, including some sort of summary of the ’relevant facts’ of the event in question (e.g. the final tally of points, who made them when, and any injuries and other specific incidents, which are accessible from some sort of API). The situation would look something like figure 2. We can complicate this picture, however, by adding more context. The article will not reach publication without an editor, and the hidden labour that has resulted in the LLM is entirely absent in our initial figure, as is the work to set up the “sports API” and the later work to feed it with the ’relevant facts’ from observations of the event. A more realistic picture emerges, as in figure 3.

Compare this to a situation where a human writer is the author of the same article. Though the plain (abstract) facts of the event in question may be the same, the human will also have access to an infinitely larger context as part of their writing process, both through direct experience and memory, and through communications with other humans, computer systems, and physical objects such as books and recordings, nevermind the sights, sounds, and smells of the event itself if the writer was also present at the event. In this case, the situation will look more like figure 4. This too can be made more complex, particularly if we imagine the writer to make use of an LLM for writing assistance of some sort, yielding a situation as in figure 5.

5. The path forward

Though primarily based in computing science, the nature of the problems addressed by the work call for an interdisciplinary approach. Notably, by building on existing work in Science and Technology Studies, as well critical marxist literature, it is possible to better situate and analyse the information flows and AI sociotechnical systems inside existing societal power structures and socioeconomic realities. In addition to the various theories of information mentioned in Section 4.1, we will also distinguish between different types of information work, as outlined in Section 2. The distinctions between data, information, and knowledge have been explored e.g. in [23], [24], and these perspectives will also be considered.

We will primarily be building on existing frameworks for the analysis of program structure and information flow through software. Notably, the theory and practice of Quantitative Information Flow (QIF) analysis in computer security, though focused on detecting and plugging information leaks between public and private variables under static source code analysis conditions, offers a range of useful tools for modelling intentional information flows as well. From a software engineering lens, constructing program flow graphs and clearly and consistently delineating components in a software system is a well established practice, with a host of frameworks available for use. An example of an abstract framework for describing program and information flows that has been specifically developed for purposes of describing Hybrid AI systems is the boxology of Harmelen and Teije [25].

For situating a software system in an organisational context, tools from business modelling are available as well, with well-established frameworks such as BPMN, STS-ml and various derivatives having seen extensive use to analyse information flows and decision processes in business contexts. Concretely, the near parts of this work will consist a phase of conceptual and theoretical grounding, studying and comparing existing frameworks for information flow analysis, to arrive at a rigorous and flexible framework for modelling information flows in sociotechnical systems, incorporating the above distinctions and specificities, and giving specific attention to questions of agency and valuation. This work will aim at identifying connections and distinctions in how different frameworks frame decisions, and how labour is considered within them.

In the course of this development, a metadata schema for information and information flows will be developed that can describe and categorise information both in terms of its qualities, its different information contents, as well as its role at a specific point in a described sociotechnical process. Tracing the changes of these properties as attached to a particular piece of information will be an important complement to the analyses of the flows themselves, and of the various transformations imposed on and driven by the information.

During and after these developments, the framework and schema will be empirically applied to real-world cases, both existing ones from the literature, and new and comparable studies of previously understudied sociotechnical contexts. Modelling these flows will be accomplished through direct study of technical artefacts and their documentation, as well as organisational policies and descriptions of their surrounding sociotechnical contexts. These will be supplemented by interviews and surveys of involved stakeholders to elicit new and undocumented perspectives not previously represented even in internal documents.

Through comparative analysis across multiple cases (to be selected to reflect diversity in AI architecture and deployment and across different domains, e.g. public-sector automation, language models, decision-support tools) the framework will be further refined to capture how different system configurations mediate flows of information, labour, and power across different AI configurations. Ultimately, we aim for a formal, extensible modelling framework for analyzing information flows in sociotechnical systems involving AI, as well as a richly annotated library of concrete case studies. Additionally, we aim to make both conceptual and methodological contributions to the study of accountability, power, and labour in AI, as well as help drive further developments in related fields.

References

[1] H. J. Jeon, B. V. Roy, Information-Theoretic Foundations for Machine Learning, 2024. URL: http://arxiv.org/abs/2407.12288. doi:10.48550/arXiv.2407.12288, arXiv:2407.12288 [stat].

[2] Y.-H. Tseng, P.-E. Chen, D.-C. Lian, S.-K. Hsieh, The semantic relations in LLMs: An informationtheoretic compression approach, in: T. Dong, E. Hinrichs, Z. Han, K. Liu, Y. Song, Y. Cao,C. F. Hempelmann, R. Sifa (Eds.), Proceedings of the Workshop: Bridging Neurons and Symbolsfor Natural Language Processing and Knowledge Graphs Reasoning (NeusymBridge) @ LRECCOLING-2024, ELRA and ICCL, Torino, Italia, 2024, pp. 8–21. URL: https://aclanthology.org/2024.neusymbridge-1.2/.

[3] M. Yin, C. Wu, Y. Wang, H. Wang, W. Guo, Y. Wang, Y. Liu, R. Tang, D. Lian, E. Chen, EntropyLaw: The Story Behind Data Compression and LLM Performance, 2024. URL: http://arxiv.org/abs/2407.06645. doi:10.48550/arXiv.2407.06645, arXiv:2407.06645 [cs].

[4] M. Dantas, Information as Work and as Value, tripleC: Communication, Capitalism & Critique. Open Access Journal for a Global Sustainable Information Society 15 (2017) 816–847. URL: https://www.triple-c.at/index.php/tripleC/article/view/885. doi:10.31269/triplec.v15i2.885.

[5] J. Warner, Labor in information systems, Annual Review of Information Science and Technology39 (2005) 551–573. URL: https://asistdl.onlinelibrary.wiley.com/doi/10.1002/aris.1440390120. doi:10.1002/aris.1440390120.

[6] J. Warner, The spectrum of semantic and syntactic labour, Journal of Documentation 80 (2024)649–664. URL: https://www.emerald.com/insight/content/doi/10.1108/JD-03-2023-0057/full/html.doi:10.1108/JD-03-2023-0057.

[7] A. Nguyen, A. Mateescu, Generative AI and Labor: Power, Hype, and Value at Work, Technical Report, Data & Society Research Institute, 2024. URL: https://datasociety.net/library/generative-ai-and-labor. doi:10.69985/gksj7804.

[8] O. F. Davis, Artificial Intelligence and Worker Power (2024).

[9] K. Crawford, Atlas of AI: power, politics, and the planetary costs of artificial intelligence, Yale University Press, New Haven London, 2021.

[10] M. Miceli, J. Posada, The Data-Production Dispositif, Proceedings of the ACM on Human-Computer Interaction 6 (2022) 1–37. Publisher: ACM New York, NY, USA.

[11] M. L. Gray, S. Suri, Ghost work: how to stop Silicon Valley from building a new global underclass, Houghton Mifflin Harcourt, Boston, 2019.

[12] B. Merchant, Blood in the machine: the origins of the rebellion against big tech, first edition ed., Little, Brown and Company, New York, 2023. OCLC: on1389775757.

[13] J. Sadowski, The mechanic and the luddite: a ruthless criticism of technology and capitalism, University of California Press, Oakland, California, 2025. doi:10.1525/9780520398085.

[14] U. A. Mejias, N. Couldry, Data grab: the new colonialism of big tech and how to fight back, WHAllen, London, 2024.

[15] P. P.-Y. Wu, C. Fookes, J. Pitchforth, K. Mengersen, A framework for model integration and holistic modelling of socio-technical systems, Decision Support Systems 71 (2015) 14–27. URL: https://www.sciencedirect.com/science/article/pii/S016792361500007X. doi:10.1016/j.dss.2015.01.006.

[16] E. Paja, F. Dalpiaz, P. Giorgini, Modelling and reasoning about security requirements in socio-technical systems, Data & Knowledge Engineering 98 (2015) 123–143. URL: https://www.sciencedirect.com/science/article/pii/S0169023X1500052X. doi:10.1016/j.datak.2015.07.007.

[17] C. Negri-Ribalta, R. Noel, N. Herbaut, O. Pastor, C. Salinesi, Socio-Technical Modelling for GDPR Principles: an Extension for the STS-ml, in: 2022 IEEE 30th International Requirements Engineering Conference Workshops (REW), 2022, pp. 238–243. URL: https://ieeexplore.ieee.org/document/9920163/?arnumber=9920163. doi:10.1109/REW56159.2022.00052, iSSN: 2770-6834.

[18] M. Gutierrez Lopez, S. Halford, Explaining machine learning practice: findings from an engaged science and technology studies project, Information, Communication & Society 28 (2025) 616–632. URL:

https://www.tandfonline.com/doi/full/10.1080/1369118X.2024.2400130. doi:10.1080/1369118X.2024.2400130.

[19] C. E. Shannon, A mathematical theory of communication, The Bell System Technical Journal 27(1948) 379–423. URL: https://ieeexplore.ieee.org/document/6773024. doi:10.1002/j.1538-7305.1948.tb01338.x, conference Name: The Bell System Technical Journal.

[20] A. N. Kolmogorov, On Tables of Random Numbers, Sankhy?: The Indian Journal of Statistics, Series A (1961-2002) 25 (1963) 369–376. URL: http://www.jstor.org/stable/25049284, publisher: Springer.

[21] G. Bateson, Form, substance and difference, Essential readings in biosemiotics 501 (1970). Publisher: Springer.

[22] P. A. Corning, Control information theory: the ‘missing link’ in the science of cybernetics, Systems Research and Behavioral Science 24 (2007) 297–311. URL:https://onlinelibrary.wiley.com/doi/abs/10.1002/sres.808. doi:10.1002/sres.808, _eprint:https://onlinelibrary.wiley.com/doi/pdf/10.1002/sres.808.

[23] L. Businska, I. Supulniece, M. Kirikova, On Data, Information, and Knowledge Representation in Business Process Models, in: R. Pooley, J. Coady, C. Schneider, H. Linger, C. Barry, M. Lang (Eds.),Information Systems Development, Springer, New York, NY, 2013, pp. 613–627. doi:10.1007/978-1-4614-4951-5_49.

[24] L. Businska, I. Supulniece, Towards Systematic Reflection of Data, Information, and Knowledge, Scientific Journal of Riga Technical University. Computer Sciences 43 (2011). URL: https://content.sciendo.com/doi/10.2478/v10143-011-0002-9. doi:10.2478/v10143-011-0002-9.

[25] F. v. Harmelen, A. t. Teije, A Boxology of Design Patterns for Hybrid Learning and Reasoning Systems, Journal of Web Engineering 18 (2019) 97–124. URL: http://arxiv.org/abs/1905.12389.doi:10.13052/jwe1540-9589.18133, arXiv:1905.12389 [cs].

Keywords (comma separated):

information theory, information flow, socio-technical system modelling

Related URL (if any):

https://people.cs.umu.se/~pettter/tracing_information_figures.pdf

How to cite this article:

Ericson P. (2025). Tracing labour, power, and information in Artificial Intelligence Systems. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/AUHD8541

Time Out of Joint: Historical reflections on AI

Somya Joshi (Stockholm Environment Institute) & Remi Paccou (Schneider Electric)


Published on 27 May 2025

Artificial Intelligence. The very term conjures images of futuristic robots and sentient machines for some, and images of climatic collapse and existential risk to others. This AI hype represents a disjoint in time with both risks and promises. It signals a paradigm shift marked by unprecedented capabilities in information processing, autonomous reasoning, and pattern recognition, challenging traditional notions of progress and sustainability while demanding a nuanced approach to harness its potential responsibly & ethically.

The Three Technological Paradigms: From water wheels to apps:

Human technological evolution can be understood through three major paradigms. The first focused on the transformation of materials, spanning from the Stone Age through the Bronze and Iron Ages(1), where humans developed increasingly sophisticated ways to manipulate their physical environment. The second paradigm, also known as the Industrial Revolution, centered on the transformation of energy. The first industrial revolution (1770–1850), as identified by Schumpeter(2), was driven by water-powered mechanization, including mills and irrigation systems. The following long wave (1850–1900) was enabled by steam-powered technology, revolutionizing transportation with trains and transforming industrial machinery. Around 1900, the Third Kondratieff Cycle began(3), marked by the electrification of society and production from 1900 to 1940. Each revolution introduced new tools, industries, and fundamentally impacted lifestyles.

Today, we stand at the cusp of an era defined by the transformation of information. Late 20th-century digital electronics fueled ICT digitalization, leading to AI disruption. But what does this disjoint in time truly entail? History reveals three fundamental mechanisms that have been central to major technological transitions: transmission, storage, and processing. These mechanisms have propelled every major technological shift: from the wheel and rope of transport to smoke signals and the internet for transmission; from containers and reservoirs to photography and magnetic media for storage; and from fire-making to electronic computation for processing(4).

In 1990, less than 0.05% of the global population used the internet. By 2020, over 59% of humanity was connected (10). Networks now move exabytes monthly, enabling unprecedented global information flow. Storage has mirrored this progression: from physical media like books, we’ve advanced to digital systems that store humanity’s collective knowledge on infinitesimal footprints—a leap from 1% digital in the late 1980s to 99% by 2012. AI compute has completed the picture, with processing power showcasing the most striking leap forward. Today’s supercomputers operate at exaflop speeds, solving in seconds problems that would take humans decades. These leaps in transmission, storage, and processing power form the bedrock of AI, enabled by infrastructure that facilitate information transmission, storage, and processing at unprecedented levels. However, these accelerations come at a cost – both to human societies and the planet.

Continuity and Discontinuity in AI Development

Unlike past technologies that built upon human abilities, AI promises autonomous reasoning, planning, and pattern detection beyond human limits. This shift, especially with the rise of agentic AI systems, challenges traditional augmentation concepts, introducing self-referential mechanisms that redefine intelligence, creativity, and technological agency.

This transformation can be framed through the concept of autopoiesis, where technological systems evolve to create themselves, or sympoiesis, where AI is built upon human knowledge to enable novel futures(6). These theoretical lenses help us understand not only the abstract nature of AI’s development but also its tangible manifestations in the evolution of computational hardware. While computational hardware has experienced profound changes, marked by incremental efficiency gains and increased capabilities, the nature of AI’s advancements, particularly its generative capacity, introduces a new dimension. AI’s generative capacity, as it currently stands, challenges human cognitive boundaries and increases technological opacity, introducing a fundamental break from previous technological trajectories. It is not merely an extension of human capabilities but a transformative force capable of generating insights and futures untethered from human precedent.

Untethered from Planetary Health: Rebound Effects and Sustainability Challenges

Current research warns of potential “rebound effects,” where gains in efficiency paradoxically lead to higher overall consumption—an abundance without limits that could undermine sustainability goals by constraining decarbonization efforts or generating waste through unrestricted growth in AI development(7). Addressing this requires policy interventions and investments in sustainable infrastructure prioritizing accuracy, frugality, proven impact assessments for electricity demand growth—and circular economy practices for both hardware and software. To align AI development with planetary (and by virtue of that human) resilience, guardrails need to be designed within the architecture of AI technologies, at the very heart instead of as an afterthought. This would entail a shift away from a focus on efficiency and optimisation alone, towards a more integrated perspective that considers the entire value chain of AI(8). Furthermore, the environmental impact of AI, including the energy & water consumption of large language models and resource depletion from hardware production, must be addressed via caps and transparent open architecture for data sharing.

From Extraction to Global Common: Resetting AI Development

Another critical discontinuity stems from historical notions—dating back to early industrial revolutions—that “human progress” exists outside of nature, which then reduces our environment to a resource for extraction. Today’s dominant discourse around scaling larger AI models risks perpetuating this extractive mindset despite rising environmental costs like energy & water crises caused by mismatched demand on infrastructure or resource depletion.

We call for a “Global Commons” approach (drawing on the seminal work of Elinor Ostrom(9), which would mean sharing benefits across borders while challenging protectionist development paradigms through sustainable practices. This includes optimizing software, improving models, evaluating environmental impacts, and promoting circular economies. We must also in parallel build global governance, set AI standards, and boost digital literacy through international collaboration. The fundamental question remains: when not to use AI? In other words, we must dare to imagine futures with and without AI, rather than accept it as a fait accompli.

To responsibly leverage AI, we must center its design and direction towards nature aligned principles, address potential risks and harms head on, and foster global collaboration in the face of an increasingly polarizing world. Sustainable strategies require long-term vision, while short-term profits shackle us to false promises of shared progress, which history reveals to be mere mirages.

References

  1. Roos, R. A. (2019). The Stone Age, Bronze Age and  Iron Age Revisited. HISTO THOMSEN V2-. J. Terrestrial Electrostatics.
  2. Schumpeter, J. A. (1939). Business Cycles: A Theoretical, Historical, and Statistical Analysis of the Capitalist Process. McGraw-Hill.
  3. Korotayev, A. V., & Tsirel, S. V. (2010). A Spectral Analysis of World GDP Dynamics: Kondratieff Waves, Kuznets Swings, Juglar and Kitchin Cycles in Global Economic Development, and the 2008–2009 Economic Crisis. Structure and Dynamics, 4.
  4. Arthur, W. B. (2009). The nature of technology: What it is and how it evolves. Simon and Schuster.
  5. Maturana, H. R., & Varela, F. J. (1980). Autopoiesis and cognition: The realization of the living. D. Reidel Publishing Company.
  6. Haraway, D. J. (2016). Staying with the trouble: Making kin in the Chthulucene. Duke University Press.
  7. Paccou, R., & Wijnoven, F. (2024). Artificial intelligence and electricity: A system dynamics approach. ResearchGate.
  8. Crawford, K. (2021). Atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University 1 Press
  9. Ostrom, E., Burger, J., Field, C. B., Norgaard, R. B., & Policansky, D. (1999). Revisiting the commons: local lessons, global challenges. science, 284(5412), 278-282.
  10. “Data Page: Share of the population using the Internet”, part of the following publication: Hannah Ritchie, Edouard Mathieu, Max Roser, and Esteban Ortiz-Ospina (2023) – “Internet”. Data adapted from International Telecommunication Union (via World Bank). Retrieved from https://ourworldindata.org/grapher/share-of-individuals-using-the-internet [online resource]

Keywords:
Artificial Intelligence, Sustainability, Geopolitics, Environment, Automation, Equity

How to cite this article:

Joshi S. (2025). Time Out of Joint: Historical reflections on AI. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/DNPK4001

Will university teachers become obsolete in times of AI?

Elin Kvist (Department of Sociology, Umeå University)


Published on 22 May 2025

We live in times of endless crisis alarms, a general state of uncertainty, combined with an orchestrated and intentional undermining of established academic institutions. Science and academic research are under questioning around the world, and universities’ position as a legitimate source of knowledge and critical thinking are under attack. Adding to this sense of instability and uncertainty, there is an ongoing digital and technological development that some argue has become a major risk for technical unemployment – fortelling the “end of work” (Brynjolfsson & McAfee, 2014; Danaher, 2017).

Automation, history and current threats

Historically, we have seen how new technology has changed the way we work and our everyday work practices. Machines have revolutionised and increased efficiency in agriculture and industrial production, and reduced the number of workers demanded in the process. In light of today’s ever-growing improvements in computer power, artificial intelligence and robotics, the gloomiest predictors are now once again convinced that we are moving towards a jobless future (Brynjolfsson & Mcafee, 2014). This time, technology is substituting more cognitively advanced and emotionally demanding jobs – ones previously performed by professionals in technical and managerial professions, including university teachers (Autor, 2015).

However, the past two centuries of automatisation and technological change have not made human labour obsolete. Even though unemployment rates have fluctuated cyclically, there has been no long-run increase in unemployment (Autor, 2015). Through governmental programs of re-education and reorientation, most exempted workers have been able to move to other forms of labour, and new areas of work have opened up in the wake of technological transformation. In previous technological automation processes the focus have been on replacing the most dangerous, bodily harming, and repetitive tasks, and in doing so contributing to a more human friendly labour market.

The digital transformation of academic work

University teachers have seen their work tasks and everyday work practices changed dramatically with digitalisation. Through digital aids and tools, they book rooms, coordinate and organise lectures and seminars, examine and grade students digitally, do research, apply for research funding and ethical approval, publish in academic outlets, manage conference bookings, calculate budgets for research proposals and develop data management plans. These are just a few examples. Their everyday work practices include a significant amount of digital administration, interacting with large numbers of different digital platforms. In the name of efficiency, an increasing number of tasks that have been previously assigned to administrative employees, have gradually been reassigned to the university teachers.

However, when trying to understand the consequences of technological changes of the everyday work situation for university employees, it is important to also take into consideration that during the same time these organisations have also been subject to New Public Management (Thomas & Davies, 2002). Which has also entailed increases in the number of students with diverse needs, less preparation time for teaching, and continuous monitoring of performance through audits and performance evaluations. In result, this left each university teacher with less time to do research due to increased demands and shrinking resources. Consequently, this has spurred even more administrative work as researchers constantly need to apply for research grants in highly competitive, complex and time-consuming funding processes, resulting in additional time and resource-consuming processes. This is also important to take into consideration when trying to understand the implications of genAI on the future of university teachers, illustrating the importance of moving beyond a techno-deterministic understanding (Lindberg et al., 2022). Automation and digitalisation are often presented as neutral, a consequence of technological progress, and as something inevitable. The ideological and material consequences remain hidden (Lindgren, 2024).

AI’s role and data dependency

We have to understand what distinguishes AI technology from previous types of technology, and in doing so, understand what consequences it will have on the everyday practices of university teachers. First, how can they use AI in their everyday work? What tasks do they have that could be suitable for genAI tools? Tasks such as compiling large amounts of text or getting an overview of a new research field for teaching or research, writting summaries, and polishing research applications, compiling CVs and creating concise bios, conducting  text analysis of ethnographic materials, supporting peer-review and expert processes, when assessing exams and essays, supporting development of lectures, conference and seminar presentations. The possibilities are endless. In the modern universities that the New Public Management have constructed, with its endless rounds of evaluations, constant applications and assessments, genAI tools can function to support and facilitate the everyday administrative work, making the work practices more manageable. However, it is also important to keep in mind that AI needs large amounts of data to be able to learn from the environment in which it will operate. To be able to help the teachers in their professional practices, genAI needs access to information and data, and the employees must assist and train the systems. Algorithmic systems depend on humans performing a certain kind of digital work, data labeling and moderation, breaking down the work into smaller components for autonomous decisions (Lindgren, 2024). This work is not always visible or even seen as actual work (Moore & Woodcock, 2021).

Hidden labour and digital capitalism

In digital capitalism, we all are involved in generating this type of data. When we move around in digital environments, we perform a lot of work for free that contributes to generating profits for the system, often without us being aware of it. As users we contribute to training the AI systems. Those who are involved in creating content online leave behind data traces, and it is these traces that the large digital media giants (Meta, X, etc.) exploit and capitalise on. The work that people do in the borderland between AI and society is often hidden (Lindgren, 2024; Moore & Woodcock, 2021; Taylor, 2018). For example, when you order a pizza for home delivery via an app, you might perceive it as a digital process. However, the actual physical work behind is invisible. Someone is standing and making the pizza. Another person is delivering it to your home. Foodora and Deliveroo’s apps are part of the complex socio-technical ecosystem of digital society. The pizza delivery people use their own bicycles to deliver the pizza. Foodora does not own the bicycles. Therefore, the company is not responsible for them. The companies claim to offer “flexible and free work”. The couriers can work whenever they want. The work is clearly fragmented, and the workers are interchangeable. The work schedule is individualised. The workers have to deal on their own with all the challenges, including icy roads, angry customers, unclear directions, and other issues. The digital platforms pay for the result, not for the time in-between. In many ways, these working conditions resemble those at the beginning of industrialisation, before union mobilisation, labor protection, sick leave pay, and the right to vacation (Ilsøe & Söderqvist, 2023). What is presented as high-tech and new, is in fact a regression in labour law. Historically, we have seen how every technological leap favors the emergence of armies of marginalised workers, who would take jobs that are not considered jobs anymore. In this respect, automation processes are often much less impressive than the big tech companies and large digital platforms want us to believe. Some tasks may disappear and wages will be reduced, though people continue working alongside the machines for lower pay or even sometimes without pay (Taylor, 2018).

To understand work under digital capitalism, we need to go back to the basic question formulated within  the socialist feminist tradition “What is work?” (Ferguson, 2020). How digital capitalism has not only survived but prospered while certain types of work have been hidden and unpaid (Fraser, 2016). The unrecognised work performed by most of us in the borderline between genAI and society, mirrors digital capitalism’s historical and current approach to reproductive work (Jarrett, 2018). The indispensable affective and material labour, mostly cast as “women’s work”, often performed without recognition and pay. In other words, a work that is not regarded as work and is not considered as having any social or economic value. This work in practice is extremely important, while ideologically seen as completely unimportant. Departing from this reasoning, we can conclude that the capitalist system have an inherent desire to devalue and hide socially important work. As participants of the digital capitalism, we often ignore the work that takes place behind the applications, and buy the myth of “success”. This way, we give automation more credence than it deserves. We ignore the work that lies behind the shiny facades of digitalisation. Making the machines appear smarter than they are (Taylor, 2018).

If the discussion about technology continues to focuse only on the narrative that technology drives humanity’s development forward and humans have to keep up, there is an imminent risk of missing the social contexts in which these technical devices are created. When considering the consequences of genAI on university teachers and their daily work, it is important to understand that it is not the technology that will make the teachers obsolete. Technology is developed within a specific economic and social system, where certain resourceful organisations and individuals invest in developing technology that will benefit their personal interests, including control, power, and immense financial returns.  The technology is designed to replace human labor to some extent, but it is developed within a digital capitalism that thrives on making people feel that they are constantly replaceable and vulnerable.

Conclusion and final reflections

To conclude, will genAI make university teacher obsolete? There is a need to acknowledge both the advantages and disadvantages of these tools. With the current university climate orchestrated by the New Public Management with its constant demands for auditing, evaluations, counting, and compiling information, daily tasks of university teachers might become more manageable with the help of genAI. In a better of world, genAI tools might be used to feed the insatiable New Public Management system, freeing up time for research, ciritical thinking and teaching. On the other hand, AI needs data and other resources to be able to learn from the environment in which it will operate. When university teachers participate in training the algorithmic systems, the digital capitalism thrives. This work is often not recognised as work. Digital capitalism wants us to believe that technological development is unstoppable and that we need to accept that our work is exploited. As educators and citizens, we need to be aware that there is an inherent mechanism in the system that actively benefits from hiding work tasks and treats them as non-work.

References

Autor, D. H. (2015). Why Are There Still So Many Jobs? The History and Future of Workplace Automation. Journal of Economic Perspectives, 29(3), 3–30. https://doi.org/10.1257/jep.29.3.3

Brynjolfsson, E., & McAfee, A. (2014). The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies. W. W. Norton & Company.

Danaher, J. (2017). Will Life Be Worth Living in a World Without Work? Technological Unemployment and the Meaning of Life. Science and Engineering Ethics, 23(1), 41–64. https://doi.org/10.1007/s11948-016-9770-5

Ferguson, S. J. (2020). Women and work: Feminism, labour, and social reproduction. Between the lines.

Fraser, N. (2016). Contradictions of Capital and Care. New Left Review, 100, 99–117.

Ilsøe, A., & Söderqvist, C. F. (2023). Will there be a Nordic model in the platform economy? Evasive and integrative platform strategies in Denmark and Sweden. Regulation & Governance, 17(3), 608–626. https://doi.org/10.1111/rego.12465

Jarrett, K. (2018). Laundering women’s history: A feminist critique of the social factory. First Monday. https://doi.org/10.5210/fm.v23i3.8280

Lindberg, J., Kvist,E., & Lindgren, S. (2022). The Ongoing and Collective Character of Digital Care for Older People: Moving Beyond Techno-Determinism in Government Policy. Journal of Technology in Human Services, 40(4), 357–378. https://doi.org/10.1080/15228835.2022.2144588

Lindgren, S. (2024). AI – ett kritiskt perspektiv (Upplaga 1). Studentlitteratur.

Moore, P. V., & Woodcock, J. (2021). Augmented Exploitation: Artificial Intelligence, Automation, and Work. For Work / Against Work. Pluto Press. https://onwork.edu.au/bibitem/2021-Moore,Phoebe+V-Woodcock,Jamie-Augmented+Exploitation+Artificial+Intelligence,Automation,and+Work/

Taylor, A. (2018). The Automation Charade. Logic(s) Magazine. https://logicmag.io/failure/the-automation-charade/

Thomas, R., & Davies, A. (2002). Gender and New Public Management: Reconstituting Academic Subjectivities. Gender, Work & Organization, 9(4), 372–397. https://doi.org/10.1111/1468-0432.00165

Keywords:

Technological unemployment, genAI, work, gender, digital capitalism, university teachers, reproductive work

How to cite this article:

Kvist E. (2025). Will university teachers become obsolete in times of AI? AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/CBGU8049

Civil Sector Vulnerabilities and NATO’s Strategic Role: The Case for International AI Governance

Jason Tucker (Researcher, Institute for Futures Studies, Sweden. Adjunct Associate Professor, AI Policy Lab, Department of Computing Science, Umeå University)


Published on 14 May 2025

Adapted from a presentation given at the NATO Science for Peace and Security Programme, Advanced Research Workshop “Clicking the Pause: The Role of Transatlantic Cooperation in AI Supervision”, Salamanca, Spain, 8-9 May 2025.

As AI becomes increasingly embedded in critical societal functions, the need for robust, internationally coordinated governance grows more urgent. While some national and regional regulation of AI is emerging, applications in defence and international security often remain exempt from these initiatives. This historical separation between civil and defence sectors is understandable given the unique operational requirements of the military. However, it risks creating a false dichotomy—suggesting that AI use in civil domains is largely divorced from international security concerns. However, the geopolitical implications of AI in the civil sector are profound and escalating (Schaake, 2024).

To illustrate this, healthcare provides a concrete and urgent example. Across NATO members and partners, localized and largely disconnected decisions are being made to adopt small-scale AI solutions in healthcare. With states having limited capacity to develop in-house solutions, they often turn to external actors. Doing so means that they are then subject to a complex and opaque web of global supply chains and international actors. This poses substantial risks, including vulnerabilities to cyber-attacks, dependencies on potentially hostile states or corporations, and strain on critical infrastructure to support its adoption.

The growing instability of the international order compounds these challenges. The United States has recently exhibited unpredictability in both its Administration and its corporate tech sector. Even if diplomatic relations are maintained, trust at the local level is harder to rebuild. Working with partners whose long-term reliability is in question introduces significant risk, and other non-traditional partners become more appealing. Where these actors are not aligned with NATO, this could be a vulnerability.

Moreover, the adoption of AI in the civil sector has been driven by techno-solutionism — the prioritisation of technological fixes that neglects broader societal and security trade-offs, as well as potentially more appropriate non-technical solutions. It glosses over the reality that AI, as a socio-technical system is embedded in cultural, institutional, and ethical contexts and requires participation from a broad range of actors to function at its best.

Healthcare systems are particularly susceptible to this narrative (Strange and Tucker, 2024). They face resource constraints that limit the capacity to develop, implement, and secure AI technologies. Combined with the dominant discourse being that AI is the only and best way to solve a broad range of healthcare issues, everyday actors in healthcare are facing pressure to adopt AI where they can. At the same time, NATO’s security infrastructure is drawing from the same limited resource pool—particularly in terms of skills, energy, data infrastructure, and cybersecurity capacity. Without careful coordination, this could lead to a zero-sum scenario, undermining societal resilience and military advantage.

Cybersecurity threats to healthcare are well documented. The World Health Organization has recognized that cyber-attacks targeting health systems have considerable consequences in terms of public health and international security (WHO, 2024). In 2021, WHO reported that one-third of global healthcare institutions had suffered at least one ransomware attack in the preceding year (Mishra, 2024). The European Union reported that in 2023, healthcare was the most targeted critical sector in cyber-attacks (WHO, 2024). During the COVID-19 pandemic, healthcare was not just a target but a vector for disinformation and destabilisation by state and non-state actors alike. Given these risks, decisions about AI adoption in critical civil sectors like healthcare cannot be made in isolation from geopolitical and security considerations. Yet most local actors are not equipped to understand or navigate these complex dynamics. The absence of coherent guidance or frameworks linking AI adoption to national and international security exacerbates vulnerability, weakens societal resilience, and increases dependence on untrustworthy partners.

Global AI governance is essential. It can establish the guardrails necessary to manage these risks and guide responsible adoption of AI technologies across sectors. NATO has a critical role to play here. By integrating civil sector AI governance into its strategic thinking, and engaging with the Allies on this, NATO can help ensure that AI adoption enhances—not undermines—resilience and collective security. This will allow for a more realistic assessment of the trade-offs involved in AI adoption, especially in sectors like healthcare that are both vital to public well-being, are particularly vulnerable to attack and a conduit for hostile actors to cause societal disruption. NATO’s role here should be seen as complementing other international AI governance efforts, such as those by UNESCO, OECD and the EU etc. This would ensure that these governance structures do not become dominated by military priorities and bridge the gap between the defense and civil sector. Democratic safeguards, such as civil society oversight or public reporting, for any NATO-related initiatives affecting the civil sector, would also be essential. As would multidimensional and multidisciplinary views on civil resiliency frameworks.

AI in the civil sector is not a technical or administrative matter alone—it is a strategic issue with implications for the stability, security, and cohesion of NATO members’ and partners’ societies. Only through coordinated, international governance, can we navigate this new terrain with the prudence and foresight it demands.

References

Mishra, V., (2024) Cyberattacks on healthcare: A global threat that can’t be ignored, UN News, https://news.un.org/en/story/2024/11/1156751.

World Health Organization., (2024), Ransomware Attacks on Healthcare Sector ‘Pose a Direct and Systemic Risk to Global Public Health and Security’, Executive Tells Security Council, https://press.un.org/en/2024/sc15891.doc.htm.

Schaake, M., (2024). The Tech Coup. Princeton University Press.

Strange, M. and Tucker, J., 2024. Global governance and the normalization of artificial intelligence as ‘good’ for human health. AI & SOCIETY, 39(6), pp.2667-2676.

Further Information

This article is part of the Politics of AI & Health: From Snake Oil to Social Good funded by The Wallenberg AI, Autonomous Systems and Software Program – Humanity and Society (WASP-HS).

Keywords (comma separated):

NATO, Civil Sector, Artificial Intelligence, Security, Healthcare, Governance

Related URL (if any):

https://www.iffs.se/en/research/research-projects/politics-of-ai-health-from-snake-oil-to-social-good/

How to cite this article:

Tucker J. (2025). Civil Sector Vulnerabilities and NATO’s Strategic Role: The Case for International AI Governance. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/UJML2174

Beyond the AI race: why global governance is the greatest innovation

Virginia Dignum (Wallenberg Chair, Professor Responsible Artificial Intelligence, Director of AI Policy Lab at Umeå University)


Published on 8 April 2025

As artificial intelligence (AI) evolves rapidly, the urgency to govern it responsibly becomes more pressing by the day. We are standing at a pivotal moment, one where the choices we make today will shape not just technological outcomes, but also the foundations of our societies, economies, and planetary well-being.

Even as the UN and other international agencies advocate for global AI regulation, major players – particularly the US, UK, and China – seem increasingly hesitant to fully commit. While the US and UK are currently moving towards light-touch, innovation-driven approaches that prioritize industry leadership over binding rules, China leans toward a state-controlled model aligned with its national priorities. Their reluctance undermines efforts to build the effective, inclusive governance frameworks we urgently need, and may encourage others to also sideline global cooperation in favour of fragmented, self-serving strategies.

But AI governance is not optional, it is essential. It protects rights, upholds global values, and ensures long-term economic stability and sustainable innovation. Without global governance, we open the door to a race to the bottom, marked by short-term thinking, ethical shortcuts, and growing global inequality. We cannot allow geopolitical competition to derail the collective responsibility required to ensure AI serves the common good. Now is the moment to strengthen our commitment to global, values-driven governance, not to stall it. Meanwhile, AI governance shouldn’t chase every new technology, but instead follow clear principles: transparency, fairness, explainability, and accountability. These form a foundation for adaptable policy that protects rights and safety. Tools like regulatory sandboxes, public engagement, and stronger international coordination support this flexible yet high-standard approach as AI evolves.

The best competitive advantage is not ruthless speed, but wise collaboration, especially when the stakes include trust, stability, and the health of our planet. In this context, the European Union’s €200 billion investment in regulated, human-centered AI stands out. This visionary approach demonstrates how regulation can act not as a brake on innovation but as a stepping stone for it. The EU’s commitment to ethics, inclusion, and sustainability offers a powerful alternative to the more narrowly competitive models pursued by the US and China. Yet, funding alone is not enough. Investment must be accompanied by sustainable practices, equitable access, and strengthened social cohesion. Other countries—Canada, Japan, Brazil—are also making important strides. But this is not a race with a single winner. It’s a collective effort, and meaningful progress depends on a globally aligned framework that ensures AI serves all of humanity.

Still, I am deeply concerned about the growing competition to dominate the AI landscape. China, the US, and others are increasingly viewing AI as a tool of economic and military supremacy. This race risks concentrating power in a handful of nations or corporations, sidelining most of the world and worsening inequality. China, the US, and others view AI through the lens of strategic dominance. But AI is not a zero-sum game. True progress requires transparency, ethical alignment, and shared governance.

One of the greatest ethical challenges today is the erosion of human agency through opaque, unaccountable AI systems. That’s why I advocate for Earth alignment, as we introduced in a recent article in Nature Sustainability. This framework emphasizes the need for AI governance to be anchored in environmental sustainability, global justice, and societal cohesion. These goals cannot be achieved in isolation or through regional silos. They require a shared commitment to values that transcend borders, and the democratization of governance. A small group of governments and companies cannot be allowed to shape society through their control over AI development, nor solely through the lens of existential threats and geopolitical rivalry.

Responsible development requires systemic change, not just technical fixes. Ethics must be embedded from the outset, but we must do so through systemic change, not fear-mongering. This is not just a question of innovation; it’s a matter of justice. AI should not be a tool that widens global divides or undermines democracy and social cohesion. It should be a force for empowerment, equity, and resilience. That’s only possible through shared governance, transparency, and ethical alignment across all borders, including those of the most powerful players.

This is why global governance of AI is not optional, it is urgent.

Looking to the future, what excites me most about AI is its potential to empower us, not to replace us. If we govern it well, AI could become one of our most powerful tools for addressing climate change, improving healthcare and education, and advancing equity and social cohesion. But that future is not guaranteed. It depends on the choices we make now. The future of AI is not just about building smarter machines and software, it is about working together towards a wiser humanity. One that values cooperation over competition, solidarity over supremacy. One that uses AI not to dominate, but to heal and uplift.

There is no alternative: in the long run, only responsible AI will lead to innovation that truly benefits society. Anything else will not only undermine trust and human rights but will also lead to technically weaker systems and a loss of true innovation. Irresponsible AI may promise short-term advantages, but it will cost us our long-term future.

Responsible AI is not the finish line. It is the only viable path forward.

Keywords (comma separated):

AI governance, global cooperation, responsible AI, sustainability, transparency, regulation

How to cite this article:

Virginia D. (2025). Beyond the AI race: why global governance is the greatest innovation. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/LNQA3726

Potential impact of the EU Platform Work Directive on AI labelers

Mariia Lesina (Lund University)


Published on 25 March 2025

Introduction

The gig economy has revolutionized labor markets, introducing high flexibility while simultaneously raising critical concerns about workers’ rights and protections. Across the European Union (EU), 3% of people aged 15-64 rely on digital platforms to access employment, yet many remain trapped in a legal grey zone, lacking social security, job stability, and transparency in how their labor is managed (Eurostat 2024).
While the attention has been largely drawn to the impact of AI algorithms on the workflows of such gig workers, most notably drivers or delivery people, one group has been largely overlooked – the invisible workforce behind AI. These workers perform essential tasks such as data tagging, annotation, and content moderation, which help train AI models (Muldoon et al. 2024, 9-10). Their work is often fragmented and distributed through digital labor platforms, making them susceptible to unfair algorithmic control, precarious employment conditions, and opaque remuneration structures. AI labelers work asynchronously across different time zones and are also subject to algorithmic management, where automated systems monitor their accuracy, determine their pay, and can even suspend or terminate their accounts without clear justification (Ibid, 12-15). Evidence of this can be easily found on community discussions on Reddit where stories of such workers are widespread, telling how their work was declined and payment delayed or cancelled altogether (Nagaraj Rao et al. 2025, 23). One of the recent attempts to address this systematic issue was taken by the EU via the Platform Work Directive, features and implications of which will be analyzed in this article.

EU Platform Work Directive

In 2024 the EU introduced the Platform Work Directive, a regulatory framework designed to enhance labor protections, redefine employment classifications, and establish oversight mechanisms for AI-driven algorithmic management. The EU Directive recognizes AI labeling as a form of platform work if it is conducted through a digital platform within the EU and based on a contractual relationship. Article 19 of Introduction specifically mentions tagging as a form of crowd work that can be conducted remotely. This recognition aims at allowing AI labelers to benefit from the Directive’s employment presumption when platforms exercise control over their workflows, task assignments, or performance evaluations. More importantly, it states that it is platforms that have to prove employment status of their platform staff, as the latter usually lacks the means and leverage to do so (Articles 30, 31).
The core of the regulations is that “platform work should be provided with rights, with a view to promoting transparency, fairness, human oversight, safety and accountability” (Article 14). These goals will be achieved through a number of legal changes, translated into the national legislation from the EU level. To begin with, Article 10 of Chapter III enforces human oversight in algorithmic decision-making and mandates greater transparency in how these systems operate, granting AI labelers the right to request detailed explanations of algorithmic decisions that impact their work. This means that platforms cannot solely rely on automated systems to suspend or deactivate an AI labeler’s account. Instead, these decisions must “ensure human oversight and regularly carry out an evaluation of the impact of individual decisions taken or supported by automated monitoring systems”: essentially demanding human in the loop, ensuring that workers are not unfairly penalized by flawed algorithms (Article 47). Additionally, AI labelers now have the right to contest algorithmic decisions, demand explanations, and request human reviews of automated rulings that affect their employment status, pay, or continued access to work (Article 8). The Directive also states that platform workers’ representatives “should be involved in the evaluation process” of these automated systems (Article 44).
The mental and physical well-being of platform workers is another crucial aspect addressed by a separate Article 12 of Chapter III. Due to the nature of their work – repetitive tasks, exposure to harmful content, and tight deadlines – AI labelers face unique risks to their mental and physical health. The Directive requires platforms to assess and mitigate these risks, ensuring that AI systems do not “put undue pressure on platform workers or otherwise puts at risk their safety and physical and mental health”. Platforms must now provide effective information and consultation for workers, while Member States – ensure that digital labour platforms take “preventive measures, including providing for effective reporting channels” (Article 12 of Chapter III).

Possible shortcomings of the Directive

While the Directive provides a theoretically effective legal framework, its success is based on effective enforcement. One major concern is that companies will exploit national law loopholes, restructuring their business models to avoid classifying workers as employees. This issue has been explored by an organization called Fairwork, which evaluates the work conditions of digital labour platforms, and whose extensive analysis highlights large reliance on contextual enforcement in the country-specific legislation. Therefore, as Fairwork experts state, in member states where the “power of labour unions is undermined”, like, for instance, in Italy, the workers can remain unprotected and “self-employed” (Fairwork Project 2024, 2).
Moreover, as experts from International Labour Organization (ILO) explore, outsourcing work to foreign workers in countries where the cost of labour is lower is common as it enables businesses to optimize their costs (Rani et al. 2021, 22). Hence, the Directive, which only protects workers within the EU, does not address the issue of exploitation of non-EU based platform staff.

Conclusion and further discussion

The EU Platform Work Directive is a crucial step toward recognizing AI labelers as platform workers, granting them employment protection, transparency, and human oversight in algorithmic management. However, its impact depends on consistent enforcement across Member States and preventing platforms from exploiting loopholes or outsourcing labor beyond the EU’s reach.
As the December 2026 deadline for national implementation approaches, the fight for fair AI labor practices is only just beginning. With platforms already pushing back, arguing that regulation stifles innovation and contradicts the inherent advantage of flexibility that digital work provides (Copenhagen Economics 2021, 24), the real test will be in Directive’s practical integration. Whether the EU emerges as a pioneer in ethical AI labor governance or struggles with unintended consequences will depend on the ability of each state to implement the Directive in a way that is both principled and pragmatic. Hence, the most topical question remains: “Will this policy set a global precedent for fair AI labor practices, or will fragmented enforcement and corporate resistance weaken its impact?”
To achieve the first outcome, EU policymakers, researchers, and labor rights advocates will need to continue pushing for a regulatory environment that ensures that this overlooked groups in the gig economy – AI labelers – receive the protections they deserve.

References
1. Copenhagen Economics. The value of Flexible work for local delivery couriers. Study for Delivery Platforms Europe. November 2021. 28 p.
2. European Parliament and Council Directive (EU) 2024/2831 of 23 October 2024 on improving working conditions in platform work [2024] OJ L283/1. Internet source. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32024L2831 (accessed 7 March 2025).
3. Eurostat. Experimental statistics on digital platform employment. 18 July 2024. Internet source.URL: https://ec.europa.eu/eurostat/web/products-eurostat-news/w/ddn-20240718-1#:~:text=In20223.0ofpeople,countriesand1EFTAcountry (accessed 7 March 2025).
4. Fairwork Project. Fairwork’s Response to the EU Directive on Platform Work. March 2024. Internet source. URL: https://fair.work//srv/www/wp-content/test.aipolicylab.se/uploads/sites/17/2024/03/Fairworks-Response-to-the-EU-Directive-on-Platform-Work.pdf (accessed 7 March 2025).
5. Muldoon, J., Graham, M., Cant, C. Feeding the Machine: The Hidden Human Labour Powering AI. Canongate Books. 2024. 288 p.
6. Nagaraj Rao, V. Dalal, S., Agarwal, E., Calacci, D., and Monroy-Hernández, A. Rideshare Transparency: Translating Gig Worker Insights on AI Platform Design to Policy. ACM Hum.Comput. Interact. No. 9, 2. April 2025. Pp. 1-49.
7. Rani, U., Rishabh, K.D., Furrer, M. G?bel, N. Moraiti, A. and Cooney, S. World employment and social outlook: the role of digital labour platforms in transforming the world of work. Geneva: International Labour Office. 2021. 283 p.
8. Silberman, M.S., Adams-Prassl, J., Abraha, H. and Suresh, R., Doth the Platform Protest Too Much? Uber, Employment Status, and the EU’s Proposed Platform Work Directive. Oxford Law Blogs, 28 September 2023. Internet source. URL: https://blogs.law.ox.ac.uk/oblb/blog-post/2023/09/doth-platform-protest-too-much-uber-employment-status-and-eus-proposed (accessed 7 March 2025).

Keywords:
EU Platform Work Directive, AI labeling, Gig Economy, Human Rights

How to cite this article:

Lesina M. (2025). Potential impact of the EU Platform Work Directive on AI labelers. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/APJD5342

Metadata/README elements for synthetic structured data made with GenAI: Recommendations to data repositories to encourage transparent, reproducible, and responsible data sharing

Ericka Johnson (Dept of Thematic Studies, Linköping University)
David Rayner (Swedish National Data Service, University of Gothenburg)
Jools Kasmire (University of Manchester), Victor Hennetier (Dept of Thematic Studies, Linköping University), Saghi Hajisharif (Dept of Science & Technology, Linköping University), Helene Ström (Fair AI Data)


Published on 20 March 2025

Introduction

Publication of AI-generated synthetic structural data in data repositories is beginning to reveal the specific documentation elements that need to accompany synthetic datasets so as to ensure reproducibility and enable data reuse.
 
This document identifies actions that research repositories can take to encourage users to provide AI-generated synthetic datasets with appropriate structure and documentation. The recommendations are specifically for AI generated data, not (for example) data produced using pre-configured models or missing data created by statistical inference. Additionally, this document discusses metadata/README elements for synthetic structured datasets (tabular and multi-modal) and not textual data from LLMs or images for computer vision. 

The document is the result of a workshop held on 23rd January 2025, with participants from the Swedish National Data Service, Linköping University and Manchester University. It also draws on survey responses about current practice from 17 data repositories and a review of existing metadata and README requirements. 

Background

AI-generated synthetic structured datasets are generated using machine learning techniques with the aim of reproducing the essential elements of an existing dataset (Guépin et al., 2024 Jacobsen, 2023; Li et al., 2023; Offenhuber, 2024; Savage 2023). Synthetic data generation may be driven by the need to ensure privacy or to expand, enhance or substitute for real-world datasets which may be insufficient or non-existent. Sometimes synthetic data is produced to create a portable or shareable dataset that is considered safe for open access, for example to share via a data repository. 

While synthetic structured data may reproduce the essential elements of an original dataset, it will also inevitably introduce “intersectional hallucinations”, which refer to anomalous inter-attribute relations within a dataset (Lee, Hajisharif & Johnson 2025). AI generated synthetic data also have a known tendency to minimize minority elements and amplify majority elements (Chen et al., 2024; Johnson & Hajisharif 2024). Thus, knowing in what ways a synthetic dataset demonstrates fidelities and in what ways it is ‘different’ from the original data is essential for successful and responsible re-use of synthetic data. Given that the goal of many data repositories is to provide access to data that is replicable and/or reusable, there is a clear need to establish protocols for documenting synthetic data.

Primary recommendations

Our primary recommendations are:

a) that data repositories establish a standardized way to label data as synthetic data, and that this information is prompted-for or required in the metadata or READMEs associated with synthetic datasets. 

b) that data repositories provide users with a guide that explains how to properly document synthetic data. The extent to which documentation should be provided with the dataset or provided in associated articles or publications linked to the data will depend on the policies of the repository. An example is the guide provided by the Swedish National Data Service (2025). 

c) that domain experts be prompted to document the context and motivation for generating synthetic data.

Documenting synthetic data – process and product

Reusability often refers to the data as a product. In the case of synthetic structured data, however, it may be the method of data generation (the data as a process) that  is reusable, not the data itself. We therefore suggest that data repositories require information about both the process of data generation and details about the actual synthetic data.

Data as a process

The following elements should be included to describe the technical details of the synthetic structured data generation process: 

  • A description of the workflow.
  • The generative model used (i.e. GAN, Diffusion, etc.). As techniques are constantly evolving, these requirements should be formulated in such a way to allow for and capture new techniques. The structure and hyperparameters (learning rate, number of epochs, etc.) of the generative model are also important factors for reproducibility and should be included.
  • What raw data or inputs, if any, were used, including its mode of collection. A link to the source of the raw data should be provided where appropriate.
  • Which (random) seeds were used.
  • If a subset of raw data were reserved for testing, how was this subset selected?
  • Versions of the software and packages used.
  • Operating system information, values for relevant environment variables.
  • A link to the source code (we suggest keeping code in a separate repository so it can be reviewed, improved, and re-released) and if appropriate a link to the weights of the trained model.
  • Citation details (including DOIs) for related documents or the release versions of code.

Additionally, some cases of synthetic data are not based on raw data (e.g. agent based modeling/multi agent systems, digital twins). In such cases, this should also be clearly stated in the description of the data generation process. 

If a repository considers that publishing the data generation model is out-of-scope, we suggest providing information on how models can be deposited in either a more generic repository or in a specific repository for models. Links to the model can then be provided in the dataset metadata and/or README.

Product

Synthetic structured datasets inevitably contain stochastic variability, meaning that different datasets can be obtained by running the same code multiple times with different random seeds. We therefore suggest that metadata/READMEs also contain information about:

  • whether the dataset is entirely synthetic or augmented. If it is augmented, what are the proportions of real and synthetic data?
  • missing edge cases at the single-attribute level and inter-attribute level. 
  • inter-attribute combinations in the raw data that have diminished frequency in the synthetic data.
  • inter-attribute hallucinations that have been observed in the synthetic data. 
  • details of the verification/validation process: how was the model tested, etc.
  • how the synthetic data are structured at the file-level: are the input data in a folder marked “raw” or “input”, and output in an “output” folder?

Privacy and specific circumstances

A common use-case for synthetic data is when privacy assurance is necessary for sensitive data. In such cases, we recommend the metadata/README contain information about disclosure risk, indication risk, reidentification risks, K-anonymity, etc. This type of synthetic data requires extra care and should only be made freely available if specific individuals cannot be re-identified by any reasonably likely means.

We also suggest that repositories include instructions on creating the metadata/README that will prompt domain experts to explain the specific circumstances of their synthetic data. Why was it generated? What is the fundamental hypothesis behind the synthetic dataset’s use? What is its subject and purpose(s)? Data creators should be encouraged to disclose, for example, if the dataset was created for exploratory research, to represent sensitive data, to allow for work by a distributed team, to enable data portability, to create categories or support classification decisions, etc.. Encourage data submitters to consider sensitive areas and intersections within the data, as well as how many relational intersections are valuable to combine when using the dataset for new research purposes.

Summary and discussion

Synthetic structured data may be produced where scientific research requires data with no personal information, data that are portable and shareable, data which are not obtainable for practical or ethical reasons, or large datasets for machine learning. However, the details of the generation process and the variations inherent in synthetic data need to be documented, either in a dataset’s metadata/README or in the articles accompanying the dataset.

Many aspects of synthetic data are still emerging, and in some cases, we lack established routines or even vocabularies for them. We hope the recommendations in this policy document will serve as a starting point for further discussions. In particular, we aim to encourage those working with data repositories to collectively establish best practices for managing synthetic data and developing vocabularies to describe them. For example, we might promote an accepted keyword or subheading, such as SYNTHETIC_DATA, or suggest appending “_synth” to filenames containing synthetic data. Additionally, controlled vocabularies should include subcategories to distinguish between fully synthetic and blended/augmented data.

With a well-defined vocabulary and clear metadata guidelines, repositories can help researchers to describe both their datasets and the processes used to create them in an open, transparent, and reproducible manner, ensuring responsible data sharing within the scientific community.

References

??Chen W, Yang K, Yu Z, et al. (2024) A survey on imbalanced learning: latest research, applications and future directions. Artificial Intelligence Review 57(6): 137.

Guépin F, et al. (2024) Synthetic Is All You Need: Removing the Auxiliary Data Assumption for Membership Inference Attacks Against Synthetic Data. In: Katsikas, S., et al. Computer Security. ESORICS 2023 International Workshops. ESORICS 2023. Lecture Notes in Computer Science, vol 14398. Springer, Cham. https://doi.org/10.1007/978-3-031-54204-6_10.

Jacobsen BN (2023). Machine learning and the politics of synthetic data. Big Data & Society. 10(1).  

Johnson E and Hajisharif S (2024) The intersectional hallucinations of synthetic data. AI & Society. https://doi.org/10.1007/s00146-024-02017-8

Lee, Hajisharif & Johnson (2025) The ontological politics of synthetic data: normalities, outliers, and intersectional hallucinations. Big Data & Society.

Li X, Wang K, Gu X, Deng F, Wang FY (2023) Parallel eye pipeline: An effective method to synthesize images for improving the visual intelligence of intelligent vehicles. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 53(9), 5545-5556.

Offenhuber D (2024) Shapes and Frictions of Synthetic Data. Big Data & Society. 11 (2): 20539517241249390. https://doi.org/10.1177/20539517241249390.

Savage, N (2023) Synthetic data could be better than real data. Nature Machine Intelligence. doi: https://doi.org/10.1038/d41586-023-01445-8.

Swedish National Data Service. (2025). Managing and publishing synthetic research data (Version 1). Zenodo. https://doi.org/10.5281/zenodo.14887525 

How to cite this article:

Johnson, E., Rayner, D., Kasmire, J., Hennetier, V., Hajisharif, S., & Ström, H. (2025). Metadata/README elements for synthetic structured data made with GenAI: Recommendations to data repositories to encourage transparent, reproducible, and responsible data sharing. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/MPEW5336

Towards Successful Industrial Policy on AI in Healthcare: Establishing the Conditions for Future Public Benefit

Erez Maggor (Assistant Professor, Ben-Gurion University and the Institute for Futures Studies). 
Jason Tucker (Researcher, Institute for Futures Studies, Adjunct Associate Professor, AI Policy Lab @Umeå University and Visiting Research Fellow, AI & Society, LTH, Lund University).

Published on 18 March 2025

Abstract

This paper explores how pro-active government policies could promote artificial intelligence (AI) in healthcare for the public good. Building on insights from the literature on industrial policy, we argue that without clear conditions and guardrails to ensure future public benefit, state assistance and subsidies will be nothing more than corporate welfare with unpredictable, if any, societal benefit. We provide a few concrete examples of this and then conclude by reflecting on how industrial policy can be a useful conceptual lens to challenge techno-solutionism, increase accountability and situate AI healthcare policies in the broader political-economy.

Keywords: Industrial Policy, Corporate Welfare, Artificial Intelligence, Healthcare, Public Interest, Conditionality, Futures.

Introduction

States and regions are increasingly turning to interventionist policies to try and realise the benefits of the development and deployment of artificial intelligence (AI). This is highlighted by the growing number of national and regional AI strategies, where bold visions for how AI will solve a plethora of societal problems are set out (OECD, 2025). For example, just recently the EU announced the 200 billion Euro regional InvestAI, the largest AI investment in history “to make Europe the continent of AI” (European Commission, 2025). This optimism is possible as AI is a useful empty signifier for various visions of technological innovation (Kak, 2024). These visions are often built upon previous AI policies and initiatives, thus reflecting a continuation of socio-technical imaginaries of AI (Bareis & Katzenbach, 2022). 

Healthcare has featured prominently in these strategies (Hoff, 2023). States are promising that the technology can solve a range of societal issues related to health, fix crumbling and underfunded healthcare systems, and improve our individual health (Tucker, 2023). Often, existing policies are market-based, which requires states to collaborate with the private sector. To get private firms on board, governments provide various incentives, including increased public funding of R&D and procurement, access to public health infrastructure and data, and cutting so-called red tape that allegedly “stifles” innovation. Regarding AI in healthcare, the recent increase in interest in industrial policies coincides with fluctuations in private sector investment in the medical and health sector – which rose sharply between 2019-2021, then plummeted afterwards (North, 2025). The sharp decline post 2021 means that relying on the private sector to address healthcare challenges without incentives is not likely to achieve states´ promised future visions. 

Such policies represent a gamble for states, as these technoscientific futures, which are based on the notoriously unpredictable development trajectory of AI, also rely on the alignment of private and public interests if this is realised. This begs the question of how we can improve the odds for these policies to succeed? To answer this question, we draw lessons from the literature on industrial policy. One of the central insights of this body of work is that when government subsidies are provided without proper conditionalities they are unlikely to produce desired socially beneficial results (Bulfone et al, 2023; Bulfone et al, 2024; Mazzucato & Rodrik, 2023).  We argue, therefore, that a critical element to the success of AI policies is the centralizing of the public good in the conditions of future benefit of AI. Without this in place these policies will end up as nothing more than corporate welfare in the guise of public interest innovation. 

Healthcare is arguably one of the sectors with the highest risk and reward in this regard. We have seen an “AI turn” in global and national health discourses (Strange & Tucker, 2024). AI is posited as being the best, and often only means, to address a broad range of individual, societal and systemic healthcare issues. Health is also intricately intertwined with social stability and democracy (Johnson & Longmore, 2023). In addition, the potential benefits of AI in the healthcare sector is often used to justify broader political agendas on AI, as we saw in the case of the EU’s InvestAI. As such, the success or failure of industrial policy on AI in healthcare has ramifications well beyond the sector. 

Industrial Policies versus Corporate Welfare

The historical record of industrial policy has been mixed. States like Japan, South Korea, Taiwan, Israel, France, and, most recently, China, have had remarkable success using industrial policies to upgrade their industries and ‘catch-up’ to more economically developed nations (Amsden, 1989; Johnson, 1982; Zysman, 1984; Wade, 2004; Maggor, 2021; Ang, 2018). However, in India, Turkey, and across Latin America, industrial policies are considered to have been a relative failure, leading to waste and corruption and resulting in these countries failing to advance economically or, at best, becoming stuck in the ‘middle-income trap’ (Doner & Schneider, 2016). The main lesson emerging from this mixed comparative-historical experience has been that rather than representing a dangerous or wrongheaded economic policy – as many on the ideological right often argue – industrial policies are, first and foremost, a daunting political challenge. 

While profit-maximizing firms will gladly accept various government subsidies and assistance that often accompany industrial policy efforts, they will always prefer this assistance be provided with limited strings and conditions. Policymakers, on the other hand, understand that subsidies without conditionalities are a form of corporate welfare, i.e., a gift from the public to the private sector. As a result, the main political challenge for policymakers is designing industrial policies that incorporate institutional mechanisms that dictate clear conditions regarding the future benefit of such public-private collaborations, ensuring the public benefit is protected, and equipping the state with the capacity to ‘discipline’ uncooperative firms (Amsden, 1989; Chibber, 2023; Maggor, 2021). Crucially, this needs to be implemented from the outset, as this is the stage at which policymakers have the greatest leverage over the private sector. To ground and contextualise this we can turn to two recent cases, the development of the COVID-19 vaccine in the USA and the NHS Google-DeepMind health data scandal in the UK. 

Developing the COVID-19 Vaccine in the USA 

One useful example to demonstrate the aforementioned logic is the development of vaccines for COVID-19. In 2020, in response to the pandemic, the Trump White House approved “Operation Warp Speed,” to develop an mRNA vaccine. In addition to federal support for the promotion of R&D, the program also provided government subsidies for scaling-up manufacturing (in the case of Moderna), offered strategic government procurement (in the case of Pfizer), and delivered various government assistance to industry across multiple sectors and regions (Adler, 2021). On the one hand, this experience represented an effective and comprehensive industrial policy program that saw the government partner with the private sector to produce a significant social good– a safe and effective vaccine that helped end a global health emergency. On the other hand, it showed that when state-supported innovation is not governed for the common good via strict terms and conditionalities, many people remain excluded from its benefits. 

For example, even though Moderna used public investments and research to develop its vaccine, they refused to share intellectual property and know-how with less-developed countries (Mazzucato, 2023). The experience  also demonstrates that without imposing strict limitations on profit-taking, private firms that succeeded due to public assistance are likely to retain “astronomical and unconscionable profits”, in this case due to their monopolies of mRNA COVID vaccines — upwards of 69% profit margins in the case of Moderna and BioNTech (Wilson, 2021; Emergency USA, 2021).

The NHS Google-DeepMind Health Data Scandal

There are also examples of industrial policies on AI and health more specifically, such as the UK National Health Service (NHS) Google-DeepMind case. In 2015 the Royal Free London NHS Foundation Trust shared the personal health records of 1.6 million patients with Google’s AI firm DeepMind. This was to support the development of a system that could potentially better diagnose acute kidney injury. Successive UK governments have long courted Google to try and attract more of the firm’s investment in the UK market, so this public-private partnership came as no surprise. 

However, while the diagnostic application showed early signs of success in detecting kidney injury, the partnership hit the headlines in 2017 due to a significant public backlash. There were serious concerns about data privacy and the lack of transparency in how the sharing of publicly funded health data with DeepMind was decided upon. It was also unclear what DeepMind was using the data for, as well as there being an uncertain public benefit from the data transfer (Dickens, 2021). The Information Commissioner’s Office eventually ruled that the NHS had failed to comply with the Data Protection Act by transferring the data (Information Commissioner’s Office, nd), though a subsequent class action lawsuit against DeepMind itself failed in the UK courts. With the introduction of the UK’s AI Opportunities Action plan in January 2025, the case has raised its head once again (Milmo & Stacey, 2025). The legacy of the NHS DeepMind scandal thus lives on, impacting public trust in the next generation of industrial policies on AI. 

Establishing the Conditions for Future Public Benefit

The unpredictable nature of AI development poses challenges in terms of establishing the conditions of future benefits in industrial policies. However, these are not insurmountable, and as argued above, need to be overcome if the social benefit of the technological investment is to be realised. We should remember that conditions do not always need to be very specific (such as the quick development of a vaccine), or indeed directly related to healthcare. For example, supporting a flourishing MedTech sector that creates high skilled jobs, and pays corporation tax into the state coffers, could be seen as an acceptable outcome. Yet, there needs to be clear guardrails about how the public are to benefit from any technological innovation that an industrial policy has facilitated. Assuming private actors will act in the public good if a discovery is made has proven to be wishful thinking time and time again. 

The economist Mariana Mazzucato and her colleagues have outlined several conditionalities that could ensure the public shares the returns of public investment in health. These include charging royalties from companies who profit from technologies developed with public funding, with funds earmarked to finance future innovation. Another strategy could be for states to retain a “golden share” of patents developed with public assistance while incorporating weak and narrow (rather than strong and broad) intellectual property protections to ensure greater access for marginalised members of the community as well as developing nations. Finally, rather than paying exorbitant prices, public health systems should pay prices that reflect their contribution to the development of new therapeutic or health technologies (Mazzucato & Li, 2019; Mazzucato & Roy, 2019).

One should also remember that industrial policies do not only support the growth of certain sectors but can also be used to reduce redundant or harmful ones. Indeed, this insight has been highlighted in the context of the green transition, as policymakers seek to promote green technologies while, at the same time, phasing out the carbon economy (Ergen & Schmitz, 2023). Industrial policies on AI in health can be considered in a similar fashion by situating them in relation to the broader healthcare sector. Healthcare actors, patient groups, unions, and other civil society actors should play a key role here in deciding where to advance and reduce different public health infrastructures and services. This raises the issue of defining “public interest” and classifying the “public(s)”, which is a process of “power, politics and truth seeking” in AI (Sieber et al 2024, p634). How this impacts the establishment of future public interest in AI industrial policy requires context specific interrogation.    

Finally, with this piece we hope to encourage the further use of industrial policy as a conceptual lens to analyse the increase in interventionist policies around AI and healthcare. While this was only an initial foray into the potential of doing so, industrial policy proved to be a useful approach for several reasons. First, the need for conditionality in the future public benefit to be established pushes back against techno-solutionist narratives. It also facilitates a broader understanding of the purpose of these policies within the wider political economy. Thus, we are better able to wrestle with the reality that policies about AI in healthcare are not just about AI or indeed healthcare. Rather, they are about prioritising certain health futures, closing down others and advancing broader national and regional AI visions. 

References

Adler, D. (2021). Inside Operation Warp Speed: A new model for industrial policy. American Affairs, 5(2), 3-32.

Amsden, A. H. (1989). Asia’s next giant: South Korea and late industrialization. Oxford University Press.

Amsden, A. H. (1989). Asia’s next giant: South Korea and late industrialization. Oxford University Press.

Ang, Y. Y. (2018). How China escaped the poverty trap. Cornell University Press.

Bareis, J., & Katzenbach, C. (2022). Talking AI into being: The narratives and imaginaries of national AI strategies and their performative politics. Science, Technology, & Human Values, 47(5), 855-881. https://doi.org/10.1177/01622439211030007

Bulfone, F., Ergen, T., & Kalaitzake, M. (2023). No strings attached: Corporate welfare, state intervention, and the issue of conditionality. Competition & change, 27(2), 253-276.

Bulfone, F., Ergen, T., & Maggor, E. (2024). The political economy of conditionality and the new industrial policy (No. 24/6). MPIfG Discussion Paper.

Chibber, V. (2003). Locked in Place: State-building and Late Industrialization in India. Princeton University Press;

Dickens, AD (2021) The right to health implications of data-driven health research partnerships. PhD thesis, University of Essex.

Doner, R. F., & Schneider, B. R. (2016). The middle-income trap: More politics than economics. World Politics, 68(4), 608-644.

Emergency USA. (2021, September). Pharmaceutical companies reaping immoral profits from COVID vaccines, yet paying low tax rates. Emergency USA. https://www.emergencyusa.org/2021/09/pharmaceutical-companies-reaping-immoral-profits-from-covid-vaccines-yet-paying-low-tax-rates/?doing_wp_cron=1739796679.7930979728698730468750

Ergen, T., & Schmitz, L. (2023). The sunshine problem: Climate change and managed decline in the European Union (No. 23/6). MPIfG Discussion Paper.

European Commission. (2025, February 18). EU 200 billion Euros for AI, largest AI investment in history. European Commission. https://ec.europa.eu/commission/presscorner/detail/en/ip_25_467

Hoff, J. L. (2023). Unavoidable futures? How governments articulate sociotechnical imaginaries of AI and healthcare services. Futures, 148, 103131. https://doi.org/10.1016/j.futures.2023.103131

Information Commissioner’s Office. (n.d.). Google DeepMind and class action lawsuit. Information Commissioner’s Office. https://ico.org.uk/for-the-public/ico-40/google-deepmind-and-class-action-lawsuit/

Johnson, C., (1982). MITI and the Japanese Miracle: The Growth of Industrial Policy: 1925–1975, Stanford: Stanford University Press.

Johnson, D., & Longmore, S. (2023) How Healthy Is Democracy: The Role of Healthcare and Social Equity Considerations in the Governance of People. In Rethinking Democracy and Governance (pp. 345-369). Routledge.

Kak, A. (2024). What does “AI for the public good” really mean? AI Now Institute. https://ainowinstitute.org/general/ai-nationalisms-executive-summary#h-what-does-ai-for-the-public-good-really-mean 

Maggor, E. (2021). Sources of state discipline: lessons from Israel’s developmental state, 1948–1973. Socio-economic review, 19(2), 553-581.

Maggor, E. (2021). The politics of innovation policy: Building Israel’s “neo-developmental” state. Politics & society, 49(4), 451-487.

Mazzucato, M. (2023). Health for All: Transforming economies to deliver what matters. bmj, 381.

Mazzucato, M., & Li, H. L. (2019). Health innovation re-imagined to deliver public value.

Mazzucato, M., & Rodrik, D. (2023). Industrial policy with conditionalities: a taxonomy and sample cases.

Mazzucato, M., & Roy, V. (2019). Rethinking value in health innovation: from mystifications towards prescriptions. Journal of Economic Policy Reform, 22(2), 101-119.

Milmo, D & Stacey, K, (2025, January 13). Labour AI action plan for NHS patient data: Why it’s causing concern. The Guardian. https://www.theguardian.com/politics/2025/jan/13/labour-ai-action-plan-nhs-patient-data-why-causing-concern

North, M. (2025, January 22). 5 ways AI is transforming healthcare. World Economic Forum. https://www.weforum.org/stories/2025/01/ai-transforming-global-health/.

OECD. (2025). National strategies, agendas, and plans. OECD AI Policy Observatory. https://oecd.ai/en/dashboards/policy-instruments/National_strategies_agendas_and_plans

Sieber, R., Brandusescu, A., Adu-Daako, A., & Sangiambut, S. (2024). Who are the publics engaging in AI? Public Understanding of Science, 33(5), 634–653. https://doi.org/10.1177/09636625231219853 p.634

Strange, M., Tucker, J. Global governance and the normalization of artificial intelligence as ‘good’ for human health. AI & Soc 39, 2667–2676 (2024). https://doi.org/10.1007/s00146-023-01774-2

Tucker, J. (2023). The future vision (s) of AI health in the Nordics: comparing the national AI strategies. Futures, 149, 103154. https://doi.org/10.1016/j.futures.2023.103154

Wade, R. (2004). Governing the Market: Economic Theory and the Role of Government in East Asian Industrialization, Princeton University Press.

Wilson, M R. (2021, November 4). Vaccine manufacturers are profiteering. History shows how to stop them. Politico. https://www.politico.com/news/magazine/2021/11/04/vaccine-manufacturers-are-profiteering-history-shows-how-to-stop-them-519504;

Zysman, J., (1984). Governments, Markets, and Growth: Financial Systems and the Politics of Industrial Change. Ithaca: Cornell University Press.

Other

The authors contributed equally to this work.

Funding

JT’s contribution was made possible with the support of the Wallenberg AI, Autonomous Systems and Software Program – Humanity and Society (WASP-HS) funded by the Marianne and Marcus Wallenberg Foundation and the Marcus and Amalia Wallenberg Foundation.

How to cite this article:

Maggor, E., & Tucker, J. (2025), Towards Successful Industrial Policy on AI in Healthcare: Establishing the Conditions for Future Public Benefit, AI Policy Exchange Forum (AIPEX), https://doi.org/10.63439/PFRX3762

Multilevel oversight of AI systems in line with the AI Act

Diana M. Popa (Delft University of Technology)

Abstract

The AI “oversight – control” nexus is a matter of debate in both scientific literature and policy papers, given the complex and disruptive nature of the technology and the intricate legislative systems regulating the deployment of AI systems. Most approaches give precedence to the need for a multilayered governance model, while in the same time taking into account the lifecycle stages of the AI product. Oversight over the ecosystem in which the AI is deployed (the governance layer) should not be confused with the process level human oversight. At process level, human oversight is needed in the case of “high-risk systems”, and AI literacy is a precondition for this oversight to be relative. Effective oversight requires many resources at both ecosystem and organisational levels. Additionally, oversight of AI systems is also highly regulated in democratic societies and therefore can be seen as a reflection of the importance that democratic values have in the way disruptive technologies are deployed within society.   

The need for control and human oversight of AI systems is acknowledged and set down in both legislation texts and research and policy papers, but operationalisation of both concepts is either broad or diverse and in practice address different levels of applicability in relation to the given AI system. Control and oversight of or over the functioning of the AI system itself should not be equalled to how the legislative framework address measures of control and oversight of the ecosystem in which the AI systems are deployed. The multitude of initiatives aiming to establish generally accepted definitions and determine the practical limitations of the “meaningful human oversight” – control nexus (Verdiesen, et al. 2021) underline the difficulty of determining  the control capacity over the AI system. Efforts for identifying the correct balance of human oversight and control are meant to counter the “black box” effect that the disruptive nature of the technology brings with itself, especially when used as a decision support tool with a high impact on the individuals.

The Artificial Intelligence Act (AIA) itself mentions both control and human oversight as risk mitigation measures, using terms such as “relevant” / “appropriate”/ “meaningful” human oversight. What the specific implementation measures are remain at a general level (Article 14 primarily) while in the same time, the scientific literature addresses the fact that there is ambiguity in the way human oversight is operationalized and implemented, even more so when it comes to “meaningful human oversight”. This is not necessarily a drawback of the legislation, given the fact that the AI Act was just approved by the European Parliament and also considering that, like in the case of the GDPR, the AI Act wants to be an overarching legislation, leaving room for personalized application at Member State level and remaining broad enough so that the AIA itself needs not reviewing or updating every year, taking into consideration also the alert rhythm of technological development in the field.

From the policy and research approaches, different initiatives have addressed the operationalization of human oversight and the “control – oversight” relationship, either as a nexus or as opposed measures, with approaches depending on the applications field of the concepts. Both control and oversight are defined and embedded within the given broader ecosystem in which the AI model is implemented, and most approaches adhere to a three layer ecosystem: the governance layer (the supra-national and/or national ecosystem in which the AI applications are developed), the socio-technical or organisational layer (based on internal regulations and sector rules) and the technical or process layer (also addressing the product safety regulations) (Verdiesen et al., 2021; Adams et al., 2024; Novelli et al.,2024). Oversight takes place at European level, within dedicated structures (still in development), at national level, with data protection authorities, at organizational level and at process level. Multilayer oversight overlaps (even if one might argue not perfectly) with a multistakeholder governance system of AI systems implementation within a certain regional or national context. Oversight is implemented both vertically and horizontally: the former within national supervisory authorities, expanding their structures to take on formal oversight and control roles with sanctioning capabilities and the latter within the organisations deploying the AI system themselves.

From a component perspective, building blocks for an effective oversight system include good functioning of the democratic rule of law principles, sound legislative frameworks accompanied by binding powers (both for the pre and post deployment phase), appropriate financial and human resources in all three layers and technical capabilities (technological maturity). In line with democratic principles, an oversight system should address the principles of transparency, accountability and responsibility regarding the way the AI systems are used in public context and incorporate human rights standards. Binding involvement of oversight bodies (in the sense of consultation before the implementation of a tool) and binding powers post factum are necessary for oversight to be effective (Wetzling, 2024), the latter with supervisory/ investigative and sanctioning powers.

Not addressing here prohibited practices that are equally regulated by the legislation, in the case of high-risk or impactful AI systems, the risk approach managed through measures and degrees of oversight is also influenced by the type of deployer, the sector it activates in and the process in question. While in the case of internal processes, it is a reflection of the risk appetite of an organisation, in the case of processes that have an impact on individuals outside the organisation, such as is the case with AI used by public authorities in the execution of the public administration act, oversight is tighter regulated from the governance level, since it should also be in line with the social values of the system it operates in. Although actual alignment with public values is not required by the legislation, it is a practice in line with democratic values that increases transparency of the governing act and public decision making and is a reflection of the governing style of a certain nation state, such as is the case of the Netherlands.

The ecosystem division is also often overlapped with the stages of the product’s lifecycle, given that AI is a product deployed on the unique market and also has to comply with product safety regulations, approach which identifies key points during the life of the AI system when human control or oversight is needed, also in relation to the inherent risks of that certain stage. Therefore, within each layer, different control and oversight measures are put in place, at different moments in the AI product’s lifecycle: before deployment, during deployment and after deployment.

A well-structured oversight model therefore includes both ex-ante and ex-post elements. Expert assessment in the form of expert bodies should be included in both stages, with pre-deployment expert advice going in the design phase of the system and in the high level regulating frameworks and with ex-post oversight in the form of expert assessment and democratic scrutiny (Oetheimer, 2024). These recommendations are also valid in the case of regulatory AI frameworks.

From this temporal perspective, control is implemented at three different points in time: ex-ante or pro-active control, on-going or simultaneous control, and ex-post control. Ex-ante or pro-active controls are (or should be) implemented by way of the AIA or by the means of the AI national strategy or organisational policies and practices through: 

  • bans on prohibited (high risks) systems (AIA);
  • enforcement through market surveillance and control (AI Act);
  • (quick) scans/ risk analysis for selection of trustworthy suppliers and safe acquisitions (NCTV, 2024) and identification of supply chain risks (Bluebird & Hawk BV, et. al, 2024);
  • DPIAs and FRIAs, (AI Act, National AI strategy, GDPR, Organisational policies);
  • attribution of supervisory roles and compartmentalisation;
  • establishment of internal ethics committees addressing issues such as data ethics compliance or data “pedigree”.

Ongoing or simultaneous control:

  • supervisory authorities at national and international levels with sanctioning powers;
  • Meaningful human oversight within the process (organisational policies);
  • Process- related activity of ethics committees for deviating cases.

Equally relevant, for oversight to be effective, practical and technical capabilities giving external parties (either supervisory authorities of the broader public) relevant insight into log frames to evaluate how the data processing took place, details on the way the training data was processed and altered at every step should also be made available. These are forms of ex-post control, giving external parties the possibility to get insight into the system, either through information made public by the deployer beforehand or through standard access requests. A different form of ex-post control would be the requirement to make the AI models FAIR (Findable, Accessible, Interoperable and Reusable) and have metadata or a more extensive documentation package made available, either openly or upon request. This is partially addressed by the requirement to have high-risk AI systems registered in the European data base for High Risk systems that is under development (Article 71 of the AIA).

Article 14 of the AIA specifically addresses human oversight measures at the process level, placing responsibilities on both developer and deployer: on the developer during the design phase (therefore an ex-ante measure) for the identified risks, and for the deployer for the attribution of the actual human oversight during the process to human resources with the appropriate AI literacy. For high-risk systems, technical provisions include measures similar to kill switches (Article 14, point 4, e.) and oversight measures on supervision of two separate natural persons with appropriate training, competence and authority. Thus, for human oversight to be meaningful, AI literacy is a precondition (Recital 20, Articles 14, 26, 91). Adequate human and financial resources at eco-system and socio-technical levels are also paramount for a functioning oversight system. This is where fragmentation at European level is foreseen to remain high, For example, the Dutch Data Protection Authority (AP), as the dedicated supervisory authority for the use of AI systems that have an impact on personal data, has had an increased budget of 3.6 million euro for 2026 and 2027 (AP, 2024) to address the new tasks under its coordination streaming from the upcoming of the AIA. Yet, despite the budget increase, the Dutch AP is still challenged by finding the necessary numbers of trained specialists for the exercise of the given tasks. While the oversight and governance systems involve multiple entities, data protection authorities, as the responsible oversight entities of the AI also have the role of curbing the data appetite of public authorities. At the implementation level, human oversight remains both challenging and fragmented, influenced by level of digital maturity within a certain society, financial capabilities of each deployer (hyper-scalers having the higher ground), the availability of a trained workforce or lack thereof, the difficulty to keep up with technological and legislative developments, efforts by central government to harmonize implementation of EU legislation in a top bottom approach within national context  and uniformise/change already established daily practices.

The fact that AI is a disruptive technology is also reflected by the intense debates and analyses in legislative and policy-making circles in order to set it into a unified, comprehensive and EU level actionable text. Oversight of AI systems is highly regulated in democratic societies and while democratic states spend years and a great deal of financial resources to harmonize these legislative frameworks, less democratic states or ones that rather focus on market  competition principles seem more inclined to invest in and focus on the development and implementation of the technology self, focusing on capitalizing on the benefits without the restrictions of ethical and robust legislative frameworks, and as such gaining larger market shares in the world competition stage. 

Addressing the “why” of the need for human oversight includes the fact that AI is a disruptive technology, the level of trust in technology in general (at social level for example – for example the World Values Survey) and AI technology in particular, and the approach within a certain society towards rules and regulations. As Adams (2024) suggests, AI governance does not translate into responsible AI. Oversight structures are a form of foresight and a practice of a healthy ecosystem, in which decision makers, weather at supra-national, national or sectorial and even company levels implement checks and balances in a proactive manner.  Foresight is limited though to the overview of the known variables of the given environment at a certain moment and unforeseen uses remain a reality and very much in the control and responsibility of the deployer. Taking the discussion on when human oversight is needed further, it comes down to the cases and moments of high risk situations. Despite the fact that automated instruments for decision making have been hailed for the benefits they bring regarding time saving with repetitive tasks, it is in “life or death” situations where they failed and when ultimate responsibility was passed to the human. Identifying such “life or death” situations should be addressed in the design phase of the AI system, despite the limitations or impossibility of mapping out all possible real life situations in a design/ test phase or lab setting.  And while the AIA does address ex-ante or pre-deployment controls, lists prohibited AI practices, and identify highly regulated sectors and exceptions, in practice, specific sector level knowledge and also case by case analysis are needed in order to identify what qualifies as such and to identify the correct moment and forms of human oversight.

Empirical case studies involving the use of responsible AI in line with the AIA within different contexts are much awaited. Paradoxically, despite extensive regulatory systems, what gives rise to the need for case by case approaches is the uniqueness stemming from the intersection of legislations (for an overview see Annex 1 of the AI Act: list of Union Harmonization legislation), technological regulations, ethics, organizational settings and target groups affected by the AI. Harmonizing legislation attempts such as those in the case of products for the single market are underway. However, if the harmonization process will not work, it is foreseeable that the EU commission will come with its own standards, with the consideration that technical safety is easier to implement and standardize, whereas human oversight is more difficult both to regulate and ethical implications even more difficult to evaluate, reflecting also political choices. As practice evolves, it is expected that in the next years the scientific literature will abound with these case studies. It will be interesting to see how early adaptors, independently of their size, will set trends, how fast good practices will be absorbed and, as was the case with the GDPR, how case law will evolve. These will also serve as measures for measuring the effectiveness of the human oversight and control nexus, currently a challenge that will take the form of consistent practices in the coming years.  

References

Adams, R., Adeleke, F., Florido, A., de Magalhaes Santos, L.G., Grossman, N., Junck, L., Stone, K. (2024) Global Index on Responsible AI 2024  (1st Edition). South Africa: Global Center on AI Governance.

Autoriteit Persoonsgegevens. (2024). Directie Coordinatie Algoritmes Werkagenda coördinerend algoritmetoezicht in 2024. Accessed Mey 2024 at: https://www.autoriteitpersoonsgegevens.nl/documenten/werkagenda-coordinerend-algoritmetoezicht-2024

Bluebird & Hawk BV., De Nederlandse Vereniging van Banken, ICT Group, Nederlandse Spoorwegen en Technolution;  Nationaal Coördinator Terrorismebestrijding en Veiligheid (NCTV), Nationaal Cyber Security Centrum (NCSC); Algemene Inlichtingen en Veiligheidsdienst (AIVD); CIO Rijk (2024). Cybercheck: ook jij hebt supply chain risico’s. Available at: https://www.ncsc.nl/documenten/publicaties/2024/april/18/cybercheck-ook-jij-hebt-supply-chain-risicos

Oetheimer, M. (2024) Presentation during the European Data Protection Summit. Rethinking data in a democratic society. Session 3: Zooming out onto democracy and the rule of law. How to build a functioning democratic oversight. Available at:  https://20years.edps.europa.eu/en/summit/media

Nationaal Coordinator Terrorismebestrijding en Veiligheid (2024). Quick Scan nationale veiligheid bij inkoop en aanbesteding. Accessed June 2024 at: https://www.nctv.nl/onderwerpen/economische-veiligheid/documenten/publicaties/2024/02/01/quick-scan-nationale-veiligheid-bij-inkoop-en-aanbesteding

Novelli, Claudio and Hacker, Philipp and Morley, Jessica and Trondal, Jarle and Floridi, Luciano, A Robust Governance for the AI Act: AI Office, AI Board, Scientific Panel, and National Authorities (May 5, 2024). Available at SSRN: https://ssrn.com/abstract=4817755 or http://dx.doi.org/10.2139/ssrn.4817755

Verdiesen, I.; Santoni de Sio, F., Dignum, V. (2021).  Accountability and Control Over Autonomous Weapon Systems: A Framework for Comprehensive Human Oversight. Minds and Machines (2021) 31:137–163. https://doi.org/10.1007/s11023-020-09532-9

Wetzling, T. (2024) Presentation during the European Data Protection Summit. Rethinking data in a democratic society. Session 3: Zooming out onto democracy and the rule of law. How to build a functioning democratic oversight. Available at:  https://20years.edps.europa.eu/en/summit/media

How to cite this article:

Popa, D. M. (2024). Multilevel oversight of AI systems in line with the AI Act. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/XEFY3802

A reappraisal of the AI Act in light of the qualitative requirements of the Law

Inês Neves (Faculty of Law, University of Porto / CIJ - Centre for Interdisciplinary Research on Justice)

Published on 29 June 2024

As the Artificial Intelligence Act (‘AI Act’ or ‘Regulation’) approaches its entry into force, it is tempting to conclude that the “job is (well) done”. However, a closer examination of the final text (signed on 13 June 2024) reveals a more nuanced reality. In addition to being implemented gradually over time and extending until 2026 (Article 113), the Regulation forms an integral part of the “New Legislative Framework” (‘NLF’) (Recitals 9, 46, 64, 83, 84). As such, it operates similarly to a ‘framework’ law, setting out a minimum set of essential requirements and high-level obligations applicable to a defined group of AI systems (those deemed high risk and specific risk).
As a result, before the Regulation is implemented in an administrative and jurisdictional manner, a process of normative creation must be initiated. In short, the actual law-making process – whereby the essential requirements of the AI Act will be specified and concretised through standards and other instruments, specifying the broad and open-texture wording of the AI Act through technical or quality specifications – is yet to commence (along with the governance structure and paraphernalia).

Nonetheless, it would be erroneous to assume that no progress has been made thus far or that nothing existed before the AI Act. There are ethical guidelines, standards that have already been published or are currently under development, and of course, there is all the European Union (‘EU’) and national legislation that is not affected by the AI Act (in terms of consumer rights, product safety, labour law, privacy and data protection, competition law, digital services, financial services, among others). Indeed, the existing legislation even raises doubts about the usefulness of the AI Act, and the EU legislator is not agnostic about the expectations and challenges of misalignment and coherence (see, among others, Recitals 46, 64, 158, and Articles 8(2), 72(4)).
While it is accurate to reflect that the AI Act is not intended to reinvent the wheel, the existing framework is deemed insufficient to provide a comprehensive framework of essential requirements specifically applicable to AI systems (high risk and specific risk). With this in mind, the AI Act follows the logic of product safety legislation, appearing as a hybrid between the need to promote trade and reduce barriers to innovation and the need to guarantee the protection of fundamental rights in the face of the specific risks of AI.

The AI Act thus exhibits a shared genetic identity with other acts of the NLF. However, it can be neither summarised nor equivalated to product safety legislation. On the one hand, this is due to the evolution and dynamism of AI systems, which make them very different from products typically associated with this type of legislation (toys, radio equipment, medical devices, machinery, among others). On the other hand, in addition to concerns for safety and health, the Regulation expressly includes fundamental rights among its objectives.
While the connection with fundamental rights does not extend to the point of turning the AI Act into an instrument that enshrines or guarantees new or specific fundamental rights per se, this link requires us to question whether greater demands on the applicable legal frameworks shall be ensured.

As is well known, the principles of legal certainty and legality demand that the law prescribing duties and obligations fulfil specific requirements of accessibility, foreseeability and precision. These requirements ensure that those obliged to (binding) legal frameworks understand what is expected of them in order to comply with such demands. Of course, these qualitative or substantive requirements do not go so far as to demand absolute certainty, which is incompatible with the recurrence of indeterminate concepts and vague terms (and the pace of technology). Furthermore, they do not negate the potential need for legal counsel or case law to comprehend the obligations prescribed by the legal system comprehensively.
However, if this is the case, there will be limits to the openness of the law, mainly when it is associated with burdens and requirements that restrict fundamental rights. This is the case of the freedom to conduct a business (Article 16 of the Charter of Fundamental Rights of the European Union), which is limited by the requirements and obligations applicable to the systems covered by the AI Act. Alternatively, the Regulation provides for sanctions that may be manifestly harmful to public interests, such as innovation and the very realisation of fundamental rights (whose realisation, at least in the current era, may depend on AI solutions).

A review of the AI Act’s recitals reveals a clear awareness of the necessity for legal certainty (see, for example, Recitals 3, 12, 83, 84, 97, 139 and 177) and the importance of defining its terms and requirements in a way that is consistent with the broader regulatory framework. However, in contrast to the limited references to this general principle of EU law, searching for the qualifier “appropriate” yields 230 results, which clearly demonstrates the broad and generic way in which the requirements for operators and systems are laid down and designed in the AI Act.
The necessity to reconsider the limitations of openness and the high-level approach adopted by the AI Act is highlighted by the fact that the AIA challenges the traditional remit of the requirements of legal certainty and the quality of the law, essentially on two levels.
Firstly, by referring to guidelines, codes of conduct, standards and common specifications, the question arises as to whether this connection between basic-fundamental law and implementation, not just by the European legislator but by other bodies, including private ones, will comply with the requirements of legitimacy and substance imposed on law in the material sense.
Secondly, insofar as the AI Act does not appear in a regulatory vacuum but instead leaves relevant European legislation (equally imposing obligations) untouched, it is essential to question whether the Regulation’s entry into force will have an impact on the predictability of what is expected of operators (in the face of perhaps contradictory expectations arising from different legal acts). It is believed that potential misalignments may be solved through a revision of the legislation in force, to identify conflicts and, if necessary, ensure that the combination of sectoral (old) and transversal (new) obligations and requirements does not result in greater entropy than advantage.

In this piece, we focus on the first area of concern.
Indeed, the Regulation leaves the implementation of a significant portion of requirements to the European Commission through a variety of means, including guidelines (Article 96), delegated acts (Article 97), and implementing acts (Articles 41(1) and 50(7)), among which codes of practice, where the AI Office acquires prominence (Articles 50(7) and 56).
The Commission is but one of many relevant players, however. Among other actors (including at the national level), the European Artificial Intelligence Board (‘AI Board’) will have the task of issuing recommendations and written opinions on issues relating to the implementation of the AI Act, aimed at ensuring its consistent and effective application (Article 66(e)). Furthermore, the role of the Advisory Forum (Article 67) and the Scientific Panel of Independent Experts (Article 68) must not be neglected.

In this regulatory landscape, harmonised standards must be recognised as primus inter pares (Article 40), as they will provide detailed technical specifications on how to comply with and meet the public interest objectives and the high-level requirements in the AI Act. The shortcomings of this co-regulation or delegation become evident when one considers that the acts in question are not those of EU institutions, bodies, offices, or agencies, and thus are not subject to judicial scrutiny (Article 263 TFEU). Additionally, fundamental rights are not a subject that can be technically specified. Finally, the fact that the institutions responsible for law-making are private entities and do not offer adequate guarantees of inclusion, representativeness and transparency creates the risk of regulatory capture and reinforces competitive foreclosure (which is already a consequence of the exclusion of AI systems already placed on the market or put into service (Article 111(2)).
In addition to standardisation, the Regulation also provides for common specifications (Article 41) in the event of i) non-acceptance of the standardisation request by any of the European standardisation organisations; ii) insufficient coverage by the harmonised standard of fundamental rights concerns, or iii) failure by the standard to meet the requirements of the request (promptly).
In both cases, voluntary compliance with harmonised standards or common specifications is associated with a presumption of conformity, particularly relevant for high-risk AI systems and general-purpose AI models. Although this presumption contributes to legal certainty, it merely guarantees a reversal of the burden of proof. The presumption is rebuttable, and the degree of legal certainty depends on the material content of the standards and specifications that will be adopted.
The preceding analysis leads to the conclusion that, in terms of meeting the requirements of accessibility and foreseeability, the AI Act does not represent an exemplar piece of legislation. The model adopted here is collaborative, with the intervention of various other players, and where the law and technology are inextricably linked. This does not imply that it is inherently inadmissible.
In fact, this way of co-regulation may be the only viable means of ensuring that the law aligns with reality. However, if this is indeed the case, the requirements for legitimising the processes for drawing up the intervening acts and actors should ensure high(er) levels of inclusion, participation, representativeness, and expertise. About the adoption of standards, it is crucial to bear in mind that the representativeness of smaller players and civil society may be impeded by factual inequalities (resources and expertise), even when abstract equality is provided for.

The interconnection between the AI Act and the fundamental rights renders these demands particularly urgent. Indeed, in this regard, the EU Regulation can be seen as both a virtue and a vice. On the one hand, its concern with fundamental rights is a distinctive feature, setting it apart from other acts within the NLF. On the other hand, however, this approach presents a tremendous challenge, as it is not aligned with the complexity and non-technical nature of fundamental rights, which are best understood as positions of value that neither the vagueness of the AIA nor the loopholes of standardisation fully consider.


In our view, there is hope in the subsidiary role of common specifications (Article 41(1)(a)(iii)). These are tailored to address the shortcomings of harmonized standards with regard to fundamental rights concerns and should be drawn up by the European Commission (an EU institution) in consultation with the consultative forum (Article 67). According to the provisions of the AI Act, this forum should represent a balanced selection of stakeholders, including industry, start-ups, SMEs, civil society and academia. Furthermore, it should be balanced about commercial and non-commercial interests, and within the category of commercial interests, with regard to SMEs and other businesses. In addition to the European Union Agency for Cybersecurity (‘ENISA’), the European Committee for Standardisation (‘CEN’), the European Committee for Electrotechnical Standardisation (‘CENELEC’), and the European Telecommunications Standards Institute (‘ETSI’), the Fundamental Rights Agency is also a permanent member of the advisory forum.

In our view, common specifications provide an additional and important safeguard. Firstly, they ensure that gaps in protecting fundamental rights are filled. Secondly, their adoption addresses the lack of representation and inclusiveness in the standardisation process, including regarding the participation of business and civil society. Thirdly, implementing acts (just as delegated acts) are subject to judicial review by the Court of Justice of the EU, as a result of which there is no lack of control.

As we can see, the Artificial Intelligence Act is not without flaws. It introduces particular challenges to the basic principles of the rule of law and the substantive requirements applicable to legal acts. This is not only because of the regulatory model it adopts but also because of the (lack of) legitimacy and democratic deficit of the players and procedures participating in the normative creation entailed by the AI Act.

It is acknowledged that the law and legislative procedures are not impervious to technological advancement and that they must, therefore, be capable of adapting frameworks and embracing new procedures and instruments (of a technical nature if needed). The key is to ensure that, in this process of openness, the law is not conflated with technology, but that technology is shaped to align with the requirements and fundamental principles of the (rule of) law.

How to cite this article

Neves, I. (2024). A reappraisal of the AI Act in light of the qualitative requirements of the law. AI Policy Exchange Forum (AIPEX). https://doi.org/10.63439/TJZR2589