Report | February 2022 | Download PDF
The incredible increase in digital data covering every aspect of our lives, both within and outside the health system, provides a myriad of new opportunities to monitor our health, learn about causes of disease, develop new therapies, and influence lifestyles. At the moment, artificial intelligence (AI) systems are penetrating the health sector and are increasingly successful at diagnosis using scans and images; repurposing or development of new therapies; patient triaging; care planning; and increasing administrative efficiency. However, its use in clinical practice remains limited to date, and the breakthrough developments that will revolutionise healthcare are yet to be seen.
Expanding boundaries of the type of data that can make inferences about an individual’s health, and who has access to these data, raises concerns about our autonomy and privacy (Ada Lovelace Institute, 2020; IDB, 2020). An example of this is the use of our health data by pharmaceutical and tech companies for commercial purposes. On top of that, healthcare data is historically full of flaws and biases. When it is used to build AI systems, it can cause discrimination and perpetuate inequities (Leslie et al., 2021). Finally, an important concern is the question who benefits from technological advancements and how they impact on national and international systematic inequities.
It is essential to protect citizens from the potential harms of AI technologies and realise their safety and accountability. That is why, in April 2021, the European Commission presented a regulatory proposal for AI in the European Union (EU). Today, more than ever, individual health and public health systems are of paramount importance to EU Member States, so it is crucial to understand what impact any new AI regulation would have on both the health sector and AI innovation itself. In this report, we will dismantle and interpret the regulatory proposal to determine when a health-related AI system would be managed and controlled and what regulatory requirements it would be subject to, in order to identify the gaps and weaknesses of the EU Artificial Intelligence Act (AIA) regulatory proposal.
Throughout this paper we use the terms health-related AI and AI systems used in the health sector interchangeably to mean a broad collection of all AI systems used within the formal healthcare sector and those used outside formal healthcare with the aim to influence people’s health and wellbeing.
Download the report here (PDF)
We have also produced a short handout on the recommendations for addressing the risk and harms of health-related AI. Download a copy