24 June 2022 | by Hannah van KOLFSCHOOTEN & Janneke van OIRSCHOT
A first win for HAI in the AI Act negotiation process: Many amendments we proposed to make the AI Act health-centric were tabled by committee members. This blog explains why health deserves priority and why MEPs should keep it squarely in sight during upcoming negotiations and votes.
Last year, the European Commission published its long awaited legislative proposal on artificial intelligence (AI): the Artificial Intelligence Act (AIA). Since then, a heated debate has been going on about its contents, as many civil society organisations believe the proposal falls short on fundamental rights protection. Various committees within the European Parliament have proposed amendments to the Commission’s AI proposal (e.g. ENVI, JURI). After an initial report with amendments from the two rapporteurs, it was up to the LIBE/IMCO Committee members to table amendments, and well, they did. In total, MEPs tabled the astronomical number of 3312 amendments. We were glad to find amongst them multiple amendments proposed by HAI. While this is a promising first step to bring health into the picture, it doesn’t mean that they will make the cut to the final act, as there are multiple negotiation and voting processes ahead. It is essential that the AI Act takes a more health-centric approach and that the health amendments make it to the final Act. We call on MEPs to prioritise health when negotiating their amendments over the coming months. Read below why the health sector needs special attention.
What do we talk about when we talk about health-related AI?
AI is the simulation of human intelligence by machines. AI systems are software-based technologies that use certain data-driven approaches to solve specific problems. In the health sector, AI has various applications. One can think of the use of AI systems to predict diagnosis and the best course of treatment, but also for allocation of hospital beds and best placement of ambulances. Governments can use AI systems for the prediction of the spread of infectious diseases and health insurers can deploy AI systems for fraud detection. What all AI systems have in common, is that they recognise patterns in enormous amounts of data. The outcome of the system – e.g. the prediction or decision – depends on the data that was used for development. It’s often difficult to determine how the system reached a specific outcome.
What are the key risks of health-related AI for patients and health?
• Bias in training data may lead to discrimination and individual injury/death (i.e., racial bias may lead to incorrect diagnoses); and deepen existing socio-economic inequalities;
• Technical system errors in AI could lead to mass patient injuries because of widespread use;
• Increased use and sharing of health data threatens privacy and data protection rights of patients;
• Lack of transparency and explainability threatens patients’ rights to information and to informed consent to medical treatment;
• Lack of clarity on accountability threatens patients’ rights to legal redress and compensation;
• Lack of regulation of health-related AI may lead to a lack of trust in healthcare; affecting access to healthcare;
• Issues with cybersecurity threaten patients’ health in the case of cyberattacks on for example insulin pumps and pacemakers.
How should we address these risks in the AI Act Proposal?
Health is one of the industries that will be most affected by AI and one of the most popular sectors for AI deployment in the EU. Yet, the AIA Proposal does not specifically address the health sector and does not provide solutions for the key risks of health-related AI for the health and rights of patients.
To address these risks, we propose the following main amendments to the AIA Proposal:
- Signal risks of AI for health, patients’ rights and access to healthcare in the preamble.
- Adhere to the 1948 WHO definition of health: “Health is a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity.”
- Classify all health-related AI systems as ‘high risk’ in Annex III (i.e. public health, pharmaceuticals and wellbeing).
- Include individual rights in the AIA + mirrored corresponding obligations for providers (i.e. the right to object to automated decision-making in healthcare, the right to an effective remedy, obligation to conduct a fundamental rights impact assessment).
- Extend the prohibition on social scoring to include private actors (i.e. health insurance companies).
- Extend the prohibition on remote biometric identification to all actors (i.e. public health authorities).
We are happy that many of these amendments are now tabled. We hope to gather broad support for these amendments in the coming months, before the vote in plenary takes place at the end of 2022. Read our report: https://haiweb.org/publication/interpreting-the-eu-artificial-intelligence-act-for-the-health-sector/