The increasing digitalisation of healthcare is reshaping how healthcare professionals approach clinical tasks and patient interaction. This is occurring against the backdrop of systemic pressures that the healthcare sector is facing due to workforce shortages and an aging population, among other factors. In this context, generative AI (GenAI) has the capacity to help healthcare providers with clinical documentation, decision-making, and patient communication through automated processes. However, despite the potential, GenAI also carries fundamental risks to patients’ rights.

Risks to patients

As with AI more widely, the increasing use of GenAI poses certain threats to patients’ rights. One of these risks relates to data privacy, whereby GenAI models may leak private health data, or other forms of data breaches. Read the full background brief for more risks to patients. 

Reminders for healthcare providers and for patients

• Patients are entitled to know where their health data is stored and for what purposes.
• Patients have the right to object to their health data being shared and used for secondary purposes (e.g., research and commercial use).
• Healthcare institutions should be encouraged to develop internal policies and guidelines regulating the use of GenAI. These should align with existing frameworks and should ensure patients’ rights.
• There is a need for transparency procedures so that patients can easily obtain information on how and where GenAI is being used in care.
• Healthcare institutions must always state where clinical documentation was generated through the use of GenAI, and alert patients to the possibility of mistakes. Systems may misinterpret information, relay false information, be inconsistent, or even hallucinate and produce information that had not been previously discussed.
• Healthcare institutions must ensure that consent procedures disclose the use of GenAI and provide clear opportunities for patients to object.
• Sensitive health data must not be shared with commercial GenAI parties.

See also: HAI’s work on AI and mental health, including on the use of chatbots.

Download the full background brief (PDF)