Use of Generative AI in Internal Medicine
Requires strong safeguards to maintain the security of patient data.
Machine learning, the use of self-adjusting computer-generated algorithms to recognise patterns in bodies of data, is widely known as artificial intelligence (AI).
Generative AI, like all AI, relies on pattern recognition but is a step up from earlier forms of AI by its use of “large language models” to scan enormous bodies of data for whole phrases and concepts rather than just single words. Examples of generative AI include Open AI’s ChatGPT series and Google’s Bard.
The true uniqueness of generative AI and the reason that it is described as “generative” is because it has the ability to create text, images and other content from the data on which it has been trained, which in practice includes much of the internet.
However this data is uncurated, meaning that outdated, biased and untrue information is often included in the data being analysed by the generative AI algorithms.
It remains to be seen whether, as some think, generative AI will increase and democratise human creativity and innovation. The benefits of other technologies in improving industrial productivity in the recent past although significant, have been hard to measure compared with investment in human and physical capital, and some in the commercial world have serious doubts about the practical usefulness of generative AI technology to them.
Further, we know that technological advances often end up, for good or ill, being used for purposes other than those for which they were conceived. Indeed the internet itself, a prime example of repurposing, is a long way from its original development as a method of creating a secure distributed network of military computers.
If one adds to these uncertainties the rapid rate of development of generative AI, the “black box” uncertainties of how it reaches its conclusions as well as differences amongst its developers about how stringently the technology should be controlled (reflected in recent organisational chaos in the OpenAI company which developed the ChatGPT series of generative AIs) then the future usefulness and uses of generative AI become hard to predict with any accuracy.
Generative AI carries a significant risk of producing misinformation
Generative AI creates a wide range of content by drawing on much of the internet but currently does not have the ability to curate out conspiracy theories, wishful thinking and deliberate misinformation that may be present in social media and other sources. These biases may contribute to absurd outputs (called “hallucinations” by data scientists) and release of false probabilities by the AI. The biased products of the generative AI algorithms are then fed back into the internet to act as part of the dataset for other generative AI searches, creating a self-amplifying spiral of bias and falsehood.
It is easy to see how generative AI, applied uncritically to situations of clinical uncertainty, can set the stage for a spiral of misinformation and wrong conclusions.
Use of generative AI in internal medicine
AI of all types can be used in internal medicine for clinical purposes ie for the direct benefit of individual patients, for analysis of “big data” and for research, both clinical and laboratory.
The continuing relevance of the statement by Sir William Osler well over a century ago that medicine is a science of uncertainty and an art of probability acknowledges that the ability to make decisions based on incomplete information is a key skill in clinical practice. The best doctors possess that skill.
Generative AI appears to offer an attractive means of resolving much clinical uncertainty. However while widely-available generative AIs may produce (for example) excellent management plans and discharge letters based on templates, such AIs will feed the patient data in such plans and letters back into the internet because the generative AI’s developers have designed it to do so.
If the generative AI could confine its activity to analysis of the electronic patient records (EPRs) and other patient data systems of an individual hospital or group of hospitals, the risk of data escape to the internet would be reduced. To prevent such data escape might require redesign of some hospital IT systems. Additionally, the training database of hospital EPRs and other patient data systems is tiny (although much more specific and accurate) compared with the internet.
In contrast if one puts a “hypothetical” question (ie without giving any patient details beyond gender, approximate age, symptoms, a medication list and a curated past medical history) to an openly-accessible generative AI such as ChatGPT or Bard about a group of mystifying symptoms, that AI will create an excellent differential diagnosis. Indeed the GPT-4 generative AI applied to complex medical case challenges published in the New England Journal of Medicine between 2017-2023 performed at least as well as, and probably better than, the human readers of that journal in identifying the final diagnosis.
Clinical confidentiality and data security
As described earlier, it is already possible in some cases using non-generative AI (or no AI) to re-identify patients whose identity has been pseudonymised or even in some cases anonymised, by combining their clinical data with open source data. Generative AI with its greater analytical power and ability to scan huge amounts of internet data may lower the barriers to patient re-identification.
For example, an entry in a patient’s EPR on a specific date for depression because their “ son is on trial for murder” could lead to reconstruction of the patient’s identity by searching media or court records for murder cases around the time of that consultation.
Thus to ensure anonymity, patient data must be curated and de-identified before machine analysis to remove not only patients’ personal identifiers such as their name, date of birth and address, but also their wider social and clinical associations including family members, country of origin (and sometimes names of countries visited), the names of their doctors and hospitals. Additionally some very rare or unusual conditions or causes that could lead to identification of an individual may also need to be excluded.
Wearable devices
The confidentiality and ownership of personal biodata gathered by wearable devices that download data to mobile phone apps linked to the manufacturer’s databases for analysis by AI is a complex topic. Many users may be unaware when purchasing these devices that their personal data may be sold on to advertisers, or on occasions inadvertently released.
However similar apps supplied by the NHS that interact with an individual’s hospital EPR are covered by the stringent data protections in the UK Data Protection Act 2018.
Generative AI and accountability of doctors undertaking clinical research
Researchers who analyse patient data for purposes other than informing decisions about individual patients are required by research ethics committees to de-identify and where possible to aggregate the data from a number of patients to exclude any possibility of the re-identification of the data of those individuals.
Breaches of confidentiality are not the only risk of using generative AI in research. It has long been a requirement for anyone involved in scientific research and analysis to maintain their professional integrity by actively avoiding invention of facts and deliberate bias in their work or the work of those whom they supervise.
Anyone straying from this standard risks professional condemnation or censure. If they are a medical professional they face disciplinary measures (including possible erasure) by their medical licensing body.
This is why generative AI does not make human expertise redundant. As is the case with medical use of any machine or device, doctors who use generative AI are responsible for the adverse consequences of its use. Those consequences may be considerable given the technology’s proneness to nonsensical outputs and bias.
Thus before generative AIs can be safely used in healthcare, all results derived with their assistance need to be “sense checked” by people knowledgeable and experienced in the field. Such results should contain some form of “watermarking”, and be identified as AI-generated in publications and presentations.
Keeping data safe when using generative AI in clinical practice
As discussed earlier, the only safe way to use any form of AI in healthcare is to ensure that patient data cannot leave healthcare systems if there is any possibility of re-identifying patients, including when their data is combined with open source data. That requires the ability to curate and anonymise all patient data before analysis.
For this reason although generative AI may ultimately prove to be far superior to any other means of making complex diagnoses it is difficult to see how generative AI can safely be used to create individual patient discharge summaries or case notes without robust, easily enforceable measures to prevent breaches of patient data confidentiality. This becomes particularly-relevant when NHS patient data on the NHS federated data platform is analysed by organisations outside the NHS.
Additionally, in order to fulfil their duty to maintain patient confidentiality doctors who release their patients’ data for electronic analysis have to satisfy themselves that it is protected against re-identification and leakage. This could add to their administrative burden. A recent Canadian study (performed on data gathered before the introduction of generative AI) showed an association between usage of existing EPRs and clinician burnout, although that might be less of a problem in EPR systems like those in the NHS where clinicians are not currently required to enter copious billing information.
Difficulties with large-scale patient data collection and curation
Curation of digitally-stored patient data is currently difficult in the UK because although 90% of NHS patient notes have now been digitised in some way, it is not clear if any of the EPRs used in the NHS are properly set up for automated curation of data that could be used to re-identify patients. GP electronic records have their own curation challenges.
Large-scale clinical data curation is undertaken in the UK for patient registries that are databases set up for analysis of the course and consequences of specific diseases, exposures, or for monitoring treatments. Data in such registries is often pseudonymised (ie patients can be re-identified if appropriate identity codes are available) but some may be anonymised (ie re-identification is impossible).
Examples include specialty society registries such as the British Society for Rheumatology Rheumatoid Arthritis Register, the UK National Joint Registry and the British Spine Registry and many others.
As any clinician will attest, data entry into these registries is often time consuming and laborious, sometimes requiring the recruitment of specially-trained assistants.
In an ideal world automatically-curated patient data would be easily transferred from the EPR to the registry or for use in other research, allowing safe large scale analysis. In reality such automated data curation (albeit not yet of free text) has only recently come into very limited use.
For these reasons, although the NHS as a unitary service with a high level of usage of EPRs should in theory be well set up for large-scale data analysis, there is still a long and obstacle-strewn road to travel before clinical data in the NHS can realise its potential as a vast source of easily-analysable data with a negligible risk of breaches of confidentiality.
In internal medicine we should start by using generative AI to pick the lowest-hanging fruit, producing differential diagnoses on the anonymised data of challenging individual cases, while researchers gradually develop and refine the technology of automated curation of complex electronic patient records and production of forms of generative AI that are safe to use in closed data systems like those of the NHS.
The use of generative AI in internal medicine will only benefit patients if does not threaten the security of their data.
Generative AI was not used in writing or editing this blog.