Ability of researchers' new AI tool to predict medical events
A new study led by the Institute of Psychiatry, Psychology & Neuroscience (IoPPN) at King’s College London and using South London and Maudsley patient data has demonstrated the potential of an AI tool developed to predict the health trajectory of patients by forecasting future disorders, symptoms, medications and procedures.
According to the team of researchers at King’s College London, UCL, King’s College Hospital NHS Foundation Trust and Guy’s and St Thomas’ NHS Foundation Trust, these results indicate the tool could be used to support clinical decision-making, monitoring in healthcare settings and to improve clinical trials.
The tool – called Foresight – is trained on existing healthcare data and uses a deep learning approach to recognise complex patterns in both the structured and unstructured data of electronic health records to produce insights and predictions.
It belongs to the same family of AI models as ChatGPT but, in contrast to ChatGPT which is trained on publicly available information, Foresight is trained on information from NHS electronic health records.
The study, published in The Lancet Digital Health, investigated the accuracy of Foresight’s medical predictions by comparing them to what actually happened to the patients as described in their records.
Using data from over 811,000 patients across King’s College Hospital NHS Foundation Trust, South London and Maudsley NHS Foundation Trust, and MIMIC-III – a publicly available dataset of patients from Beth Israel Deaconess Medical Center in the US, researchers trained three different models of Foresight.
Researchers extracted and processed the unstructured (free-text) and structured data (age, ethnicity and sex) within electronic health records using CogStack. The datasets were used to train Foresight and performance was measured by comparing its predictions on true outcomes in a smaller subset of these data. Foresight was trained on data under NHS and patient governance and inside the hospital NHS firewall.
Precision forecasting
When forecasting the next 10 possible disorders that could appear next in a patient timeline, the tests showed that Foresight correctly identified the next disorder 68 per cent and 76 per cent of the time in two UK NHS Trusts (King’s College Hospital NHS Foundation Trust and South London and Maudsley NHS Foundation Trust) and 88 per cent of the time in the US MIMIC-III dataset. Similarly, when forecasting the next new biomedical concept which could be a disorder, symptom, relapse or medication, the precision achieved by Foresight was 80 per cent, 81 per cent and 91 per cent respectively.
Zeljko Kraljevic, first author and Research Fellow in Health Informatics at King's IoPPN, said:
Our study shows that Foresight can achieve high levels of precision in predicting health trajectories of patients, demonstrating it could be a valuable tool to aid decision making and inform clinical research. The proposed purpose of Foresight is not to enable patients to self-diagnose or predict their future, but it could potentially be used as an aid by clinicians to make sure a diagnosis is not missed or for continual patient monitoring for real-time risk prediction. One of the main advantages of Foresight that it can easily scale to more patients, hospital or disorders with minimal or no modifications, and the more data it receives the better it gets.
The accuracy of Foresight’s predictions was also assessed by clinicians. Five clinicians developed 34 mock patient timelines with simulated scenarios. When all five clinicians agreed on the forecasted medical event, the predictions Foresight provided were 93 per cent relevant, meaning they made sense from a clinical perspective.
Uses of Foresight
Foresight can be used for real-world risk forecasting, emulating trials and clinical research to study the progression of disorders, simulate interventions and lifestyle factors and educational purposes.
Professor Richard Dobson, senior author, Professor of Medical Informatics at King’s IoPPN and Theme Lead for Informatics at the NIHR Maudsley BRC, said:
Foresight opens the door for many applications such as digital health twins, synthetic dataset generation, real world risk forecasting, longitudinal research, emulation of trials, medical education and more. It is an exciting time for AI in healthcare and to develop effective tools we must ensure that we use appropriate data to train our models and work towards a shared purpose of supporting healthcare systems to support patients.
Professor Dobson, who is also supported by the NIHR BRC at University College London Hospitals NHS Foundation Trust (UCLH), added: “Foresight will improve with more real-world data, and we are now looking for more hospitals to be involved in developing Foresight 2, a more accurate language model.”
Co-author Professor James Teo, Joint Director for Data Science and Artificial Intelligence at King's College Hospital NHS Foundation Trust and Guy's & St Thomas' Hospital NHS Foundation Trust, and Professor of Neurology, King's College London said "Our study demonstrates the future potential of AI technologies in healthcare and how AI can be developed by NHS and universities in a safe regulated way for society's benefit; I am very proud of the support that our patients, our staff and our NHS provided CogStack along the way."
Professor Andrew Morris, Director of Health Data Research UK (HDR UK), said:
AI has immense potential to enable scientific discovery, support the prevention, diagnosis and treatment of disease, and improve care pathway management and education. This work demonstrates the potential that using electronic health records and the latest advancements in AI could deliver for improving clinical decision making. But this is all dependent on the quality and representativeness of data. Sustained investment in the UK's data infrastructure is needed to enable this in safe and secure ways to maintain privacy and anonymity of people's health data.
HDR UK aims to accelerate trustworthy data use. This study illustrates that the free-text portion of electronic health records, which are currently underutilised in research, hold vital information for understanding health outcomes.
Foresight is part of CogStack, an information retrieval and extraction platform developed by researchers at King’s College London, King’s College Hospital NHS Foundation Trust, Guy’s and St Thomas’ NHS Foundation Trust, South London and Maudsley NHS Foundation Trust, UCL and UCLH. It uses natural language processing to harness NHS electronic health record data to support clinical decision-making and health research.
This paper was part of a programme of work that received funding from the NHS AI Lab, National Institute for Health and Care Research and Health Data Research UK. It has infrastructure support from King’s College Hospital NHS Foundation Trust, NIHR Maudsley BRC and the London AI Centre for Value-Based Healthcare.
‘Foresight - Generative Pretrained Transformer (GPT) for Modelling of Patient Timelines using EHRs: A Retrospective Study’ was published in The Lancet Digital Health.
