A computer-assisted needle misses its target, puncturing the spine. A diabetic patient goes rapidly downhill after a computer recommends an incorrect insulin dosage. An ultrasound fails to diagnose an obvious heart condition that is ultimately fatal.
These are just a few examples of incidents reported to the United States’ Food and Drug Administration involving health technology assisted by artificial intelligence (AI), and Australian researchers say they are an “early warning sign” of what could happen if regulators, hospitals and patients don’t take safety seriously in the rapidly evolving field.
“This is essentially showing us that when we’re putting in AI systems, we just need to be taking the safety of these systems really seriously,” Professor Farah Magrabi said.
Her team at the Australian Institute of Health Innovation at Macquarie University this month published a review of 266 safety events involving AI-assisted technology reported to the US watchdog. The article appeared in the Journal of the American Medical Informatics Association.
Only 16 per cent actually led to patients being harmed, but two-thirds were found to have the potential to cause harm, and 4 per cent were categorised as “near-miss events” in which users intervened.
Co-author Dr David Lyell said issues arose most commonly when users failed to enter the correct data, leading to an incorrect result, or misunderstood what the AI was actually telling them when it produced a result.
For example, one patient suffering a heart attack delayed medical care because an over-the-counter electrocardiogram device – which is not capable of detecting a heart attack – told them they had “normal sinus rhythm”.
“AI isn’t the answer; it’s part of a system that needs to support the provision of healthcare. And we do need to make sure that we have the systems in place that supports its effective use to promote healthcare for people,” Lyell said.
The researchers chose to analyse cases in the US, where the implementation of AI-enabled health devices is more advanced than in Australia. The US regulator has, to date, approved 521 artificial intelligence and machine learning-enabled medical devices, with 178 of those added in 2022.
In Australia, the Therapeutic Goods Administration (TGA) does not collect data about the number of approved devices in Australia that have AI or machine-learning components, but Magrabi said the regulator was taking the issue “very, very seriously”.
“AI isn’t the answer, it’s part of a system that needs to support the provision of health care.”
David Lyell, Australian Institute of Health Innovation, Macquarie University
AI has been used in healthcare devices for decades, but an explosion in data collection, computing power and advanced algorithms has opened up new frontiers, said David Hansen, chief executive of the CSIRO’s Australian E-Health Research Centre.
Sydney-based medical device start-up EMVision is one Australian company taking advantage of these advancements to develop a portable device for diagnosing stroke without the need for an MRI.
In the development stage, the company is using an advanced algorithm and high-powered computers to simulate stroke in numerous places in the brain, building up a database of synthetic images similar to MRIs which are then compared with real-life MRI and CT results from hospital clinical trials at Royal Melbourne, Liverpool and Princess Alexandra hospitals.
“We wouldn’t be able to do what we’re doing today if we didn’t have the [high-powered computer] infrastructure for the simulation,” said head of product development Forough Khandan.
Co-founder Scott Kirkland said the intention was not to completely replace CT and MRI scans, but to diagnose stroke in the first hour when treatment is most effective. The bedside device, set to be launched in 2025, will use a “traffic light” system based on a probability algorithm to help determine what type of stroke might have occurred.
“It’s better for an algorithm to give an ‘I don’t know’ than an incorrect answer, and have the wrong treatment and or triage process followed,” Kirkland said.
Radiology is at the forefront of the rapid adoption of AI in healthcare, especially in breast cancer screening and analysis of chest X-rays.
“A couple of years ago, almost no radiologist would say they use it, now a fair percentage would say that they use it in their daily work,” said clinical radiologist and AI safety researcher Dr Lauren Oakden Rayner.
Rayner, a member of the college, said the technology had many potential benefits, but Australian regulators and clinicians needed to better understand the risks of fully autonomous systems before putting them into hospitals, clinics and homes.
“Humans are legally and morally responsible for decision-making, and it’s taking some of that out of human hands,” she said. “There’s no reason autonomous AI systems can’t exist … but they obviously have to be tested very, very tightly.”
News
New pancreatic cancer drug carries a $39,800 monthly list price
A groundbreaking treatment for one of the most common forms of pancreatic cancer has been approved in pill form by the FDA. Revolution Medicines’ oral tablet daraxonrasib, branded as Rasonque, reduced the risk of [...]
Researchers Have Discovered a New Way To Reduce Chronic Nerve Pain
A cancer-linked protein called BRAF may help drive chronic nerve pain, and existing cancer drugs targeting it reduced pain sensitivity in preclinical models. Chronic nerve pain can persist long after an injury and often [...]
Our books now available worldwide!
Online Sellers other than Amazon, Routledge, and IOPP Indigo Global Health Care Equivalency in the Age of Nanotechnology, Nanomedicine and Artifcial Intelligence Global Health Care Equivalency In The Age Of Nanotechnology, Nanomedicine And Artificial [...]
Quantum-Enabled Regenerative Health: Reimagining Wellness, Precision Health and Longevity Medicine
Introduction Healthcare is approaching a frontier where the quantum portfolio could influence not only how disease is diagnosed and treated, but how health itself is measured, modeled, predicted and preserved. Quantum computing, quantum simulation, [...]
FDA Clears First-of-Its-Kind Nonmedication Treatment for PTSD
The FDA has cleared a system that uses brain activity data to personalize magnetic stimulation for PTSD, adding a new nonmedication treatment option. Every day in the United States, approximately 17.5 veterans die by suicide, [...]
FDA approves breakthrough drug to treat advanced pancreatic cancer
The Food and Drug Administration (FDA) approved on Wednesday a drug that could extend the survival of those with metastatic pancreatic cancer. The drug, called daraxonrasib, will be sold under the brand name Rasonque [...]
AI Decodes a Hidden DNA Signal Linked to Disease-Causing Mutations
Machine learning identifies the likely “initiator” and enables new predictions about DNA mutations that can cause disease. Every human cell depends on tens of thousands of genes being switched on at the right time [...]
Pope Leo Urges Global Response to Congo’s Deadliest Ebola Outbreak
Pope Leo called for international action to address the Ebola outbreak in the Democratic Republic of Congo. The epidemic has claimed over 2,500 lives and is the nation's largest recorded outbreak. The Pope emphasized [...]
Is there a summer COVID-19 surge this year? Yep, it’s ramping up again
Hantavirus. Ebola. West Nile. Measles. And, of course, cyclospora — that stomach parasite making people miserable across the country. Americans have plenty to worry about this summer. But remember COVID-19? It may not be [...]
Natural Compound in Broccoli Could Help Treat a Rare, Incurable Neurological Disease
Swinburne researchers have discovered how broccoli could help treat an extremely rare and incurable disease that affects only about 200 Australians. A naturally occurring compound in broccoli is being investigated as a possible treatment [...]
Antibody recycling, FcRn and the next generation of biologics
FcRn plays a central role in regulating the half-life of IgG antibodies and albumin, making it a critical target in both antibody engineering and autoimmune disease therapy. This article explores the biology of FcRn, [...]
Ebola kills 2,300 in three months with no vaccine available for Congo’s deadliest ever outbreak
It’s Congo’s 17th outbreak, with a toll that has eclipsed that of the country’s 2018-2020 outbreak when 2,299 deaths out of 3,481 cases were recorded. The latest data shows 101 new cases and 33 [...]
Beyond an Agency: How AI, Creativity and Commercialization Are Converging Into a New Growth Model
At a moment when AI is forcing a fundamental rethink of how work gets done, the life sciences industry is grappling with a deeper question: what role should human creativity play in an increasingly [...]
New $60 Blood Test Detects Lung Cancer Without Costly DNA Sequencing
A simple blood test that detects chemical changes in DNA identified more than 90% of stage 2-4 lung cancers in an early study, without relying on costly DNA sequencing. Researchers at Tel Aviv University and collaborating [...]
Reducing drug development failures with human-relevant models and AI
Professor Joseph C. Wu of Stanford University explains how stem cells, human-relevant models and AI are helping researchers predict which drug candidates are most likely to succeed before clinical trials. Researchers now have access [...]
Half of vaccines are binned – ‘fridge-free’ versions could change that
Vaccines that do not need to be kept cool in a fridge have been successfully trialed in patients for the first time, say UK scientists. Currently, most vaccines need to be refrigerated or frozen [...]
