A computer-assisted needle misses its target, puncturing the spine. A diabetic patient goes rapidly downhill after a computer recommends an incorrect insulin dosage. An ultrasound fails to diagnose an obvious heart condition that is ultimately fatal.
These are just a few examples of incidents reported to the United States’ Food and Drug Administration involving health technology assisted by artificial intelligence (AI), and Australian researchers say they are an “early warning sign” of what could happen if regulators, hospitals and patients don’t take safety seriously in the rapidly evolving field.
“This is essentially showing us that when we’re putting in AI systems, we just need to be taking the safety of these systems really seriously,” Professor Farah Magrabi said.
Her team at the Australian Institute of Health Innovation at Macquarie University this month published a review of 266 safety events involving AI-assisted technology reported to the US watchdog. The article appeared in the Journal of the American Medical Informatics Association.
Only 16 per cent actually led to patients being harmed, but two-thirds were found to have the potential to cause harm, and 4 per cent were categorised as “near-miss events” in which users intervened.
Co-author Dr David Lyell said issues arose most commonly when users failed to enter the correct data, leading to an incorrect result, or misunderstood what the AI was actually telling them when it produced a result.
For example, one patient suffering a heart attack delayed medical care because an over-the-counter electrocardiogram device – which is not capable of detecting a heart attack – told them they had “normal sinus rhythm”.
“AI isn’t the answer; it’s part of a system that needs to support the provision of healthcare. And we do need to make sure that we have the systems in place that supports its effective use to promote healthcare for people,” Lyell said.
The researchers chose to analyse cases in the US, where the implementation of AI-enabled health devices is more advanced than in Australia. The US regulator has, to date, approved 521 artificial intelligence and machine learning-enabled medical devices, with 178 of those added in 2022.
In Australia, the Therapeutic Goods Administration (TGA) does not collect data about the number of approved devices in Australia that have AI or machine-learning components, but Magrabi said the regulator was taking the issue “very, very seriously”.
“AI isn’t the answer, it’s part of a system that needs to support the provision of health care.”
David Lyell, Australian Institute of Health Innovation, Macquarie University
AI has been used in healthcare devices for decades, but an explosion in data collection, computing power and advanced algorithms has opened up new frontiers, said David Hansen, chief executive of the CSIRO’s Australian E-Health Research Centre.
Sydney-based medical device start-up EMVision is one Australian company taking advantage of these advancements to develop a portable device for diagnosing stroke without the need for an MRI.
In the development stage, the company is using an advanced algorithm and high-powered computers to simulate stroke in numerous places in the brain, building up a database of synthetic images similar to MRIs which are then compared with real-life MRI and CT results from hospital clinical trials at Royal Melbourne, Liverpool and Princess Alexandra hospitals.
“We wouldn’t be able to do what we’re doing today if we didn’t have the [high-powered computer] infrastructure for the simulation,” said head of product development Forough Khandan.
Co-founder Scott Kirkland said the intention was not to completely replace CT and MRI scans, but to diagnose stroke in the first hour when treatment is most effective. The bedside device, set to be launched in 2025, will use a “traffic light” system based on a probability algorithm to help determine what type of stroke might have occurred.
“It’s better for an algorithm to give an ‘I don’t know’ than an incorrect answer, and have the wrong treatment and or triage process followed,” Kirkland said.
Radiology is at the forefront of the rapid adoption of AI in healthcare, especially in breast cancer screening and analysis of chest X-rays.
“A couple of years ago, almost no radiologist would say they use it, now a fair percentage would say that they use it in their daily work,” said clinical radiologist and AI safety researcher Dr Lauren Oakden Rayner.
Rayner, a member of the college, said the technology had many potential benefits, but Australian regulators and clinicians needed to better understand the risks of fully autonomous systems before putting them into hospitals, clinics and homes.
“Humans are legally and morally responsible for decision-making, and it’s taking some of that out of human hands,” she said. “There’s no reason autonomous AI systems can’t exist … but they obviously have to be tested very, very tightly.”
News
Cancer’s Deadly Paradox: How Tumors Break Their Own DNA To Keep Growing
Cancer’s strongest gene switches push DNA into damaging overdrive, creating repeated breaks and repairs that may fuel tumor evolution while exposing possible therapeutic weak spots. A new study indicates that cancer can harm its own genetic [...]
NanoMedical Brain/Cloud Interface – Explorations and Implications. A new book from Frank Boehm
New book from Frank Boehm, NanoappsMedical Inc Founder: This book explores the future hypothetical possibility that the cerebral cortex of the human brain might be seamlessly, safely, and securely connected with the Cloud via [...]
Our books now available worldwide!
Online Sellers other than Amazon, Routledge, and IOPP Indigo Global Health Care Equivalency in the Age of Nanotechnology, Nanomedicine and Artifcial Intelligence Global Health Care Equivalency In The Age Of Nanotechnology, Nanomedicine And Artificial [...]
Ryugu asteroid samples contain all DNA and RNA building blocks, bolstering origin-of-life theories
All the essential ingredients to make the DNA and RNA underpinning life on Earth have been discovered in samples collected from the asteroid Ryugu, scientists said Monday. The discovery comes after these building blocks [...]
Is Berberine Really a “Natural Ozempic”?
Often labeled a “natural Ozempic,” berberine is widely discussed as a metabolic aid. Yet research suggests its influence may lie deeper. In recent years, berberine has gained significant attention as a supposed “natural way” [...]
Viagra Ingredient Shows Promise for Rare Childhood Brain Disease in Surprising Study
A rare childhood disease with no approved treatment may have an unexpected new therapeutic candidate. Sildenafil, the active ingredient also sold under the brand name Viagra, may help reduce symptoms in people with Leigh [...]
In a first for China, Neuracle’s implantable brain-computer interface wins approval
In a landmark development, Neuracle Medical Technology has secured the country’s first-ever approval for an implantable brain-computer interface (BCI) system designed to restore hand motor function in patients with spinal cord injuries, in a [...]
A Cambridge Lab Mistake Reveals a Powerful New Way to Modify Drug Molecules
A surprising lab discovery reveals a light-powered way to tweak complex drugs faster, cleaner, and later in development. Researchers at the University of Cambridge have created a new technique for altering complex drug molecules [...]
New book from NanoappsMedical Inc – Molecular Manufacturing: The Future of Nanomedicine
This book explores the revolutionary potential of atomically precise manufacturing technologies to transform global healthcare, as well as practically every other sector across society. This forward-thinking volume examines how envisaged Factory@Home systems might enable the cost-effective [...]
Scientists Discover Simple Saliva Test That Reveals Hidden Diabetes Risk
Researchers have identified a potential new way to assess metabolic health using saliva instead of blood. High insulin levels in the blood, known as hyperinsulinemia, can reveal metabolic problems long before obvious symptoms appear. It is [...]
One Nasal Spray Could Protect Against COVID, Flu, Pneumonia, and More
A single nasal spray vaccine may one day protect against viruses, pneumonia, and even allergies. For decades, scientists have dreamed of creating a universal vaccine capable of protecting against many different pathogens. The idea [...]
New AI Model Predicts Cancer Spread With Incredible Accuracy
Scientists have developed an AI system that analyzes complex gene-expression signatures to estimate the likelihood that a tumor will spread. Why do some tumors spread throughout the body while others remain confined to their [...]
Scientists Discover DNA “Flips” That Supercharge Evolution
In Lake Malawi, hundreds of species of cichlid fish have evolved with astonishing speed, offering scientists a rare opportunity to study how biodiversity arises. Researchers have identified segments of “flipped” DNA that may allow fish to adapt rapidly [...]
Scientists Discover Why Some COVID Survivors Still Can’t Taste Food Years Later
A new study provides the first direct biological evidence explaining why some people continue to experience taste loss long after recovering from COVID-19. Researchers have uncovered specific biological changes in taste buds that could help [...]
Catching COVID significantly raises the risk of developing kidney disease, researchers find
Catching Covid significantly raises the risk of developing deadly kidney disease, research has shown. The virus was found to increase the chances that patients will develop the incurable condition by around 50 per cent. [...]
New Toothpaste Stops Gum Disease Without Harming Healthy Bacteria
Researchers have developed a targeted approach to combat periodontitis without disrupting the natural balance of the oral microbiome. The innovation could reshape how gum disease is treated while preserving beneficial bacteria. The human mouth [...]
