A computer-assisted needle misses its target, puncturing the spine. A diabetic patient goes rapidly downhill after a computer recommends an incorrect insulin dosage. An ultrasound fails to diagnose an obvious heart condition that is ultimately fatal.
These are just a few examples of incidents reported to the United States’ Food and Drug Administration involving health technology assisted by artificial intelligence (AI), and Australian researchers say they are an “early warning sign” of what could happen if regulators, hospitals and patients don’t take safety seriously in the rapidly evolving field.
“This is essentially showing us that when we’re putting in AI systems, we just need to be taking the safety of these systems really seriously,” Professor Farah Magrabi said.
Her team at the Australian Institute of Health Innovation at Macquarie University this month published a review of 266 safety events involving AI-assisted technology reported to the US watchdog. The article appeared in the Journal of the American Medical Informatics Association.
Only 16 per cent actually led to patients being harmed, but two-thirds were found to have the potential to cause harm, and 4 per cent were categorised as “near-miss events” in which users intervened.
Co-author Dr David Lyell said issues arose most commonly when users failed to enter the correct data, leading to an incorrect result, or misunderstood what the AI was actually telling them when it produced a result.
For example, one patient suffering a heart attack delayed medical care because an over-the-counter electrocardiogram device – which is not capable of detecting a heart attack – told them they had “normal sinus rhythm”.
“AI isn’t the answer; it’s part of a system that needs to support the provision of healthcare. And we do need to make sure that we have the systems in place that supports its effective use to promote healthcare for people,” Lyell said.
The researchers chose to analyse cases in the US, where the implementation of AI-enabled health devices is more advanced than in Australia. The US regulator has, to date, approved 521 artificial intelligence and machine learning-enabled medical devices, with 178 of those added in 2022.
In Australia, the Therapeutic Goods Administration (TGA) does not collect data about the number of approved devices in Australia that have AI or machine-learning components, but Magrabi said the regulator was taking the issue “very, very seriously”.
“AI isn’t the answer, it’s part of a system that needs to support the provision of health care.”
David Lyell, Australian Institute of Health Innovation, Macquarie University
AI has been used in healthcare devices for decades, but an explosion in data collection, computing power and advanced algorithms has opened up new frontiers, said David Hansen, chief executive of the CSIRO’s Australian E-Health Research Centre.
Sydney-based medical device start-up EMVision is one Australian company taking advantage of these advancements to develop a portable device for diagnosing stroke without the need for an MRI.
In the development stage, the company is using an advanced algorithm and high-powered computers to simulate stroke in numerous places in the brain, building up a database of synthetic images similar to MRIs which are then compared with real-life MRI and CT results from hospital clinical trials at Royal Melbourne, Liverpool and Princess Alexandra hospitals.
“We wouldn’t be able to do what we’re doing today if we didn’t have the [high-powered computer] infrastructure for the simulation,” said head of product development Forough Khandan.
Co-founder Scott Kirkland said the intention was not to completely replace CT and MRI scans, but to diagnose stroke in the first hour when treatment is most effective. The bedside device, set to be launched in 2025, will use a “traffic light” system based on a probability algorithm to help determine what type of stroke might have occurred.
“It’s better for an algorithm to give an ‘I don’t know’ than an incorrect answer, and have the wrong treatment and or triage process followed,” Kirkland said.
Radiology is at the forefront of the rapid adoption of AI in healthcare, especially in breast cancer screening and analysis of chest X-rays.
“A couple of years ago, almost no radiologist would say they use it, now a fair percentage would say that they use it in their daily work,” said clinical radiologist and AI safety researcher Dr Lauren Oakden Rayner.
Rayner, a member of the college, said the technology had many potential benefits, but Australian regulators and clinicians needed to better understand the risks of fully autonomous systems before putting them into hospitals, clinics and homes.
“Humans are legally and morally responsible for decision-making, and it’s taking some of that out of human hands,” she said. “There’s no reason autonomous AI systems can’t exist … but they obviously have to be tested very, very tightly.”
News
Lab-grown corticospinal neurons offer new models for ALS and spinal injuries
Researchers have developed a way to grow a highly specialized subset of brain nerve cells that are involved in motor neuron disease and damaged in spinal injuries. Their study, published today in eLife as the final [...]
Urgent warning over deadly ‘brain swelling’ virus amid fears it could spread globally
Airports across Asia have been put on high alert after India confirmed two cases of the deadly Nipah virus in the state of West Bengal over the past month. Thailand, Nepal and Vietnam are among the [...]
This Vaccine Stops Bird Flu Before It Reaches the Lungs
A new nasal spray vaccine could stop bird flu at the door — blocking infection, reducing spread, and helping head off the next pandemic. Since first appearing in the United States in 2014, H5N1 [...]
These two viruses may become the next public health threats, scientists say
Two emerging pathogens with animal origins—influenza D virus and canine coronavirus—have so far been quietly flying under the radar, but researchers warn conditions are ripe for the viruses to spread more widely among humans. [...]
COVID-19 viral fragments shown to target and kill specific immune cells
COVID-19 viral fragments shown to target and kill specific immune cells in UCLA-led study Clues about extreme cases and omicron’s effects come from a cross-disciplinary international research team New research shows that after the [...]
Smaller Than a Grain of Salt: Engineers Create the World’s Tiniest Wireless Brain Implant
A salt-grain-sized neural implant can record and transmit brain activity wirelessly for extended periods. Researchers at Cornell University, working with collaborators, have created an extremely small neural implant that can sit on a grain of [...]
Scientists Develop a New Way To See Inside the Human Body Using 3D Color Imaging
A newly developed imaging method blends ultrasound and photoacoustics to capture both tissue structure and blood-vessel function in 3D. By blending two powerful imaging methods, researchers from Caltech and USC have developed a new way to [...]
Brain waves could help paralyzed patients move again
People with spinal cord injuries often lose the ability to move their arms or legs. In many cases, the nerves in the limbs remain healthy, and the brain continues to function normally. The loss of [...]
Scientists Discover a New “Cleanup Hub” Inside the Human Brain
A newly identified lymphatic drainage pathway along the middle meningeal artery reveals how the human brain clears waste. How does the brain clear away waste? This task is handled by the brain’s lymphatic drainage [...]
New Drug Slashes Dangerous Blood Fats by Nearly 40% in First Human Trial
Scientists have found a way to fine-tune a central fat-control pathway in the liver, reducing harmful blood triglycerides while preserving beneficial cholesterol functions. When we eat, the body turns surplus calories into molecules called [...]
A Simple Brain Scan May Help Restore Movement After Paralysis
A brain cap and smart algorithms may one day help paralyzed patients turn thought into movement—no surgery required. People with spinal cord injuries often experience partial or complete loss of movement in their arms [...]
Plant Discovery Could Transform How Medicines Are Made
Scientists have uncovered an unexpected way plants make powerful chemicals, revealing hidden biological connections that could transform how medicines are discovered and produced. Plants produce protective chemicals called alkaloids as part of their natural [...]
Scientists Develop IV Therapy That Repairs the Brain After Stroke
New nanomaterial passes the blood-brain barrier to reduce damaging inflammation after the most common form of stroke. When someone experiences a stroke, doctors must quickly restore blood flow to the brain to prevent death. [...]
Analyzing Darwin’s specimens without opening 200-year-old jars
Scientists have successfully analyzed Charles Darwin's original specimens from his HMS Beagle voyage (1831 to 1836) to the Galapagos Islands. Remarkably, the specimens have been analyzed without opening their 200-year-old preservation jars. Examining 46 [...]
Scientists discover natural ‘brake’ that could stop harmful inflammation
Researchers at University College London (UCL) have uncovered a key mechanism that helps the body switch off inflammation—a breakthrough that could lead to new treatments for chronic diseases affecting millions worldwide. Inflammation is the [...]
A Forgotten Molecule Could Revive Failing Antifungal Drugs and Save Millions of Lives
Scientists have uncovered a way to make existing antifungal drugs work again against deadly, drug-resistant fungi. Fungal infections claim millions of lives worldwide each year, and current medical treatments are failing to keep pace. [...]
