A computer-assisted needle misses its target, puncturing the spine. A diabetic patient goes rapidly downhill after a computer recommends an incorrect insulin dosage. An ultrasound fails to diagnose an obvious heart condition that is ultimately fatal.
These are just a few examples of incidents reported to the United States’ Food and Drug Administration involving health technology assisted by artificial intelligence (AI), and Australian researchers say they are an “early warning sign” of what could happen if regulators, hospitals and patients don’t take safety seriously in the rapidly evolving field.
“This is essentially showing us that when we’re putting in AI systems, we just need to be taking the safety of these systems really seriously,” Professor Farah Magrabi said.
Her team at the Australian Institute of Health Innovation at Macquarie University this month published a review of 266 safety events involving AI-assisted technology reported to the US watchdog. The article appeared in the Journal of the American Medical Informatics Association.
Only 16 per cent actually led to patients being harmed, but two-thirds were found to have the potential to cause harm, and 4 per cent were categorised as “near-miss events” in which users intervened.
Co-author Dr David Lyell said issues arose most commonly when users failed to enter the correct data, leading to an incorrect result, or misunderstood what the AI was actually telling them when it produced a result.
For example, one patient suffering a heart attack delayed medical care because an over-the-counter electrocardiogram device – which is not capable of detecting a heart attack – told them they had “normal sinus rhythm”.
“AI isn’t the answer; it’s part of a system that needs to support the provision of healthcare. And we do need to make sure that we have the systems in place that supports its effective use to promote healthcare for people,” Lyell said.
The researchers chose to analyse cases in the US, where the implementation of AI-enabled health devices is more advanced than in Australia. The US regulator has, to date, approved 521 artificial intelligence and machine learning-enabled medical devices, with 178 of those added in 2022.
In Australia, the Therapeutic Goods Administration (TGA) does not collect data about the number of approved devices in Australia that have AI or machine-learning components, but Magrabi said the regulator was taking the issue “very, very seriously”.
“AI isn’t the answer, it’s part of a system that needs to support the provision of health care.”
David Lyell, Australian Institute of Health Innovation, Macquarie University
AI has been used in healthcare devices for decades, but an explosion in data collection, computing power and advanced algorithms has opened up new frontiers, said David Hansen, chief executive of the CSIRO’s Australian E-Health Research Centre.
Sydney-based medical device start-up EMVision is one Australian company taking advantage of these advancements to develop a portable device for diagnosing stroke without the need for an MRI.
In the development stage, the company is using an advanced algorithm and high-powered computers to simulate stroke in numerous places in the brain, building up a database of synthetic images similar to MRIs which are then compared with real-life MRI and CT results from hospital clinical trials at Royal Melbourne, Liverpool and Princess Alexandra hospitals.
“We wouldn’t be able to do what we’re doing today if we didn’t have the [high-powered computer] infrastructure for the simulation,” said head of product development Forough Khandan.
Co-founder Scott Kirkland said the intention was not to completely replace CT and MRI scans, but to diagnose stroke in the first hour when treatment is most effective. The bedside device, set to be launched in 2025, will use a “traffic light” system based on a probability algorithm to help determine what type of stroke might have occurred.
“It’s better for an algorithm to give an ‘I don’t know’ than an incorrect answer, and have the wrong treatment and or triage process followed,” Kirkland said.
Radiology is at the forefront of the rapid adoption of AI in healthcare, especially in breast cancer screening and analysis of chest X-rays.
“A couple of years ago, almost no radiologist would say they use it, now a fair percentage would say that they use it in their daily work,” said clinical radiologist and AI safety researcher Dr Lauren Oakden Rayner.
Rayner, a member of the college, said the technology had many potential benefits, but Australian regulators and clinicians needed to better understand the risks of fully autonomous systems before putting them into hospitals, clinics and homes.
“Humans are legally and morally responsible for decision-making, and it’s taking some of that out of human hands,” she said. “There’s no reason autonomous AI systems can’t exist … but they obviously have to be tested very, very tightly.”
News
Does Space-Time Really Exist?
Is time something that flows — or just an illusion? Exploring space-time as either a fixed “block universe” or a dynamic fabric reveals deeper mysteries about existence, change, and the very nature of reality. [...]
Unlocking hidden soil microbes for new antibiotics
Most bacteria cannot be cultured in the lab-and that's been bad news for medicine. Many of our frontline antibiotics originated from microbes, yet as antibiotic resistance spreads and drug pipelines run dry, the soil [...]
By working together, cells can extend their senses beyond their direct environment
The story of the princess and the pea evokes an image of a highly sensitive young royal woman so refined, she can sense a pea under a stack of mattresses. When it comes to [...]
Overworked Brain Cells May Hold the Key to Parkinson’s
Scientists at Gladstone Institutes uncovered a surprising reason why dopamine-producing neurons, crucial for smooth body movements, die in Parkinson’s disease. In mice, when these neurons were kept overactive for weeks, they began to falter, [...]
Old tires find new life: Rubber particles strengthen superhydrophobic coatings against corrosion
Development of highly robust superhydrophobic anti-corrosion coating using recycled tire rubber particles. Superhydrophobic materials offer a strategy for developing marine anti-corrosion materials due to their low solid-liquid contact area and low surface energy. However, [...]
This implant could soon allow you to read minds
Mind reading: Long a science fiction fantasy, today an increasingly concrete scientific goal. Researchers at Stanford University have succeeded in decoding internal language in real time thanks to a brain implant and artificial intelligence. [...]
A New Weapon Against Cancer: Cold Plasma Destroys Hidden Tumor Cells
Cold plasma penetrates deep into tumors and attacks cancer cells. Short-lived molecules were identified as key drivers. Scientists at the Leibniz Institute for Plasma Science and Technology (INP), working with colleagues from Greifswald University Hospital and [...]
This Common Sleep Aid May Also Protect Your Brain From Alzheimer’s
Lemborexant and similar sleep medications show potential for treating tau-related disorders, including Alzheimer’s disease. New research from Washington University School of Medicine in St. Louis shows that a commonly used sleep medication can restore normal sleep patterns and [...]
Sugar-Coated Nanoparticles Boost Cancer Drug Efficacy
A team of researchers at the University of Mississippi has discovered that coating cancer treatment carrying nanoparticles in a sugar-like material increases their treatment efficacy. They reported their findings in Advanced Healthcare Materials. Over a tenth of breast [...]
Nanoparticle-Based Vaccine Shows Promise in Fighting Cancer
In a study published in OncoImmunology, researchers from the German Cancer Research Center and Heidelberg University have created a therapeutic vaccine that mobilizes the immune system to target cancer cells. The researchers demonstrated that virus peptides combined [...]
Quantitative imaging method reveals how cells rapidly sort and transport lipids
Lipids are difficult to detect with light microscopy. Using a new chemical labeling strategy, a Dresden-based team led by André Nadler at the Max Planck Institute of Molecular Cell Biology and Genetics (MPI-CBG) and [...]
Ancient DNA reveals cause of world’s first recorded pandemic
Scientists have confirmed that the Justinian Plague, the world’s first recorded pandemic, was caused by Yersinia pestis, the same bacterium behind the Black Death. Dating back some 1,500 years and long described in historical texts but [...]
“AI Is Not Intelligent at All” – Expert Warns of Worldwide Threat to Human Dignity
Opaque AI systems risk undermining human rights and dignity. Global cooperation is needed to ensure protection. The rise of artificial intelligence (AI) has changed how people interact, but it also poses a global risk to human [...]
Nanomotors: Where Are They Now?
First introduced in 2004, nanomotors have steadily advanced from a scientific curiosity to a practical technology with wide-ranging applications. This article explores the key developments, recent innovations, and major uses of nanomotors today. A [...]
Study Finds 95% of Tested Beers Contain Toxic “Forever Chemicals”
Researchers found PFAS in 95% of tested beers, with the highest levels linked to contaminated local water sources. Per- and polyfluoroalkyl substances (PFAS), better known as forever chemicals, are gaining notoriety for their ability [...]
Long COVID Symptoms Are Closer To A Stroke Or Parkinson’s Disease Than Fatigue
When most people get sick with COVID-19 today, they think of it as a brief illness, similar to a cold. However, for a large number of people, the illness doesn't end there. The World [...]
