A computer-assisted needle misses its target, puncturing the spine. A diabetic patient goes rapidly downhill after a computer recommends an incorrect insulin dosage. An ultrasound fails to diagnose an obvious heart condition that is ultimately fatal.
These are just a few examples of incidents reported to the United States’ Food and Drug Administration involving health technology assisted by artificial intelligence (AI), and Australian researchers say they are an “early warning sign” of what could happen if regulators, hospitals and patients don’t take safety seriously in the rapidly evolving field.
“This is essentially showing us that when we’re putting in AI systems, we just need to be taking the safety of these systems really seriously,” Professor Farah Magrabi said.
Her team at the Australian Institute of Health Innovation at Macquarie University this month published a review of 266 safety events involving AI-assisted technology reported to the US watchdog. The article appeared in the Journal of the American Medical Informatics Association.
Only 16 per cent actually led to patients being harmed, but two-thirds were found to have the potential to cause harm, and 4 per cent were categorised as “near-miss events” in which users intervened.
Co-author Dr David Lyell said issues arose most commonly when users failed to enter the correct data, leading to an incorrect result, or misunderstood what the AI was actually telling them when it produced a result.
For example, one patient suffering a heart attack delayed medical care because an over-the-counter electrocardiogram device – which is not capable of detecting a heart attack – told them they had “normal sinus rhythm”.
“AI isn’t the answer; it’s part of a system that needs to support the provision of healthcare. And we do need to make sure that we have the systems in place that supports its effective use to promote healthcare for people,” Lyell said.
The researchers chose to analyse cases in the US, where the implementation of AI-enabled health devices is more advanced than in Australia. The US regulator has, to date, approved 521 artificial intelligence and machine learning-enabled medical devices, with 178 of those added in 2022.
In Australia, the Therapeutic Goods Administration (TGA) does not collect data about the number of approved devices in Australia that have AI or machine-learning components, but Magrabi said the regulator was taking the issue “very, very seriously”.
“AI isn’t the answer, it’s part of a system that needs to support the provision of health care.”
David Lyell, Australian Institute of Health Innovation, Macquarie University
AI has been used in healthcare devices for decades, but an explosion in data collection, computing power and advanced algorithms has opened up new frontiers, said David Hansen, chief executive of the CSIRO’s Australian E-Health Research Centre.
Sydney-based medical device start-up EMVision is one Australian company taking advantage of these advancements to develop a portable device for diagnosing stroke without the need for an MRI.
In the development stage, the company is using an advanced algorithm and high-powered computers to simulate stroke in numerous places in the brain, building up a database of synthetic images similar to MRIs which are then compared with real-life MRI and CT results from hospital clinical trials at Royal Melbourne, Liverpool and Princess Alexandra hospitals.
“We wouldn’t be able to do what we’re doing today if we didn’t have the [high-powered computer] infrastructure for the simulation,” said head of product development Forough Khandan.
Co-founder Scott Kirkland said the intention was not to completely replace CT and MRI scans, but to diagnose stroke in the first hour when treatment is most effective. The bedside device, set to be launched in 2025, will use a “traffic light” system based on a probability algorithm to help determine what type of stroke might have occurred.
“It’s better for an algorithm to give an ‘I don’t know’ than an incorrect answer, and have the wrong treatment and or triage process followed,” Kirkland said.
Radiology is at the forefront of the rapid adoption of AI in healthcare, especially in breast cancer screening and analysis of chest X-rays.
“A couple of years ago, almost no radiologist would say they use it, now a fair percentage would say that they use it in their daily work,” said clinical radiologist and AI safety researcher Dr Lauren Oakden Rayner.
Rayner, a member of the college, said the technology had many potential benefits, but Australian regulators and clinicians needed to better understand the risks of fully autonomous systems before putting them into hospitals, clinics and homes.
“Humans are legally and morally responsible for decision-making, and it’s taking some of that out of human hands,” she said. “There’s no reason autonomous AI systems can’t exist … but they obviously have to be tested very, very tightly.”
News
GHCE Concept
From the preface of the book Global Health Care Equivalency in the Age of Nanotechnology, Nanomedicine and Artificial Intelligence, Edited by Frank Boehm: Since the publication of my first book (Nanomedical Device and Systems [...]
Human Bodies Are Producing Two Billion Tons of CO2 a Year
A new study reveals that the amount of carbon dioxide produced by human metabolism has increased dramatically over three decades. The carbon dioxide produced by human bodies as they convert food into energy increased [...]
A Hidden DNA Mechanism Could Explain How 20% of Human Cancers Develop
Researchers have identified an unexpected molecular vulnerability in cancers carrying BAF mutations. Mutations found in roughly 20% of human cancers may promote tumor growth by changing how DNA is organized inside cells. Researchers have [...]
Ebola kills over 5,300 in fastest-growing outbreak on record
Africa’s latest Ebola outbreak has killed nearly 5,400 people, the vast majority of them in the Democratic Republic of Congo (DRC), according to the continent’s top health agency. Africa Centres for Disease Control and [...]
China makes early progress in using sound waves to stimulate damaged brains
Chinese researchers are testing whether focused ultrasound can help restore consciousness in patients with severe brain injuries, adding to a growing international effort to use sound waves to stimulate deep brain structures without surgery. [...]
Micro-drones fly through the body thanks to ultrasound
At the EPFL, micro-drones have been developed that can be moved without a motor or battery.No motors, no batteries, no electronics on board, just small cavities that convert sound waves into thrust: this is [...]
A Single Enzyme Could Hold the Key to Treating a Devastating Brain Disorder
Blocking an enzyme’s production restored brain growth and healthy behaviors in mice engineered to develop Angelman syndrome, pointing to a possible treatment target. Children with Angelman syndrome begin to show difficulties with thinking and [...]
Scientists Develop a Clever New Way To Kill Cancer Cells From the Inside
Scientists have developed an experimental strategy that exploits a chemical byproduct of cancer cells to produce anticancer drugs directly inside them. Scientists in Japan have developed an experimental method that uses a chemical produced [...]
Our books are available worldwide!
Online Sellers other than Amazon, Routledge, and IOPP Indigo Global Health Care Equivalency in the Age of Nanotechnology, Nanomedicine and Artifcial Intelligence Global Health Care Equivalency In The Age Of Nanotechnology, Nanomedicine And Artificial [...]
After only a short training, people in a coma managed to send targeted signals
Researchers have succeeded in communicating with brain-injured patients who are unconscious. They see enormous potential in this for future treatments. Patients who lie motionless in bed after a severe brain injury and do not [...]
MIT Scientists Reprogram Immune Cells Inside the Body To Destroy Cancer
Natural killer cells reprogrammed inside mice eliminated lymphoma tumors and helped damaged livers heal, though whether this approach could help people with cancer or autoimmune disease remains unknown. To make a cancer treatment from [...]
Melanoma mRNA Vaccine Shows Promise in Preventing Disease Recurrence
Key takeaways: Phase 3 trial finds personalized intismeran plus pembrolizumab may better prevent high-risk melanoma recurrence after surgery The trial enrolled 1,137 patients and builds on phase 2 results showing about half the recurrence [...]
Scientists Discover a Powerful New Weapon Against a Dangerous Gut Infection
The promise of an experimental treatment for C. difficile gut infections lies partly in the bacteria it leaves alive, according to early laboratory and mouse research. Broad-spectrum antibiotics can disrupt the gut’s microbial community, [...]
Your Bones May Hold a Hidden Clue to Blood Cancer Survival
For people with blood cancers called myelodysplastic syndromes, osteoporosis may hold an overlooked clue to survival, a three-year observational study suggests. In blood cancers that develop in the bone marrow, an important clue to [...]
Researchers Discover What Reading for Pleasure May Be Doing to Your Brain
Reading for pleasure may offer surprising benefits for the brain, with research linking the habit to better mental health, stronger cognitive abilities, and a lower risk of dementia. Reading for pleasure may do far [...]
Scientists Discover a Hidden Trigger That Keeps Inflammation Alive
Inflammation persists after proteins detach from immune receptors, revealing a potential target for autoimmune treatments. Since a Nobel Prize-winning discovery in the 1990s identified molecules called Toll-like Receptors as the guardians of the human [...]
