By providing plausible label maps for one medical image, the Tyche machine-learning model could help clinicians and researchers capture crucial information.
In biomedicine, segmentation involves annotating pixels from an important structure in a medical image, like an organ or cell. Artificial intelligence models can help clinicians by highlighting pixels that may show signs of a certain disease or anomaly.
However, these models typically only provide one answer, while the problem of medical image segmentation is often far from black and white. Five expert human annotators might provide five different segmentations, perhaps disagreeing on the existence or extent of the borders of a nodule in a lung CT image.
Embracing Uncertainty in Diagnosis
“Having options can help in decision-making. Even just seeing that there is uncertainty in a medical image can influence someone’s decisions, so it is important to take this uncertainty into account,” says Marianne Rakic, an MIT computer science PhD candidate.
Rakic is lead author of a paper with others at MIT, the Broad Institute of MIT and Harvard, and Massachusetts General Hospital that introduces a new AI tool that can capture the uncertainty in a medical image.
Introduction of Tyche
Known as Tyche (named for the Greek divinity of chance), the system provides multiple plausible segmentations that each highlight slightly different areas of a medical image. A user can specify how many options Tyche outputs and select the most appropriate one for their purpose.
Importantly, Tyche can tackle new segmentation tasks without needing to be retrained. Training is a data-intensive process that involves showing a model many examples and requires extensive machine-learning experience.
Because it doesn’t need retraining, Tyche could be easier for clinicians and biomedical researchers to use than some other methods. It could be applied “out of the box” for a variety of tasks, from identifying lesions in a lung X-ray to pinpointing anomalies in a brain MRI.
Ultimately, this system could improve diagnoses or aid in biomedical research by calling attention to potentially crucial information that other AI tools might miss.
“Ambiguity has been understudied. If your model completely misses a nodule that three experts say is there and two experts say is not, that is probably something you should pay attention to,” adds senior author Adrian Dalca, an assistant professor at Harvard Medical School and MGH, and a research scientist in the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL).
Their co-authors include Hallee Wong, a graduate student in electrical engineering and computer science; Jose Javier Gonzalez Ortiz PhD ’23; Beth Cimini, associate director for bioimage analysis at the Broad Institute; and John Guttag, the Dugald C. Jackson Professor of Computer Science and Electrical Engineering. Rakic will present Tyche at the IEEE Conference on Computer Vision and Pattern Recognition, where Tyche has been selected as a highlight.
Addressing Ambiguity With AI
AI systems for medical image segmentation typically use neural networks. Loosely based on the human brain, neural networks are machine-learning models comprising many interconnected layers of nodes, or neurons, that process data.
After speaking with collaborators at the Broad Institute and MGH who use these systems, the researchers realized two major issues limit their effectiveness. The models cannot capture uncertainty and they must be retrained for even a slightly different segmentation task.
Some methods try to overcome one pitfall, but tackling both problems with a single solution has proven especially tricky, Rakic says.
“If you want to take ambiguity into account, you often have to use an extremely complicated model. With the method we propose, our goal is to make it easy to use with a relatively small model so that it can make predictions quickly,” she says.
The researchers built Tyche by modifying a straightforward neural network architecture.
A user first feeds Tyche a few examples that show the segmentation task. For instance, examples could include several images of lesions in a heart MRI that have been segmented by different human experts so the model can learn the task and see that there is ambiguity.
The researchers found that just 16 example images, called a “context set,” is enough for the model to make good predictions, but there is no limit to the number of examples one can use. The context set enables Tyche to solve new tasks without retraining.
For Tyche to capture uncertainty, the researchers modified the neural network so it outputs multiple predictions based on one medical image input and the context set. They adjusted the network’s layers so that, as data move from layer to layer, the candidate segmentations produced at each step can “talk” to each other and the examples in the context set.
In this way, the model can ensure that candidate segmentations are all a bit different, but still solve the task.
“It is like rolling dice. If your model can roll a two, three, or four, but doesn’t know you have a two and a four already, then either one might appear again,” she says.
They also modified the training process so it is rewarded by maximizing the quality of its best prediction.
If the user asked for five predictions, at the end they can see all five medical image segmentations Tyche produced, even though one might be better than the others.
The researchers also developed a version of Tyche that can be used with an existing, pretrained model for medical image segmentation. In this case, Tyche enables the model to output multiple candidates by making slight transformations to images.
Better, Faster Predictions
When the researchers tested Tyche with datasets of annotated medical images, they found that its predictions captured the diversity of human annotators, and that its best predictions were better than any from the baseline models. Tyche also performed faster than most models.
“Outputting multiple candidates and ensuring they are different from one another really gives you an edge,” Rakic says.
The researchers also saw that Tyche could outperform more complex models that have been trained using a large, specialized dataset.
For future work, they plan to try using a more flexible context set, perhaps including text or multiple types of images. In addition, they want to explore methods that could improve Tyche’s worst predictions and enhance the system so it can recommend the best segmentation candidates.
Reference: “Tyche: Stochastic In-Context Learning for Medical Image Segmentation” by Marianne Rakic, Hallee E. Wong, Jose Javier Gonzalez Ortiz, Beth Cimini, John Guttag and Adrian V. Dalca, 24 January 2024, Electrical Engineering and Systems Science > Image and Video Processing.
arXiv:2401.13650
This research is funded, in part, by the National Institutes of Health, the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard, and Quanta Computer.
News
An easy pill to swallow—new 3D printing research paves way for personalized medication
A new technique for 3D printing medication has enabled the printing of multiple drugs in a single tablet, paving the way for personalized pills that can deliver timed doses. Researchers from the University of [...]
Revolutionary “Quartet Nanocage” Vaccine Effective Against Coronaviruses That Haven’t Even Emerged Yet
Researchers have created a new vaccine technology that, in tests with mice, has demonstrated protection against a wide variety of coronaviruses, including those that could cause future disease outbreaks—even those not yet identified. This [...]
Researchers Warn: AI Systems Have Already Learned How To Deceive Humans
Numerous artificial intelligence (AI) systems, even those designed to be helpful and truthful, have already learned how to deceive humans. In a review article recently published in the journal Patterns, researchers highlight the dangers of [...]
MIT’s New MRI Technique Reveals Hidden Light Deep in the Brain
A novel MRI technique developed at MIT allows for detailed imaging of bioluminescence deep within the brain, offering new insights into how brain cells develop and communicate with each other. Scientists often label cells with proteins [...]
Revolutionary mRNA Cancer Vaccine Shows Immense Promise in First-Ever Human Clinical Trial
In a first-ever human clinical trial of four adult patients, an mRNA cancer vaccine created at the University of Florida rapidly reprogrammed the immune system to target glioblastoma, the deadliest and most aggressive form of brain [...]
Human brain map contains never-before-seen details of structure
A cubic millimeter of brain tissue may not sound like much. But considering that tiny square contains 57,000 cells, 230 millimeters of blood vessels, and 150 million synapses, all amounting to 1,400 terabytes of [...]
AI Ethics Surpass Human Judgment in New Moral Turing Test
AI’s ability to address moral questions is improving, which prompts further considerations for the future. A recent study revealed that when individuals are given two solutions to a moral dilemma, the majority tend to [...]
AstraZeneca removes its Covid vaccine worldwide after rare side effect
AstraZeneca withdraws COVID-19 vaccine: The pharmaceutical giant AstraZeneca's COVID-19 vaccine is being withdrawn worldwide after the company acknowledged for the first time in court documents that it can cause a rare and dangerous side [...]
AI Transforms Drug Discovery With Faster, Safer Cancer Treatments
The new platform helped UC San Diego scientists synthesize 32 potential multi-target cancer drugs. Scientists at UC San Diego have developed a machine learning algorithm to simulate the time-consuming chemistry involved in the earliest phases of [...]
Nanotech opens door to future of insulin medication
An international team, led by researchers from Australia, have developed a system using nanotechnology that could allow people with diabetes to take oral insulin in the future. The researchers say the new insulin could [...]
COVID-19’s Impact on Child Development More Subtle Than Expected
Researchers at Johns Hopkins Children’s Center express that the study results offer a sense of reassurance, instilling “guarded optimism” regarding the developmental outcomes of young children who have been exposed to restrictions stemming from [...]
New Hope for Neurological Disorders: Scientists Have Discovered How an Essential Nutrient Enters the Brain
Researchers have discovered the process by which dietary choline crosses the blood-brain barrier. This breakthrough has potential applications in enhancing drug delivery to the brain for treating neurological disorders. A researcher from the University of [...]
Developing a tech platform for fast, robust series production of nanoparticles
Nucleic acid-based medications such as mRNA vaccines offer tremendous potential for medicine and are opening up new therapeutic approaches. These active ingredients must be enclosed inside nanoparticles to ensure that they get to where [...]
Unlocking Medical Mysteries: Tyche AI’s Revolutionary Insight Into Ambiguity
By providing plausible label maps for one medical image, the Tyche machine-learning model could help clinicians and researchers capture crucial information. In biomedicine, segmentation involves annotating pixels from an important structure in a medical [...]
Scientists Decode the Molecular Impact of Exercise
MoTrPAC examined the molecular effects of exercise on 2,600 volunteers, incorporating factors like age, race, and gender diversity. Building upon research in rats, MoTrPAC discovered over 35,000 biological molecules responding to endurance exercise and [...]
Prostate Cancer Breakthrough: Urine Test Avoids Unnecessary Biopsies
A study in JAMA Oncology reveals that MyProstateScore 2.0, a new urine test analyzing 18 genes, surpasses PSA in detecting significant prostate cancers and could reduce unnecessary biopsies by up to 42%. A new urine test [...]