Medical imaging is a fancy field where interpreting results might be difficult.
AI models can assist doctors by analyzing images that may indicate disease-indicating anomalies.
However, there’s a catch: these AI models often give you a single solution when, in point of fact, medical images often have multiple interpretations.
If you ask five experts to stipulate an area of interest, like a small lump in a lung scan, you would possibly find yourself with five different drawings, as they may all have their very own opinions on where the lump starts and ends, for instance.
To tackle this problem, researchers from MIT, the Broad Institute of MIT Harvard, and Massachusetts General Hospital have created Tyche, an AI system that embraces the paradox in medical image segmentation.
Segmentation involves labeling specific pixels in a medical image that represent essential structures, like organs or cells.
Marianne Rakic, an MIT computer science PhD candidate and lead creator of the study, explains, “Having options might help in decision-making. Even just seeing that there’s uncertainty in a medical image can influence someone’s decisions, so it will be important to take this uncertainty under consideration.”
Named after the Greek goddess of likelihood, Tyche generates multiple possible segmentations for a single medical image to capture ambiguity.
Each segmentation highlights barely different regions, allowing users to decide on essentially the most suitable one for his or her needs.
Rakic tells MIT News, “Outputting multiple candidates and ensuring they’re different from each other really gives you an edge.”
So, how does Tyche work? Let’s break it down into 4 easy steps:
- Learning by example: Users give Tyche a small set of example images, called a “context set,” that show the segmentation task they wish to perform. These examples can include images segmented by different human experts, helping the model understand the duty and the potential for ambiguity.
- Neural network tweaks: The researchers modified a typical neural network architecture to permit Tyche to handle uncertainty. They adjusted the network’s layers in order that the potential segmentations generated at each step could “communicate” with one another and the context set examples.
- Multiple possibilities: Tyche is designed to output multiple predictions based on a single medical image input and the context set.
- Rewarding quality: The training process was tweaked to reward Tyche for producing the perfect possible prediction. If the user asks for five predictions, they’ll see all five medical image segmentations produced by Tyche, even when one may be higher.
At the highest, human annotators show variations in segmenting medical image outputs, as there are multiple interpretations. Traditional automated techniques (middle) are generally designed for specific tasks, generating a single segmentation per image. In contrast, Tyche (bottom) adeptly captures the range of annotator disagreements across various modalities and anatomical structures, eliminating the necessity for retraining or adjustments. Source: ArXiv.
One of Tyche’s biggest strengths is its adaptability. It can tackle recent segmentation tasks without having to be retrained from scratch.
Normally, AI models for medical image segmentation use neural networks that require extensive training on large datasets and machine learning expertise.
In contrast, Tyche might be used “out of the box” for various tasks, from spotting lung lesions in X-rays to identifying brain abnormalities in MRIs.
Numerous studies have been conducted in AI medical imaging, including major breakthroughs in breast cancer screening and AI diagnostics that match and even beat doctors in interpreting images.
Looking to the long run, the research team plans to explore using more flexible context sets, possibly including text or multiple sorts of images.
They also wish to develop ways to enhance Tyche’s worst predictions and enable the system to recommend the perfect segmentation candidates.
