By recognizing and separating different tissues, organs, or regions of interest, medical image segmentation is important to studying medical pictures. For more exact diagnosis and therapy, clinicians can use accurate segmentation to assist them locate and accurately pinpoint disease regions. Moreover, thorough insights into the morphology, structure, and performance of varied tissues or organs are provided through quantitative and qualitative evaluation of medical pictures, enabling the study of illness. As a consequence of the peculiarities of medical imaging, reminiscent of its wide range of modalities, complicated tissue and organ architecture, and absence of annotated data, most existing approaches are restricted to certain modalities, organs, or pathologies. 

For this reason restriction, algorithms are difficult to generalize and modify to be used in various clinical contexts. The push towards large-scale models has recently generated excitement among the many AI community. The event of general AI models like ChatGPT2, ERNIE Bot 3, DINO, SegGPT, and SAM makes employing a single model for various tasks possible. With SAM, probably the most recent large-scale vision model, users may create masks for certain regions of interest by interactively clicking, drawing bounding boxes, or using verbal cues. Significant attention has been paid to its zero-shot and few-shot capabilities on natural photos across various fields. 

Some efforts have also focused on the SAMs’ zero-shot capability within the context of medical imaging. Nevertheless, SAM finds it difficult to generalize to multi-modal and multi-object medical datasets, resulting in variable segmentation performance across datasets. It is because there’s a substantial domain gap between natural and medical images. The cause may be linked to the methods used to assemble the info: resulting from their specific clinical purpose, medical pictures are obtained using particular protocols and scanners and displayed as various modalities (electrons, lasers, X-rays, ultrasound, nuclear physics, and magnetic resonance). Because of this, these images deviate substantially from real images since they rely on various physics-based features and energy sources. 

Natural and medical images differ significantly by way of pixel intensity, color, texture, and other distribution features, as seen in Figure 1. Because SAM is trained on only natural photos, it needs more specialized information regarding medical imaging, so it can’t be immediately applied to the medical sector. Providing SAM with medical information is difficult resulting from the high annotation cost and inconsistent annotation quality. Medical data preparation needs subject expertise, and the standard of this data differs greatly between institutions and clinical trials. The quantity of medical and natural images varies significantly resulting from these difficulties. 

The bar chart in Figure 1 compares the info volume of publicly available natural image datasets and medical image datasets. As an example, Totalsegmentor, the biggest public segmentation dataset within the medical domain, also has a major gap in comparison with Open Image v6   and SA-1B. On this study, their objective is to transfer SAM from natural images to medical images. This can provide benchmark models and evaluation frameworks for researchers in medical image evaluation to explore and enhance. To attain this goal, researchers from Sichuan University and Shanghai AI Laboratory proposed SAM-Med2D, probably the most comprehensive study on applying SAM to medical 2D images.

Try the Paper and Github. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to hitch our 29k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the newest AI research news, cool AI projects, and more.

For those who like our work, you’ll love our newsletter..