[Paper] Performance of a Deep Learning-Based Segmentation Model for Pancreatic Tumors on Public Endoscopic Ultrasound Datasets

Source: arXiv - 2601.05937v1

Overview

A new study evaluates a Vision‑Transformer (ViT) based deep‑learning model for automatically segmenting pancreatic tumors in endoscopic ultrasound (EUS) images. By training on over 17 k publicly available scans and testing on an independent set, the authors demonstrate that transformer‑powered segmentation can reach clinically relevant accuracy while dramatically reducing the subjectivity inherent to manual EUS interpretation.

Key Contributions

• ViT‑backed segmentation pipeline – Introduces the USFM framework that couples a Vision Transformer encoder with a lightweight decoder for pixel‑wise tumor delineation.
• Large‑scale public‑dataset training – Utilizes 17,367 EUS frames from two open repositories, making the work reproducible and benchmarkable.
• Robust cross‑validation & external testing – Reports 5‑fold cross‑validation results and validates on a completely separate public dataset (350 images) annotated by radiologists.
• Comprehensive performance metrics – Provides Dice similarity coefficient (DSC), Intersection‑over‑Union (IoU), sensitivity, specificity, and accuracy, enabling direct comparison with other medical‑image segmentation approaches.
• Error analysis – Highlights a 9.7 % failure mode where the model produces multiple disjoint predictions, pointing to practical challenges for deployment.

Methodology

1. Data preprocessing – All EUS frames are converted to grayscale, centrally cropped, and resized to a uniform 512 × 512 px resolution. Simple intensity normalization is applied to reduce scanner‑specific bias.
2. Model architecture – The USFM pipeline uses a Vision Transformer as the encoder, which captures long‑range spatial dependencies across the image. A shallow convolutional decoder upsamples the transformer embeddings back to the original resolution, producing a binary mask for tumor vs. background.
3. Training strategy – The authors perform 5‑fold cross‑validation on the combined training set (≈ 17 k images). AdamW optimizer, a cosine‑annealing learning‑rate schedule, and a combined Dice + binary‑cross‑entropy loss are employed to balance region overlap and pixel‑level classification.
4. Evaluation – Standard segmentation metrics (DSC, IoU) are computed per fold, along with sensitivity (true‑positive rate), specificity (true‑negative rate), and overall accuracy. An independent test set of 350 images, manually segmented by expert radiologists, serves as external validation.

Results & Findings

• Consistency – Performance on the unseen test set mirrors cross‑validation results, indicating good generalization despite dataset heterogeneity.
• Error mode – Approximately 9.7 % of test images contain “multiple predictions,” i.e., the model outputs several disconnected tumor masks, which could confuse downstream analysis.

Practical Implications

• Computer‑assisted diagnosis (CAD) – Integrating this ViT‑based segmenter into EUS workstations could provide instant, objective tumor outlines, helping endoscopists make faster, more consistent decisions.
• Workflow automation – The model’s high specificity (> 97 %) means false alarms are rare, allowing developers to build pipelines that automatically flag suspicious regions for radiologist review without overwhelming them with false positives.
• Dataset‑agnostic training – Because the authors rely solely on publicly available data, other teams can fine‑tune the same architecture on institution‑specific scans, accelerating adoption across hospitals.
• Research acceleration – The released code and pretrained weights (if provided) give AI engineers a solid baseline for exploring multimodal fusion (e.g., combining EUS with CT) or for extending the model to other gastrointestinal lesions.

Limitations & Future Work

• Dataset heterogeneity – The training data come from different sources with varying acquisition settings; while the model generalizes reasonably, a more diverse, multi‑center corpus could improve robustness.
• Limited external validation – Only one independent public dataset (350 images) was used; larger prospective clinical trials are needed to confirm real‑world performance.
• Multiple‑prediction errors – The 9.7 % failure rate suggests the decoder may need stronger spatial regularization or post‑processing (e.g., connected‑component analysis) to enforce a single tumor mask.
• Explainability & latency – Future work should explore attention‑map visualizations for clinician trust and benchmark inference speed on edge devices to assess feasibility for real‑time EUS assistance.

Bottom line: This Vision‑Transformer segmentation model pushes the envelope for AI‑driven pancreatic tumor detection in EUS, offering a reproducible, high‑specificity tool that could soon move from research notebooks into everyday endoscopic practice.

Authors

• Pankaj Gupta
• Priya Mudgil
• Niharika Dutta
• Kartik Bose
• Nitish Kumar
• Anupam Kumar
• Jimil Shah
• Vaneet Jearth
• Jayanta Samanta
• Vishal Sharma
• Harshal Mandavdhare
• Surinder Rana
• Saroj K Sinha
• Usha Dutta

Paper Information

• arXiv ID: 2601.05937v1
• Categories: cs.CV, cs.AI, cs.LG
• Published: January 9, 2026
• PDF: Download PDF