Deep Learning Driven Breast Cancer MalignancyPrediction in Ultrasound Leveraging Multiscale FeatureFusion and Self Supervised Learning

Abstract

Accurate malignancy prediction in breast ultrasound imaging is challenged by limited annotated data, high inter-observer variability, and inherent noise in sonographic textures. To address these limitations, we propose a deep learning framework that synergistically integrates multiscale feature fusion and self-supervised learning (SSL) to improve diagnostic performance while minimizing reliance on labeled datasets. The architecture employs a hierarchical convolutional backbone with multiscale feature extractors that capture both coarse contextual semantics and fine-grained morphological cues of lesions. Features across multiple receptive fields are fused via a cross-scale attention mechanism, enhancing the model’s ability to localize and characterize malignant regions. For robust pretraining, we adopt a self-supervised contrastive learning paradigm tailored for medical ultrasound, incorporating spatial transformation invariance and anatomical context preservation to learn domain-relevant representations from unlabeled data. The pretrained encoder is fine-tuned with a supervised classification head using a limited set of annotated images. Extensive experiments on two publicly available breast ultrasound datasets demonstrate that our model achieves higher performance over state-of-the-art baselines, yielding significant improvements in AUC, F1-score, and sensitivity. Ablation studies confirm the individual and combined efficacy of the multiscale fusion and SSL modules. This work establishes a scalable and label-efficient pipeline for ultrasound-based malignancy prediction, with implications for real-time clinical decision support.