Weakly-Supervised Hair SEM Microscope Image Segmentation Using a Priori Structure Information
DOI:
https://doi.org/10.5566/ias.3406Keywords:
SEM segmentation, hair microscopic segmentation, Radon transform, weakly-supervised segmentationAbstract
The analysis of microscopic images of hair has a wide range of applications in the domains of cosmetics, healthcare and forensics. The segmentation of the hair represents the initial step of automatic large-scale quantitative analysis of microscopic images of hair. It ensures that subsequent quantitative measurements are performed in an appropriate area corresponding to the hair, avoiding artifacts near the boundaries. This process can be time-consuming, tedious and susceptible to subjective errors when conducted by a human operator. Deep learning methods represent a promising solution; however, obtaining pixel-level accurate masks is a costly process. This paper presents a novel weakly supervised pipeline for the segmentation of hair SEM (Scanning Electron Microscope) microscopic images, which requires only simple image-level annotations for training. The proposed method incorporates the Radon transform, the Sobel operator and a novel Boundary Discrimination Module (BD-module) for the estimation of the presence of boundaries. The proposed pipeline was evaluated on a recently collected hair SEM dataset (429 images). Furthermore, it is benchmarked with methods including Unet and Segment Anything Model (SAM). The results demonstrated a mean Hausdorff Distance improvement of over 30% and a standard deviation improvement of over 50% in comparison with the Unet and SAM. Moreover, we proposed additional refinement modules to address boundary nonlinear cases and conducted Grad-CAM analysis to enhance the interpretability of the BD-module. Additionally, we proposed a novel quality estimation metric based on gradient map for self-quality assessment. The SEM hair dataset is accessible to the research community in an open-source format.
References
Ahn J, Kwak S (2018). Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition.
Angulo J, Velasco-Forero S, Chanussot J (2009). Multiscale stochastic watershed for unsupervised hyperspectral image segmentation. In: 2009 IEEE international geoscience and remote sensing symposium, vol. 3. IEEE.
Bhaskar H, Werghi N (2011). Comparing Hough and Radon transform based parallelogram detection. In: 2011 IEEE GCC Conference and Exhibition (GCC). IEEE.
Bilodeau A, Delmas CV, Parent M, De Koninck P, Durand A, Lavoie-Cardinal F (2022). Microscopy analysis neural network to solve detection, enumeration and segmentation from image-level annotations. Nat Mach Intell 4:455–66.
Chen H, Wang J, Chen HC, Zhen X, Zheng F, Ji R, Shao L (2021). Seminar learning for click-level weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision.
Chu F, Anex DS, Jones AD, Hart BR (2020). Automated analysis of scanning electron microscopic images for assessment of hair surface damage. R Soc Open Sci 7:191438.
Coroaba A, Chiriac AE, Sacarescu L, Pinteala T, Minea B, Ibanescu SA, Pertea M, Moraru A, Esanu I, Maier SS, et al. (2020). New insights into human hair: SAXS, SEM, TEM and EDX for Alopecia Areata investigations. PeerJ 8
.
Dong M, Liu D, Xiong Z, Chen X, Zhang Y, Zha ZJ, Bi G, Wu F (2020). Towards neuron segmentation from macaque brain images: a weakly supervised approach. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part V 23. Springer.
Dutta A, Zisserman A (2019). The VIA annotation software for images, audio and video. In: Proceedings of the 27th ACM International Conference on Multimedia, MM ’19. New York, NY, USA: ACM.
Galliano A, Ye C, Su F, Wang C, Rakshit R, Guerin M, Flament F, Steel A (2023). Assessing the effect of cleansing products on artificially polluted human hairs and skin through in vivo and in vitro models. Skin Res Technol 29
.
Galliano A, Ye C, Su F, Wang C, Wang Y, Liu C, Wagle A, Guerin M, Flament F, Steel A (2017). Particulate matter adheres to human hair exposed to severe aerial pollution: consequences for certain hair surface properties. Int J Cosmet Sci 39:610–6.
He K, Zhang X, Ren S, Sun J (2016). Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition.
Huang Z, Wang X, Wang J, Liu W, Wang J (2018). Weakly-supervised semantic segmentation network with deep seeded region growing. In: Proceedings of the IEEE conference on computer vision and pattern recognition.
Kim YD, Jeon SY, Ji JH, Lee WS (2010). Development of a classification system for extrinsic hair damage: Standard grading of electron microscopic findings of damaged hairs. Am J Dermatopathol 32:432–8.
Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, Xiao T, Whitehead S, Berg AC, Lo WY, et al. (2023). Segment anything. In: Proceedings of the IEEE/CVF International Conference on Computer Vision.
Kniesel H, Sick L, Payer T, Bergner T, Devan KS, Read C, Walther P, Ropinski T, Hermosilla P (2023). Weakly supervised virus capsid detection with image-level annotations in electron microscopy images. In: The Twelfth International Conference on Learning Representations.
Kulharia V, Chandra S, Agrawal A, Torr P, Tyagi A (2020). Box2seg: Attention weighted loss and discriminative feature learning for weakly supervised segmentation. In: European Conference on Computer Vision. Springer.
Lee H, Jeong WK (2020). Scribble2label: Scribble-supervised cell segmentation via self-generating pseudo-labels with consistency. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part I 23. Springer.
Lee I, Du X, Anthony B (2017). Hair segmentation using adaptive threshold from edge and branch length measures. Comput Biol Med 89:314–4.
Lee S, Choi A, Baek J, Kim H, Shin M, Koh J (2016). Twelve-point scale grading system of scanning electron microscopic examination to investigate subtle changes in damaged hair surface. Skin Res Technol 22:406–11.
Downloads
Published
Data Availability Statement
The dataset and its annotation is available here.
Issue
Section
License
Copyright (c) 2024 Xiaohu Liu, Thérèse Baldeweck, Pierre Dupuis, Thomas Bornschloegl, Etienne Decencière, Beatriz Marcotegui
This work is licensed under a Creative Commons Attribution 4.0 International License.
