Automatic Joint Teeth Segmentation in Panoramic Dental Images using Mask R-CNN with Residual Feature Extraction
Can it be useful in Oral Cancer Diagnosis and Management?
DOI:
https://doi.org/10.59667/sjoranm.v12i1.18Keywords:
Panoramic radiography, Tooth segmentation, Mask R-CNN, Residual learning, Medical image analysisAbstract
Introduction
Panoramic dental radiographs provide an overview of the teeth, jaws, and surrounding structures. Automated segmentation may support the quantitative analysis of dental images. This study evaluated a modified Mask R-CNN approach with residual feature extraction for the segmentation of individual teeth in panoramic radiographs.
Material and Methods
A sequence of residual blocks was used to construct a reported 62-layer feature extraction network for a modified Mask R-CNN (MRCNN). The UFBA-UESC and Tufts dental image datasets comprised 2,500 panoramic radiographs: 1,800 were assigned to training, 448 to validation, and 252 to testing.
Results
The modified model achieved a reported training accuracy of 99.67% and validation accuracy of 98.94%. Averaged across validation runs, the author-reported corrected Dice score was 98.67%, intersection over union (IoU) was 97.8%, and pixel accuracy was 96.53%. On the separate test set of 252 radiographs, the Dice scores were 96.0% for the modified model and 83.0% for the original model. Training, validation, and test metrics are reported separately.
Conclusion
The findings support the potential of the proposed approach for automated tooth segmentation in panoramic dental radiographs. Accurate delineation of individual teeth may provide a useful component for future computer-assisted dental image analysis. Its possible contribution to oral cancer assessment, treatment planning, or follow-up requires dedicated clinical validation, as these applications were not evaluated in the present study.
References
1. G. Silva, L. Oliveira, and M. Pithon, “Automatic segmenting teeth in x-ray images: Trends, a novel data set, benchmarking and future perspectives,” Expert Systems with Applications, vol. 107, pp. 15 – 31, 2018. doi: https://doi.org/10.1016/j.eswa.2018.04.001 DOI: https://doi.org/10.1016/j.eswa.2018.04.001
2. A. Lurie, G. M. Tosoni, J. Tsimikas, and F. Walker Jr., “Recursive hierarchic segmentation analysis of bone mineral density changes on digital panoramic images,” Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, vol. 113(4), pp. 549–558.e1, 2012. https://doi.org/10.1016/j.oooo.2011.10.002 DOI: https://doi.org/10.1016/j.oooo.2011.10.002
3. Y.Y. Amer and M. J. Aqel, “An efficient segmentation algorithm for panoramic dental images,” Procedia Computer Science, vol. 65, pp. 718–725, 2015, international Conference on Communications, management, and Information technology (ICCMIT’2015). https://doi.org/10.1016/j.procs.2015.09.016 DOI: https://doi.org/10.1016/j.procs.2015.09.016
4. M. K. Alsmadi, “A hybrid fuzzy c-means and neutrosophic for jaw lesions segmentation,” Ain Shams Engineering Journal, vol. 9, no. 4, pp. 697–706, 2018. https://doi.org/10.1016/j.asej.2016.03.016 DOI: https://doi.org/10.1016/j.asej.2016.03.016
5. M. R. M. Razali, N. S. Ahmad, R. Hassan, Z. M. Zaki, and W. Ismail, “Sobel and canny edges segmentations for the dental age assessment,” in 2014 International Conference on Computer Assisted System in Health, 2014, pp. 62–66. https://doi.org/10.1109/CASH.2014.10 DOI: https://doi.org/10.1109/CASH.2014.10
6. G. Jader, J. Fontinele, M. Ruiz, K. Abdalla, M. Pithon, and L. Oliveira, “Deep instance segmentation of teeth in panoramic x-ray images,” in 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), 2018, pp. 400–407. https://doi.org/10.1109/SIBGRAPI.2018.00058 DOI: https://doi.org/10.1109/SIBGRAPI.2018.00058
7. K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp.2980–2988. https://arxiv.org/abs/1703.06870 DOI: https://doi.org/10.1109/ICCV.2017.322
8. T. L. Koch, M. Perslev, C. Igel, and S. S. Brandt, “Accurate segmentation of dental panoramic radiographs with U-Nets,” in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 2019, pp. 15–19. https://doi.org/10.1109/ISBI.2019.8759563 DOI: https://doi.org/10.1109/ISBI.2019.8759563
9. O. Ronneberger, P.Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention (MICCAI), ser. LNCS, vol. 9351. Springer, 2015, pp. 234–241. https://doi.org/10.1007/978-3-319-24574-4_28 DOI: https://doi.org/10.1007/978-3-319-24574-4_28
10. R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Region-Based Convolutional Networks for Accurate Object Detection and Segmentation” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 1, pp. 142–158, Jan. 2016, https://doi.org/10.1109/TPAMI.2015.2437384 DOI: https://doi.org/10.1109/TPAMI.2015.2437384
11. R. Girshick, “Fast R-CNN,” in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1440–1448. https://doi.org/10.1109/ICCV.2015.169 DOI: https://doi.org/10.1109/ICCV.2015.169
12. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137–1149, Jun. 2017. https://doi.org/10.1109/TPAMI.2016.2577031 DOI: https://doi.org/10.1109/TPAMI.2016.2577031
13. T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature Pyramid Networks for Object Detection”, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2117–2125. https://doi.org/10.1109/CVPR.2017.106 DOI: https://doi.org/10.1109/CVPR.2017.106
14. F. Deng, H. Hu, S. Chen, Q. Guan, and Y. Zou, “Rich feature hierarchies for cell detecting under phase contrast microscopy images”, in 2015 Sixth International Conference on Intelligent Control and Information Processing (ICICIP), Nov. 2015, pp. 348–353. https://doi.org/10.1109/ICICIP.2015.7388195 DOI: https://doi.org/10.1109/ICICIP.2015.7388195
15. N. Atif, M. Bhuyan, and S. Ahamed, “A Review on Semantic Segmentation from a Modern Perspective,” in 2019 International Conference on Electrical, Electronics and Computer Engineering (UPCON), Nov. 2019, pp. 1–6. https://doi.org/10.1109/UPCON47278.2019.8980189 DOI: https://doi.org/10.1109/UPCON47278.2019.8980189
16. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014) https://doi.org/10.48550/arXiv.1409.1556
17. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015) https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Szegedy_Going_Deeper_With_2015_CVPR_paper.html DOI: https://doi.org/10.1109/CVPR.2015.7298594
18. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016) https://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html DOI: https://doi.org/10.1109/CVPR.2016.90
19. K. Panetta, R. Rajendran, A. Ramesh, S. P. Rao and S. Agaian, "Tufts Dental Database: A Multimodal Panoramic X-Ray Dataset for Benchmarking Diagnostic Systems", in IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 4, pp. 1650-1659, April 2022, https://doi.org/10.1109/JBHI.2021.3117575 DOI: https://doi.org/10.1109/JBHI.2021.3117575
20. Mask-RCNN implementation for Tensorflow 2.7.0 and Keras 2.7.0. https://github.com/Kamlesh364/Mask-RCNN-TF2.7.0-keras2.7.0
21. Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested U-net architecture for medical image segmentation,” in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Berlin, Germany: Springer, 2018, pp. 3–11. https://doi.org/10.1007/978-3-030-00889-5_1 DOI: https://doi.org/10.1007/978-3-030-00889-5_1
22. H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 2881–2890. https://openaccess.thecvf.com/content_cvpr_2017/html/Zhao_Pyramid_Scene_Parsing_CVPR_2017_paper.html DOI: https://doi.org/10.1109/CVPR.2017.660
23. L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,” 2017, https://doi.org/10.48550/arXiv.1706.05587.
24. L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-Decoder with atrous separable convolution for semantic image segmentation,” in Proc. Eur. Conf. Comput. Vis., 2018, pp. 801–818. https://openaccess.thecvf.com/content_ECCV_2018/html/Liang-Chieh_Chen_Encoder-Decoder_with_Atrous_ECCV_2018_paper.html
25. Su, B., Zhang, Q., Gong, Y. et al. Deep learning-based classification and segmentation for scalpels. Int J CARS. 2023;18(5):855-864. https://doi.org/10.1007/s11548-022-02825-7 DOI: https://doi.org/10.1007/s11548-022-02825-7
26. Wang, H., Xiao, N., Luo, S. et al. Multi-scale dense selective network based on border modeling for lung nodule segmentation. Int J CARS. 2023;18(5):845-853. https://doi.org/10.1007/s11548-022-02817-7 DOI: https://doi.org/10.1007/s11548-022-02817-7
27. Manjunatha, Y., Sharma, V., Iwahori, Y. et al. Lymph node detection in CT scans using modified U-Net with residual learning and 3D deep network. Int J CARS. 2023;18(4):723-732. https://doi.org/10.1007/s11548-022-02822-w DOI: https://doi.org/10.1007/s11548-022-02822-w
Downloads
Published
Data Availability Statement
The public datasets are identified in references 1 and 19, and the cited software repository in reference 20. Complete study-specific split files, the exact modified-model implementation, and individual validation-run results are not available for verification in this revision.
Issue
Section
License
Copyright (c) 2024 Raghavendra H. Bhalerao, Abhijeet Ashok Salunke, Shristi Sharan, Kamlesh Kumar, Priyank Rathod, Prince Kumar, Manish Chaturvedi, Nandlal Bharwani, Krupa Shah, Dhruv Patel, Keval Patel, Vikas Warikoo, Manisha Abhijeet Salunke, Shashank Pandya
This work is licensed under a Creative Commons Attribution 4.0 International License.
This license requires that reusers give credit to the creator. It allows reusers to distribute, remix, adapt, and build upon the material in any medium or format, even for commercial purposes.
