The Automatic Joint Teeth Segmentation in Panoramic Dental Images using Mask Recurrent Convolutional Neural Networks with Residual Feature Extraction:

Can it be useful in Oral Cancer Diagnosis and Management?

Authors

  • Raghavendra H. Bhalerao Dept of Electrical Engineering, IITRAM, Ahemdabad, India
  • Abhijeet Ashok Salunke Department of Surgical Oncology, The Gujarat Cancer and Research Institute, Ahemdabad, India https://orcid.org/0000-0003-0103-8599
  • Shristi Sharan Dept of Electrical Engineering, IITRAM, Ahemdabad, India
  • Kamlesh Kumar Dept of Electrical Engineering, IITRAM, Ahemdabad, India
  • Priyank Rathod Dept of Electrical Engineering, IITRAM, Ahemdabad, India
  • Prince Kumar Dept of Electrical Engineering, IITRAM, Ahemdabad, India
  • Manish Chaturvedi Dept of Electrical Engineering, IITRAM, Ahemdabad, India
  • Nandlal Bharwani Department of Surgical Oncology, The Gujarat Cancer and Research Institute, Ahemdabad, India
  • Krupa Shah Dept of Electrical Engineering, IITRAM, Ahemdabad, India
  • Dhruv Patel Department of Surgical Oncology, The Gujarat Cancer and Research Institute, Ahemdabad, India
  • Keval Patel Department of Surgical Oncology, The Gujarat Cancer and Research Institute, Ahemdabad, India
  • Vikas Warikoo Department of Surgical Oncology, The Gujarat Cancer and Research Institute, Ahemdabad, India
  • Manisha Abhijeet Salunke Dental Surgeon, Ahmedabad, India
  • Shashank Pandya Department of Surgical Oncology, The Gujarat Cancer and Research Institute, Ahemdabad, India

DOI:

https://doi.org/10.59667/sjoranm.v12i1.18

Keywords:

Medical image segmentation, Image segmentation using MRCNN Medical Imaging, Teeth segmentation, AI in Oral Cancer, CNN in Oral Cancer

Abstract

Introduction

Panoramic dental images gives an in-depth understanding of the tooth structure, both lower and upper jaws, and surrounding structures throughout the cavity in our mouth.The Panoramic dental images provided have significance for dental diagnostics since they aid in the detection of an array of dental disorders, including oral cancer. We propose a novel approach to automatic joint teeth segmentation using the pioneer Mask Recurrent Convolutional Neural Network (MRCNN) model for dental image segmentation.

Material and Methods

In this study, a sequence of residual blocks are used to construct a 62-layer feature extraction network in lieu of ResNet50/101 in MRCNN. To evaluate the efficacy of our method, the UFBA-UESC and Tufts dental image dataset (2500 panoramic dental x-rays) were utilised. 252 x-rays were used in test set, rest of the x-rays were utilised as training(1800 images) and validation datasets(448images) in ratio of 8:2 of the modified MRCNN model.

Results

Modified MRCNN achieved the final training and validation accuracies as 99.67% and 98.94%, respectively.The achieved accuracy of Dice coefficient (97.8%), Intersection over Union, (98.67%), and Pixel Accuracy(96.53%) respectively over the whole dataset. We also compare the performance of proposed model and other well established networks such as FPN, UNet, PSPNet, and DeepLabV3. The Modified MRCNN provides better results segmenting any two teeth which are close.

Conclusion

Our proposed method will serve as a valuable tool for automatic segmentation of individual teeth for medical management. This current method leads to higher accuracy and precision. Segmented images can be used to evaluate periodic changes, providing valuable data for assessing the progression of oral cancer and the efficacy of management.Future research should focus on developing  less complex, lightweight, and faster vision models while maintaining high accuracy.

References

1. G. Silva, L. Oliveira, and M. Pithon, “Automatic segmenting teeth in x-ray images: Trends, a novel data set, benchmarking and future perspectives,” Expert Systems with Applications, vol. 107, pp. 15 – 31, 2018. doi: https://doi.org/10.1016/j.eswa.2018.04.001

2. A. Lurie, G. M. Tosoni, J. Tsimikas, and W. Fitz, “Recursive hierarchic segmentation analysis of bone mineral density changes on digital panoramic images,” Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, vol. 113(4), pp. 549–558, 2012. https://doi.org/10.1016/j.oooo.2011.10.002

3. Y.Y. Amer and M. J. Aqel, “An efficient segmentation algorithm for panoramic dental images,” Procedia Computer Science, vol. 65, pp. 718–725, 2015, international Conference on Communications, management, and Information technology (ICCMIT’2015). https://doi.org/10.1016/j.procs.2015.09.016

4. M. K. Alsmadi, “A hybrid fuzzy c-means and neutrosophic for jaw lesions segmentation,” Ain Shams Engineering Journal, vol. 9, no. 4, pp. 697–706, 2018. https://doi.org/10.1016/j.asej.2016.03.016

5. M. R. M. Razali, N. S. Ahmad, R. Hassan, Z. M. Zaki, and W. Ismail, “Sobel and canny edges segmentations for the dental age assessment,” in 2014 International Conference on Computer Assisted System in Health, 2014, pp. 62–66. https://doi.org/10.1109/CASH.2014.10

6. G. Jader, J. Fontineli, M. Ruiz, K. Abdalla, M. Pithon, and L. Oliveira, “Deep instance segmentation of teeth in panoramic x-ray images,” in 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), 2018, pp. 400–407. https://doi.org/10.1109/SIBGRAPI.2018.00058

7. K. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in ´ 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp.2980–2988.

8. T. L. Koch, M. Perslev, C. Igel, and S. S. Brandt, “Accurate segmentation of dental panoramic radiographs with U-Nets,” in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 2019, pp. 15–19. https://doi.org/10.1109/ISBI.2019.8759563

9. O. Ronneberger, P.Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention (MICCAI), ser. LNCS, vol. 9351. Springer, 2015, pp. 234–241. https://doi.org/10.1007/978-3-319-24574-4_28

10. R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Region-Based Convolutional Networks for Accu-rate Object Detection and Segmentation” IEEE Transactions on Pattern Analysis and Machine In-telligence, vol. 38, no. 1, pp. 142–158, Jan. 2016, https://doi.org/10.1109/TPAMI.2015.2437384

11. R. Girshick, “Fast R-CNN,” in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1440–1448. https://doi.org/10.1109/ICCV.2015.169

12. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137–1149, Jun. 2017. https://doi.org/10.1109/TPAMI.2016.2577031

13. T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature Pyramid Networks for Object Detection”, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2117–2125. https://doi.org/10.1109/CVPR.2017.106

14. F. Deng, H. Hu, S. Chen, Q. Guan, and Y. Zou, “Rich feature hierarchies for cell detecting under phase contrast microscopy images”, in 2015 Sixth International Conference on Intelligent Control and Information Processing (ICICIP), Nov. 2015, pp. 348–353. https://doi.org/10.1109/ICICIP.2015.7388195

15. N. Atif, M. Bhuyan, and S. Ahamed, “A Review on Semantic Segmentation from a Modern Pers-pective,” in 2019 International Conference on Electrical, Electronics and Computer Engineering (UPCON), Nov. 2019, pp. 1–6. https://doi.org/10.1109/UPCON47278.2019.8980189

16. Simonyan, K., Zisserman, A.: Very deep convo-lutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)https://doi.org/10.48550/arXiv.1409.1556

17. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015) https://www.cv-foundation.org/openaccess/content_cvpr_2015/html/Szegedy_Going_Deeper_With_2015_CVPR_paper.html

18. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016) https://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html

19. K. Panetta, R. Rajendran, A. Ramesh, S. P. Rao and S. Agaian, "Tufts Dental Database: A Multimodal Panoramic X-Ray Dataset for Benchmarking Diagnostic Systems", in IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 4, pp. 1650-1659, April 2022, https://doi.org/10.1109/JBHI.2021.3117575

20. Mask-RCNN implementation for Tensorflow 2.7.0 and Keras 2.7.0. https://github.com/Kamlesh364/Mask-RCNN-TF2.7.0-keras2.7.0

21. Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested U-net architecture for medical image segmentation,” in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Berlin, Germany: Springer, 2018, pp. 3–11. https://doi.org/10.1007/978-3-030-00889-5_1

22. H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 2881–2890. https://openaccess.thecvf.com/content_cvpr_2017/html/Zhao_Pyramid_Scene_Parsing_CVPR_2017_paper.html

23. L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,” 2017, arXiv:1706.05587.

24. L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoderdecoder with atrous separable convolution for semantic image segmentation,” in Proc. Eur. Conf. Comput. Vis., 2018, pp. 801–818. https://openaccess.thecvf.com/content_ECCV_2018/html/Liang-Chieh_Chen_Encoder-Decoder_with_Atrous_ECCV_2018_paper.html

25. Su, B., Zhang, Q., Gong, Y. et al. Deep learning-based classification and segmentation for scalpels. Int J CARS (2023). https://doi.org/10.1007/s11548-022-02825-7

26. Wang, H., Xiao, N., Luo, S. et al. Multi-scale dense selective network based on border modeling for lung nodule segmentation. Int J CARS (2023). https://doi.org/10.1007/s11548-022-02817-7

27. Manjunatha, Y., Sharma, V., Iwahori, Y. et al. Lymph node detection in CT scans using modified U-Net with residual learning and 3D deep network. Int J CARS (2023). https://doi.org/10.1007/s11548-022-02822-w

Published

2024-09-25

How to Cite

The Automatic Joint Teeth Segmentation in Panoramic Dental Images using Mask Recurrent Convolutional Neural Networks with Residual Feature Extraction:: Can it be useful in Oral Cancer Diagnosis and Management?. (2024). Swiss Journal of Radiology and Nuclear Medicine, 12(1), 5-14. https://doi.org/10.59667/sjoranm.v12i1.18

Similar Articles

21-26 of 26

You may also start an advanced similarity search for this article.

Most read articles by the same author(s)