Deep learning model using YOLO-v5 for detecting and numbering of teeth in dental bitewing images

INTRODUCTION

Making an accurate diagnosis is one of the most criti­cal tasks in dental practice. In addition to dentist’s experience, there are complementary technologies, which can provide essential data to evaluate clinical situation [1]. Radiographic examination plays a cornerstone role in a patient’s dental journey [2, 3].
In the field of dentistry, different scanning techniques have been implemented in the same anatomical region of the same person [4]. Intra-oral radiography methods, such as bitewing and periapical radiographs, are also frequently used. When compared with periapical radiographs, bitewing images offer views of the crown, root, and part of the bone surrounding the teeth in the upper and lower jaw, and more teeth can be analyzed on a single image. Bitewing radiographs can identify interproximal caries, alveolar bone loss, and calculus [5]. Although they are undoubtedly helpful, their interpretation will always depend on the expertise of the person analyzing them. More experienced examiners were found to have better diagnostic accuracy than less experienced ones [6]. Any skilled physician, however, might suffer from both mental and visual weariness that may lead to missing important radiographic indicates, which can influence their final diagnostic conclusions, and result in misdiagnosis [7].
Convolutional neural networks (CNNs), an advancement in artificial intelligence (AI) as well as deep learning (DL), which promise to enhance and expedite diagnosis, have grown in popularity in recent years. Therefore, the afore-mentioned issues may be resolved by automated dental radiography diagnostic systems, which can eliminate the variations caused by subjectivity of different practitioners [4].
In general, dental imaging (both 2D and 3D) as innovative technology, has demonstrated exceptional performance [8-10]. Majority of studies assess how well CNN performs in identifying anatomical features, medical procedures, and disorders [11-13]. When reporting CNNs results, the detection and classification tasks, which refer to the process of object identification and categorization, respectively, are very frequently investigated [14]. One of them, tooth detection and numbering, is a fundamental subject that has been explored in research projects with the application of AI in evaluation of dental radiographs [15]. This task is important for medico-legal experts in identification of an individual, but can also be used to create statistical data and dental charts [16, 17]. To our knowledge, few studies have used DL techniques to detect and number teeth in bitewing images [18-20].

OBJECTIVES

The current study aimed to develop a DL model for detecting and numbering of teeth in bitewing images, and to validate the diagnostic performance of the developed model.

MATERIAL AND METHODS

This study was approved by the Institutional Review Board at Saveetha Dental College (decision date and approval number: SRB/SDC/Faculty/22/Endo/060), and complied with the Declaration of Helsinki and ethical guidelines of the responsible committee.

IMAGE DATASET PROTECTION AND SAMPLING

The study sample consisted of 3,491 anonymized bitewing radiographs, exported in JPEG, and acquired randomly using various intra-oral phosphor plates with different sizes and resolutions. Radiographs were taken using three different X-ray units, according to the manufacturer’s instructions. Their exposure conditions are summarized in Table 1. Radiographs of patients with permanent dentition (without missing teeth or dental implants) and only good diagnostic quality bitewing images were included in the study, while radiographs with mixed or deciduous teeth, motion artifacts, or root fragments were excluded.

PROCESSING IMAGE DATASETS AND REFERENCE TESTING

Ground truth labeling was performed with CranioCatch annotation software (CranioCatch, Eskisehir, Turkey) by 2 dentomaxillofacial radiologists (CG, ISB) in agreement on all images. Experts were instructed to draw minimum-size boxes around every tooth (including the whole crown and root), and to number each box with two digits from FDI system (13-18, 43-48 for right-side bitewing radiographs, and 23-28, 33-38 for left-side bitewing radiographs).

ARCHITECTURE OF CNN

Methods for detecting objects involve one-stage and two-stage techniques. Single-stage approaches are popular for their speed and accuracy, as they are able to perform both classification and position detection tasks simultaneously [21]. You only look once (YOLO) is the most common one-step object detection method, and is characterized by simple data processing network that perform both object detection and classification at the same time [22].
Several versions of YOLO architecture exist, but tests in the present study were carried out with YOLO-v5. This is because YOLO-v5 is lighter and faster than previous version, YOLO-v4 [23]. Additionally, it has been proven to be very fast and achieving good results. The YOLO-v5 model consists of three parts: CSP-DarkNet53 as the backbone, SPP, and PANet in the model neck and head used in YOLO-v4 [24].
To describe these models, it is necessary to clarify some technical terms. Several hyper-parameters need to be set before starting of the learning process. One of these is the learning rate that controls the speed at which the network updates its parameters. Epochs define how many times one complete instance traverses this network during training. Total number of datasets trained in a single batch is defined as the batch size.

ESTABLISHMENT OF DATASETS

CLASSIFICATION MODEL

All images were re-sized to 224 × 224 pixels. The 3,491 anonymized bitewing images were randomized into training, validation, and test sets of 2,792, 367, and 332 radiographs, respectively. The proposed AI model approach for classifying bitewing radiographs as the left or right was trained with YOLO-v5. A batch size of 32, learning rate of 0.01, and epoch number of 100 were employed.

TOOTH DETECTION AND NUMBERING MODEL OF THE RIGHT BITEWING RADIOGRAPHS

The following categories for 1,887 anonymous bitewing images were chosen at random – train: 1,496 im-ages, 12,950 labels; validation: 195 images, 1,598 labels; test: 196 images, 1,584 labels.
The proposed AI model approach to detect and number teeth in the right bitewing images using labels including 13-18 and 43-48 was based on deep CNN. The hyper- parameters applied to train YOLO-v5 were a batch size of 32, learning rate of 0.01, and epoch number of 300.
Contrast limited adaptive histogram equalization (CLAHE), and data augmentation were utilized to train and validate. The total number of images was 5,269. Train: 1,496 × 3 = 4,488 images, 38,850 label; validation: 195 × 3 = 585 images, 4,794 labels; test: 196 images, 1,584 labels.

TOOTH DETECTION AND NUMBERING MODEL OF THE LEFT BITEWING RADIOGRAPHS

The following categories for 1,604 anonymous bitewing images were chosen at random – train: 1,273 im-ages, 10,907 labels; validation: 165 images, 1,316 labels; test: 166 images, 1,362 labels.
The proposed AI model approach to detect and number teeth in the left bitewing radiographs using la-bels including 23-28 and 33-38 was based on deep CNN. The hyper-parameters applied to train YOLO-v5 were a batch size of 32, learning rate of 0.01, and epoch number of 300.
For the training and validation groups, CLAHE and data augmentation were utilized. The total number of images was 4,480. Train: 1,273 × 3 = 3,819 images, 32,721 labels; validation: 165 × 3 = 495 images, 3,948 labels; test: 166 images, 1,362 labels.

METRICS FOR VALIDATION AND STATISTICAL ANALYSIS

The most common metrics used in studies on tooth detection and numbering are precision, sensitivity, and F1 score. The true positive (TP), false positive (FP), and false negative (FN) numbers were calculated. After the calculation of TP, FP, and FN, the following metrics were derived: sensitivity = TP/(TP + FN), precision = TP/(TP + FP), and F1 score = 2TP/(2TP + FP + FN).

RESULTS

The Eskisehir Osmangazi University Faculty of Dentistry Dental-AI Laboratory’s computer equipment with Dell PowerEdge T640 GPU Calculation Server and NVIDIA Tesla with a V100 16G Passive GPU (Dell Inc., Texas, USA), was used for the experiments. For the classification of bitewing radiographs, promising outcomes were obtained with deep CNN architecture-based AI models. The classification performance of YOLO-v5, including accuracy, sensitivity, specificity, and NPV on the test set is presented in Table 2. The sensitivity and precision for the classification task were 0.9940 and 1, respectively. The estimated F1 score was 0.99970, indicating good recall/precision balance. The proposed DL performed well in detecting and numbering permanent teeth in bitewing images. The results are presented in Table 3.
Figures 1 and 2 demonstrate the confusion matrix, showing the diagnostic results. Higher diagonals and darker blue in confusion matrices indicate higher accuracy. On the right side, the diagnostic accuracy was the highest for upper premolars (97.0%) and the lowest for upper third molars (76%). On the left side, the diagnostic accuracy was the highest for maxillary second premolars (98.0%) and the lowest for mandibular canine (71%). The graphics of F1-confidence curve, precision-confidence curve, precision-recall curve, and recall-confidence curve are shown in Figures 3 and 4.
Figures 5 and 6 describe the training and validation loss curves for the right and left sides, respectively.
The goal of training an object detection model is to minimize total loss, which is a combination of box loss, object loss, and class loss. The loss values should tend to decrease as training proceeds, which indicates improvement in the ability of the model to detect teeth in images. Examples of automatic tooth numbering using the YOLO-v5 model are illustrated in Figures 7 and 8.

DISCUSSION

Applying AI to medicine is regarded as one of the most impressive advancements accomplished in recent years. Restorative dentistry [25-27], oral and maxillofacial surgery [28, 29], and forensic dentistry [30, 31] are some of dental specialties, which have already adopted this novelty.
In dental radiology, several AI systems were proposed for 2D imaging, and many studies attempted to automate the diagnosis of pathology and therapy in panoramic [32-34] and intra-oral radiographs [25, 35, 36]. Furthermore, AI-based software was employed to analyze and automatically recognize anatomical components in 3D images [37-39].
Bitewing radiography is a technique that can obtain parts of maxillary and mandibular teeth on a single film. This method diagnoses inter-proximal caries, which are difficult to detect, and secondary caries under restorations on extra-oral radiographs [5]. AI applications trained with deep learning-trained AI systems started to identify a wide range of dental disorders, including missing teeth, crowns, caries, periodontal dis-eases, and periapical pathologies. Some AI research employed bitewing radiographs for dental diagnostic process [5, 18, 40].
Furthermore, several studies using Bayesian approaches, linear models, or binary support vector ma-chines have been conducted to detect and number teeth [41-43]. However, manual curation of image features is often required, and majority of these systems usually perform an image-enhancing procedure before features’ ex-traction and segmentation. It is time-consuming, producing high workload. The extracted feature quality has a significant impact on the performance of image recognition [44].
Lin et al. [43] developed a model to number and classify teeth based on the tooth region and contour in-formation from bitewing images. The study showed that tooth and pulp shapes were an important part of the classification for correct tooth numbering; therefore, segmentation improved classification. Morpho-lo­gical transformations are usually not considered in dental radiographs due to noise, low contrast, and non-uniform exposure. Although good results have been demonstrated, the classification of poor images, such as those with only 3 or 4 teeth, inconsistent tooth arrangement in the maxilla and mandible, or over-lapping teeth in both jaws, is still a major challenge.
A study by Aeini and Mahmoudi [42] proposed an algorithm to number and classify teeth using the universal numbering scheme based on mesiodistal distance. Their data consisting of 476 dental radio-graphs is superior to several published studies on the number of teeth, number of fractured teeth, and number and type of edentulous teeth. It was found that contour-based classification was preferable when there was less than a 0.3 percent difference between different teeth. However, if only one type of tooth was present in the jaw, the type of tooth could not be determined from the mesiodistal data.
Recently, AI and DL have made profound developments in computer vision, including object, face, and activity recognition, tracking, 3D mapping, and locali­zation. There are recent advancements in DLs, which can automatically extract image characteristics by utilizing original pixel data as input. These systems drastically minimize the amount of work required, and are able to detect some features, which even human eye finds difficult to identify. The classification of medical images using CNN-based DL has recently gained im-portance [45]. For tooth detection and numbering, Chen et al. [35] utilized a faster R-CNN in TensorFlow framework. To test the method, the authors gathered 1,250 periapical radiographs in total. The study’s outcome measurements were insufficient before the application of post-processing procedures. Even the cases, where two “half teeth” were incorrect for one tooth, were considered insufficient, despite their improvement.
Yasa et al. [18] reported an AI model technique to detect and number teeth with Faster R-CNN algorithm. In the test data of the dataset, 697 of the 715 teeth identified in 109 bitewing images utilized in the research were precisely numbered and classified. The confusion matrix values were applied to measure the neural network accuracy. However, it was not possible to identify the exact borders of teeth using the proposed method.
In a study published by Tekin et al. [19], their DL approach had a neural network structure that operated by region. A region-based automated segmentation system that supported the analysis with masking, de-creased the effort on the expert. One technique for segmentation is Mask R-CNN that, when employed on each ROI, predicts a pixel-by-pixel segmentation mask. The segmented teeth are classified and numbered using a FDI notation by the tooth numbering module. Its experimental results showed a precision of 100% and a mAP of 97.49%.
For the tooth numbering, an accuracy of 94.35% and a mAP of 91.51% were achieved. Imaging and analyzing the dental structure are essential for diagnosing disease, monitoring oral health, and planning treatment [3, 18]. To help professionals automatically identify and number teeth in bitewing radiographs, this study evaluated the clinical usefulness of an AI in terms of accuracy. This research suggests that AI-based software can help improving efficiency in detecting teeth in bitewing images. However, due to the anatomical symmetry of the mouth and the alignment of the upper and lower teeth, it can be misclassified. Unlike panoramic radiographs, the bitewing radiographs used in the study only allow for imaging of certain areas of the mouth, not the whole mouth. In particular, canines and third molars can appear in a small area in some bitewing radiographs and larger areas in others. This reduces the detection capability of the model, as YOLO-v5 is not optimized to detect small objects [46].
Several AI research papers have suggested that the use of multiple neural networks for different detection tasks could improve CNN performance. This was proven to help learning faster, therefore producing more accurate results [7, 32]. A study conducted by Leite et al. [7] showed positive results in detecting, numbering, and segmenting teeth in panoramic radiographs. In particular, the researchers noted that the best performance came from combining DeepLabv3 CNN with ResNet101 CNN. Two different types of neural networks have also been used in previous studies to improve the performance of CNN. Researchers employed the latest version of Faster R-CNN model to detect teeth from radiographs, and VGG-16 architecture to number the detected dental structures [14, 32].
The potential of YOLO-v5 for automated diagnostic processes in bitewing radiographs and dental image interpretation were demonstrated in this study. In terms of tooth detection and numbering, the model showed high performance. However, the model performed worst in areas partially visible in bitewing radio-graphs (especially canines and third molars), and within the image area of varying size. In future studies, this problem may be overcome by using a combination of one-stage and two-stage object detection algorithms.
The current study have several limitations. First, there were no complex images in our dataset. Further studies can include primary teeth, implants, root fragments, and teeth with severely damaged crowns. Second, the performance of the experts was not assessed. More research are needed to support clinicians and provide meaningful clinical implications.

CONCLUSIONS

The impact of YOLO-v5 on tooth detection and classification was demonstrated using the experimental results of the proposed system. Even with good results obtained, the classification of poor images, such as inconsistent tooth arrangement in both jaws and overlapping teeth, is still a major challenge. Future research should test different types of neural networks and architectures, and find a better way to improve the results of tooth detection.

DISCLOSURES

1. Institutional review board statement: The study was approved by the Institutional Review Board at Sa­veetha Dental College (decision date and approval number: SRB/SDC/Faculty/22/Endo/060).
2. Assistance with the article: None.
3. Financial support and sponsorship: None.
4. Conflicts of interest: The authors declare no potential conflicts of interest concerning the research, authorship, and/or publication of this article.

References