Deep Learning Technique for Non-Invasive Periodontitis Classification from Oral Thermal Images Using Vision Transformer

Nearly 3.5 billion people are suffering from oral gum health diseases [1, 2] according to World Health Organization (WHO) records. Recently, many people, irrespective of age, are affected by gum disease due to unhealthy food habits [3, 4]. Unhealthy food eating and improper maintenance of teeth lead to gum diseases starting from mild infectious gum and heart strokes [5], diabetes [6, 7]. Teeth are the first organ to be infected due to eating junk foods. The tooth infection arises due to unhealthy activities such as smoking, poor nutrition, and hormonal changes [8-10]. Tiny food particles deposit between gums and teeth, causing bacterial infections and leading to plaque formation [11]. The texture of plaque is a sticky and soft layer, which causes bacterial infections on the teeth. This infection should be treated immediately; otherwise, it leads to the calculus stage. Calculus, also known as tartar, is a hard texture deposited over the teeth due to the chemical reaction caused by the minerals and saliva inside the mouth. Calcium and phosphate are the main reasons for the hardened texture in teeth [12, 13]. Tartar is very rough and porous due to the chemical components such as Calcium Phosphate $\left(\left(\mathrm{Ca}_3 \mathrm{PO}_4\right)_2\right)$ Calcium Carbonate $\mathrm{CaCO}_3$ Magnesium Phosphate $\left(\mathrm{Mg}_3\left(\mathrm{PO}_4\right)_2\right)$ and Hydroxyapatite $\left(\mathrm{Ca}_{10}\left(\mathrm{PO}_4\right)_6(\mathrm{OH})_2\right)$. The above components are formed due to the process of crystallization with saliva and form a hard layer in the teeth. The calculus is of two types, such as Supragingival (above the gum) and subgingival (below the gum), as depicted in Figure 1.

Normally, supra-gingival calculus occurs near the openings of salivary glands (i.e., behind the lower teeth or the upper molars. The sediment appears yellow or white. In contrast, subgingival calculus is located at the root of the gums, where there is interaction with blood. It is often in dark brown or black. This condition results in swelling or irritation of the gums, medically termed as gum inflammation or gingivitis. This gum inflammation damages the soft tissues of the teeth and creates problems such as reddish, swollen gums [14], bleeding gums [15], pain while chewing, halitosis [16] (bad breath), bad taste [17], deep pockets between teeth and gums [18], receding gums [19], and tooth loss [20-22]. These sustained conditions lead to Periodontitis [23-25], which is a chronic inflammatory disease. The accumulation of sticky thin film damages the bones and tissues surrounding the teeth and gums. Periodontitis is the starting stage of tooth bone rupture and tissue damage. It is reversible with the removal of the contaminated particles from the teeth. The appearance of the teeth changes at different stages of infection, as shown in Figure 2. Through continuous dental checkups and the scaling process, the above condition can be reversed without proceeding to further growth of dental disease.

Figure 1. Differences between supergingival and subgingival

Figure 2. Comparison of healthy teeth versus different stages of infectious teeth

The infection worsens if it is not controlled at the early stages, affecting the roots of the teeth and leading to the loss of teeth. The growth of periodontitis disease is not consistent among individuals; it can be managed with proper self-care and periodic dental checkups [26-28]. While dental checkups can be expensive, they are recommended for severely affected patients to reinforce personal hygiene and plaque control.

Untreated Periodontitis not only affects the teeth and gums but also causes severe respiratory syndromes [29], heart strokes [30], low blood pressure [31], and diabetes [32, 33]. Hence, continuous monitoring and an effective treatment plan are essential. Some common treatments for gingivitis and Periodontitis include scaling, antibiotics, root planning, and surgical options. Surgical operations depend on the severity of the infection. The early stages of tooth decay are corrected with surgical procedures such as fillings, replacing crowns, and root canal therapy. The gingivitis disease is reversed through professional cleaning and continuous maintenance of oral hygiene. The Periodontitis is corrected with scaling, the utilization of antibiotics, and root canal surgery. The worst case of Periodontitis is irreversible and leads to the loss of a tooth. Predicting the plaque and gingivitis at an earlier stage helps to treat the infection effectively, and their respective symptoms are listed in Figure 3.

Figure 3. Common symptoms of oral disease

This proposed methodology classifies oral diseases such as coronal plaque, gingivitis, i.e., supragingival and subgingival, periodontitis, and healthy teeth using a created thermal image dataset. Thermal image differentiates the degradation of the gums precisely by comparing the visible images. The heat contours of the affected teeth are used to identify the severity of the oral infection.

3.1 Data collection

A total of 500 thermal images were collected for experimental purposes from Kaviya dental clinic, and they were converted to thermal images using the HIKIMICRO thermal camera. Among 500 images, 400 images are used for training, and 100 images are used for testing purposes. The teeth thermal images are captured from the predefined datasets that are available online. Apart from this, nearly 300 visible and thermal images have been collected from Mendeley [20] and Kaggle [27] online dataset is used for training the classification model. The overall process and flow of this research methodology are depicted in Figure 4.

Figure 4. Architecture of the proposed model for classifying the coronal plaque, gingivitis, and Periodontitis using thermal image

For teeth thermal data collection and for validation of the proposed model, Dr. R.S. Kaviya, B.D.S., Dental Surgeon, Reg.No: 22049, has helped with clinical diagnosis and extended her full support for this research. As a part of the data collection totally 30 patients were subjected to testing, and they were interviewed. The patients were informed prior to, and their consent was obtained as per the ethical guidelines. This proposed study has been granted ethical clearance under the Reference Number 366/IEC/2024 by the Institutional Ethics Committee of Kaviya Dental Clinic, Chennai, Tamil Nadu, India. This proposed study utilized a content analysis approach to categorize the clinical and patient data, later used for classification of the gingivitis, periodontics, and plaque with the proposed classification model. The interviews were conducted by the treating doctor, with whom patients had an existing clinical relationship.

Participants were briefed about the research proposal, and proper consent was obtained. The interviewer (doctor) is a qualified medical professional with experience of 8 years in dental patient’s care and clinical research. This doctor had a clinical interest in improving the health status of the patients. Participants were selected using consecutive sampling, where all the eligible patients visited during the clinical study period were included with their consent. Since the doctor – patient relationship was smooth due to the trust, all the patients agreed to participate in this study after signing the consent form. To ensure privacy and confidentiality, the interview was conducted exclusively between the doctor and patients without any intervention of a third party. The dataset has been collected through face-to-face, semi-structured interviews, which were carried out in a clinic.

Among 30 patients, 5 were healthy, 10 were affected with plaque, 10 were infected with gingivitis, and 5 were affected with periodontitis. There were 15 men and 15 women in the age group between 18 and 60 years. A semi-structured interview with predefined sample questions was created with the joint effort of the authors and the clinical doctor. Each patient was interviewed (10–15 minutes) once during their clinical visit, and their consent was obtained for only picturing their teeth with a thermal camera. The verbal conversation was not recorded, and only the thermal images were acquired for our research study. We observed data saturation by the 25^th patient, where no new symptoms were identified. This proposed study used the collected thermal images for earlier diagnosis of gum diseases using the proposed classification CNN - ViT model.

3.2 Teeth thermal image pre-processing

The acquired thermal images are pre-processed to enhance the pixel quality and perform the analysis. Pre-processing of teeth thermal images eradicates the noisy pixels and improves the proposed model’s classification accuracy rate. In acquired thermal images, the histogram equalization is applied and highlights the subtle features around the teeth and gums. Thermal image pixels are visualized in terms of red (hot regions) and blue (cold regions) colors. There are errors in the distribution of pixels due to the sensors. To rectify it, thermal images are subjected to the CLAHE process that tunes the image contrast as shown in Figure 5. The unevenly distributed pixels are focused and neutralized to improve the visibility of the image features. Further to observe the degradation of teeth and gums, the proposed pre-processing CLAHE method is applied to thermal images and corrected using gamma correction. It equalizes the contrast and brightness level of the pixels and improves the quality of the teeth thermal image. The medical images have noise signals due to the patient's movement and electronic interference.

The variation of the image pixels between the original thermal images (Figure 6(a)) and the noisy thermal image is as in Figure 6(b). and their corresponding histogram are shown in Figure 6. Each bin of the histogram represents a particular range of pixel intensity, and its height represents how many times that particular range of pixel are present in the teeth thermal image. When comparing these three histograms, it is clear that the noisy thermal image has an uneven distribution of pixels compared to the original and de-noised image, as shown in Figure 6(c). To analyze the pixel distribution, the gum regions are chosen across three images, and their pixel intensity profiles are compared. The intensity of the pixels is compared across the original input image, noisy image, and pre-processed image, and shown in Figure 7. In Figure 7, the ‘x-axis’ represents the pixel position in the selected region and the ‘y-axis’ represents the corresponding intensity value. This approach is used to observe the changes in the patterns, texture, and contrast of the image. It is inferred that the noisy image has an uneven distribution of pixels when compared to the original image. However, the de-noised image has a pixel variation similar to that of the original image. Hence, it is evident that the proposed pre-processing method is suitable for thermal image processing.

Figure 5. Proposed pre-processing methods using Contrast Limited Adaptive Histogram Equalization (CLAHE), gamma correction, and a Gaussian filter

Figure 6. Analyzing the variation of image pixels using the histogram

Figure 7. Intensity profile comparison of the original, noisy, and de-noised image

3.3 Image feature extraction and segmentation

The pre-processed thermal images have enhanced distinct features that distinguish the object from the background. The features, such as edges and contours of the teeth and gums, are extracted efficiently to classify the diseases. In this proposed work, the features from the pre-processed thermal image are extracted using the TLBP Feature Extraction algorithm. The TLBP algorithm performs effectively for teeth thermal images because it is rotation invariant, highlights, and distinguishes the subtle patterns. The proposed TLBP algorithm compares the neighboring pixel intensity and analyzes its pattern in binary format. If the intensity of the neighboring pixel ($I_p$) is greater than or equal to the central pixel's intensity $I(x, y)$ then that region is considered as a brighter region (1). In case it is dark and less than the central pixel, the value is assigned as '0 '.

The mathematical representation for this TLBP is given in Eq. (1).

$L B P_{P, R}(x, y)=\sum_{p=0}^{P-1} s\left(I_p-I(x, y)\right) \cdot 2^p$                 (1)

The intensity of the center pixel $I(x, y)$ is compared with neighboring pixel $I_p$ and further undergoes step function $s(x)$ in which the brighter region is assigned as '1' and the darker region is assigned as '0'. Finally, the regions of interest across thermal images are combined in the form of a histogram as shown in Figure 8.

Figure 8. Tuned Linear Binary Pattern (TLBP) based feature extraction and Canny edge-based contour segmentation

Table 2. Tuning and analyzing the parameters of the linear binary pattern

In this proposed TLBP method, the LBP parameters such as radius (R) and number of circularly symmetric neighbors (P) are tuned, and the features are extracted precisely. These parameters are tuned, and their observed values are tabulated in Table 2. It is inferred that the alignment of the pixel with its corresponding grid location is not possible with the higher R value. Therefore, it requires the bilinear interpolation process, which is computationally expensive. The larger P value needs more neighbor comparisons; therefore, it results in a longer binary encoding process. Considering all the above factors, radius R is chosen as ‘1’ and the number of neighbors P is chosen as ‘8’in this proposed TLBP method. The Tuned LBP converts the resultant binary patterns into feature vectors that denote the texture of the teeth image. Though it extracts the meaningful information from thermal images, particular features are used for the classification of gun disease. The remaining non-important features may introduce noise during the classification of gum disease. To avoid this, the extracted feature vectors are further optimized using the Pelican Optimization Algorithm (POA) and hippopotamus optimization algorithm (HOA). The above algorithm searches the feature space and focuses on the promising areas in the feature vector. These methods narrow down the image features by filtering out the redundant and less important details. The obtained LBP feature vector is given as input to both the POA and HOA. The essential texture information is selected accurately through several iterative optimizations. The performance of both POA and HOA is shown in Figure 9 as follows. The pixels are distributed as in Figure 9(a), and the important features are extracted using the POA algorithm. The HOA algorithm has uneven pixel distribution as shown in Figure 9(b).

Figure 9. Comparison of the performance of the Pelican Optimization Algorithm and the hippopotamus optimization algorithm (HOA) using a histogram

The POA algorithm selects 56 features from the input thermal image and predicts gingival infection in 4.21 seconds. The HOA algorithm selects 73 features from the thermal image and predicts the infection in 6.87 s. The prediction accuracy rate for POA is 93.5%, whereas 90.2% for the HOA method. It is evident that the prominent features are selected better using POA comparing the HOA in this proposed work. When comparing the HOA and POA, the POA balances the precision and speed, focusing on the critical features. Hence, POA is preferred for this proposed study.

3.4 Pelicon optimization algorithm

The POA algorithm selects the most discriminative features from the teeth image and reduces the dimensions. LBP algorithm extracts a high-dimensional feature vector, which makes the classification harder. The LBP algorithm acquires the textures and temperature variations in the obtained teeth image. However, it generates high-dimensional feature vectors that contain redundant or irrelevant thermal features, which increases the computational complexity. In order to improve the classification model’s performance, the POA algorithm is used sequentially after the LBP feature extraction process, as in Eq. (2).

$F($LBP features$)=\left\{f_1, f_2, \ldots f_n\right\}$            (2)

In Eq. (2), let $f_1, f_2, \ldots f_n$ be the LBP feature vector extracted from the input thermal teeth image. Similarly, the POA features are extracted from the input thermal image using Eq. (3).

$($POA features$)=\left\{p_1, p_2, \ldots p_m\right\} \subseteq \mathrm{F}$              (3)

where, $m<n$.

In this proposed work, each pelican in the POA algorithm denotes a binary feature mask $X_i$ as in Eq. (4), where:

$X_i=\left\{x_{i 1}, x_{i 2}, \ldots x_{i n}\right\}$             (4)

If $x_{i j}=1$ then it denotes that $j^{\text {th}}$ LBP feature is selected from thermal image. Else if $x_{i j}=0$ then that particular irrelevant feature is discarded. This process is repeated until the fitness of each pelican is evaluated and its positions are updated through the optimization process. Once the optimal features are obtained, they are masked using the sigmoid function, and it is given in Eq. (5).

$S(x)=\frac{1}{1+e^{-x}}$, then $x_{i j}=\left\{\begin{array}{c}1, \text { if } S(x)>\operatorname{rand}() \\ 0, \quad \text { Otherwise }\end{array}\right.$              (5)

In this proposed method, the POA algorithm efficiently handles high-dimensional LBP feature vectors by reducing the redundant and noisy features. The extracted POA features are passed into the Canny edge detection model, which detects the contour of the teeth from the thermal image. The edge detector primarily focuses on the infected gingival region and segments the gum region for classification. Canny detector enhances the strong gradients and de-noises the irrelevant pixels. Canny edge detection is superior to other algorithms because it predicts even the prominent edges using the gradient intensity and orientation. As a result, the boundaries and structures of human teeth and gums are visualized clearly and avoid false edge identification. It focuses on both the dark and bright regions of edges and defines the boundary effectively without any compromise, as shown in Figure 10. The Gaussian filters are used as the pre-processing method prior to the Canny edge segmentation process. It helps to smooth the thermal image features and reduces noise. The Canny edge detector employs both upper threshold and lower threshold values for focusing on the prominent features of teeth. These gradients are essential in edge-linking steps for determining the edges and contours of the gingiva.

Figure 10. Effect of tuning the Gaussian filter size on edge detection using the Canny method

Figure 11. Heat map visualization of the Canny edge detector for extracting features from the gingivitis thermal image

The Gaussian filter size is tuned up to ‘value = 5’ and enhances the edges and boundaries (Figure 10). Further, the edge pixel count of dental thermal images is computed subject to different threshold values ranging from ‘0.1’ to ‘0.9’. This pixel count plays an important role in defining the contours of a pre-processed image. Figure 11 provides an insight into the allocation of pixel count with its respective threshold value. This heat map elaborates the pixel allocation and helps to determine the position of the meaningful features with less noise. The yellow regions (left corner) have more texture details and noise, whereas the blue or purple regions have low noise with few edge points (right corner). Hence, the Canny edge detection algorithms and Gaussian filters identify the important edges and contours for the classification of the coronal plaque, gingivitis, and Periodontitis.

Flow of the proposed CNN-ViT classification algorithm.

3.5 Image optimization and classification

In this proposed work, the extracted contours are analyzed using the hybrid Convolutional Neural Network with Vision Information Transformers (CNN-ViT) for gum disease classification. The CNN model extracts the low-level features such as corners, contours, edges, and textures from the segmented thermal image. The convolution and pooling operations detect small abnormalities in the gum regions of teeth. In contrast, ViT models are capable of capturing the global spatial relationship by understanding the structure of the mouth and teeth. Once the local features are extracted using the CNN, it is passed to the ViT layers, and the images are segregated into patches. From continuous analysis, the patterns across the mouth regions are learned, and this process is repeated until it reads all the regions of the mouth. The combination leverages the strength of the classification model and enhances the medial image processing. The overall working of the proposed CNN-ViT model is briefed using algorithmic steps in the section below.

This proposed hybrid model focuses on both the local and global features without any compromise. Initially, the pre-processed image is divided into 196 patches. Patches are flattened linearly and reduces its dimension and convert into 1D vector. This 1D vector retains the essential information without any loss, and it is embedded as a discrete token. Each patch of the image is considered as a discrete token and fed as input to the transformer layers. The Multi-Head Self Attention (MHSA) mechanism observes the dependencies between the tokens to order the discrete tokens. The heads of the MHSA are tuned up to 12, and inferred that it requires more training and inference time. So, the number of heads is assigned as ‘8’and its respective attention score is calculated using the dot product. After this process, a residual connection is created to extract the information from previous layers, and the outputs are standardized using the layer normalization parameters (mean & variance). The vanishing gradient problem is addressed using residual connections, whereas the internal covariate is stabilized with the normalization process. Each token is subjected to the non-linear transformations through a Feed-Forward Neural Network (FFN) and enhances the performance of the model. The Rectified Linear Unit (ReLU) activation function is used in this network because it is nonlinear, computationally efficient, and mitigates the vanishing gradient problem. The performance of the proposed CNN-ViT algorithm during the training process is shown in Figure 12. The convolution layers study the features effectively with a learning rate of 0.001% and achieve an accuracy rate of 96.78% approximately. The learning process continues for 30 iterations, and finally, the classification layer classifies the types of oral disease. Its validation is done using statistical metrics.

Figure 12. Training performance of the proposed Convolutional Neural Network with Vision Information Transformers (CNN-ViT) algorithm

The presence of Coronal plaque, Gingivitis, and Periodontitis affects both physical and mental health of the person. It leads to social embarrassment and isolation due to halitosis (bad breath). Considering the above problem, the proposed study differentiates the healthy teeth and infectious teeth due to coronal plaque, gingivitis, and Periodontitis using the Tuned CNN-ViT classification method. Thermal images are acquired using a smartphone attached with a thermal USB camera and then pre-processed using the proposed hybrid model (CLAHE, gamma correction & Gaussian Noise filter) to eradicate the noise signal and improve the image quality. The pre-processing techniques equalize the image pixels and tune the contrast of the image using gamma correction. Then the features are extracted using the Tuned LBP algorithm, and further, it is optimized using POA to acquire vital features of the thermal image. Based on the severity of the tooth infection, the prognosis is analyzed and determined. This avoids unnecessary tooth loss, bone loss, and malignant diseases. The early prediction of Coronal plaque and Gingivitis avoids the occurrence of Periodontitis, tooth loss, and other malignant diseases. The proposed CNN-ViT method identifies the healthy teeth and infectious teeth, i.e., coronal plaque, gingivitis, and periodontitis, using the Tuned CNN-ViT classification method with an accuracy of 98.95% approximately. The Proposed CNN-ViT method assists the dentists and analyzes the oral thermal images efficiently, and supports immediate diagnosis, as shown in Figure 20. For the experimental set up HIKIMICRO thermal camera is attached to the RealMe Narzo 70 smartphone to diagnose the plaque, gingivitis, and periodontitis. As future work, different machine learning algorithms such as tuned Support Vector Machine (SVM), Random Forest, and XGBoost can be used for extraction or alternate feature selection, and their performance can be investigated for optimal results. Different thermal image palettes can be utilized, and their performance can be analyzed through experimentation using the proposed real-time setup. As future work, different machine learning and deep learning algorithms will be investigated to find the optimal model. Also, different thermal images from the Seek compact Pro and FLIR thermal camera can be utilized, and their performance can be analyzed through experimentation.

5.1 Clinical relevance

5.1.1 Scientific rationale for the study

• Effective classification of coronal plaque, gingivitis, and periodontitis helps in understanding the disease progression and its risk.
• The advanced technologies (CNN & ViT) improve the accuracy of clinical diagnostics.

5.1.2 Principal findings

• Thermal image-based feature extraction and segmentation identify the plaque density well by focusing on the prominent feature.
• Predicting the disease at an early stage improves the treatment process and the patient’s prognosis.

5.1.3 Practical implications

• The AI-based classification streamlines the diagnostic processes that save time for clinicians, and also prevents severe diseases like diabetes and cardiovascular issues.