Enhanced Karyotyping Through Deep Learning-Assisted Segmentation and Classification of Chromosomal Cells

Recent improvements in science and engineering have led to the automation of numerous examination procedures, particularly in the examination of biological data. The improvements in computing facilities and the accessibility of recent computational tools have empowered us to implement a wide range of diagnostic and detection procedures. This enables accurate evaluation of biological and medical information collected from the human subjects [1, 2].

In the current era, Artificial Intelligence (AI) plans are widely employed to solve a number of complex engineering tasks including medical image processing problems [3]. The advancements in the AI also helps to develop and implement various procedures to examine the complex medical images. As a result of the current research in the biological and medical fields, it has been confirmed that AI frameworks such as heuristic algorithms, Machine-Learning Methods (MLMs) as well as Deep-Learning Mechanisms (DLMs) are being utilized to increase automatic diagnosis accuracy [4, 5].

As part of a genetic disorder examination, chromosome cell (CC) karyotyping is a common biological test. In this task, a person can be examined using the following information; number of available CCs, deletion of a chromosome, duplication of a chromosome, and chromosomes with unusual structures. These analyses play a vital role during the clinical level diagnosis of Edwards syndrome, Turner syndrome, and Down syndrome are examples of genetic illnesses. As an automatic assessment of individual cells, segmentation and classification are involved. As far back as the early cytogeneticists were concerned, the CC was classified as twenty-two pairs of autosomes and one set of sex chromosomes. The automatic classification of the CC is based on its texture and shape features. There are two types of features in a CC: Handcrafted (traditional) Features (HF) as well as Learned Features (LF) derived with the help of a chosen Artificial Intelligence scheme. The past few years, the availability of modern computing facilities has led to the implementation of DLM in a variety of domains, including CC karyotyping, because of the accessibility of the modern computing facility. In order to achieve a higher level of clinical significance, a DLM can be implemented to perform automatic segmentation and classification. As a result, DLM schemes have been widely applied in research works related to CC karyotyping that have been conducted recently [6, 7]. Earlier works on CC karyotyping suggests that the existing procedures provides a satisfactory result on this task and it needs to be improved further with a modified Convolutional Neural Network (CNN) scheme. In this research, we plan to create a framework by integrating the Convolutional Neural Network -supported Segmenting and classification to enhance the results of CC karyotyping by integrating CNN-supported segmentation and classification approaches. A new feature of this work is the implementation of VGG-UNet (VGG19 acts as the backbone) for segmenting the CC region and extracting the HF, including GLCMs and DWTs [8-10]. In this scheme, the pretrained VGG19 functions as the backbone (encoder) while the decoder is constructed based on the encoder structure. In order to form a dominant hybrid feature vector (HF+LF), the HF from the segmented images is combined with the LF from the VGG19 scheme to form a dominant hybrid feature vector (HF+LF). This feature vector will then be used in a cross-validation process for performing the classification task. With the help of different classifiers chosen by the authors of this study, the experimental outcome of this work is verified and compared with the recent work by Al-Kharraz et al. [11]. The proposed scheme has higher levels of classification accuracy than the other approaches in the literature, according to the experimental findings of this research.

The contributions of this research involve;

(i) Application of a CNN segmentation method to extract CCs and compute their shape and texture features based on the extracted CC.

(ii) Improvement of CC karyotyping accuracy through HF and LF classification.

(iii) Evaluation of the scheme's performance and assessment of its clinical relevance through comprehensive performance appraisal and validation.

An essential facet of early-stage genetic disease diagnosis is the automatic karyotyping of chromosome cells (CC). This procedure entails the examination of CC using a carefully selected methodology to optimize results. Previous literature has firmly established that deep learning methods (DLM) serve as effective tools for improving the accuracy of CC examinations. Consequently, the proposed scheme incorporates a DLM-supported segmentation and classification system to enhance the evaluation of CC.

Figure 1 illustrates the procedural framework adopted in this research, encompassing the following pivotal stages: (i) Image acquisition and preprocessing; (ii) Utilization of VGG-UNet for the segmentation of chromosome cells (CCs); (iii) Extraction of handcrafted features (HF) from the segmented images; (iv) Extraction of learned features (LF) from a pre-trained deep learning model (DLM); and (v) Integration and classification of the derived sequential features. Within this study, the assessment of classification accuracy, facilitated by a carefully chosen classifier, serves as the benchmark for evaluating the efficacy of the proposed methodology. A central component of this framework is the VGG-UNet, as depicted in Figure 1(b), which significantly contributes to enhancing the precision of CC mining.

1a.png

(a) Proposed scheme

1b.png

(b) VGG-UNet

Figure 1. Structure of the proposed CC detection process

2a.png

(a) Original image

2b.png

(b) Complement image

2c.png

(c) Detected chromosomes

2d.png

(d) Karyotyping

Figure 2. Sample images available for segmentation task

3.1 Database

The performance evaluation of the developed chromosome cell (CC) detection tool necessitates rigorous validation using a benchmark database. In this study, we have utilized the test images available from the well-established Biomedical Imaging Laboratory (BioImLab) database, and comprehensive details regarding this dataset can be accessed from Laboratory of Biomedical Imaging [15]. This dataset encompasses two distinct image sets designed to support both segmentation and classification tasks. The images presented in Figure 2 serve a dual purpose, catering to both segmentation and karyotyping. Specifically, Figure 2(a) showcases a Q-band prometaphase image for assessment, while Figure 2(b) represents the complementary image. Figure 2(c) portrays the contrast-adjusted and segmented chromosome images, and Figure 2(d) exhibits the karyotype information included within the dataset.

In the context of this research proposal, we have directed our focus towards the classification database, implementing a systematic approach that includes the following key steps: (i) resizing, (ii) mask generation utilizing ITK-Snap software [16, 17], and (iii) segmentation and classification, guided by the VGG-UNet architecture. For the segmentation task, we have specifically considered 5474 images available in this database for assessment and classification. Within this set, a subset of 500 images, each accompanied by a binary mask, has been selected to perform the segmentation process. These images are then considered for training and validating the proposed VGG-UNet algorithm. By using the trained VGG-UNet to extract the CC section (binary image) from the selected images from all 5474 images, the binary image of the CC is extracted, which is necessary to extract the HF, such as GLCM and DWT. The purpose of this study is to use the pretrained VGG-UNet scheme in order to achieve both

3.2 VGG-UNet

UNet is one of the commonly employed image segmentation practices to extract chosen regions from the digital image. The advantages of the UNet and its variations are supported by earlier research in the literature [18, 19]. The application of VGG-UNet in image extraction can be found in the studies [20, 21]. Khan et al. [9] proposed the VGG-UNet scheme that can be used to extract CC from selected test images. This scheme uses the VGG19 as a backbone (Encoder) and its inverted structure to function as the decoder. There is also the addition of another layer, which is a SoftMax, which extracts the CC section of the test image. The extracted section will be in the form of binary; therefore, it can be used later for the mining of GLCM and DWT features. Information about the VGG-UNet can be found in the studies [22-25]. The initial values for VGG-UNet are assigned as follows; Adam optimizer, MaxPooling layer and ReLu.

The final layer within the decoder is equipped with SoftMax, which plays a pivotal role in extracting the binary representation of the target image section.

A commonly used CNN scheme is image segmentation with UNet (UNet), which can be used to extract parts of a digital image for analysis. Earlier publications in the literature [18, 19] support the UNet and its derivatives. The application of VGG-UNet in image extraction can be found in the studies [20, 21]. Khan et al. [9] proposed the VGG-UNet scheme that can be used to extract CC from selected test images. This scheme uses the VGG19 as a backbone (encoder) and its inverted structure to function as the decoder. There is also the addition of another layer, which is a SoftMax, which extracts the CC section of the test image. The extracted section will be in the form of binary; therefore, it can be used later for the mining of GLCM and DWT features. Information about the VGG-UNet can be found at the studies [22-25]. The following are the initial values for VGG-UNet: Adam optimizer, MaxPoolinglayer, and andReLu. The final section of the decoder is associated with SoftMax to extract the binary image.

3.3 Feature extraction and integration

The proposed scheme emphasizes the importance of feature extraction and the importance of feature supported detection. In this work, both the HF and LF are considered. GLCM [26, 27] and DWT [28, 29] were used so as to obtain the necessary HF. The outcomes of these methods can be obtained by using Eq. (1) and Eq. (2).

$\operatorname{GLCM}_{(1 \times 1 \times 44)}=$ GLCM 1, GLCM $2, \ldots$, GLCM 44         (1)

$D W T_{(1 \times 1 \times 26)}=D W T 1, D W T 2, \ldots, D W T 26$        (2)

This scheme proposes a LF vector of dimension 1×1×1000 that is then reduced by 50% to 1×1×500, resulting in a new hybrid feature vector (HF+LF) of dimension as presented in equation. The classifier is then trained and validated using a five-fold cross-validation procedure based on Eq. (3), which is referred to as the hybrid feature vector (HF+LF).

$\begin{gathered}(H F+L F)_{(1 \times 1 \times 570)}=G L C M_{(1 \times 1 \times 44)}+ D W T_{(1 \times 1 \times 26)}+L F_{(1 \times 1 \times 500)}\end{gathered}$     (3)

3.4 Performance assessment and endorsement

A number of classifiers, such SoftMax, k-NN, Random decision forests (RF), and support-vector networks (SVM), are employed to evaluate the suggested approach, and the results obtained are compared to and justified against those published in the literature. In this study, only classification accuracy is evaluated, and its expression is given in Eq. (4) [30-33].

$Accuracy =A C=\frac{T P+T N}{T P+T N+F P+F N}$           (4)

This research introduces a novel approach for the automated recognition of chromosome cells (CC) through the integration of Convolutional Neural Networks (CNNs) for segmentation and classification tasks. Specifically, the VGG19 architecture is harnessed to execute these essential functions. The initial phase of the proposed tool employs the VGG-UNet model to effectively segment the requisite CC section, while the subsequent phase leverages the VGG19 model for classification. The CNN-based segmentation process results in a binary representation of the CC, which is subsequently utilized for the extraction of Handcrafted Features (HF) such as Gray-Level Co-occurrence Matrix (GLCM) and Discrete Wavelet Transform (DWT).

To enhance the discriminative power of our approach, this research generates a hybrid feature vector by sequentially combining Deep Features (DF), GLCM, and DWT-derived features. This feature vector is then employed to provide a binary classification outcome with the aid of a chosen classifier. The empirical findings of this study unequivocally establish the superiority of the K-Nearest Neighbor (KNN) classifier, achieving classification accuracy surpassing 98%, which outperforms existing methods documented in the literature.

However, it's important to acknowledge a limitation inherent in this scheme, which primarily relies on CNN-based segmentation and classification. In future investigations, alternative approaches for CC classification, distinct from segmentation, could be explored to further enhance the versatility and robustness of the proposed methodology.