Development of a Comprehensive Lung Scan Dataset for Machine Learning-Based Lung Cancer Detection

Tasnim et al. [21] employed deep learning (DL) techniques on patient's 1190 CT scan images from the Kaggle IQ-OTH lung cancer dataset. Following extensive image preprocessing, this method was discovered by augmented images comprised of benign, malignant, and high-risk cases to identify individuals to target for early intervention to prevent long-term consequences and identify lung cancer. A comprehensive analysis of the relative performances of multiple classifiers, such as InceptionV3, Resnet50, and the traditional CNN, has been provided. Gaussian noise, affine transformation, and other thorough picture pre-treatment methods were applied here. The contribution reduced the complexity of the model with the prior pre-processing step and achieved a 98% validation accuracy. The proposed technique was validated by the comparison method, which produced a higher F1-Score value of 97% for the suggested pre-processing procedure.

Nathan and Rithani [22] incorporated fully connected layers for genomic data, recurrent neural networks (RNNs) for sequential clinical data, and Convolutional Neural Networks (CNNs) for picture analysis. The approach enabled more accurate forecasts by capturing complex patterns and correlations by combining these many data modalities. presents a novel deep-learning method for predicting the prognosis of lung cancer that makes use of standardized pre-processing and diverse data improvement. This algorithm gave clinicians an effective tool for better patient outcomes through more accurate prognostic predictions by combining medical pictures, clinical records, and genomic data. With new opportunities for early intervention and individualized treatment plans, this research advanced personalized medicine in the management of lung cancer.

Khattar et al. [23] introduced techniques for detecting lung cancer, such as CT scans and biopsies, which have disadvantages like being pricy, invasive, and radiation-exposing for patients. Deep learning, Convolutional Neural Networks (CNN), and other machine learning techniques have lately shown encouraging results in the analysis of medical images, including the detection of lung cancer. A dataset of 1,010 lung nodules was used by the publicly available Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI). The nodules in the dataset varied in size and shape and could be either benign or cancerous. The images were pre-processed by standardizing the pixel values to be between 0 and 1 and uniformly resizing them to 32 by 32 pixels. A data-validating set (20%) and a data-training set (80%) were randomly selected from the database.

Li et al. [24] used a hybrid feature extraction technique that combines autoencoder features with an autoencoder and Gray-level co-occurrence matrix (GLCM) with Haralick. Following that, supervised machine learning techniques were trained using these features. SVM polynomial provided an accuracy of 99.89% when using GLCM with an autoencoder, Haralick, and autoencoder features, whereas Support Vector Machine (SVM) Radial Base Function (RBF) and SVM Gaussian reached flawless performance metrics. SVM RBF, using GLCM with Haralick features, obtained an accuracy of 99.35%, whereas SVM Gaussian obtained an accuracy of 99.56%. These outcomes show how the suggested strategy may be used to create better prognostic and diagnostic tools for lung cancer treatment planning and decision-making.

Venkatesh et al. [25] introduced a brand-new approach to lung cancer diagnosis was put forth that used deep learning algorithms to accurately detect the disease while consuming less processing time. Since CT pictures had less noise than MRI and X-ray images, they were used in this investigation. Patch processing and median filtering were applied to these CT scans to enhance image quality. Following their pre-processing, these images were sent into a CNN classifier use a segmentation procedure called clustering. CNN architecture was used to classify and extract features. High-level and low-level features were extracted in the section on future extraction. The supplied image's categorization layer was responsible for the identification of whether the tumor was malignant, benign, or normal.

Sujatha et al. [26] examined whether sophisticated deep learning algorithms are suitable for accurately diagnosing lung cancer using MRI scans. To ascertain their distinct contributions, recurrent neural networks (RNN), Convolutional Neural Networks (CNN), K-Nearest Neighbours (KNN), and ResNet50 were all rigorously assessed. With an accuracy of 92.3%, CNN demonstrated its strong performance and ability to identify complex patterns in lung pictures. KNN showed competitive outcomes, highlighting the versatility of non-parametric techniques for classify medical images. Surprisingly, ResNet50 performed remarkably well, demonstrating an astounding accuracy of 94.8% and confirming the usefulness of deep residual networks in distinguishing between complex properties. RNNs added a time dimension to the study and helped it achieve an accuracy of 89.5%.

Mishra et al. [27] suggested a novel deep learning-based method, namely utilizing Generative Adversarial Networks (GANs), to modernize the use of scientific imaging for the diagnosis and localization of pulmonary cancers. The models, which were trained on a variety of datasets, showed a good accuracy rate of 70% in the sample set, indicated that they can distinguish between cancerous and non-cancerous cases in scientific photographs with ease. Although the findings point to a notable increase in lung cancer detection, they also point out areas that still required improvement. Future research would continue to focus on finding a balance between scientific value and technological prowess, as measured by criteria like specificity and sensitivity. Karthikeyan and Ali [28] presented a unique Convolutional Neural Network (CNN) framework that was painstakingly created for the interpretation of CT scan pictures to detect lung cancer early. The research emphasized the improved performance of CNN over traditional diagnostic techniques through thorough comparison evaluations with other models. The outcomes highlight the effectiveness of the suggested deep learning model and confirm that it was a more reliable and powerful diagnostic tool than current methods for the early detection of lung cancer [29]. To ensure the model's robustness and applicability across a range of clinical settings, future research paths may investigate the integration of larger and more diversified datasets. This would ultimately advance the landscape of lung cancer diagnostics toward improved patient outcomes and healthcare practices [30].

The key challenges of the existing works are given below:

Developing extensive lung screening data for device-based lung cancer screening has been hampered by a number of significant obstacles in the past. These encompass challenges such as acquiring varied top-notch data from numerous references as well as having it accurately interpreted by expert radiologists to ensure correctness; also, there is the constant need to deal with alterations in imaging methodologies and standards that affect data pre-processing while retaining original details.

The proposed method for developing comprehensive lung screening datasets for lung cancer detection based on machine learning Another approach using federal learning-driven data collection enhancement (FL-DAE) addresses key challenges such as data privacy, diversity, and quality management The integrated process begins with the establishment of an integrated curriculum in which multiple medical institutions participate in collaborative modelling training without informed consent not in the long run. Each institution trains a local model on its data set, which includes various imaging modalities such as CT, X-ray, MRI, etc., and only shares model updates (gradients) with a central server. This approach ensures that patient data at any institution is secure and confidential. The central server collects these updates and creates global instances, which are then redistributed to organizations for further local training. This iteration process is ongoing, enabling the global model to learn from multiple datasets from all participating organizations, thus ensuring that patient populations, stages, and types of disease are studied and available for detailed information. A lot of data is augmented with more sophisticated data augmentation techniques, such as generative adversarial networks, which are used to synthesize high-quality lung scans. Synthetic scans can increase the diversity of datasets and balance out underrepresented classes by mimicking the statistical features of real scans.

1.png

Figure 1. Block diagram of the proposed FL-DAE

The Block diagram (Figure 1) represents the Federated Learning framework for the public databases that generates the data and augment it to proceed the Federated Learning-Drive Data Aggregation and Enhancement (FL-DAE).

Global model and synthetic data are validated and the quality control is strict with expert radiologists reviewing and noting the synthetic scans to make sure they meet clinical standards. Federated learning also allows for de-identified clinical metadata, such as patient demographics, medical histories, treatment outcomes, etc., which can be used to enhance machine learning models. After the global model has been sufficiently trained and validated, the dataset, which is enriched with high-quality synthetic data and detailed annotations, is released for public use. A comprehensive record of the data collection, enhancement, validation, user support channels, and regular updates is available. Federated learning drives data privacy and provides a comprehensive, reputable, and clinically relevant lung scan dataset, enabling the development of machine learning models for lung cancer detection.

4.1 Process of the proposed FL-DAE

The Federal Learning-Driven Data Collection Enhancement (FL-DDE) is used in this paper. For analysis, several models can be employed to achieve better results.

4.1.1 Data collection

To create a complete set of lung scan images for using machine learning to find lung cancer, collect different lung scan pictures from many hospitals. Make sure the images use different types of scans (like CT scans and X-rays) and come from people of different ages, genders, and ethnicities. Employ advanced techniques to securely merge and utilize data from various sources while maintaining patient confidentiality. Enhance the collection of images by incorporating additional details such as doctor's notes, test outcomes, and patient history, and by refining the images to improve their clarity. This collaborative approach while preserving privacy will result in a high-quality, clear set of images that can be used to train robust machine-learning models for detecting lung cancer.

$B(\mu ,v)=\eta \sum\limits_{(k,n)\in A}{[{{\mu }_{k}}}\ne {{\mu }_{n}}]\exp (-\alpha ||{{L}_{k}}-{{L}_{n}}|{{|}^{2}})$    (1)

4.1.2 Preprocessing

The initial step was standardizing the lung scan data from the LUNA16 and Kaggle datasets to guarantee uniformity and compatibility across various imaging sources. While LUNA16 offers volumetric images in.mhd and.raw forms, the Kaggle dataset includes CT scans in DICOM format. To ensure model compatibility and consistent downstream processing, all image files were transformed into standard 3D NumPy array structures. A fixed voxel spacing of 1 mm × 1 mm × 1 mm was then applied to all scans during resampling. Because different medical institutions utilize varied scanner settings, which result in non-uniform spatial resolutions, this step is crucial. In order to ensure spatial consistency of anatomical features throughout the datasets, resampling was accomplished through the use of linear interpolation techniques.

Intensity normalization was done after resampling. The pixel values were clipped to the range of [-1000, 400], which accurately depicts the density range of lung tissues and any lesions, because CT image intensities are expressed in Hounsfield Units (HU). In order to improve contrast and guarantee numerical stability during neural network training, the clipped values were then scaled to the range [0, 1] utilizing min-max normalization.

Intensity normalization was done after resampling. The pixel values were clipped to the range of [-1000, 400], which accurately depicts the density range of lung tissues and any lesions, because CT image intensities are expressed in Hounsfield Units (HU). In order to improve contrast and guarantee numerical stability during neural network training, the clipped values were then scaled to the range [0, 1] utilizing min-max normalization. The segmented lung areas were then used to mask the original scans in order to eliminate background and irrelevant anatomical features. While 3D patches (usually 64×64×64) were extracted around annotated nodule coordinates in the LUNA16 dataset, axial slices with visible nodules were chosen from the Kaggle volumes for better focus on diagnostically significant locations. This focused extraction minimizes needless computational work and guarantees that training samples are rich in pertinent features.

4.1.3 Federated learning framework

The federated learning system for creating a large, detailed dataset of lung scans to help machines detect lung cancer works by having several hospitals train their versions of a model using their lung scan data. These hospitals regularly send the information their models have learned, not the actual patient data, to a main computer, which combines this information to improve a single, overall model. This way, patient privacy is protected and data protection laws are followed while using a wide range of data from various places. The system has strong safeguards like secure messaging, privacy techniques, and encryption to keep the data safe and accurate. It also improves the model over time through repeated updates and checks, leading to a very accurate model that can be used widely for detecting lung cancer.

The following equation shows how the feature map $N_j$ is calculated:

${{N}_{j}}={{d}_{j}}+\sum\limits_{i}{{{V}_{ji}}}*{{Y}_{i}}$    (2)

where, $Y_i$ is the $i^{\text {th }}$ input channel $V_{j i}$ is its sub-kernel, and $d_j$ is a biased term, and * is the convolution operator. In other words, each feature map's convolution operation consists of applying $i$ separate 2D squared convolution features in addition to a bias term.

4.1.4 Public databases

To create a complete set of lung scan images for using machine learning to find lung cancer with a method called federated learning, we can use important public databases like the Lung Image Database Consortium image collection (LIDC-IDRI), The Cancer Imaging Archive (TCIA), and the National Lung Screening Trial (NLST). These databases give us access to many lung scan images with notes, including CT and X-ray pictures, that show different stages and types of lung cancer. Using these free datasets helps us assemble various imaging data, which helps make a strong and general model. Also, adding datasets like ChestX-ray14 and the NIH Clinical Centre’s Chest-ray dataset can make the dataset bigger and better, giving a good base for using federated learning to gather and improve data.

4.1.5 Synthetic data generation

Creating artificial data is very important for building a complete set of lung scan images for using machine learning to find lung cancer. This is especially true when combined with a special way of sharing and improving data called federated learning. Methods like generative adversarial networks (GANs) and variational autoencoders (VAEs) can be used to make high-quality fake lung scans that look like real ones from patients. These fake images can add to the real ones we already have, fixing problems like not having enough data or having too much of one type and not enough of another. This makes the group of training images more varied and stronger. By adding fake data, the federated learning method gets more data and more kinds of it without breaking rules about keeping patient information private. This helps the model work better at finding lung cancer in different groups of people and under different conditions for taking pictures of the lungs. This gives a strong base for using federated learning to bring together and improve data.

$p(a)=\max (0,a)$    (3)

where, a represents the training images, which receives a set of input images, and produces the results.

4.1.6 Data augmentation

Data augmentation is an important method for creating a large and varied set of lung scan images for use in machine learning to detect lung cancer. This is especially useful when combining data from different sources using a method called federated learning. At each federated client, data augmentation was used during the local training stage. Among the augmentation techniques were zooming, contrast modifications, random rotations, vertical and horizontal flips, and elastic deformations. These adjustments enhance the model's capacity to generalize across various patient profiles and scanner circumstances in addition to preventing overfitting. This helps create a better set of data that can help the computer learning model work well in various situations and with different patients. More advanced methods, like slightly changing the shapes of the images and adjusting the colors, can also be used to make the training data even more diverse and high-quality, which helps the model better identify small differences in lung tissue and possible signs of cancer.

$f(e)=({{t}^{*}}u)(e)$    (4)

4.1.7 Data anonymization

A crucial step in producing a big collection of lung scan images for computer-assisted lung cancer detection is data anonymization. To respect HIPAA regulations and protect patient privacy, this procedure entails removing or concealing personal information from medical records. Sensitive information like names, social security numbers, and precise birthdates are kept secret by techniques including name-changing, using false identities, and obscuring specifics. Additionally, unique numbers are utilized in place of names so that researchers can link disparate data sets without being aware of the identity of the patients. In addition to safeguarding patients' privacy, anonymizing the data makes it easier for researchers to collaborate and share findings.

${{g}_{n}}(z)=\frac{{{F}^{z}}n}{\sum\nolimits_{r}{{{F}^{z}}R}}$    (5)

Using strong methods to hide personal information in lung scan data is very important. This means we need to mix the need for useful data with keeping people's privacy. We must make sure the data without names still has the important parts that help train and test computer models. Special methods like adding small random changes to the data can protect patient privacy while keeping the data useful for analysis. By focusing on hiding personal details in the data, researchers can make a safe, ethical, and complete set of data that helps improve computer models for finding lung cancer early and accurately.

The present investigation uses sophisticated privacy-preserving mechanisms in addition to traditional anonymization techniques to improve patient confidentiality, particularly in the context of federated learning. Differential privacy is one such method that adds precisely calibrated random noise to the data processing pipeline, particularly while model updates are being transmitted from local clients to the central server. This guarantees that, even in cooperative machine learning environments, no single data point can be identified or reverse-engineered. All institutional participants followed stringent ethical guidelines in order to comply with international data protection requirements, including the General Data Protection Regulation (GDPR) and the Health Insurance Portability and Accountability Act (HIPAA). These included following data minimization guidelines, getting the appropriate institutional review board (IRB) approvals, and, when necessary, gaining informed consent. All storage systems were secured with authentication and access control procedures to avoid unwanted access, and data transmission was carried out over encrypted channels.

Furthermore, because patient data never leaves the local institution, the federated learning-driven solution naturally supports privacy by design. The risk of data leakage is greatly decreased because only model updates not raw images are transmitted. Strong anonymization and differential privacy procedures, along with this decentralized training paradigm, guarantee both data value for precise lung cancer detection and strong patient identity protection. Because of this, researchers can work together across institutions with confidence without sacrificing moral principles or legal obligations.

4.1.8 Federated Learning-Driven Data Aggregation and Enhancement (FL-DAE)

Federated Learning Driven Data Aggregation and Enhancement is crucial for creating large, detailed datasets of lung scans to help machines detect lung cancer. This method uses data from different places like hospitals and clinics to gather a variety of datasets while keeping the data private and safe. By using federated learning [31-33], the data stays where it was collected, and only the changes to the model are shared between different locations, which helps protect privacy and follow rules.

$G(\mu ,v,\lambda ,L)=RE(\mu ,v,\lambda ,L)+B(\mu ,L)$    (6)

This process includes training a model together where each local dataset helps improve a main model without having to put all the sensitive medical data [34] in one place. This method of sharing information among many locations not only protects patient confidentiality but also increases the variety of data, showing differences in lung images from different groups of people and health situations. Additionally, this technique called federated learning lets the models get better over time as the local data changes, making models that can adapt to new information and improve how well they can spot lung cancer early. This way of gathering and improving data using federated learning is a path to making machine learning tools in healthcare better and more ethical, especially for finding and diagnosing lung cancer early.

4.1.9 Validation and testing

In the validation stage, the data is split into two groups: one for teaching the model and one for testing it. This helps adjust the settings of the model and prevents it from becoming too specialized for the training data. This process ensures that the model works well with new, unseen data. Methods like k-fold cross-validation are used to check how consistently the model performs across different parts of the data, giving a thorough evaluation. The testing stage uses a completely new part of the data that the model hasn't seen. This is important to see how accurately the model can predict outcomes and how well it can handle real-world situations. Performance measures like accuracy, precision, recall, F1-Score, and the area under the ROC curve are used to evaluate how well the model works. It's also important to check if the model can correctly identify different stages and types of lung cancer, making sure it can give accurate results for a wide range of patients. By thoroughly testing and validating the data and the machine learning model, we can make sure it is reliable and effective for detecting lung cancer early, which helps improve patient health.

4.1.10 Deployment

During this stage, the goal is to incorporate the model into the current healthcare setups so that it works well in different medical settings. Important factors to consider are how well the model can work with various medical imaging devices, how it fits with hospital computer systems, and how it follows healthcare rules and guidelines. To make the integration go smoothly, strong software development methods are used, such as APIs, cloud services, and simple interfaces that help doctors use the model and understand its findings easily. After the model is put into use, it's important to keep an eye on it and maintain it to make sure it stays accurate and useful. This includes gathering data and feedback from doctors to find any problems and ways to improve the model. It's important to regularly update and train the model with fresh data to ensure it stays up-to-date with the newest medical information and imaging techniques. Also, creating a system where healthcare professionals can give feedback helps improve the model and fix any problems or shortcomings found in everyday use. By effectively using the lung scan data and the related machine learning model in medical practice, we can greatly improve the early identification of lung cancer, which leads to improved patient results and more effective healthcare services.

${{H}_{sol}}=D\times {{H}_{press}}$    (7)

4.1.11 Fitness function

When creating a large set of lung scan images to help computers find lung cancer, the need of a special tool called a fitness function. This tool helps us check and improve how well the computer model can tell the difference between scans showing lung cancer and those that don't. Here, several ways are used to measure this, like accuracy, precision, recall, F1-score, and something called the area under the ROC curve. These ways give us a full picture of how good the model is at its job, making sure it can correctly spot both cancer and non-cancer cases.

$\begin{align}& F=\eta \sum\limits_{(k,n)\in A}{[{{\mu }_{k}}}\ne {{\mu }_{n}}]\exp (-\alpha ||{{L}_{k}}-{{L}_{n}}|{{|}^{2}}) \text{ }+(1-(b-1)/U-1){{)}^{\frac{1}{\beta }}} \\ \end{align}$    (8)

where, (1-(b-1)) denotes the threshold value, U denotes the maximum number of generations, β denotes the mutation factor.

Algorithm:

Step 1: Setup

At the central server, start a global Feature-Adaptive CNN model.

Give each participating medical facility (client) a copy of the initialized model.

Step 2: Preparing Local Data

Every client creates a local lung scan dataset. To get rid of noise and artifacts, do preprocessing and data cleaning.

Use data augmentation strategies to improve the balance and diversity of your dataset.

Verify normal and abnormal classifications to ensure annotation accuracy.

Step 3: Client-side local training

Every client uses its local dataset to train the Feature-Adaptive CNN model.

To enhance representation, the model adaptively adjusts its convolutional parameters during training in response to feature pattern.

The modified model is returned to the central server after training for a predetermined number of local epochs.

Step 4: Server-side global aggregation

All updated models from participating clients are gathered by the server.

Combines the models in a safe manner to create a better global model.

For the upcoming round, all customers receive an updated version of the global model.

Step 5: Iteration

To gradually improve the global model, repeat Steps 2-4 for a predetermined number of communication rounds.

Step 6: Concluding Assessment

Use a validation dataset to assess the finished global model after training is finished.

Metrics such as accuracy, precision, recall, F1-Score, and false rate can be used to gauge performance.

Step 7: Implementation and Evaluation

To detect lung cancer, use the trained model as a reference.

Make the dataset and model available for further research and application in clinical environments.

2.png

Figure 2. Flowchart representation

The flowchart (Figure 2) represents that the detection of lung cancer by collecting the datasets from two different datasets and train the local model available for further research and clinical results.