Touchless Fingerprint Recognition with Capsule Networks and PCA Filtration Using Dual-Cross Generative Adversarial Networks

ABSTRACT


INTRODUCTION
The World Health Organization (WHO) has advised avoiding contact with objects in public places to prevent transmission of the new coronavirus 2019 (COVID- 19), which is now causing widespread outbreaks.Capturing touchless fingerprints, in which the finger is not brought into direct contact with the sensor, is a promising and exciting development in the field of security [1] and hygiene.By eliminating the need for direct finger-to-scanning-device contact, touchless fingerprint recognition provides a secure biometric authentication mechanism.Fingerprint recognition technology in the cyber security realm has the potential to significantly improve user authentication and authorization procedures.Historically, passwords have served as a means of verifying a user's identity.However, it is important to note that it is susceptible to theft or unauthorized access, therefore introducing potential security vulnerabilities, so it is inadequate in terms of security.In contrast, biometric methods such as touchless fingerprint recognition may be both more reliable and more convenient for authenticating users.A rapidly expanding area of research that has been investigated for more than ten years.Verifying a person's identity through this method involve using the unique characteristics of their fingerprint.Fingerprints, which are the ridge friction patterns on the tips of one's fingers, are a common form of biometric identification.Automated fingerprint recognition systems have had tremendous success for a variety of applications after several years of study [1].The typical touch-based fingerprint recognition method faces difficulties while acquiring fingerprints [2].Low fingerprint quality is caused by problems such as a latent fingerprint left on the sensor surface by a previous user during the acquisition of a touch-based fingerprint [3,4].Additionally, the pressure applied to the sensor surface causes fingerprint deformation and distortion [3].Distortions may result from uneven finger pressure on the device, changes to the finger ridge from strenuous activity or accidents, changing lighting conditions on the finger skin, or motion artifacts during image capture.Ridge flow can be interrupted when fingerprints come into contact with the scanner.During capture, a significant amount of background noise may also be added [5].Instead of having to physically touch a fingerprint reader, this technology can record and analyze the unique patterns and properties of a person's fingerprint using modern image processing techniques and algorithms.The use of touchless fingerprint identification technologies [2] could make biometric identification one of the most trustworthy processes.As an alternative to conventional touch-based fingerprint identification systems, the first touchless system was presented in 2004 [3].In order to overcome the limitations of touch-based fingerprint recognition systems, it is necessary to improve the acquisition process by reducing restrictions.This will allow for the development of new applications and improve their overall usability and acceptance by users.Furthermore, a touchless fingerprint image does not include any latent fingerprints and does not exhibit any distortion.
Touchless fingerprint recognition is commonly characterized as a sequential procedure comprising four distinct stages: capturing the fingerprint image, preprocessing, feature extraction, and ultimately recognizing the fingerprint, as shown in Figure 1.

Figure 1. Touchless fingerprint recognition model a) Touchless fingerprint image capturing
In touchless fingerprint capturing one or more fingers can be presented to an optical device like a camera or lens.The National Institute of Standards and Technology [6] produced a document to evaluate techniques of touchless fingerprint capturing, the document also includes appropriate instructions for equipment that may do touchless fingerprint capturing.Typically, when the fingerprint image has been obtained, it will be analyzed.
b) Preprocessing Perform cleaning and preprocessing on the fingerprint images.Typical preprocessing procedures involve reducing noise, enhancing images, and normalizing data.Perform cleaning and preprocessing on the fingerprint images.Typical preprocessing procedures involve reducing noise, enhancing images, and normalizing data.The majority of touch-based fingerprint images obtained from the devices are in grayscale and suitable for feature extraction.However, the majority of touchless finger-imaging solutions offer color RGB images that need to undergo preprocessing prior to feature extraction [7,8].
The illumination sources have an impact on the ability of a camera to capture fingerprints.Touchless fingerprints typically have a low contrast between the ridges and valleys of the fingerprint, which can lead to poor performance when attempting to extract feature points.it's hard to control how fingers are placed on sensors during acquisition, which can cause images to be distorted and in different poses.To solve such problem preprocessing is done.The primary challenges of preprocessing touchless finger are Image segmentation, low contrast [2] and low resolution [3].In this step, the background area of an image is excluded and only those areas that contain pertinent information are taken into account.Through average squared image gradients, an orientation field is estimated.A fingerprint image should have smooth lighting after being enhanced.
c) Recognition of fingerprint A Deep learning model is employed to acquire and derive features from the preprocessed touchless fingerprint images.This comprise of several steps such as feature extraction, feature matching and so on.After the features have been retrieved, they will be compared to a database of known fingerprints.This could involve analyzing the fingerprint's pattern and shape in addition to its individual minutiae in order to verify a person's individuality.This requires determining the precise location, direction, and nature of each minute detail, including a ending of ridge, a bifurcation, or an island.To Extracting the Minutiae-Based Feature [4,5,9], Images need to be changed from RGB to greyscale, Consequently, learning must also be used to ROI [10], which includes tasks like identifying fine details, extracting ridges and valleys, and estimating the direction of fingers.Once the finger has been located, the ROI can be retrieved by adjusting the image's size and resolution to their baseline values.
d) Evaluation It is critical to ensure that the fingerprint recognition method satisfies the particular requirements of your application, be it for identity verification, access control, or any other use case, by conducting exhaustive testing and evaluation.
In this paper, a novel Dual-Cross GAN [11][12][13][14] click or tap here to enter text.framework with Capsule Network [15] based PCA filtration is proposed, for accurately recognize the touchless fingerprint images.Capsule Network based PCA filtration is utilized for fast feature embedding with convolutional architecture to collect spatial information and important differentiating characteristics in order to offset the information loss caused by pooling operations.A filtration method is used to find the precise match of the fingerprint based on the extracted features.This is done to construct uncorrelated variables, which limit the loss of information and optimize variation over time.These goals are accomplished by optimizing variation throughout time and minimizing the loss of information.Dual-Cross GAN is modeled to restore and recognize the fingerprint so that it may be used to manage all of the variation.Consequently, the effectiveness of the framework based on accurately recognizing the fingerprint from the generated fingerprint is assessed Touchless Fingerprint Recognition with Capsule Networks and PCA Filtration using Dual-Cross Generative Adversarial Networks.

RELATED WORK
Significant research has been done in the rapidly developing field of touchless fingerprint identification over the past few years.The efficacy of deep convolutional neural networks in addressing many challenges in computer vision has motivated researchers to employ trained networks for direct predictions of fingerprint details.There have been a number of promising studies that feed raw fingerprint images into untrained deep neural networks in exchange for the output of directly gathered information.
Developed a technique to color-based segmentation [16] that was used for the extraction of ROI.The authors utilized this strategy in conjunction with a frequency estimation map.In addition to that, in order to extract the ROI, they carried out a region expanding operation and utilized a Gaussian probability density function [17].A similar strategy took advantage of the fingertip's ridge line properties.The finger was separated into separate, non-overlapping blocks in this instance.The ROI included ridge-line characteristics that could be seen inside a block, further demonstrate that a ROI extraction based on finger shape features is also feasible in constrained scenarios [18].By identifying distinguishing features like fingertips and discontinuities, the authors were able to statically calculate the ROI.Deep touchless fingerprint unwrapping [19] involves the segmentation of the input image into patches of varying sizes, followed by the training of a neural network to classify the labels associated with each patch.This approach employs a patch-based prediction technique, while additionally integrating JudgeNet to anticipate the presence of minutiae and LocateNet to accurately determine their locations.There exist intriguing scenarios wherever the complete fingerprint image is employed to forecast the minute elements of the score map.A sophisticated convolutional neural network [20] is used to effectively analyze grayscale touchless fingerprint images.The primary objective is to simultaneously acquire accurate position detection and direction calculation through the network's learning process.This approach employs a combination of offline learning and online testing.A novel loss function is introduced to facilitate the joint learning of granularity detection and direction regression using complete fingerprint data [9].A novel approach to person authentication, which involves the utilization of touchless fingerprint selfies alongside palmprint selfies, hence enhancing the efficiency of the multi-biometric system [21].In this study, three local descriptors, namely "local phase quantization" (LPQ), "local ternary patterns" (LTP), and "binarized statistical image features" (BSIF), were employed to obtain key characteristics from touchless fingerprint and palmprint selfies.The selection of these descriptors was motivated by their ability to provide simplicity and efficiency in the feature extraction process.The multibiometric score level system was created, merges the matching scores utilizing two distinct fusion strategies The improved Gabor filters along with intrinsic image decomposition and filtering for enhancement [22] is used.While an improvement over traditional touched-based fingerprint methods, these techniques are still vulnerable to low-contrast images and prone to false positives due to the aforementioned spurious detail.Additionally, this uses a cross-database evaluation and COTS comparison.An integrated Gabor filter framework [23], approach is utilized to extract and analyze both finger vein and fingerprint information.A novel method called "supervised local-preserving canonical correlation analysis" (SLPCCAM) for generating feature vectors of fingerprint-vein (FPV) patterns through feature-level fusion Ultimately, the process of personal identification is executed by the utilization of the nearest neighbor classifier, which relies on the utilization of FPVFVs.A method a method in which isolate regions of interest and normalize uneven lighting, the obtained images are first put through rigorous preprocessing stages than gather localized feature data and efficiently add this regional data to the matching stage [3].
Table 1 shows a comparison of various touchless fingerprint identification techniques.

Author Approach Accuracy
Attrish et al. [24] The fingerprint image is transformed into a fixed length embedding by the Siamese network, which is then used to determine a similarity score between the reference and probing images.EER=2.19%[24] Grosz et al. [25] TPS spatial transformer for 500 ppi scaling and deformation correction of touchless fingerprints.Fusion of minutiae and CNN texture representations.
Kumar and Zhou [3] Retrieve localized feature data and seamlessly integrate it into the matching phase.EER=3.95%[3] Labati et al. [27] The neural networks are utilized to assess the disparity in orientation across two touchless fingerprint captures.

PROPOSED WORK
The Dual-Cross GAN framework with PCA filtration based on Capsule Network is discussed in this section.first present a summary of the suggested algorithm.After that, a detailed description of our Dual-Cross GAN network's design follows.Finally, we provide implementation information and validate fingerprints.

Overview of the proposed algorithm
When extracting features using the prior restoration approach, some data was lost due to pooling layers, which negatively impacted recognition accuracy.In addition, the recognition system needs to be automated so that it can cope with the variations in fingerprint images and produce reliable results.Furthermore, there is only one sort of error that can be handled by the earlier conventional models that have been published in the literature.It is not possible to handle all artefacts and recognize fingerprint images with complete accuracy.Therefore, it is necessary to create a framework that can effectively address all of the aforementioned problems.Therefore, this proposal, proposes a unique Dual-Cross GAN framework with Capsule Network based PCA filtration framework as illustrated in Figure 2 for accomplishing precise fingerprint identification automatically by restoring the region of interest.At the initial step, a touchless fingerprint image preprocessing technique is employed that extracts the input, detects the fingerprint using the modified Haar-like pattern using the SVM algorithm [28], and performs optimized alignment and cropping of the required portion of the image, allowing the complexity of processing unwanted regions to be deduced at an early stage, after that, a dual normalization of size and pixels is carried out so that the appropriate input can be provided to the network throughout the succeeding step.

b) Capsule network based PCA filtration
An optimum Capsule Net-based PCA filtration is performed after the pre-processing stage, using a capsule neural network to capture spatial information and major distinguishing factors to compensate for the loss of information inherent in pooling processes.The collected features are then sent into a Principal Component Analysis (PCA) model.

Figure 3. Capsule network
Figure 3 displays a straightforward Capsule Network architecture [29].There are simply two convolutional layers and one fully connected layer, making this a very simple architecture.The Preprocessed image of size 28×28 is given to the first convolution layer(Conv1), which executes 256 convolutional with kernel of size 9×9, with a stride of 1, ReLU activation function.This layer converts pixel intensities to the 20 ×20 local features that are then used as inputs for the primary capsules.In order to create 32 capsule maps with an 8D vector as the capsule, the second layer additionally executes 256 convolutional operations with a stride of 2. Each 16D capsule in the final Layer, receives input from all capsules in the layer below.
For total input 'Ni', the vector output 'vi' for each capsule 'i' is given by The previous layer capsule output 'pj' is then multiplied by a 'Mji' weight matrix to generate the input 'Ni' which is a weighted sum over all 'prediction vectors' ̂ | and   is a coupling coefficient.
The coupling coefficients between capsule i and every other capsule in the layer above are calculated using the softmax of   .
Then the collected features are given to PCA filtration, here complete dataset consisting c+1 dimension.In machine learning paradigm c represents X_train and 1 represents Y_train (labels).So X_train + Y_train equals our entire training dataset.where the best possible fingerprint match is found by generating independent variables that minimize the amount of time-varying correlation.The operation is completed if the correlation is higher than a threshold; otherwise, if the fingerprint is not identified, there is a low quality that needs to be fixed.

c) Dual-cross GAN framework
This network is based on the modelling of two separate GAN networks.The Dual Generative Adversarial Network (GAN) [12] was introduced as a method for conducting imageto-image translation without the need for paired data.Dual GAN aims to develop the ability to convert images from a source domain X to a desired domain Y. Dual Gan is a framework consisting of two generators, G1 and G2, which are trained to learn the mappings from X to Y and from Y to X, respectively [30].Additionally, there are two corresponding adversarial discriminators, D1 and D2.A schematic model and information flow is depicted in Figure 4.The function  1 (, ) is subsequently transformed back into the original domain X using the function G2.Discriminator D1 is trained using y as positive samples and G1(x, n) as negative samples.Generators G1 and G2 are set up to make "fake" outputs that fool the respective discriminators D1 and D2 and to keep the two rebuilding losses to a minimum.The overall objective of Dual GAN is Expressed: where, ƞ represents a parameter that regulates the impact of losses, and Ɩ GAN represent the adversarial loss.
where,  2 (|) represents the probability that the generator has correctly classified the actual image.It could more accurately classify the fake image produced by the generator by maximizing log (1 −  2 ( 1 (|)) .In this scenario, the generator  aims to produce images that are indistinguishable from real images  by the discriminator D2.
In order to map  → , this procedure is utilized as well.On the other hand, due to the fact that adversarial loss by itself is unable to ensure the formation of the intended image, Dual_con loss was suggested as an alternative.This loss can be expressed as follows: In GAN1, a Convolutional Neural Network (CNN) with a variational encoder structure is utilized, with a leaky ReLu activation function and a tanh function in the final layer of the network, to ensure normal distribution, and a convolutional kernel is employed to convolve and deconvolve optimally with the Adam algorithm for parametric optimization.After that, a discriminator is generated with the help of VGG19, and the sigmoid function is used to perform batch normalization upon the convolutional layers of the network.The GAN2 is a DeblurGan with a Feature Pyramid network and a ResNet-like topology.Because of the cross-setup between GAN1 and GAN2, a high performance is achieved when processing fingerprint images.From the resulting fingerprint images, the fingerprint image is then correctly identified using PCA.Because of this, the suggested framework may be used to correctly identify fingerprint, resolving the problems with earlier networks.

RESULT AND DISCUSSION
In this section, the precise experimental findings that were collected are presented in order to evaluate the efficiency of the proposed framework.The proposed Dual-Cross GAN framework with Capsule Network based PCA filtration is implemented in Python.The output of the implementation and the results obtained are discussed in this section.

Dataset description
Two touchless fingerprint datasets, Benchmark 2D/3D [25] and PolyU Cross [26], are utilized in our investigations in order to examine the proposed technique and then compare it with different techniques.The data in the PolyU Cross dataset come from two different sets.2016 touchless fingerprint images from a total of 336 fingers, with six impressions taken from each finger, were used in the first session.The second set of 960 fingerprint images, which were taken between two and twenty-four months after the initial set for the same clients, includes six fingerprint impressions for each unique finger.1500 different fingers were used in the collection of 9000 touchless fingerprint scans that make up the benchmark 2D/3D dataset.There are two separate perspectives, and each perspective produces two impressions.

Implementation results
After the training dataset, the touchless fingerprint images are pre-processed to create the test set, and a ratio of 3:1 is taken into consideration while dividing the dataset between training and testing.In this paper, the data loader function was utilized for gathering fingerprint image data, with the batch size chosen as 4 and the shuffling turned on, so that the sequence of four image data from the same batch taken at various times can be randomly shuffled in order to minimize the influence of the input sequence on PCA training.Figure 5, show the processing of single fingerprint image using the proposed framework.

Experimental evaluations
The overall performance of the proposed framework in terms of accuracy and Equal Error Rate is depicted in Figure 6.The framework that was suggested has obtained an accuracy (ACU) of 99.51% for the PolyU dataset and 99.13% for the benchmark 2D/3D dataset.Additionally, the proposed model has achieved an Equal Error Rate (EER) of 1.81% for the PolyU dataset and 3.34% for the benchmark 2D/3D dataset.As a result, the proposed model is shown to operate more effectively.The model is validated in the next part by a comparison to the traditional approaches.

Comparison metrics
In Figure 7 shown the ROC curves for fingerprint recognition are compared for several techniques on two different datasets.Their related AUCs and EERs have been compared in Table 1.We can observe that for both the datasets, the suggested method's ROC curve is higher than that of other approaches.As shown in to that of previous methods.Figure 8 shows a comparison of the CMC curves of various fingerprint recognition techniques using the PolyU Cross dataset and the Benchmark 2D/3D dataset, respectively.On both sets of data, we are able to see that the CMC curve generated by proposed method has a value that is noticeably higher than that generated by the other methods.

CONCLUSION
In this paper, we have presented a framework for touchless fingerprint with Capsule Network based PCA filtration using a Dual-Cross GAN Network.Our experimental results on two publicly available databases, which were discussed in the previous section, show that they perform significantly better than conventional techniques.Over two distinct databases, the Benchmark 2D/3D dataset and the PolyU cross dataset, the proposed model has obtained accuracy (ACUs) of 99.51% and 99.13% and Equal Error Rate (EER) of 1.81% and 3.34%, respectively.
The offered method is hypothesis-driven and has the potential to be implemented in a variety of experimental touchless environments.The findings of the experiments indicate that the approach that was proposed has the potential to effectively improve the overall accuracy of touchless fingerprint recognition systems.Additionally, the results that were obtained were compared to earlier work and determined to be superior in every way.
In summary, touchless fingerprint recognition is a cuttingedge technology with revolutionary possibilities for access control and identity verification.Touchless fingerprint recognition systems have proven to be more user-friendly than their predecessors, and they are quickly becoming a mainstream biometric performance modality.Therefore, we will direct our future studies toward developing novel methods for dealing with some of the forthcoming difficulties associated with touchless fingerprinting, such as accelerating feature extraction, decreasing the time needed to process images, and detecting fingerprint liveness.

Figure 2 .
Figure 2. Proposed framework a) Image preprocessingAt the initial step, a touchless fingerprint image preprocessing technique is employed that extracts the input, detects the fingerprint using the modified Haar-like pattern using the SVM algorithm[28], and performs optimized alignment and cropping of the required portion of the image, allowing the complexity of processing unwanted regions to be deduced at an early stage, after that, a dual normalization of size and pixels is carried out so that the appropriate input can be provided to the network throughout the succeeding step.b)Capsule network based PCA filtrationAn optimum Capsule Net-based PCA filtration is performed after the pre-processing stage, using a capsule neural network to capture spatial information and major distinguishing factors to compensate for the loss of information inherent in pooling processes.The collected features are then sent into a Principal Component Analysis (PCA) model.

Figure 5 .
Figure 5. Processing of single fingerprint image using the proposed framework

Figure 6 .
Figure 6.Performance of the proposed framework (a) for Accuracy (b) for equal error rate

Table 2 ,
our proposed method is successful in achieving a drop in EER for the PolyU Cross dataset and the Benchmark 2D/3D dataset.According to these findings, the performance of our suggested method is superior

Table 2 .
AUCs and EERs of different methods on touchless fingerprint recognitions