Abderrahmane Herbadji* | Noubeil Guermat | Lahcene Ziet | Zahid Akhtar | Mohamed Cheniti | Djamel Herbadji
© 2020 IIETA. This article is published by IIETA and is licensed under the CC BY 4.0 license (http://creativecommons.org/licenses/by/4.0/).
OPEN ACCESS
Due to the COVID-19 pandemic, automated contactless person identification based on the human hand has become very vital and an appealing biometric trait. Since, people are expected to cover their faces with masks, and advised avoiding touching surfaces. It is well-known that usually contact-based hand biometrics suffer from issues like deformation due to uneven distribution of pressure or improper placement on sensor, and hygienic concerns. Whereas, to mitigate such problems, contactless imaging is expected to collect the hand biometrics information without any deformation and leading to higher person recognition accuracy; besides maintaining hygienic and pandemic concerns. Towards this aim, in this paper, an effective multi-biometric scheme for person authentication based on contactless fingerprint and palmprint selfies has been proposed. In this study, for simplicity and efficiency, three local descriptors, i.e., local phase quantization (LPQ), local Ternary patterns (LTP), and binarized statistical image features (BSIF), have been employed to extract salient features from contactless fingerprint and palmprint selfies. The score level fusion based multi-biometric system developed in this work combines the matching scores using two different fusion techniques, i.e., transformation based-rules like triangular norms and classifier based-rules like SVM. Experimental results on two publicly available databases (i.e., PolyU contactless to contact-based fingerprint database and IIT-Delhi touchless palmprint dataset) show that the proposed contactless multi-biometric selfie system can easily outperform uni-biometrics.
COVID-19, multi-biometrics, score fusion, contactless fingerprint, contactless palmprint, BSIF, person authentication
Biometrics technology refers to the employment of human’s anatomical, behavioral and/or chemical traits such as wrist vein, fingerprint, iris, face, voice, gait for person identification. Biometrics now being utilized in numerous people recognition’s applications ranging from mobile authentication to border crossing. Biometrics offers various advantages (e.g., higher efficiency, user convenience, better security) over traditional human’s identification methods relying on ID card, PIN, password, etc. [1]. These conventional mechanisms to recognize individual are easy to share or forget, which is not possible in biometrics and it is thus considered more reliable for human recognition. In fact, biometrics technologies are increasingly being deployed as an mandatory security tool [2-4], by individuals (e.g., Fujitsu NX F-04G and iPhone 5s smartphones are unlocked using iris and fingerprint, respectively), business(e.g., voice or fingerprint based access to online banking by HSBC) and governments (e.g., US-VISIT program that authenticates the identity of the visitors of US via face and ten fingerprints image, and iris images based travel’s identity recognition at the Amsterdam airport ).
Numerous biometric traits now are being utilized in different applications. The choice of biometric trait is not only related to the performance, but it also needs to meet some requirements such as stability, accuracy, cost, universality, security, acceptability, measurability, and population size [5]. Besides those mentioned requirements, other factors also play a pivotal role in choosing biometric traits like culture, perception, and environment. For instance, use of face as biometric modality is not preferred at places where females wear veils owing to religious convictions, whilst iris will be a good choice at dark places (e.g., dark coal mines) [2]. Similarly, owing to the COVID-19 pandemic, people are using face masks and avoiding touching sensors’ surfaces.
The process of recognition in biometric systems may be categorized into two categories [1], the first one is biometric verification (i.e., verifying if two samples of input biometric data are related to the same person’s identity, namely question “Are you who you say you are?”) whilst the second one is biometric identification (i.e., input data to identify the person who they say are or not, namely “Are you someone who is known to the system?” or “Are you who you say you are not?”). Furthermore, biometric systems can be either unimodal or multi-modal. The unimodal systems employ only one biometric trait. These systems are stymied by numerous issues, e.g., dark fingerprint leads to poor collected data, occluded faces make biometric information missing, difficulty of distinguishing between twins’ identities in case of using only face images to identify them. Unfortunately, systems that rely on a single distinctive biometric trait present drawback and are not able to achieve the performance criteria in terms of distinctiveness, universality, accuracy, and acceptability. In order to lessen inherent limitations of uni-biometrics and minimize the error rates, techniques for integrating multiple biometric evidences have attracted researchers’ attention with the purpose of protecting the privacy of user against attacks, guarantee interoperability, manage users’ huge databases, and able to handle incomplete information. The system using more than one biometric trait is known as multi-modal biometric system. In multi-modal biometric systems, the information can be merged at four levels namely, sensor, feature, match-score, and decision. Multi-biometric systems are expected to increase the accuracy and attain low error rates better than systems that employ a single trait.
Contact-based hand based biometrics, in particular fingerprint, is the most common biometric trait widely adopted by national ID programs, the law-enforcement departments around the world, and e-business, because provides several advantages such as permanence, accuracy, and uniqueness [6]. Various fingerprint modality matching algorithms can be grouped into three classes, (i) minutiae-based matching [7], (ii) non-minutiae-based matching [8-10], and (iii) correlation-based matching [11]. Moreover, the fingerprint recognition systems can collect the fingerprint images via different kind of sensors, e.g., capacitive or optical. Thus, the fingerprint sensor interoperability problem has attracted attention. Also, contact-based fingerprint sensors should deal with other concerns like sensor surface noise, deformation, and hygienic concerns. In order to address these shortcomings, contactless fingerprint has been recently investigated for more security and better hygienic purposes [12]. Imaging of contactless fingerprint can usually be done without any deformation, which leads to better matching performance. Like fingerprint, palmprints are located in the inner surface of the human hand, and depend on three line patterns (i.e., principal lines, wrinkles and epidermal ridges), and marks, texture, and indents.
Though many fingerprint and palmprint systems, including contactless, to recognize people have been investigated [13-16], almost all those published works are based on uni-biometrics, i.e., only one traits is used to verify user’s identity. But it is well documented that these systems are unable to provide required security performances and high accuracy. To the best of our knowledge, no contactless multimodal biometric system that integrates fingerprint and palmprint patterns has been presented in the literature. To this end, we present a contactless multi-biometric framework, which integrates the information originated from fingerprint and palmprint selfies at score level fusion. The proposed contactless multi-biometric score fusion framework was evaluated on publicly available PolyU Contactless to Contact-based Fingerprint database and IIT-Delhi touchless palmprint database. Some of the potential advantages of the presented contactless multi modal framework are: (i) it provides better hygiene and security; (ii) fingerprint and palmprint patterns offer high matching performance due to their uniqueness, permanence, and stability; (iii) it can achieve high level of accuracy than fingerprint and palmprint based uni-biometrics; (iv) fingerprint and palmprint are well accepted for recognition by users, and have a durable immunity to ageing issues, thus, making it easy to match the templates for that person after a certain time compared to other biometric modalities that are affected by ageing (e.g., face); (v) the collection of contactless fingerprint and palmprint biometric data requires no cooperation from the user, and can be collected utilizing a simple conventional cameras from distance, thereby low cost equipment is needed to capture these modalities; and (vi) it has low complexity owing to use of effective local texture descriptors for fingerprint and palmprint selfie images. The organization of the presented paper is as follows. Next section provides the related works on contactless fingerprint and palmprint recognition systems. Section 3 presents the three used local texture descriptors. Section 4 is devoted to the experimental analysis. A conclusion is drawn in Section 5.
In order to attain a reliable solution for person authentication or identification, multi-biometric systems have been proposed. Diverse set of studies proved the effectiveness of multi-biometrics authentication systems [17-19], and showed that in multi-biometric systems different biometric cues compensate inherent frailties of other uni-modal biometric systems. Although biometric features can be fused at different levels (i.e., feature, scores, decision), the fusion of match-scores is considered convenient, because while it can boost the system’s reliability, it decreases overall complexity. Researchers have presented numerous techniques to integrate match-scores [19-23]. For instance, Peng et al. [24] used t-norms to fuse multiple finger traits such as fingerprint, finger vein, finger knuckle print, and finger shape. Similarly, t-norms, sum rule, min and max rule have been employed to combine palm and wrist vein biometrics [25]. While, Shariatmadar and Faez [26] explored the combination of four different fingers’ knuckle print (i.e., multiple-instance fusion) and they achieved performance improvement in comparison with uni-modal systems (i.e., single-instance).
Besides consolidation of biometric evidence at match-score level, fusion at feature level has also explored by researchers in order to improve the recognition rate. For instance, Hezil and Boukrouche [27] proposed the use of canonical correlation analysis (CCA) and the serial feature fusion to integrate ear and palmprint. They obtained a recognition rate of 100%. While, Yang and Zhang [28] investigated a multi-biometric system to recognize an individual based on their fingerprint and finger vein. The features have been extracted using unified Gabor filter framework and combined via supervised local-preserving canonical correlation analysis method. Also, multi-biometric user recognition systems based on decision level fusion were investigated in literature, e.g., references [29, 30].
Literature analysis shows that there exist several works on multi-biometric systems based on contact based data acquisition, but contactless multi-biometric systems, specially using fingerprint and palmprint, are under explored domain. For example, Mhaske and Patanka [31] suggested a contact based palmprint and fingerprint recognition scheme using modified gabor filter to extract the features. The authors [32] presented a hierarchical minutiae strategy for contact-based palmprint and fingerprint images, which can be used in the matching step; this proposed algorithm leads to decrease in time for searching by almost 50% in comparison with traditional methodologies. Chin et al. [33] combined fingerprint and palmprint modalities at the feature level and then secure the integrated template via a hybrid protection scheme. This scheme utilized two techniques, i.e., random tiling and an equal-probable 2N discretization. Bajwa and Kumar [34] investigated a multi-biometric system that combined palmprint and fingerprint features to recognize a person, and employed thinning feature extraction strategy and Hidden Markov Model approach. While, Bellaaj et al. [35] performed possibility theory concepts on fingerprint and palmprint images and fused the match-scores via a simple sum rule; the multi-biometric system was evaluated on CASIA and FVC databases.
Majority of the works on contact based fingerprint and palmprint identification systems use location of minutiae and bifurcation points as reliable features. But it is hard to extract these features from samples of low quality due to non-uniform pressure between finger and the scanner or skin condition, etc. Some of such issues can be overcome by contactless imaging. Contactless fingerprint and palmprint images can be captured in higher-quality, and thus, rich information can be extracted. There exist no studies on the fusion of contactless-based fingerprint and palmprint. Moreover, due to COVID-19 pandemic people are using face mask and avoiding touching sensor surfaces. This motivates us to explore fusion of contactless patterns of fingerprint and palmprint for person verification.
Figure 1 depicts a schematic diagram that illustrates the procedure of proposed contactless multi-biometric system, which combines data from fingerprint and palmprint traits. In an authentication setting, the user should claim the identity and provide his/her biometric traits to the camera (i.e., sensor) in order to extract individual features using local texture descriptor (explained below in Section III-A). Once, the useful features are extracted from the collected images, the system compares these features with corresponding templates (i.e., features) of the claimed identity accumulated at enrollment stage in the data set, and yields a fingerprint and palmprint matching scores Sf and Sp, respectively. Since the produced scores are heterogeneous, therefore the scores are converted in the same range [0, 1]. In this study, min-max normalization scheme (Eq. (1)) was applied for this purpose and can be defined as:
$S^{\prime}=\frac{S-S_{\min }}{S_{\max }-S_{\min }}$ (1)
where, Smax and Smin are the maximal and minimal values of S, respectively.
Next, contactless fingerprint and palmprint normalized scores are combined via SVM classifier [36] or binary functions called t-norms [37], which generalizes the intersection at the fuzzy sets, and satisfy properties such as commutativity, associativity, and monotonicity. The employed t-norms in this study are defined as:
Hamacher: $\frac{S_{f} S_{p}}{S_{f}+S_{p}-S_{f} S_{p}}$
Schweizer-Sklar (m > 0): $\left(\max \left(S_{f}^{m}+S_{p}^{m}-1,0\right)\right)^{\left(\frac{1}{m}\right)}$
Aczel-Alsina (m > 0): $e^{-\left(\left(-\ln S_{f}\right)^{m}+\left(-\ln S_{p}\right)^{m}\right)^{\left(\frac{1}{m}\right)}}$
Frank (m >0): $\log _{m}\left(1+\frac{\left(m^{S} f-1\right)\left(m^{S_{p}-1}\right)}{m-1}\right)$
Finally, if the combined score is greater than a threshold (τ) then the user is classified as genuine, otherwise he/she is rejected as an impostor.
Figure 1. Proposed contactless fingerprint and palmprint user verification system based on local texture features
3.1 Local texture descriptors
Texture-based feature extraction strategies are very popular in image processing and computer vision fields, and they have demonstrated their effectiveness in numerous applications like object recognition and image classification due to their robustness to weak lighting and being are very distinctive. Therefore, local descriptors can be applied more effectively to contactless fingerprint and palmprint based multimodal biometric system. The presented work in this paper also exploited three local features explained below.
3.1.1 Local Phase Quantization (LPQ)
This descriptor was proposed by Ojansivu and Heikkila [38] for texture description [4]. It is based on the blur invariance property of the Fourier phase spectrum. The main idea behind LPQ method is to employ 2-D DFT to extract the local phase information. The phase information is calculated in local M-by-M neighborhoods Nx at each pixel position x of the image f(x).
$F(\boldsymbol{u}, \boldsymbol{x})=\sum_{y \in N_{x}} f(x-y) e^{-j 2 \pi u^{T} y}=w_{u}^{T} f_{x}$ (2)
where, wu is the basis vector of the 2-D DFT at frequency u, and fx is another vector containing all M2 image samples from Nx.
The local Fourier coefficients are computed at four frequency points u1 = [a; 0]T, u2 = [0; a]T, u3 = [a; a]T, u4 =[a; −a]T where a is a scalar frequency that satisfies H(ui) > 0. Note that H(u)is the original image. Let
$\boldsymbol{F}_{x}=\left[F\left(\boldsymbol{u}_{1}, \boldsymbol{x}\right), F\left(\boldsymbol{u}_{2}, \boldsymbol{x}\right), F\left(\boldsymbol{u}_{3}, \boldsymbol{x}\right), F\left(\boldsymbol{u}_{4}, \boldsymbol{x}\right)\right)$ (3)
The phase information in the Fourier coefficients is recorded by observing the signs of the real and imaginary parts of each component in Fx as follows:
$b_{j}(x)=\left\{\begin{array}{c}1, \text { if } b_{j}(x) \geq 0 \\ 0, \text { otherwise }\end{array}\right.$ (4)
where, bj(x) is the jth component of the vector Bx = [Re{Fx}; Im{Fx}].
3.1.2 Local Ternary Patterns (LTP)
This descriptor was introduced by Tan and Triggs [39] as a generalization of the Local Binary Pattern (LBP), which is more discriminant as well as less sensitive to noise in uniform regions. Note that LTP descriptor attained a high accuracy for face identification under uncontrolled lighting conditions. The LTP operator extends LBP to 3-valued codes using a user specified threshold. The indicator s(u) is computed as:
$s^{\prime}\left(u, i_{c}, t\right)=\left\{\begin{array}{c}1, \quad u \geq i_{c}+t \\ 0, \quad\left|u-i_{c}\right|<t \\ -1, u<i_{c}-t\end{array}\right.$ (5)
where, t, ic, and u are user-specified threshold, centre pixel, and neighborhood pixels value, respectively. The LTP encoding process is shown in Figure 2, where t equals 5, and thereby the tolerance interval is [49, 59].
Figure 2. Basic LTP operator
3.1.3 Binarized Statistical Image Features (BSIF)
This descriptor has been proposed by Kannala and Rahtu [40]. It is inspired by other popular local descriptors like LPQ [38] and LBP [41]. By using a small set of natural images, BSIF encodes each pixel of the input biometric image in terms of binary strings, instead of employing handcrafted filters like in LBP and LPQ descriptors. It is worth noticing that BSIF descriptor presented encouraging results in several applications such as face recognition, ear recognition, and fingerprint liveness detection. Since BSIF uses learning instead of manual setting, adjustment of the descriptor length can be done in a simple and flexible way, which can lead to achieve a statistically expressive representation of biometric data and thereby enables efficient information encoding using straightforward element-wise quantization. Therefore, we adopted BSIF for efficient personal verification from contactless fingerprint and palmprint.
Given a contactless fingerprint or palmprint image patch X and a linear filter Wi of the same size, the filter response si is calculated as
$s_{i}=\sum W_{i}(u, v) X(u, v)$ (6)
where, the binarised feature bi is obtained by setting bi = 1. If si>0 and bi= 0 otherwise. The filters Wi are learnt using independent component analysis (ICA) by maximizing the statistical independence of si. The filters Wi were learnt using different choices of parameter values and 50,000 image patches were employed to learn each set of filters.
As we mentioned above, three local texture descriptors (LPQ, LTP, and BSIF) have been employed to extract discriminant features from fingerprint and palmprint selfies, which are characterized by a rich information for robust person recognition. Here, the recognition is done in a contactless manner to avoid the issues related to contact-based biometrics such as deformation, hygienic and pandemic concerns (e.g., COVID-19 pandemic). Besides, the biometric information is collected without any contact between a finger and sensor and thereby there is no need to clean the latent left on the sensor, which leads to speeding up the process of recognition, especially in high-traffic areas. Normalized fingerprint and palmprint selfies along with their corresponding LPQ, LTP, and BSIF are displayed in Figures 3 and 4.
Figure 3. Normalised contactless fingerprint along with their corresponding local features codes. From left to right, original image, BSIF, LPQ, and LTP code features
Figure 4. Normalised contactless palmprint along with their corresponding local features codes. From left to right, original image, BSIF, LPQ, and LTP code features
4.1 Databases
The presented contactless multibiometric system based on fingerprint and palmprint were evaluated using two benchmark datasets, namely the PolyU Contactless to Contact-based Fingerprint Images Database Version 1.0 and IIT-Delhi touchless palmprint.
4.1.1 PolyU Contactless to Contact-Based Fingerprint database
In the PolyU Contactless to Contact-based Fingerprint database [12], fingerprint images (i.e., contact based and contactless) acquired from 336 volunteers are available. This database provides six fingerprint images for each user besides to their corresponding contact-based images.
The images of this dataset have been collected using contact-based and contactless traditional sensors from the students and staff at the university in 2016; some sample images from this data set are shown in Figure 5. In this study, we used only contactless images.
Figure 5. Example of normalized contactless fingerprint images
4.1.2 IIT-Delhi touchless palmprint database
In the IIT-Delhi touchless palmprint database [42], palmprint images were acquired using touchless imaging setup of 230 users. It consists of right and left hand images and have been collected during July 2006 to Jun 2007 in the IIT Delhi campus at an indoor environment. The subjects were in the age group of 12 to 57 years. The touchless palmprint images have a resolution of 800 × 600 pixels. Figure 6 displays some touchless palmprint images from IIT-Delhi database.
Figure 6. Example of normalized contactless palmprint images
4.2 Results
To analyze the proposed user verification system’s performance, the ROC (receiver operating characteristic) curve has been utilized. Note that ROC curve is a plot of FAR versus GAR. GAR is genuine acceptance rate such as GAR = 1 – FRR, FAR is false acceptance rate, and FRR is false rejection rate. In addition to ROC curve, the performances were also tested based on EER (equal error rate), where EER is measured when FAR and FRR are equal.
4.2.1 Performance of individual contactless fingerprint system
In this section, we evaluate contactless fingerprint based unimodal system. Namely, the experiments were accomplished on the PolyU Contactless to Contact-based Fingerprint database. Four contactless fingerprint images per user (i.e., two train and two test images) belonging to 230 users have been randomly chosen from this data set. Hence, we obtain 460 (230 × 2) genuine match-scores and 119,140 (230 × 29 × 2) imposter match-scores.
First, we used BSIF features for the purpose of analyzing insight into the accuracy of the contactless fingerprint system. We carried out experiments based on the poyU Contactless to Contact-based Fingerprint database. As we mentioned above that the BSIF descriptor is characterized by two parameters: the size of window filter and the length of the binary code (i.e., number of bits), thus we have explored all BSIF parameters to identify the values that can lead to the best verification rate.
In Table 1, we present the verification rates of the unimodal biometric user authentication, where BSIF descriptor was used to extract the features from the contactless fingerprint images. From this table, we can notice that the best performance is obtained in case of using a window size of 17 × 17 pixels and 12-bits.
4.2.2 Performance of individual palmprint system
To test contactless palmprint based uni-modal system, the same evaluation protocol for contactless fingerprint system was applied on IIT-Delhi touchless palmprint data set using three local features (LPQ, LTP, and BSIF).
In order to find the best parameters that lead to the optimal BSIF descriptor accuracy for palmprint modality, we have explored all the BSIF parameters on IIT-Delhi touchless palmprint data set. Then, we choose the best performance for fusion along with contactless fingerprint purpose. As can be observed in Table 2, the best parameters for BSIF descriptor is windows size of 17 × 17 pixel with 12 bits string. The obtained results indicate that BSIF works better with 12 bits string for all the window sizes. Note that the lower verification rate for uni-modal palmprint is yielded with windows size of 3 × 3 pixel and 5 bits string, where a GAR equals to 16.50% and is obtained at FAR equals to 0.01%. Thus, BSIF descriptor is less discriminative in case of using small window sizes.
Table 1. Verification rates obtained for contactless fingerprint uni-biometric system using all BSIF parameters
|
BSIF parameter |
3×3 |
5×5 |
7×7 |
9×9 |
11×11 |
13×13 |
15×15 |
17×17 |
|
5 |
70.00 |
80.02 |
83.00 |
81.30 |
82.00 |
79.00 |
72.00 |
69.00 |
|
6 |
81.20 |
83.90 |
86.00 |
87.23 |
87.15 |
85.92 |
83.15 |
83.70 |
|
7 |
83.10 |
85.00 |
86.25 |
88.10 |
89.10 |
88.30 |
87.35 |
86.00 |
|
8 |
80.45 |
88.20 |
89.12 |
90.00 |
90.75 |
91.18 |
90.00 |
88.76 |
|
9 |
/ |
88.70 |
88.80 |
89.20 |
90.57 |
91.35 |
90.40 |
90.70 |
|
10 |
/ |
89.40 |
89.67 |
90.00 |
91.10 |
90.62 |
91.61 |
91.78 |
|
11 |
/ |
89.70 |
90.00 |
91.00 |
91.90 |
93.00 |
92.67 |
91.95 |
|
12 |
/ |
90.66 |
90.07 |
91.20 |
92.27 |
93.30 |
93.00 |
93.50 |
Table 2. Verification rates obtained for contactless palmprint uni-biometric system using all BSIF parameters
|
BSIF parameter |
3×3 |
5×5 |
7×7 |
9×9 |
11×11 |
13×13 |
15×15 |
17×17 |
|
5 |
16.50 |
21.00 |
34.50 |
39.00 |
47.10 |
48.55 |
50.12 |
54.00 |
|
6 |
20.06 |
29.40 |
41.00 |
52.30 |
60.00 |
62.40 |
63.50 |
62.65 |
|
7 |
26.85 |
33.05 |
53.30 |
65.00 |
69.42 |
70.08 |
72.00 |
73.90 |
|
8 |
30.10 |
35.20 |
56.00 |
71.81 |
74.55 |
78.30 |
79.72 |
79.50 |
|
9 |
/ |
39.00 |
62.30 |
74.50 |
79.61 |
80.00 |
82.05 |
84.10 |
|
10 |
/ |
42.30 |
65.68 |
79.65 |
81.45 |
86.45 |
87.22 |
86.75 |
|
11 |
/ |
44.65 |
67.48 |
80.60 |
85.30 |
87.00 |
89.50 |
90.20 |
|
12 |
/ |
45.38 |
69.85 |
83.24 |
88.35 |
89.60 |
91.32 |
92.55 |
Table 3 compares verification rates obtained via different local features, these results show that BSIF descriptor with windows sizes of 17 × 17 pixels and 12 bits outperforms other descriptors such as LPQ and LTP.
Table 3. Comparison of verification rates obtained for contactless fingerprint and palmprint based unimodal systems
|
Descriptor |
BSIF |
LPQ |
LTP |
|
Contactless fingerprint |
93.50 |
90.15 |
84.00 |
|
Contactless palmprint |
92.55 |
81.50 |
80.80 |
4.2.3 Performance of multi-modal biometric user verification
The multi-biometrics’ effectiveness has been empirically demonstrated in the literature. A biometric multi-modal system integrates evidences of several biometric traits of the same user in order to attain enhanced accuracy. Therefore, pertinent match-scores combination method plays a serious part in the accuracy of the resulting estimation. In this work, the proposed contactless multimodal framework is composed of two traits, i.e., fingerprint and palmprint. More specifically, the scores were transformed in the same range [0, 1] by the min-max normalization scheme as shown in (Eq. (1)). The verification rate is reported in term of ROCs of uni-biometric systems using different local descriptors-based feature extraction. Also, two types of score fusion are utilized and compared: transformation-based using t-norms and classification-based using SVM classifier.
Figure 7(a) depicts ROCs of individual contactless fingerprint and palmprint and their multi-biometric system through BSIF descriptor. We can state that schweizer-Sklar t-norms fusion scheme increases the verification rate of uni-biometric systems, as GARs of fingerprint and palmprint are 93.5% and 92.55% at FAR=0.01 percent, respectively. Whilst SchweizerSklar fusion boosts the performance to 100% at the same FAR%. It is easy to observe in Figure 7(b) that 90.15%, 81.5% and 97.65% of GARs are obtained using LPQ descriptor, respectively, for fingerprint, palmprint and SVM using RBF function for score level fusion. From Figure 7(c), we can see that GARs of individual systems via LTP descriptor are 84.0% and 80.8% are attained, whereas GAR equals to 98.0% when Aczel-Alisina t-norms is utilised to combine the match-scores. Table 4 compares verification rates of the proposed contactless multi biometric systems. The results were obtained using the three local features (i.e., LPQ, LTP, and BSIF that has been implemented with a 17 × 17 pixel filter and a 12-bit string) and two types of matching score fusion namely, transformation (i.e., t-norms) and classification (i.e., SVM) based schemes. The obtained matching performances provide evidence that BSIF descriptor achieves better results than LBP and LPQ local features. More specifically, a GAR = 100% at FAR = 0.01% is achieved with Schweizer-Sklar (m = 1.8) and Aczel-Alisina (m = 0.5) t-norms. In Table 4, we also report EER of contactless fingerprint and palmprint based person verification biometric system with different local features as well as different combination of scores strategies.
It is easy to see in Table 4 that BSIF descriptor can lead to minimize the EER, where EER = 0.00% is attained and thereby achieving high accuracy.
In Figure 8, we display the probability density functions of both contactless fingerprint and palmprint based individual systems along with multi-biometric system. In Figure 8, it is easy to observe that there is an overlap between the distributions of legitimate and imposter matching scores. Whilst, fusion of the two biometric traits using proposed framework is able to achieve almost no overlap between their distributions, thereby improving the recognition.
Table 4. Comparison of VRs and EERs obtained for contactless multi-biometric system utilizing different techniques
|
Score level fusion method for FAR= 0.01% |
BSIF |
LPQ |
LTP |
|||
|
VR(%) |
EER(%) |
VR(%) |
EER(%) |
VR(%) |
EER(%) |
|
|
Hamacher t-norm |
98.91 |
0.87 |
94.30 |
1.62 |
94.53 |
1.67 |
|
Schweizer-Sklar t-norms (m=1.8) |
100.0 |
0.00 |
96.089 |
0.65 |
96.66 |
0.87 |
|
Aczel-Alisina t-norms (m=0.5) |
100.0 |
0.00 |
97.22 |
0.65 |
98.00 |
0.80 |
|
Frank t-norm (m=1.8) |
99.13 |
0.84 |
95.00 |
1.50 |
95.26 |
1.30 |
|
SVM with linear function |
91.90 |
1.90 |
88.62 |
2.41 |
88.25 |
2.85 |
|
SVM with RBF function |
99.60 |
0.02 |
97.65 |
0.24 |
98.00 |
0.80 |
(a) (b) (c)
Figure 7. ROC curves of unimodal systems along with their combination based on (a) BSIF, (b) LPQ, and (c) LTP features
(a) (b) (c)
Figure 8. Score distribution using local features of (a) contactless fingerprint, (b) palmprint and (c) multi-biometric score combination
Table 5. Comparison of related and recent study on personal verification based on the combination of fingerprint and palmprint
|
References |
Feature extraction |
Fusion level |
Fusion technique |
Performance |
|
Chin et al. [43] |
2D Gabor filters |
Features |
Features concatenation |
EER = 1.0% |
|
Geetha and Radhakrishnan [44] |
Coiflet wavelets |
Features |
SVM with RBF kernel |
Accuracy = 97.53 % F-Measure = 97.6% |
|
Mhaske and Patankar [31] |
Modified Gabor filter |
Features |
Features concatenation |
Accuracy = 88.89 % |
|
Manssor et al. [45] |
Contourlet transform |
Features Match-scores |
Features concatenation Product rule |
EER = 0.538% EER = 0.548% |
|
Proposed framework |
BSIF |
Match-scores |
Schweizer-Sklart-norms |
GAR = 100.0% |
Table 5 summarizes the archived verification rates of the proposed contactless fingerprint and palmprint along with other existing multi-biometric systems that integrate the fingerprint and palmprint traits. It is clear that our proposed system achieves the best results compared to ones reported in [31, 43, 45].
To sum up, the obtained results prove that the proposed multi-biometric system, which is based on contactless fingerprint and palmprint are robust to verify the person’s identity. By comparing our multimodal biometric under different local features, we can state the BSIF descriptor yielded a significant improvement in authentication rates for both unimodal and multimodal systems.
This paper focuses on the use of contactless patterns of fingerprint and palmprint for person recognition. During COVID-19 pandemic, employing contactless hand biometrics to identify individuals will be very helpful since most of them are wearing face masks. Moreover, contactless hand patterns can overcome the drawbacks imposed by contact-based ones such as hygienic risks, contaminated patterns due to the latent left on a scanner platen, and high failure-to-collection rate caused by moist, oily, or dry. Moreover, the proposed method is based on an efficient and compact feature extraction and recognition framework. To this aim, a contactless multimodal biometric for verifying individuals via their fingerprint and palmprint modalities has been presented. For feature extraction, three local texture descriptors (i.e., LPQ, LTP, and BSIF) have been employed. The proposed contactless multi-modal biometric system was evaluated using chimerical dataset based on publicly available PolyU Contactless to Contact-based Fingerprint database and IIT-Delhi touchless palmprint. Experimental results showed that the presented multi-modal biometric system can not only outperform unimodal systems but also prior multimodal systems. In future, we plan to explore vulnerability of the proposed method. Also, we would conduct a comparative study on different fusion strategies like sensor-, feature-, and decision-level fusion schemes.
[1] Jain, A.K., Ross, A.A., Nandakumar, K. (2011). Introduction to Biometrics. Springer Science & Business Media.
[2] Akhtar, Z., Hadid, A., Nixon, M.S., Tistarelli, M., Dugelay, J.L., Marcel, S. (2018). Biometrics: In search of identity and security (Q & A). IEEE MultiMedia, 25(3): 22-35. https://doi.org/10.1109/MMUL.2018.2873494
[3] Akhtar, Z., Fumera, G., Marcialis, G.L., Roli, F. (2011). Robustness evaluation of biometric systems under spoof attacks. In International conference on Image Analysis and Processing, Springer, Berlin, Heidelberg, pp. 159-168. https://doi.org/10.1007/978-3-642-24085-0_17
[4] Akhtar, Z., Fumera, G., Marcialis, G. L., Roli, F. (2011). Robustness analysis of likelihood ratio score fusion rule for multimodal biometric systems under spoof attacks. In 2011 Carnahan Conference on Security Technology, pp. 1-8. https://doi.org/10.1109/CCST.2011.6095935
[5] Jain, A.K., Nandakumar, K., Ross, A. (2016). 50 years of biometric research: Accomplishments, challenges, and opportunities. Pattern Recognition Letters, 79: 80-105. https://doi.org/10.1016/j.patrec.2015.12.013
[6] Maltoni, D., Maio, D., Jain, A.K., Prabhakar, S. (2009). Handbook of Fingerprint Recognition. Springer Science & Business Media.
[7] Maio, D., Maltoni, D. (1997). Direct gray-scale minutiae detection in fingerprints. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(1): 27-40. https://doi.org/10.1109/34.566808
[8] Jain, A.K., Prabhakar, S., Hong, L., Pankanti, S. (2000). Filter bank-based fingerprint matching. IEEE Transactions on Image Processing, 9(5): 846-859. https://doi.org/10.1109/83.841531
[9] Yang, J.C., Park, D.S. (2008). Fingerprint verification based on invariant moment features and nonlinear BPNN. International Journal of Control, Automation, and Systems, 6(6): 800-808.
[10] Tachaphetpiboon, S., Amornraksa, T. (2005). A fingerprint matching method using DCT features. In IEEE International Symposium on Communications and Information Technology ISCIT, 1: 461-464. https://doi.org/10.1109/ISCIT.2005.1566893
[11] Kumar, B.V., Savvides, M., Xie, C., Venkataramani, K., Thornton, J., Mahalanobis, A. (2004). Biometric verification with correlation filters. Applied Optics, 43(2): 391-402. https://doi.org/10.1364/AO.43.000391
[12] Lin, C., Kumar, A. (2018). Matching contactless and contact-based conventional fingerprint images for biometrics identification. IEEE Transactions on Image Processing, 27(4): 2008-2021. https://doi.org/10.1109/TIP.2017.2788866
[13] Labati, R.D., Genovese, A., Piuri, V., Scotti, F. (2013). Contactless fingerprint recognition: A neural approach for perspective and rotation effects reduction. In 2013 IEEE Symposium on Computational Intelligence in Biometrics and Identity Management (CIBIM), pp. 22-30. https://doi.org/10.1109/CIBIM.2013.6607909
[14] Liu, X., Pedersen, M., Charrier, C., Cheikh, F.A., Bours, P. (2016). An improved 3-step contactless fingerprint image enhancement approach for minutiae detection. In 2016 6th European Workshop on Visual Information Processing (EUVIP), pp. 1-6. https://doi.org/10.1109/EUVIP.2016.7764594
[15] Yin, X., Zhu, Y., Hu, J. (2019). Contactless fingerprint recognition based on global minutia topology and loose genetic algorithm. IEEE Transactions on Information Forensics and Security, 15: 28-41. https://doi.org/10.1109/TIFS.2019.2918083
[16] Tan, H., Kumar, A. (2020). Towards more accurate contactless fingerprint minutiae extraction and pose-invariant matching. IEEE Transactions on Information Forensics and Security, 15: 3924-3937. https://doi.org/10.1109/TIFS.2020.3001732
[17] Rattani, A., Kisku, D.R., Bicego, M.,Tistarelli, M. (2007). Feature level fusion of face and fingerprint biometrics. In 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems, pp. 1-6. https://doi.org/10.1109/BTAS.2007.4401919
[18] Kumar, A., Kanhangad, V., Zhang, D. (2010). A new framework for adaptive multimodal biometrics management. IEEE transactions on Information Forensics and Security, 5(1): 92-102. https://doi.org/10.1109/TIFS.2009.2031892
[19] Herbadji, A., Akhtar, Z., Siddique, K., Guermat, N., Ziet, L., Cheniti, M., Muhammad, K. (2020). Combining multiple biometric traits using asymmetric aggregation operators for improved person recognition. Symmetry, 12(3): 444. https://doi.org/10.3390/sym12030444
[20] Abderrahmane, H., Noubeil, G., Lahcene, Z., Akhtar, Z., Dasgupta, D. (2020). Weighted quasi-arithmetic mean based score level fusion for multi-biometric systems. IET Biometrics, 9(3): 91-99. https://doi.org/10.1049/iet-bmt.2018.5265
[21] Nandakumar, K., Chen, Y., Dass, S.C., Jain, A. (2007). Likelihood ratio-based biometric score fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2): 342-347. https://doi.org/10.1109/TPAMI.2007.70796
[22] Ma, Y., Cukic, B.,Singh, H. (2005). A classification approach to multi-biometric score fusion. In International Conference on Audio-and Video-Based Biometric Person Authentication Springer, Berlin, Heidelberg, pp. 484-493. https://doi.org/10.1007/11527923_50
[23] Walia, G.S., Rishi, S., Asthana, R., Kumar, A., Gupta, A. (2019). Secure multimodal biometric system based on diffused graphs and optimal score fusion. IET Biometrics, 8(4): 231-242. https://doi.org/10.1049/IET-BMT.2018.5018
[24] Peng, J., Abd El-Latif, A.A., Li, Q., Niu, X. (2014). Multimodal biometric authentication based on score level fusion of finger biometrics. Optik, 125(23): 6891-6897. https://doi.org/10.1016/J.IJLEO.2014.07.027
[25] Herbadji, A., Guermat, N., Ziet, L., Cheniti, M., Herbadji, D. (2019). Personal authentication based on wrist and palm vein images. International Journal of Biometrics, 11(4): 309-327. https://doi.org/10.1504/ijbm.2019.10023699
[26] Shariatmadar, Z.S., Faez, K. (2014). Finger-Knuckle-Print recognition performance improvement via multi-instance fusion at the score level. Optik, 125(3): 908-910. https://doi.org/10.1016/J.IJLEO.2013.04.134
[27] Hezil, N., Boukrouche, A. (2017). Multimodal biometric recognition using human ear and palmprint. IET Biometrics, 6(5): 351-359. https://doi.org/10.1049/iet-bmt.2016.0072
[28] Yang, J., Zhang, X. (2012). Feature-level fusion of fingerprint and finger-vein for personal identification. Pattern Recognition Letters, 33(5): 623-628. https://doi.org/10.1016/j.patrec.2011.11.002
[29] Kumar, A., Hanmandlu, M., Sanghvi, H., Gupta, H.M. (2010). Decision level biometric fusion using Ant Colony Optimization. In 2010 IEEE International Conference on Image Processing, pp. 3105-3108. https://doi.org/10.1109/ICIP.2010.5654019
[30] Sharifi, O., Eskandari, M. (2016). Optimal face-iris multimodal fusion scheme. Symmetry, 8(6): 48. https://doi.org/10.3390/sym8060048
[31] Mhaske, V.D., Patankar, A.J. (2013). Multimodal biometrics by integrating fingerprint and palmprint for security. In 2013 IEEE International Conference on Computational Intelligence and Computing Research, pp. 1-5. https://doi.org/10.1109/ICCIC.2013.6724125
[32] Chen, F., Huang, X., Zhou, J. (2013). Hierarchical minutiae matching for fingerprint and palmprint identification. IEEE Transactions on Image Processing, 22(12): 4964-4971. https://doi.org/10.1109/TIP.2013.2280187
[33] Chin, Y.J., Ong, T.S., Teoh, A.B.J., Goh, K.O.M. (2014). Integrated biometrics template protection technique based on fingerprint and palmprint feature-level fusion. Information Fusion, 18: 161-174. https://doi.org/10.1016/J.INFFUS.2013.09.001
[34] Bajwa, N., Kumar, G. (2015). Multimodal biometric system by feature level fusion of palmprint and fingerprint. International Journal of Engineering Research & Technology (IJERT), 4(7). https://doi.org/10.17577/ijertv4is070911
[35] Bellaaj, M., Boukhris, R., Damak, A., Sellami, D. (2016). Possibilistic modeling palmprint and fingerprint based multimodal biometric recognition system. In 2016 International Image Processing, Applications and Systems (IPAS), pp. 1-8. https://doi.org/10.1109/IPAS.2016.7880147
[36] Schölkopf, B., Smola, A.J., Bach, F. (2002). Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press. https://doi.org/10.1145/882116.882120
[37] Klement, E.P., Mesiar, R., Pap, E. (2013). Triangular norms (Vol. 8). Springer Science & Business Media.
[38] Ojansivu, V., Heikkilä, J. (2008). Blur insensitive texture classification using local phase quantization. In International conference on image and signal processing Springer, Berlin, Heidelberg, pp. 236-243. https://doi.org/10.1007/978-3-540-69905-7_27
[39] Tan, X., Triggs, B. (2010). Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE transactions on image processing, 19(6), 1635-1650. https://doi.org/10.1109/TIP.2010.2042645
[40] Kannala, J., Rahtu, E. (2012). Bsif: Binarized statistical image features. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 1363-1366.
[41] Ojala, T., Pietikainen, M., Maenpaa, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7): 971-987. https://doi.org/10.1109/TPAMI.2002.1017623
[42] Kumar, A. (2008). Incorporating cohort information for reliable palmprint authentication. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pp. 583-590. https://doi.org/10.1109/ICVGIP.2008.73
[43] Chin, Y.J., Ong, T.S., Goh, M.K., Hiew, B.Y. (2009). Integrating palmprint and fingerprint for identity verification. In 2009 Third International Conference on Network and System Security, pp. 437-442. https://doi.org/10.1109/NSS.2009.25
[44] Geetha, K., Radhakrishnan, V. (2013). Multimodal biometric system: A feature level fusion approach. International Journal of Computer Applications, 71(4). https://doi.org/10.5120/12347-8635
[45] Mansoor, A.B., Masood, H., Mumtaz, M. (2011). Personal identification using feature and score level fusion of palm-and fingerprints. Signal, Image and Video Processing, 5(4): 477. https://doi.org/10.1007/s11760-011-0251--7
