Integration of Two-Dimensional Kernel Principal Component Analysis Plus Two-Dimensional Linear Discriminant Analysis with Convolutional Neural Network for Finger Vein Recognition

This paper combines K2DPCA+2DLDA and CNN into a novel way to recognize finger veins. Firstly, nonlinear dimensionality reduction and feature extraction were carried out with K2DPCA+2DLDA to screen out the massive redundant information in the original image. Next, the CNN was introduced to extract and learn the higher-level semantic features (Figure 1).

For an input image I of the size r×c, our method involves the following steps:

Stage 1:

Input: Finger vein region-of-interest (ROI) image $A^{m \times n}$.

Output: Finger vein feature data $C^{d \times r}$.

Step 1. Perform K2DPCA transform of the input image $A^{m \times n}$ in the row direction to obtain the projection of the mapping $\Phi(A)$ of image $\mathrm{A}$ in feature space $\mathrm{F}$ on eigenvector, producing the corresponding feature matrix $Y^{m \times r}$.

Step 2. Perform 2 DLDA transform of the feature matrix in the column direction to obtain the transform matrix $V$, and transpose $V$ into $V^{T}$.

Step 3. Reduce the dimensionality of the data again with the transposed matrix $V^{T}$, yielding the final feature $C^{d \times r}=$ $V^{T} Y^{m \times r}=V^{T} .$

Stage 2:

Import the dimensionality reduced training set obtained in the above stage into the designed CNN, and train the network through forward propagation and error backpropagation. After the training, import the training set into the network, and obtain the recognition result through forward propagation.

1.png

Figure 1. Principle of K2DPCA+2DLDA combined with CNN

2.1 K2DPCA

The high-dimensional computation of PCA might result in the curse of dimensionality, while 2DPCA lacks the ability to extract nonlinear features. In this paper, these two problems are overcome by K2DPCA [31].

As a nonlinear feature extractor, K2DPCA nonlinearly maps each row of the input image matrix to a high-dimensional space, and performs the PCA transform in that space. However, it is very difficult to solve the eigenvalues and eigenvectors of the scatter matrix corresponding to the data mapped to the high-dimensional space. To avoid complex direct calculation, this paper overcomes the difficulty by solving the eigenvalues and eigenvectors of kernel matrix. The specific steps are as follows:

Let m be the total number of training samples; X_k be the k-th training sample; X_kⁱ be the i-th row vector of the k-th training sample; X=[X₁,X₂,……,X_m], X_i=[(X_i¹)^T,(X_i²)^T,……,(X_i^m)^T]^T; Ф is the corresponding nonlinear mapping.

By inner product kernel function K, the inner product of the projections of input data X_i and X_j in higher-dimensional space F can be calculated by:

$K\left(X_{i}, X_{j}\right)=\Phi\left(X_{i}\right) \cdot \Phi\left(X_{j}\right)=\Phi\left(X_{i}\right) \Phi\left(X_{j}\right)^{T}$                  (1)

The mapping $\widehat{\Phi}$ can be established through the centralization of all data. Then, the covariance matrix $C^{\Phi}$ of space $\mathrm{F}$ can be expressed as:

  $C^{\Phi}=\frac{1}{m} \sum_{i=1}^{m} \widehat{\Phi}\left(X_{i}\right) \widehat{\Phi}\left(X_{j}\right)^{T}$                (2)

where, $\widehat{\Phi}\left(X_{i}\right)=\left[\widehat{\Phi}\left(X_{i}^{1}\right), \widehat{\Phi}\left(X_{i}^{2}\right), \ldots \ldots, \widehat{\Phi}\left(X_{i}^{m}\right)\right]$. By solving the eigenvalues and eigenvectors of the covariance matrix, there is no need to conduct complex computation with every column vector.

$\operatorname{Each} \widehat{\Phi}\left(X_{i}^{j}\right)$ can be projected to the eigenvector x_k of space F:

$x_{k} \widehat{\Phi}\left(X_{i}^{j}\right)^{T}=\sum_{p=1}^{m} \sum_{q=1}^{n} \alpha_{l}^{p \times q}\left(\widehat{\Phi}\left(X_{p}^{q}\right) \widehat{\Phi}\left(X_{i}^{j}\right)^{T}\right)$                  (3)

$=\sum_{p=1}^{m} \sum_{q=1}^{n} \alpha_{l}^{p \times q} \widehat{K}\left(X_{p}^{q}, X_{i}^{j}\right)$            (4)

where, l=m×n-d+1, m×n-d+1,……, m×n.

Then, the projection Y_i of the i-th mapping image $\widehat{\Phi}\left(X_{i}\right)$ can be obtained as:

$Y_{i}=x_{k} \widehat{\Phi}\left(X_{i}\right)=\alpha^{T}\left(\Psi^{\Phi}\right)^{T} \widehat{\Phi}\left(X_{i}\right)$                 (5)

where,

$\alpha=\left(\alpha_{m \times n-d+1}, \alpha_{m \times n-d+2}, \ldots, \alpha_{m \times n}\right)$              (6)

$\Psi^{\Phi}=\left[\left[\widehat{\Phi}\left(X_{1}^{1}\right), \ldots, \widehat{\Phi}\left(X_{1}^{n}\right)\right], \ldots,\left[\widehat{\Phi}\left(X_{m}^{1}\right), \ldots, \widehat{\Phi}\left(X_{m}^{n}\right)\right]\right]$             (7)

The row vectors of all training images are projected to the first d eigenvectors in the feature space. Then, the features in the row direction can be extracted from the projection matrix of each image obtained through K2DPCA transform.

2.2 2DLDA

To extract features in the column direction, this paper chooses 2DLDA proposed by Ming Li et al., which follows similar ideas as 2DPCA. Like 2DPCA, 2DLDA directly reduces the dimensionality on the image matrix, and thus avoids the heavy load of vector computation, without sacrificing the dimensionality reduction performance. The steps of 2DLDA are as follows:

Suppose the training set contains M images in c classes. Let A_i,j be an m×n matrix of the j-th image in class i. Let $\bar{A}$ be the mean matrix of all training samples; $\bar{A}_{i}$ be the mean of class i images. Then, the inter-class scatter matrix $S_{b}^{r o w}$ and the intra-class scatter matrix $S_{w}^{\text {row }}$ can be respectively written as:

$S_{b}^{\text {row }}=\frac{1}{M} \sum_{i=1}^{C}\left(\overline{A_{i}}-\bar{A}\right)^{T}\left(\overline{A_{i}}-\bar{A}\right)$          (8)

$S_{w}^{r o w}=\frac{1}{M} \sum_{i=1}^{C} \sum_{j=1}^{n}\left(A_{i, j}-\bar{A}_{i}\right)^{T}\left(A_{i, j}-\bar{A}_{i}\right)$              (9)

Sorting the eigenvalues of $\left(S_{w}^{r o w}\right)^{-1} S_{b}^{r o w}$ in descending order, the eigenvectors corresponding to the top-q eigenvalues can be combined into a mapping axis:

$V=\left(v_{1}, v_{2}, \ldots, v_{q}\right)$                   (10)

Finally, the matrix is mapped to axis V to obtain the eigenmatrix of each training image, and maximize the total scatter matrix. After 2DLDA dimensionality reduction, the eigenmatrix C of Y can be obtained as:

$C=V^{T} Y$                  (11)

K2DPCA and 2DLDA have a common defect: they are only capable of eliminating the correlation in one direction via feature compression. The features can only be extracted in the row or column direction in the 2D image. It is impossible to eliminate the correlations in both column and row directions. To overcome the limitation, this paper couples K2DPCA with 2DLDA. Firstly, K2DPCA transform was applied to the image matrix in the row direction to eliminate row correlation; Next, 2DLDA transform was implemented in the column direction to eliminate column correlation. In this way, a dimensionally reduced compressed image could be obtained without row or column correlation. Figure 2 shows the process of K2DPCA+2DLDA method. In addition, supervised learning was introduced to classify the information, which simplifies the computation, improves recognition accuracy, increases compression rate, and saves storage space. As a result, the proposed algorithm K2DPCA+2DLDA is expected to have a good performance.

2.png

Figure 2. Process of K2DPCA+2DLDA transform

2.3 DL

DL, an extension of traditional machine learning, has been widely applied in many fields, because it can automatically learn the suitable representation of features. During image recognition, general DL algorithms would lose the structural information of the original image, exerting a negative impact on recognition effect. As a DL network, CNN performs convolution with the aid of local receptive fields, which preserves the structural relationship of the original signal space. Besides, weights are shared to reduce the parameters to be trained. Hence, the CNN has been extensively implemented in image recognition.

CNN is a deep feed-forward neural network, whose structure is easy to set up and train. Compared with the earlier fully-connected neural network, the CNN boasts excellent generalization and robustness. The efficient network architecture attracts wide attention from computer vision teams. The classic structure of the CNN encompasses convolutional layers and a fully-connected layer. Normally, a convolutional layer involves convolution, nonlinear activation, and pooling operations. During the learning, the training set is inputted to the convolutional layers in the front of the network for learning and training, while the overall feature is integrated in the fully-connected layer in the rear. On this basis, more abstract higher-level semantics are learned through training. The defining feature of CNN structure is the adoption of three key processes: local receptive field processing, weight sharing mechanism, and pooling technology. These processes not only reduce the computing load, but also enhance network robustness and generalization.

(1) Convolutional layer

Different features are extracted from the input image via the local receptive fields, using convolution kernels. The kernels of any dimension need to completely traverse the input, and share weights and biases through the weight sharing mechanism. In this way, the kernels of any dimension can extract the corresponding abstract information. Drawing on the principle of local image perception, the computation can be speed up by defining different strides for local receptive fields, and the bias sharing can reduce the target parameters in the network. In this paper, the kernel size 1×1 is applied before the kernel size 3×3 (Figure 3). Instead of changing height or width, the 1×1 kernel increases or reduces the original data volume by changing the number of channels. Under the premise of scale invariance, the kernel can substantially enhance nonlinear features, and deepen the network, thereby improving the expression ability and increasing the number of parameters. Meanwhile, 3×3 kernels promote information exchange between channels, making finger vein recognition more accurate.

3.png

Figure 3. Convolution operation

(2) Nonlinear layer

The nonlinear layer generally receives the results from the previous layer, and then performs nonlinearization on the results. This layer introduces more nonlinearity to the model, and enables the model to better fit nonlinear targets, making up for the defect of linear models. There are many commonly used nonlinear activation functions, including sigmoid, rectified linear unit (ReLU), and Leaky ReLU. This paper chooses ReLU as the activation function. Unlike sigmoid, ReLU has a relatively small computing load, and rarely encounter vanishing gradients. It is a desirable tool for deep network training. ReLU makes some nodes to output zero, which ensures network sparsity, reduces parameter interdependence, and mitigates overfitting (Figure 4).

4.png

Figure 4. Nonlinear activation function

(3) Pooling layer

The pooling layer essentially downs samples the output of the previous layer, aiming to preserve feature description ability while reducing image resolution. This layer often comes right after a convolutional layer to simplify the convolutional output. There are two types of operations of the pooling layer: average pooling and max pooling. Finger vein recognition mainly targets images, which only contain a few useful information in the convolution process. Most information of images is redundant. Max pooling (Figure 5) helps to prevent the interference of lots of redundant information. In addition, average pooling can reduce the first type of error (increase of estimation variance caused by the limited neighborhood size), and preserve the most of background information; max pooling can reduce the second type of error (shift of mean estimation induced by the error of convolutional parameters), and preserve texture information. This paper adopts max pooling, because finger vein recognition needs to preserve much of the texture information.

As shown in Table 1, our CNN has four learning layers, including 2 convolutional (conv) layers and 2 fully-connected (fc) layers. During the training, the probability of overfitting was reduced through dropout regularization, the training speed was increased with ReLU, and the recognition accuracy was enhanced through pooling and normalization.

5.png

Figure 5. Max pooling with stride of 1 and operator of 2×2

Table 1. CNN parameters

The original finger vein image has several defects, namely, redundant information, high-dimensional data, and lack of information representation. These defects can be effectively solved by the proposed K2DPCA+2DLDA, coupled with CNN.

Considering the characteristics of finger veins, this paper integrates K2DPCA+2DLDA with CNN to process finger vein images, and verifies the proposed approach through experiments on FV-USM dataset. The following conclusions were draw:

(1) Concerning the finger vein image, the K2DPCA+2DLDA and supervised learning can reduce dimensionality through training more accurately, extract nonlinear features more effectively, compress the image greatly, and remove redundant information from the image. In this way, the data volume is reduced, the data representation is improved, and the data are depicted accurately.

(2) The CNN was introduced to train and learn the finger vein images after dimensionality reduction, and to extract higher-level features. After that, the recognition rate could reach as high as 97.3%.

(3) The proposed K2DPCA+2DLDA+CNN is much more accurate in recognizing finger vein images than (2D)2FPCA.