An Image Classification Algorithm of Financial Instruments Based on Convolutional Neural Network

The computing power explosion has promoted the application of computer vision in various fields [1, 2]. One of the important application fields is the identification and classification of financial instruments [3]. Traditionally, these instruments are identified and classified manually by financial personnel in the following steps: First, various types of financial instruments are classified manually, including VAT (ordinary bills, electronic bills, special bills), bank receipts, toll bills (road invoice, charges, highway tolls). Then, the basic information of these financial products is manually input to the financial software to generate the corresponding accounting slip. After that, the accounting vouchers were attached to each financial instrument in turn; Finally, the financial personnel check and confirm the classification three to four times [4, 5]. The above method is undoubtedly inefficient. Considering the diversity and sheer number of financial instruments nowadays, the manual method will incur a heavy workload, and consume too much time in image classification [6, 7].

The traditional methods for image feature extraction mainly adopt such techniques as intensity histogram, feature-based filter, scale-invariant feature transform (SIFT), local binary pattern (LBP), and support vector machine (SVM) [8]. These techniques cannot accurately recognize images with highly complex scenes, and require a long time for model learning. As a computer vision technique, convolutional neural network (CNN) has achieved a good performance in image classification, due to its ability to describe and extract features.

For many years, artificial neural networks (ANNs) have been widely used to solve image classification and complex classification problems. The CNN is one of the most successful ANNs, because it has good performance in handling many problems of image classification. The defining feature of the CNN is the use of convolutional layers, which truthfully simulates the optic nerves of human. Each node can only respond to a specific image feature, such as horizontal or vertical edges. These simple nodes are arranged layer by layer. Sufficient features can be acquired by enough layers of nodes. Mimicking the human vision mechanism, the CNN boasts a good effect on image recognition. In some innovative applications, such as traffic sign detection, CNN-based systems are better than human vision in benchmark tests [9].

When it comes to the recognition and classification of financial instruments, the CNN also has super-strong control capability of the processing accuracy [10]. Therefore, this paper designs a CNN for the classification of financial instruments. The CNN consists of components like traditional CNN, shallow convolutional layers, and cropping structure. Then, the momentum weight update was combined with weight attenuation to accelerate the model learning. In addition, a preprocessing method was developed for rapid pixel-level adjustment of financial instruments, such that the proposed CNN can classify financial instruments of various types.

The overall architecture of our CNN is shown in Figure 1. The input to the network is normalized image blocks of financial instruments with a zero mean unit variance. The first layer is a convolutional layer with 16 output channels and a kernel size of 7×7 pixels. The second layer is the 2×2 pooling layer with the maximum kernel size. The third layer is a completely connected layer with 100,50 layers, and 5 nodes, respectively. The last layer is activated by softmax function.

1.jpg

Figure 1. The overall architecture of our CNN

Compared with CNN for other applications, our CNN involves multiple convolutional layers, and receives multi-class images. Thus, the network must be trained with largescale or high-level features. For the classification of financial instruments, a network with a limited number of convolutional layers has the same performance as a deep CNN. The simple structure greatly reduces the number of training parameters, which helps avoid overfitting. Besides, many techniques that improve or accelerate the network training were implemented. The learning parameters of these techniques were set to typical values or selected empirically. Each layer of the CNN is detailed in the following subsections.

3.1 Convolution-based extraction of image features

To solve the image classification problem, the input to the standard feedforward neural network can be used directly based on image pixels. But even a small image block may contain thousands of pixels, that is, numerous connection weights need to be trained. According to the theory on multi-dimensional data processing, too many weight parameters will complicate the system. In this case, more training samples are required to avoid overfitting. By combining the weights into small kernel filters, the CNN substantially simplifies the learning model, and operates faster and better than the traditional fully-connected neural network.

3.2 Convolutional layer

In image classification, convolutional layer is an important part of CNN. The number of input and output matrices may differ. One or more two-dimensional (2D) matrices serve as inputs to the convolutional layer, and the convolutional layer outputs multiple two-dimensional matrices. The calculation of a single output matrix can be defined as:

$A_{j}=f\left(\sum_{i=1}^{N} I_{i} * K_{i, j}+B_{j}\right)$    (1)

First, the corresponding kernel matrix K_i,j is convolved with each input matrix I_i and then summed, and the nonlinear activation function f(•) is added to the deviation B_j and applied to each element of the obtained matrix to obtain the output matrix A_j, and the features extracted by the local feature extractor represented by each set of kernel matrices are extracted from the input matrix. Finally, a set of kernel matrices K, is found and used to extract good discriminative features to classify the images. The connection weights of the CNN are optimized by the inverse algorithm, and the deviations and matrices are trained into network nodes with shared connection weights.

3.3 Max pooling layer

The pooling layer is responsible for dimensionality reduction of the CNN. In order to reduce the output nodes of the convolutional layer, the pooling algorithm was adopted to combine adjacent elements in each matrix outputted by the convolutional layer. Max pooling and average pooling are common pooling algorithm. In this research, a pooling layer with 2×2 kernels were adopted to select the maximum of the four adjacent elements in the output matrix usually selects the largest of the four adjacent elements in the input matrix. The gradient signal is used only for the nodes of the pool output, if the transmission is wrong.

3.4 Rectified linear unit (ReLU) activation

Neural networks often employ nonlinear transfer functions as the activation function of nodes. The sigmoid function f(x)=1/(1+e^−x) and the hyperbolic tangent function f(x)=tanh(x) are commonly used activation functions. Both are saturated nonlinear functions: The output gradient decreases and approaches zero as the input increases. To improve learning speed and classification performance, our CNN adopts the latest activation function Mish [19]: f(x)=x*tanh(ln(1+e^x)) [20], and applies it to the convolutional layer. Experimental evidence shows that Mish activation function has a faster convergence rate and a higher classification performance of 2.5% than the sigmoid activation function.

3.5 Backpropagation

Multiple techniques were adopted to accelerate and stabilize the CNN training, such as batch processing, momentum weight update, and weight attenuation. The authors batch processed 128 input samples, and the batch method was chosen to improve learning efficiency and accuracy, rather than updating the connection weights after each backpropagation, and their connection weights were updated at once.

To accelerate learning speed, weight attenuation update was coupled with momentum weight to update weight Δω_i into:

$\Delta \omega_{i}(t+1)=\omega_{i}(t)-\mu \frac{\partial E}{\partial w_{i}}+\alpha \Delta \omega_{i}(t)-\lambda \mu \omega_{i}$  (2)

where, $\omega_{i}(t)-\mu \frac{\partial E}{\partial w_{i}}$ is the standard backpropagation term; $\omega_{i}(t)$ is the current weight vector; $\frac{\partial E}{\partial w_{i}}$ is the error gradient relative to the weight vector; μ is the learning rate; $\alpha \Delta \omega_{i}(t)$ is the momentum part; $\alpha$ is the momentum rate; $\lambda \mu \omega_{i}$ is the weight attenuation part; λ is the weight attenuation rate. In order to speed up learning, the authors made the momentum weight update, while the weight attenuation stabilizes the learning by slightly reducing the weight vector in each iteration. In our experiments, the learning rate, momentum rate, and weight attenuation rate were set to 0.001, 0.9, and 0.01, respectively, for each layer. These parameter values were determined through repeated experiments.

3.6 DropOut

The network performance was improved by randomly disabling nodes in each layer [21]. At the beginning of each training iteration, a DropOut with the same size of nodes was initialized in each layer to label the on or off state of the corresponding nodes. In the iterative training process, different models are switched in each learning iteration to train different models at the same time, and non-state nodes are removed from the network by disabling forward propagation of activation signals of nodes and wrong back propagation. During the test phase, all nodes are activated, but during the training phase, the activation signal decays to the average open rate. In our experiments, the DropOut was set to 0.6 for the training.

5.1 Experimental settings

During the experiments, SIFT [24], LBP [25], and SVM were taken as the benchmarks to accurately monitor the recognition accuracy, computing speed, and operation state of our CNN in intelligent recognition and classification of financial instruments, without any manual intervention.

Our CNN model was established under the PyTorch framework, and deployed on the server operating system CentOS Linux 7. Two Nvidia Tesla v100 graphic cards, each with32G video memory, and a central processing unit (CPU) of 2.20GHz were adopted. 

The multi-class financial instruments dataset was established through long-term collection and sorting of images on Google and Baidu, and the cooperation with financial institutions, as well as transportation and taxation departments. The 100,000 plus real instruments in the dataset were divided into 482 subclasses in 12 classes. The name and subclass number of the 12 classes are presented in Table 1. A total of 100 images were selected from each class and combined into a test set of 1,200 images, which cover financial instruments like value-added tax invoices, toll bills, quota tax invoices, train tickets, taxi invoices, and bank receipts.

Table 1. The name and subclass number of each class of financial instruments

5.2 Results analysis

In our dataset, the images on financial instruments are collected by various methods from different sources. Some images contain multiple instrument areas and multiple instrument samples, while some only contain a single instrument area with a large background area. These factors impede the classification and recognition of images.

The images containing multiple instrument areas were adopted to verify the performance of our model. As shown in Figure 3, the classification accuracy averaged at 97%, indicating that our model has a good learning ability that mines the feature difference between instruments. The results also testify the reliability and practicability of the collected images.

Further, the classification results of our method were compared with those of the other three feature extraction methods in terms of classification accuracy. According to Figure 4 that the CNN achieved the best classification performance. In support vector machines, feature learning is independent of classifier training. Compared with support vector machines, this method shows the superiority of supervised learning. In addition, compared with the currently popular feature descriptors SIFT and LBP, our CNN model has obvious advantages, which indicates the feasibility of automatic learning of financial instrument image features. The main advantages of our approach include the customization adaptability of CNN, and the fast preprocessing of instrument data, which are rare in current image calculation. Moreover, the neural network in classification model learning also contributes to the superiority of our method: the final classification model is optimized through parameter finetuning on each layer by backpropagation algorithm.

3.png

Figure 3. The classification effects on multiple instrument areas

4.jpg

Figure 4. The classification performance of different methods on financial instruments

Table 2. The training time of different methods

Table 2 records the training time of different methods. It can be seen that our method consumed 10 times shorter time in training than SVM. The training time of our method was even shorter than that of SIFT, which is known for its low complexity. The time efficiency comes from two unique properties: the simple structure of our model greatly facilitates training; the model learning is accelerated by the combination between momentum weight update and weight attenuation.