Artificial Intelligence and Machine Learning Approaches to Document Digitization in the Banking Industry: An Analysis

Over the past few decades, digital technologies have been increasingly integrated into various facets of daily life. This process, known as digitization, is transforming societies, businesses, and global economies. Identified as a pivotal technology trend, digitization exerts a profound influence on both societal and industrial sectors. Consequently, organizations are under persistent pressure to adopt digital tools and adapt their strategies to thrive in the digital age. Although the transition to digital encompasses a multitude of benefits, it also necessitates substantial investments and time commitments.

In the wake of digital technology's evolution [1], a pertinent question arises regarding the utilization of digitization by professionals in the field and the extent to which scientists and researchers are keeping pace with this development. The digitization of workflow often involves the creation of application-specific software, likely based on the current workflow, thereby necessitating the replication of document structures in their digital storage. However, digitization encompasses more than this minor component; businesses may also need to update their software or hardware, expand their workforce or provide training, or even entirely redefine their business strategy. The complexity of digitization arises from the numerous variables involved, making a robust process model an invaluable tool [1]. In the absence of such a process architecture, businesses may produce suboptimal results.

Thus, our primary objective is to delineate the current state of the art and foster an enhanced understanding of the term "digitization." Although an abundance of literature exists on digital transformation, studies specifically addressing digitization are notably sparse. It is recognized that the term "digital transformation" was initially introduced by corporate leaders before academic investigations ensued. However, there exists a significant knowledge gap at the federal level, contributing a mere 1% to global research [2].

In the 2020s, artificial intelligence (AI) and machine learning (ML) have emerged as increasingly influential technologies in document digitization. This study strives to elucidate the intellectual progression of AI and ML in financial research, employing a research review with an embedded investigation to analyze the services of these concepts.

Emphasizing the importance of investigating the potential exclusionary effects of economic digitization from both social and political perspectives, this study proposes trends in the digitization of the economy [3, 4]. It draws on research on digital inequality to analyze these developments and their potential impact on individuals. This interdisciplinary approach underscores the necessity for further research on this topic and suggests avenues for exploration from both policy and academic perspectives. Additionally, it examines document digitization technologies such as Convolutional Neural Networks, Artificial Intelligence, and Machine Learning.

This review encompasses the effects of digitization on society and the financial sector, as well as the technologies developed to facilitate digitization. The literature review is divided into two parts: one addressing digitization in the banking and financial sectors and its societal effects, and the other discussing various techniques used for digitization. This review paper is organized as follows: Section 2 provides an overview of the literature on digitization in the banking and financial sectors, as well as its societal and mobility effects. Section 3 presents a survey on different technologies such as Handwritten Recognition systems using machine learning techniques, CNN techniques, and AI and ML techniques for the digitization of various financial and company documents. Section 4 presents the conclusion of the review paper, and Section 5 discusses the limitations of the literature survey.

1.png

Figure 1. Traditional approach for document digitization

3.1 Review on OCR engines used in document digitisation

To understand how OCR might be enhanced, it is necessary to first grasp its limits. Readability is affected by the following factors:

1) Handwriting or other marks that cover the text hamper its correct recognition.

Furthermore, many components in the file, such as lines and boxes, confuse OCR because it attempts to interpret the lines as part of the text. According to Legal Scans, in this situation, the success rate of OCR lowers swiftly and equals just 60%-80% successful read.

2) The quality and condition of a paper.

The first issue can be addressed by AI Machine Learning, making OCR more advanced [29].

OCR mainly has three units: text detection, text recognition, and text labeling. In the industry many OCR tools are available. There are open-source OCR tools like Tesseract, and EasyOCR and commercial tools like ABBY, and Google vision. Open-source tools like Tesseract and EasyOCR have inbuilt text detection, or we can use algorithms like EAST and CTPN for detection using pre-trained models. As the output of the text detection step, a bounding box is created around the portion of the image having text. After text detection, we use machine learning to match those identified images with a particular text in the text recognition step. Tools like Tesseract, Calamari, and CRNN can be used for this purpose. Tesseract is the most popular since it is open source and provides decent accuracy for the document. The last step of OCR is text labeling. During this step, useful information recognized from the particular image is tagged with a particular label. This step aims to develop the linkage between the desired content and its usage. Regular expressions or NER model-based spacy tools are often used for this purpose. Once the relevant data is extracted using the NER methods, we use key-value pair and reserve it in JSON or XML format or we will directly store the relevant extracted value in the database like a particular key. Tesseract is an open-source OCR solution maintained by the community supported by Google. It returns results in plain text or PDF, with text overlaid on the original image, and provides decent accuracy. Tesseract has a neural net-based OCR engine mode that gives improved accuracy even for image documents with high noise. But with Tesseract, a lot of preprocessing is needed specifically to document noise type. ABBY comes with a GUI to support OCR extraction. It comes with ABBY cloud for cloud services. ABBY also has a desktop app that starts at approximately $200 and provides access to a software development kit. ABBY returns JSON, XML, or a pdf with a text searchable inline form. But ABBY has errors due to missing character detection and has low accuracy for handwritten text.

The Google vision OCR tool can be used via API. But its setup is quite challenging. It gives JSON as an output, which contains information about character position, and it works well with handwritten text. This tool is not a good choice when working with tabular data as it misses single digits. It also does not support document sizes greater than 10MB. It is a commercial engine that gives free access to the first 1000 pages.

Microsoft OCR provides high accuracy in text extraction, even from noisy image documents. It has an API endpoint to send the image document and return it with JSON output, which contains coordinate information along with text extracted. Microsoft OCR has errors due to missing character detection. PDF2GO is a multiplatform online tool that works with file formats like pdf, Microsoft word, txt, ePub, OpenOffice, etc. But the drawback is the limited free version, which is ad-laden. There is no option for an e-signature and watermark.

Foxit Reader is a multilingual PDF tool that allows you to create, view, edit, digitally sign and print PDF files. However, this engine contains potentially unwanted programs that behave like malware, such as Ask Toolbar and OpenCandy.

From the literature different OCRs like Google cloud vision, Microsoft Azure,ABBYY FineReader, Google Tessearct and ABBYY Flexicapture were studied and compared on different aspects. As Google cloud Vision and Microsoft Azure, uses pre trained models, they have higher image classification accuracy. ABBYY FineReader has market leading OCR capabilities, excelling at digitizing documents and extracting text from images and pdfs with high acuracy. However it lacks more advanced computer vision functionality. Google Tesseract and ABBYY FlexiCapture are outperformed by ABBYY FineReader. Tessearct is open source and focuses more on structured data extraction.

Google cloud vision delivers the highest overall accuracy above 98%. ABBYY FineReader matches their accuracy on clean documents, but lags on handwriting and poor quality scans. Tesseract and Microsoft Azure has accuracy of 90-95%.

3.2 Printed and handwritten text recognition systems by machine learning techniques

To digitize documents, particularly drawings and plans, geometrical forms predominate, making text recognition a significantly more difficult process. Because a standard OCR system cannot recognize the location of text in such images, the text is usually ignored. The objects are first identified using an AI and ML-based. The text is then explicitly searched for in frames. This workflow protects the system from overlooking critical information.

Many financial documents have handwritten text. As discussed in previous section on review of OCR engines, all OCR engines provide limited support on handwritten etxt detection and recognition. Therefore, it is necessary to employ techniques for handwritten text recognition. Handwriting recognition systems are models used to collect and analyze data in the form of handwritten text from sources such as paper documents and photographs. The success of handwriting recognition systems has largely depended on optical text recognition, which is responsible for segmenting handwritten digits, and character recognition is the core of that module [21]. Number and character recognition is used to match handwritten numbers and characters with their associated computer records. This is something that any learning paradigm will do.

Using machine learning algorithms, Shamim et al. [20] suggested Handwritten Digit Recognition. Naive Bayes, Multilayer Perceptron, J48, Random Forest, Bayes Net, Support Vector Machine, and Random Tree were among the methods used. The approach detailed in this paper employs many machine learning methods to identify printed numbers without an Internet connection. This paper's main objective is to make sure that techniques for reading handwritten numbers are accurate and effective. Bayes Net, Support Vector Machine, Random Forest, Random Tree, Multilayer Perceptron, J48, and Naive Bayes are used for identification of digits. The Multilayer Perceptron used in this study increased accuracy to a maximum of 90.37%. More accuracy needs to be achieved.

CNN-ECOC was suggested by Bora et al. [21] for Handwritten Character Recognition from Pictures. This study uses a CNN and an Error Correcting Output Code (ECOC) classifier to present OCR. The ECOC classifies the features once they have been isolated using the CNN. The NIST manually written character picture dataset was utilized for preparing and approving the CNN-ECOC. According to this article, CNN-ECOC provides greater precision than the regular CNN classifier. The fact that the ECOC classifier places a higher computing burden on the system and requires more time to train is one of its drawbacks.

Hybrid evolutionary technique for Convolutional Neural Network-based Devanagari handwritten numeral identification was proposed by Trivedi et al. [18]. They employed the Sparse Autoencoder, CNN, Softmax Classifier, and Genetic Algorithm to create the model. A hybrid deep learning method utilizing the L-BFGS system and the Genetic Algorithm has been created for CNN training. The model has been tested using the Devanagari handwritten numeral dataset. This study's conclusion is that CNN training should be improved by utilizing evolutionary approaches. There will be some issues with chromosomal length as the number of iterations rises. Not all layers of the system are applied with genetic algorithms.

Manchala et al. [22] presented deep learning with TensorFlow for handwritten text recognition. Connectionist temporal classification (CTC), Recurrent Neural Network (RNN), and Convolutional Neural Network (CNN) are the strategies used in this paper to train recurrent neural networks. TensorFlow is used to implement the model. This approach achieves an accuracy of 90.3%. Only the text in this project that has the least amount of noise is given the maximum level of accuracy. The dataset determines the correctness in its entirety. The model's accuracy increases as the amount of data increases. It does not provide the highest level of precision for cursive letters.

Choudhary et al. [19] proposed an offline handwritten character recognition using features extracted from binarization technique. The main purpose of this study is to remove features learned by binarization to recognize handwritten English characters. A multi-layer feedforward neural network was used as a classifier to recognize handwritten character images. Before the characters are labelled, the images are preprocessed by doing things like thinning, reducing foreground and background noise, cropping, and normalizing size. However, compared to previous research, the precision attained by this study is relatively low.

Hallale and Salunke [24] presented Twelve Directional Feature Extraction for Handwritten English Character Recognition. This paper employs twelve directional properties to distinguish handwritten English alphabets and digits. The features of similarity measurements are studied using directed pattern matching. The detection rates of conventional and twelve directional feature extraction approaches are then compared. It is clear from the results of explore that coordinated capability extraction calculations beat more customary strategies. However, the project's precision is just 88.29%. Accuracy must be enhanced further.

3.3 Document structure recognition

3.3.1 Importance

The analysis of titles, headers, sections, and thematically cohesive pieces is the aim of document logical structure recognition. There is yet another significant benefit that an AI and ML-based OCR system offers over a traditional one, which is significant because:

• It is essential for extracting useful information;
• It permits automated indexing and storage, minimizing the need for extra information retrieval;
• It makes it possible to link related documents (for example, those describing the same object).
• Document structure recognition improves file handling flexibility. If the structure is known, you may simply locate the document as well as others that are related to it.

3.3.2 Realization

Recognizing the document structure may be done in two ways:

(1) Layout analysis

Entry forms, invoices, and other papers are all generated using the same structure and standards and have a comparable format. This frequently provides numerous hints regarding the relationships between various structural units, including headings, body text, tables, references, figures, etc.

(2) Analyzing the content itself

Use keywords to identify the relationships and meanings of text.

Why Using Algorithms is not an option

To help the programme detect certain sections of a document, several software service providers offer to include bespoke algorithms.

However, this strategy isn't flexible. In some projects, it may be necessary to manage a wide range of distinct document layouts. Also, with time, even for the same document type, layouts alter and the distinction between versions becomes increasingly obvious. Thus, using algorithms is ineffective, and a more adaptable tool is needed [30].

It takes a lot of effort, money, and time to refine algorithms so they can adapt to freshly designed document layouts. With its capacity for self-improvement, machine learning is able to adapt to changes swiftly. Document structure identification may be seen as an object detection problem that AI and ML can tackle.

3.4 Deep learning algorithms for feature extraction and document classification

Accuracy of traditional methods depend on the image preprocessing. Deep learning methods depends on basic Convolutional neural networks. For feature extraction, deep learning methods like VGG-16 and ResNet has proved to be very effective [31].

2.png

Figure 2. Deep neural network-based approach

VGG-16 (Visual Geometry Group):  Input to this network can be color image. This image is  passed through stack of convolutional layers. Every convolutional layer in this stack uses a kernel which is very small receptive field and it is of the size 3×3 with stride 1. This network has thirteen convolutional layers, five max pool layers, three fully connected layers. The activation at the last layer is softmax. One thousand channels at the last layer indicates one channel per class. As the depth of the deep neural net goes on increasing, the number of parameters to be trained also increases.

ResNet (Residual Network): This network addresses the problem of vanishing gradient. The core idea in ResNet is, it introduces skip connection or identity shortcut. It skips one or more layers and feeds the input indirectly by passing one or more intermediate layers. This bypass path is an additional path along with regular path. Figure 2 shows the Deep neural net-based approach.

From above discussion, we can conclude that the OCR accuracy on modern documents is very high upto 99% and it drops significantly for older, degraded documents. For improving accuracy of such documents more computational power is needed. OCR processing time mainly depends on image resolution, color mode, document condition and OCR engine design. Grayscale images and cleaner images process faster compared to colored images. Neural networks in OCR engines are very slow but accurate. Post processing like layout analysis and text formatting adds to computational burden. Cloud based OCR services can scale computationally but have latency trade offs [32].

3.5 Comparison of different datasets used in document digitization

The main activities in the bank are cheque clearance, deposits and withdrawals, loan approvals and death claim clearance, and insurance clearance. For automating cheque clearance, different features on the cheque need to be extracted. These features include extraction of bank logo, date, account number, IFSC code, amount, and most importantly verification of signatures on the cheque.

The success of any machine learning algorithm depends on the selection of standard datasets that are properly benchmarked and evaluated. The complete automation of the banking domain has many aspects like automated cheque clearance, data extraction from KYC documents, loan applications, account opening, and closing forms, etc. There are a lot of complex workflows involving document processing, verification, and authorization, especially in the case of customer onboarding, which is tightly coupled with regulatory requirements such as KYC. With OCR, this workflow can be expedited, and inconsistencies in data can be identified at an early stage. Any CTS (Cheque Truncation System) compliant cheque has mandatory security features like a bank logo in UV ink, void Pantograph, Micro lettering, Date, Rupee Symbol, Signatures, and re, Account number. For the accurate identification of these security features, different datasets are needed. Prateek [33], MNIST dataset of handwritten digits used to identify the legal amount that is digits. This dataset has more than sixty thousand examples to identify handwritten characters, the EMNIST dataset can be used. This dataset has more than 88800 samples. These two datasets contain balanced images. This dataset is normalized. The MNIST dataset has ten classes. The cheque images can be obtained from the IDRBT bank cheque dataset, which contains 112 cheque images.

Every dataset has certain challenges and limitations. RVL-CDIP has limited diversity of documents, also it has class imbalance as certain class have more sample data compared to other classes. The tables, figures and headings are not annoted in RVL-CDIP and the contextual data like matadata, source authorship and relationship between documents is not available for entire dataset. In MNIST dataset images are grayscaleand contains basic handwritten digits. EMNIST dataset mainly has english and latin handwritten characters. This dataset is not suitable for non english and non latin documents. It has limited variations in writing style, shapes and sizes of characters. It does not capture full range of variations found in real world handwriting This impacts models generalization capabilities. The data available for training is very limited. This dataset has alphabets written as isolated characters. But handwritten characters are often written in the context of words, sentences or paragraphs. Bank Search dataset has loan documents which has unstrcutured data and contains text heavy documents. These documents are prone to errors, inconsistencies and missing information. Therefore, data cleaning and preprocessing is necessary. Loan documents can vary in format, structure and loan types. These documents lack contextual information about borrower’s credit history, or the purpose of the loan. Turkish bank dataset does not include all types of loan documents. This dataset lacks diversity in terms of loan purposes, borrower profiles or loan terms. This dataset has inconsistencies due to missing values and errors. Therefore extensive data cleaning and preprocessing steps are necessary. As loan documents contain complex legal and financial terminology, understanding and interpreting loan documents can be complex. This dataset lacks temporal information like when the loans were issued or when the repayments occurred. NIBSS is a financial dataset related to Nigeria’s banking industry. It mainly consists of inter bank transactions, therefore has limited diversity in the dataset. This dataset has limited contexual information. The FUNSD dataset contains scanned copies of invoices, resumes and application forms. It does not cover all varieties of real world documents. As this dataset is very small , therefore it has limited generalisation capability. This dataset provides annotations at the word and character level, but it has annotations inconsistency. NIST- SPDB6 has limted diversity in the dataset with varying image quality. Tobacco 3482 is also a small dataset which has class imbalance. SROIE dataset contains images which needs preprocessing to improve quality of the images. The images are complex due to logos, stamps or graphics in the background.

Table 4 shows a comparison of different datasets used for evaluating the performance of OCR in the banking domain. Engin et al. [34] developed a multimodal deep neural network for classifying Turkish Banking Customer Order Documents. They have created a dataset with approximately twenty-seven thousand documents of forty-five different classes. The main challenge while creating this dataset was wrongly labeled documents by the back office, also many documents were visually similar. The presence of support documents like identity proofs of customers, driving licenses, property papers, and blank documents also creates problems in the development of clean datasets.

Campos et al. [7] present a dataset of Portugal’s public sector banks. This dataset contains information about stocks, loans, and, net transactions. This dataset has details like the type of banks whether they are domestic or international banks ownership details, country, balance sheet structure, liquidity, profitability, and size. Sinka and Corne [35] present a method of developing a dataset. The first step in this is to select the web documents that have some content in them. To save time, the Open Directory Project and the Yahoo! Directory are used to provide web documents that are manually categorized. This dataset contains almost one thousand documents in each of the eleven categories.

This research is about document processing in the banking and financial sector. The success of document digitization and automation lies in the quality of the dataset. For the preparation of high-quality datasets, only those documents that are easily categorized by OCR are chosen.

Oral et al. [36] used a dataset of 3500 Turkish bank documents. These documents are extracted from email, fax, and scanner. To get different layouts, these documents are collected from different customers. It is also concluded that as the dataset increase, the performance also increases. There are datasets available for different domains in banking. Malkadi et al. [30] have developed a dataset for electronic banking in the Central bank of Nigeria. The data is present in the form of tables and graphs. This data is used for reviewing the performance of e-payment channels.

The FUNSD dataset is the dataset used for text detection in scanned documents, OCR, Layout Analysis, and form understanding. This dataset consists of 199 documents, all of which are in image format. SROIE (Scanned Receipt OCR and Information Extraction) consists of around 1000 images of scanned receipts. RVL-CDIP is a dataset of letters, forms, emails, resumes, memos, etc. in scanned format. There are sixteen different classes in the RVL-CDIP dataset. But the images are low quality, noisy and low resolution.

BCSD is bank filled cheque dataset. The data is in image Jpeg format. A total of one hundred and fifty-eight cheque images of the same bank are present in the dataset.

For loan applications as well as death claim clearance, certain supporting documents are required. For example, aadhar card or PAN card copies, driving license, etc. Kaggle provides an image dataset for aadhar card images and synthetic PAN card data [36].

3.6 Performance metrics for evaluation of OCR

Performance metrics used for evaluating OCR performance are mainly Precision, Recall and F1-score. Many text classification algorithms use precision and recall as performance metrics. But these metrics mainly depend on true positives. In document classification, the count of the negative classes is very high due to many unrelated documents. Precision and recall fail to detect changes in the true negative.

Confusion Matrix: To measure classification quality by measuring the count of how correctly and incorrectly the text is classified. Consider input text has Cj number of classes.

X={C1, C2….Cj}

Correct classification of positive examples is used to determine precision, recall, F1 score, and Break-Even point.

These measures consider only the classification of positive examples but do not consider the classification of negative examples.

Accuracy: It can be calculated from the confusion matrix as the ratio of the sum of true positive and true negative with the sum of true positive, true negative, false positive, and false negative.

Recall: It is the ratio of the specific true positive document to all documents that are true. Accuracy and recall measure the effectiveness of class identification by a classifier. It is the extent of the error caused by false negatives.

Precision: It is the ratio of the true positive document to the sum of true positive and true negative documents. It is the extent of error that is caused by false positives.

F1 score: If precision and recall values are given for different models, the F1 score is used to decide which model is best. It works well for imbalanced datasets. It is the harmonic mean of precision and recall.

$F1score=2\times \frac{\text{ PrecisionxRecall }}{\text{ Precision }+\text{ Recall }}$                 (1)

Character Recognition Accuracy: This is nothing but the number of characters correctly recognised by the OCR system.

Word Recognition Accuracy: It gives number of words that are correctly recognised by OCR.

Recognition Time: This is the most important measure for comparing the time taken by OCR to process the document and recognise the text in the document.

Language Support: This metric measures versatility of the OCR engines. Different OCR engines support different multiple languages.

Font and Style Compatibility: Different documents have different fonts and styles which impact overall OCR accuracy. Therefore, OCR performance is tested on different fonts and formats for consistency.

Document Layout Acccuracy: A good OCR recognises the texts as well as different elements in the document like headers, footers, tables and images.