A Hybrid System for Handwritten Character Recognition with High Robustness

The optical character recognition (OCR) is one of the complex analysis in the field of pattern recognition, and has created a huge impact in the digital world. A prominent recognition system must demonstrate high accuracy and must be feasible in nature in order to meet the current requirement of the industry. The automation of a recognition model of handwritten scripts has a significant impact on various sectors, such as the insurance industry, the operation related to banking, postal services, etc., in the digital world. The recognition system is required not only for handwritten texts, but also for printed scripts [1]. Initially, OCR has been used for various scripts of languages, such as English, Arabic, Japanese, Chinese, etc., across the world and then it has been extended to Indian regional scripts as well [2]. In an ideal OCR system, the extraction of proper discriminant features from the characters, symbols, and words is considered the most important stage. Similarly, the selection of a classifier is considered another important aspect, which exhibits a significant impact on recognition accuracy. In the past few decades, researchers have focused on Indian regional scripts and found it more challenging. However, Indian scripts are categorized in terms of state wise distribution; among which, Odia regional language is one of the ancient languages listed in the Devanagari script. This regional language is widely used in the eastern zone of India. The recognition of handwritten characters of Odia scripts is very much interesting. Although, a considerable amount of work has been devoted for automatic classification for document analysis [3-7]. However, the earlier studied not that focused on optimize feature selection and good recognition rate is still far away. Therefore, there exists an enough space for optimal feature selection in order to get higher recognition accuracy.

In this proposed model an automated classification model for handwritten Odia characters, numerals and Bangla numerals has been studied. However, most of the Odiya character appear same in orientation, structure, and shape. Moreover, the handwritten format is considered a quite difficult task as various writers write in various ways, so it is quite difficult to predict the correct class level. Generally, all OCR models depend on two basic factors, such as feature selection and classifier used in a recognition model. Fundamentally, there are 49 numbers of characters (consonant and vowels) and 10 numbers of numerals in both Odia scripts and Bangla scripts. In addition, there are also some special characters, known as conjunct characters, which are obtained by combining one or more consonants. One advantage in this script is that it has no concepts of uppercase and lowercase systems [4]. In order to predict the correct level in the proposed model the statistical analysis of a character image set has been successful studied. After successful analysis of above mentioned issue, the proposed system have addresses an automated recognition model by selecting optimal feature section. The major contributions of this paper can be outlined as follows:

• The invariant features of the processed image using fast discrete curvelet transformation (FDCT) have been performed to select the approximation feature vector from the third level decomposition.
• The effectiveness of multi-variant feature selection in terms of principal component analysis (PCA) has been studied for achieving reduced feature set.
• The outcome of PCA is process harnessed through Linear Discriminant Analysis (LDA) to list the different ratio in terms of the number of feature vectors and treated them as key feature values.
• To analyze the effectiveness of proposed model, we consider three standard multi-script handwritten character and numerals of both Odia and Bangala script. Furthermore, the comparison study has been performed with recent existing approaches and the results reveal the potential of the suggested model.

This paper is organized as follows. Section 2 gives the description about state-of-art related work. Section 3 discusses the details of the proposed methodology adopted in current research study. The evaluation results of the standard datasets and their comparisons with the existing schemes are presented in Section 4 and 5, respectively. The concluding remarks and future research directions are presented in Section 6.

1.png

Figure 1. Proposed CR model for handwritten character and numeral

The proposed model for handwritten character recognition is illustrated in Figure 1. The proposed model aims to recognize both Odia and Bangala numerals by incorporating two vital stages such as pre-processing (Gaussian filter, morphological operation) and feature selection followed by classification using standard databases of handwritten characters and numerals are collected from various research institutions. The detailed descriptions of each stage are discussed in subsequent paragraph.

3.1 Materials preparation

In the experimental setup, the authors have used the standard database of handwritten Odia character that is taken from NIT, Rourkela [34]. However, for numerals, the authors have also used the database from IIT, BBSR [15] and NIT, Rourkela [35]. The numeral dataset consists of 10 numerals starting from 0 to 9 and the character set consists of 47 characters as depicted in Figures 2-4, respectively.

2.jpg

Figure 2. Sample of handwritten Odia numeral database from IIT, BBSR

3.jpg

Figure 3. Sample of handwritten Bangla numeral database from NIT, RKL

4.jpg

Figure 4. Sample of handwritten Odia character database from NIT, RKL

In the above discussed databases, more than 500 samples are listed for the 10 numeral categories and 350 samples are listed for the 47 characters, which were collected from various individuals. Hence, the total input size of handwritten numerals and characters was 500×10 (5000) and 350×47 (16,450) images, respectively.

3.2 Pre-processing

In the OCR system, preprocessing is considered as a crucial step in proposed model, as this state makes the image free from noise and other unwanted features and also improves the quality of the object image. In order to obtain noise-less images, the Gaussian filter selected imperially. In order to report a high percentage of the accuracy for the handwritten recognition model, the authors have also performed dilation morphological operation over the handwritten image to detect the exact location of edges and the boundary line in an Odia character and numerals.

3.3 Adopted feature selection procedure

The process of handwritten character recognition involves a well-defined approach for the segmentation and feature extraction. As this process is performed in the Odia character and numerals, it is quite interesting to report the uniqueness of each image. As there are few variations in numbers as compared to characters, there is always ambiguity in the classifier. Different handwritten patterns used by different writers make the recognition problem more complex. In order to find a feasible solution to such problems, the authors must have applied a significant feature section method for each individual character as well as a classic one. This calculates the minimum key values or characteristics, which makes it different from the others. Here, in the proposed model of the handwritten recognition system, the authors have suggested a classical technique named as curvelet analysis [36]. For the proposed experimental setting, selected fast discrete curvelet transformation (FDCT) [36, 37] as our feature extraction technique. Initially, the Fourier transformation was applied, and then the wavelet transformation was used. The wavelet transformation [38] is considered very efficient in representing 1D object, but it is quite challenging in representing 2D and multi-scale objects such as curve and lines. The basic difference between the curvelet and the wavelet transformation lies in the orientation degree of localization. Mostly, these are outcome based and concentrate on attributes such as resolution and directional selectivity, whereas localization outcomes are affected by the curvelet transformation [39, 40]. This transformation is considered an advanced version of the ridge let transform that is used for representing the curve singularity. The first implantation over the sub- band images led to the first-generation curvelet and latter, the second-generation wavelet was adopted [36, 39]. Generally, it is a block-based approach and very powerful in representing the curve, edges, and texture of the image dataset. From the literature analysis, the authors have observed that the curvelet is quite significant in representing the edges and texture and represents the inner cross-product of a signal f, which can be expressed as follow:

$(i, l, k)=\left\langle f, \lambda_{i,,}\right\rangle$        (1)

In the above equation, λ i, l, k is the generalized curvelet function that holds the respective scale (i), orientation (l) and, position (k) values. In the continuous form, the function is represented by 's' along with θ polar coordinate values and the spatial and frequency component χ and ω. Subsequently, in this proposed model can apply the continuous 2D curvelet transform with the frequency domain containing spatial variable 'z', where the respective images can be decomposed at a certain variation of scales along with the variation of orientations:

$U_{i}(s, \theta)=2^{-3 i / 4}\left(2^{-i} s\right) V\left(2^{|i / 2|} \theta / 2 \pi\right)$       (2)

In each case i>i₀, and Ui represents the respective Fourier transform as expressed in the above equation. In addition, W(s) and V(t) are defined as the radial and the other as the angular windows. While evaluating the transformation, certain basics functions are used such as:

$\sum_{t=-\infty}^{\infty} W^{2}\left(2^{i} s\right)=1, s \in(3 / 4,3 / 2)$       (3)

$\sum_{l=-\infty}^{\infty} U^{2}(t-1), t \in(-1 / 2,1 / 2)$         (4)

In this segment, we maintained the window in a smooth, nonnegative condition to a real value based on sÎ(1/2,2) and tÎ(-1,1). The above discussed transformation performed at a certain shifting orientation value of scaling, which is 2^j of scaling and it must produce a function of q=(q1,q2).

$\lambda_{i, l, k}(q)=\lambda_{i}\left(R_{\theta_{l}}\left(q-q_{k}^{i, l}\right)\right.$       (5)

In the above equation, $\mathrm{R}_{\theta}$ represents the orthogonal rotation at an angle $\theta$. Here, the rate of orientation is expressed as $\theta_{1}=2 \pi .2^{-[\mathrm{i} / 2]}$, l, l=0,1,2, …, k=(k₁,k₂)$\in$z², which indicates the sequence of shift parameters. In addition, the context of the position $\mathrm{q}^{(\mathrm{i}, \mathrm{l})}=\mathrm{R}_{\theta \mathrm{l}}^{-1}\left(\mathrm{k}_{1} \cdot 2^{-\mathrm{i}}, \mathrm{k}_{2} \cdot 2^{-\mathrm{i} / 2}\right)$ and the continuous forms of the curvelet are expressed as follows:

$C_{(i, l, k)}=\frac{1}{(2 \pi)^{2}} \oint \hat{\mathrm{f}}(\omega) \overline{\lambda_{1, j, \mathrm{k}}} \mathrm{dw}$      (6)

From the above-mentioned equation, we can easily retrieve the coefficient of the transformation over the frequency plane as follows:

$C_{(i, l, k)}=\frac{1}{(2 \pi)^{2}} \oint \hat{\mathrm{f}}(\omega) \overline{\lambda_{1, \mathrm{~J}, \mathrm{k}}} \mathrm{dw}$      (7)

Subsequently, we also employed the discrete curvelet transform as $C_{(i, l, k)}^{d}$, which is considered for a Cartesian array of any input images as expressed below:

$C_{(i, l, k)}^{d}=\sum_{0 \leq x_{1}, y_{1}<n} f\left[x_{1}, y_{1}\right] \overline{\lambda_{l, l, k}^{d}\left[x_{1}, y_{1}\right]}$    (8)

This curvelet-based approach exhibits variation in scaling along with the different orientation of the decomposition level of the frequency plane. This curvelet-based approach is generally represented in the wedges at certain scales and orientation, as shown in Figure 5 where the shaded region is the wedge one. In the proposed model, the authors have evaluated the second- order discrete curvelet, known as fast discrete curvelet transform (FDCT), up to the 3rd level of decomposition. In most cases the authors have apply two conditions for evaluating the FDCT: one is unequal space in Fourier transform (USFFT) [34] and the other is the wrapping method [41]. In order to report the desired feature vector, the authors have applied the 2d- FDCT method in both the datasets of handwritten numerals and characters, which are approximation feature values of the third level decomposition. In addition, the proposed model has applied the wrapping method owing to its robust nature and least redundancy. Here, each image of the character and numeral has been processed by the md number of the highest coefficients of FDCT for each scale and every orientation. The authors have considered N r and N c for rows and column respectively along with S c that is the scale used for FCDT, which is expressed as follows:

$S_{c}=\left\lceil\log _{2}^{\left(\min \left(N_{r}, N_{c}\right)\right)-3}\right\rceil$        (9)

Assume a scenario where an image of a certain size, say 256 *256; for such types of images, the evaluated number of scale will be 5 at various orientations but will be absent for the initial and last scales of the decomposition. However, the authors can also observe variation in the angle of orientation of 16, 32, and 32 for respective scales with values 2, 3, and 4. Here, all sub-bands of images are considered, and first one represents the approximation part and the rest the detail one. All required steps to calculate the feature vectors are well depicted in Algorithm 1. Henceforth, by obtaining such phenomenon, it can easily obtain that the desired feature vector in the matrix format for all handwritten images by applying the wrapping function of the curvelet transform which can be used for further processing in the recognition model.

5.png

Figure 5. Proposed CR model for handwritten character and numeral

Algorithm I: Estimation of Feature Vector using FDCT.

Input: Read the image from database: F[x₁,x₂];0≤x₁,x₂≥x;

Output: Report the curvelet coefficient: Coeff(i,l,k)

1. At each input of F[x₁,x₂], perform 2d-FDCT and report the coefficients of the Fourier transform as follow:

0.png

2. For every value of scale i and angel l, report the cross-product of $\hat{V j l}\left[\begin{array}{l}n, n \\ 1,2\end{array}\right] \hat{F}\left[\begin{array}{c}n, n \\ 1,2\end{array}\right]$.
3. where, $\hat{V j} l\left[\begin{array}{ll}n & n \\ 1 & 2\end{array}\right]$  represents the discrete value of the window in frequency.
4. Then perform the wrapping as $f_{i, l}^{\approx}\left[\begin{array}{cc}n & n \\ 1 & 2\end{array}\right]$=Z($\hat{V j l}\left[\begin{array}{l}n, n \\ 1,2\end{array}\right] \hat{F}\left[\begin{array}{c}n, n \\ 1,2\end{array}\right]$) .

where, $0 \leq \underset{1}{n}<\underset{1, i}{L}$  and $0 \leq \underset{2}{n}<\underset{2, i}{L}$  for $\theta \in\left(-\Pi / /_{4}, \Pi / 4\right)$  and L_1,i, L_2,i, represents the length and width of rectangle at scale j.

5. Subsequently, calculate the curvelet coefficient Coeff(i,l,k) by performing 2d Fourier transform to each $f_{i, l}^{\approx}\left[\begin{array}{cc}n & n \\ 1 & 2\end{array}\right]$.

All the relevant datasets of both numerals and characters are followed by applying Algorithm I. Along all the sub-bands it can be assumed that we can obtain ηa which may be represented as Ma where a=1,2,...ηa. We have also noted the values of curvelet at the desired angle θ which is the same as that obtained at Π+θ. Therefore, the desired feature vector obtained for both the dataset images is expressed as follows:

$F_{v e c}=\left[M_{1}^{l_{1} . . l_{c}, M_{2}} l_{1}, \ldots l_{c}, \ldots \ldots \ldots, M_{\eta b}^{l_{1}, \ldots, l_{c}}\right]$       (10)

where, ηb represents the symmetric set of excluded sub-bands and lc represents the highest value of the co-efficient along the ηb sub-band. In the proposed model, all the desired obtained values are processed for further use in phases. All the subsequent steps are followed for the two handwritten database images of characters and numerals one after another, and the reported desired feature vectors are depicted in Figure 6 and 7.

6.png

Figure 6. Handwritten Odia character and its respective frequency level of discrete curvelet

7.png

Figure 7. Handwritten Odia numeral and its respective frequency level of discrete curvelet

3.4 Reduction of feature vector dimension

The conversion of a higher feature dimension value to a smaller dimension value is known as the reduction analysis over the feature vector set. In this process, the higher dimension of the feature vector produces very high load and higher memory that exhibits poor performance. For avoiding such problems, PCA [42] is applied for mapping of these high dimensional entities to low dimensional entities, which has been widely used for reducing the dimensionality of the feature vector. PCA generally exhibits the orthogonal transformation of the input data and reports the achieved matrices of different variances, which helps in predicting the score values known as PC score or principal components. In addition to PCA, in this step, the linear discriminate analysis (LDA) is also introduced for performing the reduction in dimension. This discriminate analysis always produces a low number of feature spaces and reports the variation among classes; therefore, this analysis is widely used by researchers. This supervised-based approach provides a sample that very different from other classes but similar to the same class. Generally, the LDA discriminates the classes and also exhibits low performance for higher dataset constraints in order to obtain small sample size [43]. A suitable solution is proposed for higher in dimensions but for small sample sizes. For instance, consider a scatter matrix termed as $S_{\text {mat }}$ which is singular. Then, calculate the value of m+n, where m all total classes available and n is the required vector dimension. Generally, it is nearly impossible to obtain $S_{\text {mat }}$ in a singular form, so a hybrid approach termed as PCA is used along with LDA. In the cases of PCA+LDA, initially M dimensions are reduced to N dimensions with the help of PCA and thereafter they are reduced to ddimensions by harnessing LDA such that $\mathrm{d} \ll \mathrm{N} \ll \mathrm{M}$.

Before processing the PCA with LDA, a certain necessary action is taken. In addition, we must also ensure that the desired feature vector must be normalized one with zero mean and unit variance. This particular step must be followed so that an optimal feasible solution for the feature vector can be suggested. All the eigenvalues are arranged according to their decreasing values and the normalized cumulative sum of variance (NCSV) should also be reported in that phase.

$\operatorname{NCSV}(i)=\frac{\sum_{u=1}^{\alpha} \alpha(u)}{\sum_{u=1}^{D} \alpha(u)} 1 \leq i \leq D$       (11)

The eigenvalues of the images are denoted by α(u) and the respective dimension of the feature set is represented by D. In order to make the subsequent step successful, certain threshold values are maintained manually so that the desired relevant reduced feature vectors with suitable dimensions are considered to achieve high recognition accuracy for handwritten characters and numerals, which are termed as the basic vectors. All the essential steps are discussed below in Algorithm II.

Algorithm. II: Feature reduction using PCA+LDA.

Require: Feature matrix: F_M of size N*D

Ensure: Reduced feature matrix: Fr of size N*L

Functions PCA () and LDA () reduce the dimensions by using PCA and LDA, respectively.

1: Choose a dimension M $\triangleleft$ Reduced dimension using the PCA

2: F’(N *M) PCA(F_M; M)

3: Select a dimension L using the NCSV measure $\triangleleft$ Reduced dimension using the LDA

4: Fr(N * L) LDA(F’;L)

5: Output the reduced matrix Fr

3.5 Classification by the LS-SVM, RF classifiers

In the OCR system, the classification stage [44] is considered the most important stage in a model. It defines the robustness of the proposed model with respect to the variation in the datasets. For the classification analysis, the proposed model have employed the support vector machine (LS-SVM) and random forest (RF) as classifiers. Subsequently, the obtained basic feature sets of datasets are processed as input parameters along with the class levels of the above- mentioned two classifiers one by one. All the extracted key feature data were processed through classification implemented by using LS-SVM classifier [37] and followed by the RF classifier [38]. This process is not that much effective in the case of larger datasets, and sometime leads to complex networks, so for removing this drawback LS-SVM is used in this paper. It has been observed that LS-SVM exhibits more incremental responses as compared to the traditional ones in terms of the addition of some equality constraints. Generally, an optimization analysis is applied for solving the linier solution to the problem domain. The performance of the LS-SVM classifier can be expressed as follows:

$\min _{c, d, e_{r}} j\left(c, d, e_{r}\right)=\frac{1}{2} c^{T} c+D \frac{1}{2} \sum_{i=1}^{N} e_{r_{i}}^{2}$       (12)

With some constraints as

$Z_{i}\left[c^{T} \phi(p i)+d\right]=1-e_{r}, i=1,2,3, \ldots, N$         (13)

Here {p_i,Z_i} is the obtained key feature vector that acts as a training set of N samples with p_i data. ϕ(.) and c represent the mapping function and associated weight, respectively. D>0, d and e_r are used as the regularization factor, bias, and error, respectively. In addition, we also included the three-kernel function such as linear, polynomial, and radial basis functions, to the LS-SVM. The required solution for LS-SVM achieves the optimality by the help of the Lagrangian method, which is expressed as follows:

$\mathrm{L}\left(\mathrm{c}, \mathrm{d}, \mathrm{e}_{\mathrm{r}}, \lambda\right)=\mathrm{j}\left(\mathrm{c}, \mathrm{d}, \mathrm{e}_{\mathrm{r}}\right)-\sum_{\mathrm{i}-1}^{\mathrm{N}} \lambda_{\mathrm{i}}\left\{\mathrm{y}_{\mathrm{i}}\left[\mathrm{c}^{\mathrm{T}} \phi\left(\mathrm{p}_{\mathrm{i}}+\right.\right.\right.$

$\left.\mathrm{d})]-1+\mathrm{e}_{\mathrm{ri}}\right\}$       (14)

In the same process, some constraints or kernel functions are harnessed over the dataset to report the actual class level of each individual handwritten image. We also executed the polynomial kernel in addition to the other two kernels, namely, linear and radial basics function (RBF). Apart from LS-SVM, another random forest (RF) classifier [39] was analyzed for the handwritten characters and numerals. It was found to be quite simpler and flexible in nature, so it was considered to be very helpful for large datasets. Basically, RF is a well-defined approach of tree which follows the decision-based structured classifier and is completely based on random samples that have uniform distribution among trees. Hence, it is considered to be the most positive point of this method and works or proceeds with the tree in the depth wise location. In this random-based approach, each tree is structured considering the values of random vectors and sampled with the same distribution using the forest classifier [45]. Suppose the random forest represents the methodology for building L tree for classification then it is denoted by $\left\{\mathrm{T}_{\mathrm{j}}\left(\mathrm{X}, \theta_{\mathrm{j}}\right\}\right.$. In this tree, X is the input feature and $\theta_{\mathrm{j}}=\left\{\theta_{1}, \theta_{2}, \ldots, \theta_{\mathrm{j}}\right\}$  is the respective random vector for the j^th  tree with a training size of N.  Apart from this, $\theta_{1}, \theta_{2}, \ldots, \theta_{\mathrm{j}}$ are the integral values drawn from the random uniform distribution. Therefore, it creates a subspace method $\theta_{\mathrm{j}}$ that provides K number of integers with an interval M. For any given sample x, the chances of being classified by T are denoted by a probability function like $P\left(\frac{C}{x}, T\right)$. Moreover, it is verified that if the sample x belongs to class $C(C \in\{1,2, \ldots, C\}$, it is represented as follows:

$P\left(\frac{C}{x}, T\right)=\frac{n(c)}{n}$       (15)     

In this case, $\mathrm{n}(\mathrm{c})$ is the desired sample taken from the training sample, which makes them to fall in v(x). Apart from this, all numbers in the training sample are made to fall in v(x). At a certain step, if an unknown sample is classified then the equation is altered to belong to L numbers of trees as follows:

$\mathrm{P}\left(\frac{\mathrm{C}}{\mathrm{x}}, \mathrm{T}\right)=\frac{1}{\mathrm{~L}} \sum_{\mathrm{i}=1}^{\mathrm{L}} \mathrm{P}\left(\frac{\mathrm{C}}{\mathrm{x}}, \mathrm{T}_{\mathrm{i}}\right)$       (16)

In order to successfully implement the recognition system, the authors have applied the proposed algorithms I and II to the standard dataset of handwritten characters and numerals. Similarly it can show at third level decomposition of the discrete curvelet transformation (FCDT) in the feature extraction section. Moreover, the approximation feature vector along the sub-bands of the decomposition level of images is reported in 21*21 numbers of features in this transformation. As already discussed, higher dimensions are again processed for the next segment, that is, the dimension reduction. In order to obtain better efficiency or to make the recognition process easier, it is required to perform a reduction in the dimension vector reported from the FCDT. Hence, PCA+LDA is processed over the feature set and it produces some basic vectors that represent the reduced dimension in the feature vector in terms of NCVS. In addition, the proposed model has maintained the threshold value at 0.9 for the recognition system. In the simulation part, it has been considered 350*47 numbers of handwritten character images and 500*10 numbers of handwritten numeral images. After performing the reduction to the feature vector by using PCA+LDA, the authors have reported 50 and 60 numbers of primary feature vectors which shows the reduction in dimension of the feature vector for handwritten characters and handwritten numerals, and act as inputs to LS-SVM and RF classifiers, respectively. Here, it has also noted the recognition rate at different ranges of features. All the well-defined steps are listed in Algorithm III. The authors have performed the classification over three datasets. In this process, first all the characters are classified and then they are followed by handwritten numerals.

This proposed model primarily focused on the variant features of the handwritten images and provided an automated system for the recognition of the numerals and characters. All the essential steps, such as acquisition, preprocessing, and proper feature extraction methods, and well-defined classifiers were used to make the recognition system successful. In order to make the system robust, we trained the system with a wide range of styles of handwritten images that have been reported here as a standard database. To make the system less complex, in proposed model have performed a reduction in the dimension of the feature vector. Hence, PCA and LDA are harnessed over the feature set in order to produce the reduced version of the feature vector and to be considered as the primary key feature vector. Apart from this, the proposed model promoted three classifiers, namely, LS-SVM and RF Classifiers, in the segment of the classification phase. Initially, the recognition is performed by the LS-SVM classifier and subsequently, followed by the RF classifier. As an outcome of the proposed model, the overall accuracy of handwritten Odia character datasets is reported as 99.01%, 98.1% respectively. Similarly for Odia handwritten numerals 98.6% and 97.6%, respectively and for Bangla numerals 97.6%, 96.3% recognition rate is achieved. From the analysis of results, the proposed models have concluded that LS-SVM performs better as compared to RF based on our problem domain. Proposed model outperforms better as compared to the state-of-the-art methods in terms of both aspect of recognition accuracy and using least number of optimal features are depicted in Table 2 and Table 3 respectively.

Table 2. Analysis over certain well-defined approach for handwritten Odia characters

Table 3. Analysis over certain well-defined approach for handwritten numerals