A Deep Learning-Based Approach for Gender Prediction in Digital Forensics

Biometric systems use one or more of a person's vital characteristics for the authentication process. These systems are divided into physiological and behavioral systems. Physiological characteristics are linked to the characteristics of the individual, such as fingerprints, irises, retinas, DNA, hands, faces, and voices. while the behavioral biometric systems encompass attributes such as keystrokes and voice.

Biometric data, encompassing physiological and behavioral information, is often employed to recognize and find criminal suspects. The utilization of biometric characteristics for identification verification in forensic investigation belongs to the early 20th century [5]. A handwritten signature is a biometric measure called soft biometrics, which has a long history and is used for predicting many traits, such as the gender of the writer. Predictive capabilities instituted on handwriting open vast horizons for predicting many traits [9].

Gender prediction can be an important aspect of forensic investigations, especially when the crime scene contains samples of handwritten papers that can be used to identify the writer, as it can provide additional information for identifying a suspect or victim [10, 12, 15]. Gender can be identified from handwriting samples, which is essential in situations such as personal and gender identity that may be needed in deciding the author of a document found at a crime scene [12, 17]. For many years, handwriting has been used by document examiners, forensic analysts, and in psychoanalysis. Handwriting-based gender prediction analysis can serve as a valuable measure in forensic inquiries because of its ability to recognize the gender of a handwritten document's author, which can assist in refining the suspect pool in criminal investigations and may serve as evidence in legal activities [16].

Offline and online recognition are types of recognition processes. The first one uses the text that is found on paper or another physical medium and is then digitized, while the second deals with text generated by an electronic digitizer device [15]. Offline handwriting text analysis entails the classification of handwriting into distinct categories, including age, gender, and nationality. This process involves the extraction of features from handwriting, such as textural characteristics, letter shape, size, curvature, pen pressure, and text inclination, and the application of algorithms through various techniques to recognize the characters and words present in the text. Offline handwritten text analysis entails the transformation of handwritten characters into machine-encoded text, employing techniques such as pattern recognition, machine learning, and artificial intelligence algorithms [15]. In this context, machine learning (ML) is considered a solution. The main attractive features of it are that it streamlines the analytical transaction and provide accurate results [4, 9].

ML and artificial intelligence (AI) are the prominent technologies of the contemporary era. All processes are progressing towards automation. DF must advance concurrently with the digital world to enhance data processing capabilities. Deep learning and deep neural network methods are employed for analysis of text and digital files and are also used with multiple types of forensics applications that include linguistic, image, video, and email. It is utilized in timeline reconstruction, pattern identification, and object categorization [18, 19].

In recent years, ML has gained major interest. It is a method acquired via training and experience rather than through programming. The dataset must be trained before the application of these algorithms. The selection of algorithms is another crucial part in machine learning. Depending on the need for supervision, ML algorithms can be classified into two types: supervised learning and unsupervised learning. In the first type, it is necessary to train the algorithm on a labeled dataset for examples, SVM, DT, RF, CNN, LR, and Naïve Bayes (NB). In unsupervised learning, it is necessary to train the classifier on an unlabeled dataset. For instance, clustering and Principal Component Analysis (PCA). Another category is deep learning, which relies on unsupervised learning. It employs hierarchical structures of artificial neural networks (ANNs) [4, 6, 20]. Data analysis is conducted primarily in two phases. The initial phase involves preprocessing, during which we must input our dataset into the machine learning algorithm. If the data is gathered personally, it must first be trained, while it is filtered based on requirements if it was pre-trained. After training the dataset, the ML algorithm will be used for classification purposes. The results of this step will be considered as input for the final stage of digital forensics [9].

All stages of identification of the person and gender are illustrated in Figure 1, starting with the data collection step and ending with the classification step, passing through preprocessing, feature extraction steps. The quality of the collected data is poor, so it needs to be improved, which is done in the preprocessing step that prepares the data for the feature extraction stage. This step includes various operations such as segmentation, binarization, normalization, and noise removal (particularly applied for image or signal) [21, 22].

Figure 1. Gender identification system process

In this research, we propose a two-stage DF system that combines evidence verification and gender prediction based on handwritten documents. SHA-256 hashing is used in the first stage to verify evidence. Using a hash provides a cryptographic seal, which is useful for verifying the authenticity of handwritten documents during the investigation process. The second stage of the proposed system encompasses using a model of CNNs named ResNet 18 for gender prediction. This model has attractive features that make us use it, one of which is overseeing complex images during classification tasks, and its ability to mitigate vanishing gradient problems through residual connections. The ResNet18 structure consists of 18 layers with skip connections that allow for gradient flow, making DNN training more efficient with stable performance. Through the results that will be explained later, we demonstrate the effectiveness of the proposed model for the task of handwriting analysis for gender prediction. The structure of our comprehensive system is illustrated in Figure 2, which details the system's stages, from initial document processing through validation to gender prediction. The introduced system guarantees the security and accuracy of DF while supporting the standards required for forensic evidence processing.

2.jpg

Figure 2. Proposed system architecture

5.1 Securing evidence

In recent years, with the advancement in technology, the need for the authentication of digital images has increased, because many tools are arising that enable unauthorized people to modify the images and republish them [26]. The integrity of collected evidence must be ensured during all investigation phases, from collection until investigation completion. The way to do this is by using hashing [27]. A mathematical hash function is a cryptographic function that accepts input messages of arbitrary size and outputs a fixed-size result known as the hash value or digest. The output is usually a combination of numbers and letters that function as a distinct digital "fingerprint" for the input information. The output of the hash function must be deterministic, which means that for a given input, the output will always be the same [27]. The main powerful feature of the hash function is that the generated hash value cannot be used to retrieve the original message, so privacy and security are preserved while sharing the message. For this feature, the hash is effective for password storage and digital signature [28, 29]. There are various hash functions, like message digest (MD2, MD4, and MD5), hashed message authentication code (HMAC), and secure hash algorithm (SHA-0, SHA1, SHA2, and SHA-3). Each one has its features, but SHA-256 is highly secure and widely used in DF. This paper suggests SHA256 to legitimate DF images. Hashing can be used at distinct stages of DF. Hashing can be used at the evidence collection stage, where it is used at the time of acquiring digital evidence, such as it is used for (original handwriting image files, extracted features (e.g., HOG features or CNN embeddings), or prediction results (gender labels) [27].

The SHA-256 processing is as follows:

Padding, in this step, we arrange the final input message into the following, as Figure 3 below explains. This step is called the padding step. Then the padded message is broken into 512-bit blocks [30].

3.jpg

Figure 3. Padding

In SHA-256, the message is split into blocks, each one have 512 bits, $B^{(0)}, B^{(1)}, \ldots ., B^{(N-1)}$, which are then processed in 64 rounds on each block of data $B^{(i)}$ utilizing a variety of bitwise operations and nonlinear functions, such as logical functions (AND, OR, XOR) and arithmetic functions (ADD, SHIFT). A hash value $H_N$ with a partial 256-bit is gained as the total processing of the current $B^{(i)}$ block is done, and the last hash $H_N$ is computed and delivered after the last data block $B^{(N-1)}$ is computed. During each round, the block is partitioned into sixteen words, each one has 32 bits, and these words are used to update a set of eight 32-bit variables, known as the working variables [26, 27, 29]. Figure 4 shows the steps of the SHA-256 hash function algorithm.

where,

${Ch}(x, y, z)=(x \wedge y) \oplus(\neg x \wedge z)$                (1)

${Maj}(x, y, z)=(x \wedge y) \oplus(x \wedge z) \oplus(y \wedge z)$                 (2)

$\sum_0(x)={ROTR}^2(x) \oplus {ROTR}^{13}(x) \oplus {ROTR}^{22}(x)$                   (3)

$\sum_1(x)={ROTR}^6(x) \oplus {ROTR}^{11}(x) \oplus {ROTR}^{25}(x)$                   (4)

4.jpg

Figure 4. SHA-256 hashing algorithm

To process the $B^{(i)}$ data block, the main function uses eight working variables: a, b, c, d, e, f, g, and h, each of which is 32 bits. After the 64 loops, these variables comprise a buffer state, which is the partial hash value $H_{ {temp }}$. The result $H_{ {temp }}$ is summed with the hash of the previous data block that was previously computed to obtain the hash value corresponding to the message until the data block $B^{(i)}$. When the first data block is processed, a first hash value $H_0$ is used. At the beginning of the computation $H_{(i+1)}$, the $H_i$ value needs to be loaded into the buffer state. At every iteration in the hash algorithm, a constant $K_t$ is desired. This is constantly well known in the SHA-256 specification. The equations below compute the temporal variables $T_1$ and $T_2$.

$T_1=h+\sum_1^{256}(e)+{ch}(e, f, g)+K_t^{256}+W_t$                    (5)

$T_2=\sum_0^{256}(a)+{Maj}(a, b, c)$                   (6)

A 32-bit value $W_t$ is calculated for each iteration in the algorithm above as explained during the first 16 iterations, the first 16 values $W_t$ are taken directly from the 512-bit message, and the remaining iterations $\sigma_0$ and $\sigma_1$ are used to compute the values of $W_t$.

$W_t=\left\{M_t^{(i)} 0 \leq t \leq 15 \sigma_1^{\{256\}}\left(W_{t-2}\right)+W_{t-7}+\sigma_1^{\{256\}}\left(W_{t-15}\right)+W_{t-16}\right\} \quad 16 \leq t \leq 63$                  (7)

$\sigma_0(x)={ROTR}^7(x) \oplus {ROTR}^{18}(x) \oplus {SHR}^3(x)$                (8)

$\sigma_1(x)={ROTR}^{17}(x) \oplus {ROTR}^{19}(x) \oplus {SHR}^{10}(x)$                 (9)

In our paper, after we get the evidence (images, notes, etc.), we compute the hash value using SHA-256. We can ensure that the evidence is not altered if we get the same hash value for the processed evidence, so its integrity is preserved.

5.2 Prediction with deep learning

This section describes the details of deep learning models that are used for handwriting evidence-based gender prediction as follows:

5.2.1 Data acquisition

To assess the performance of writer identification systems, databases available online can be used. One such public database used for this purpose is QUWI, which consists of handwritten documents in both Arabic and English. 1,017 individuals of varying ages, genders, and educational levels were collected in this database. These volunteers were asked to write a distinct text and a random text of their choice in both Arabic and English. This makes the data suitable for use in text-based writer identification tasks. The most noteworthy features of these databases, which led us to use them in our proposed system, are the balanced distribution of males and females (48% and 52%, respectively) and the holding of both Arabic and English texts, enabling them to be used in various systems.

5.2.2 Preprocessing

To ensure good model results, it is important to focus on the quality of the inputs. To achieve high quality, the preprocessing of handwritten documents is required. This preprocessing plays a significant role in the gender recognition process. In this paper, we applied several preprocessing procedures to the documents to gain a high-quality input for the ResNet18 model.

Image Standardization: Dataset images are high-resolution images (600 dots Per Inch), so they will first be converted to a standard format. To keep colors, all images are converted to the RGB color space, which indicates pen pressure and writing style. ResNet18 accepts images with (224 x 224) pixels, so the images must be resized to the desired size while keeping the aspect ratio. To normalize the pixel values, we divide them by 255, and thus all values will be in the range [0,1]. Finally, the normalization was followed by standardization using the ImageNet mean and standard deviation.

Evidence quality enhancements: We'll be making several improvements to the document to gain high-quality features, such as using the histogram equalization technique for promoting contrast and using a 3×3 kernel filter to eliminate the Gaussian noise. As well as employ bilateral filtering to keep the edges that support the text's font characteristics.

Text Region Extraction: This step involves separating background from text using Otsu's threshold for initial separation. Contiguous text regions can be identified by analyzing connected components. This step also applies to morphological operations to connect nearby characters while preserving the integrity of the fonts.

Evidence Augmentation: One of the common problems is overfitting, which can be solved in this step, thus increasing the robustness of ResNet-18 by implementing various augmentation mechanisms to the evidence, such as random rotation and random gradients. The first one deals with the variance in the writing angle, whereas the second deals with the inconsistency of font size. Finally, the brightness and contrast must be adjusted because it is affected by writing tools and scanning circumstances.

Data Systemizing: This step involves creating small batches of equally distributed male and female samples. To process large-sized documents, a sliding window approach is utilized. Training, validation, and selection groups are prepared with stratified sampling while maintaining gender distribution.

These steps will be applied to process Arabic and English texts to ensure proper processing while preserving the text properties.

5.2.3 Feature extraction and model training

This stage of the proposed model is extracting the features from the preprocessed evidence to use later in training the model. Transfer learning is used for this demonstration in conjunction with the pre-trained ResNet18 model. We can obtain leveraging weights after training the model on a large-scale dataset. Low-level visual features like edges and textures, and high-level features like shapes and patterns, can be obtained from this stage. These features can be used for gender classification based on handwriting. The ResNet18 architecture processes handwritten documents through five components, as shown in Figure 5. This architecture was designed for use in handwriting analysis yet maintains its computational efficiency for other forensic applications.

5.jpg

Figure 5. ResNet18 architecture for handwritten document processing