A Comparison of Deep Learning and Machine Learning Approaches to Video Injection Detection

Numerous studies investigating video injection attacks using machine learning and deep learning have been conducted by many researchers. Zamir et al. [11] propose a Raspberry Pi-camera-based face recognition system that uses CNNs for feature extraction and classification to identify faces in real time. A bespoke dataset of 700 photos of 7 individuals yielded accuracies of 95.23% (precision: 96.51%, recall: 97.64%, F1-score: 97.07%) at a 70:30 training/testing ratio and 97.71% (precision: 98.09%, recall: 96.26%, F1-score: 97.16%) at an 80:20 ratio. The study shows that CNN outperforms HOG in face detection and recognition, especially in difficult settings, and that hardware affects training time and memory consumption.

Chaabane et al. [12] introduce a statistical feature extraction and Support Vector Machine (SVM) classification face recognition approach with a 99.37% recognition rate and 4.35% EER. Local Binary Patterns (LBP) and Principal Component Analysis (PCA) with sliding windows are used to select and extract features and compute statistical properties like mean, variance, skewness, and kurtosis from images. The experimental results reveal that the suggested method is effective, especially when using numerous postures for each individual, and outperforms existing methods, in spite of its computational requirements and noise sensitivity.

Guo [13] tests the K-Nearest Neighbors (KNN) face recognition algorithm on exposed frontal, profile, and mask-covered faces. The study indicates that KNN has a 95% success rate for exposed frontal faces but 22.2% for profile faces and 2.22% for masks. Traditional face recognition algorithms struggle to identify partially veiled faces and profile photos, stressing the requirement for broad training datasets. It also finds a high false rejection rate when the KNN model is trained on covered faces, suggesting that future applications should focus on eye and forehead identification.

Nassih et al. [14] introduce GD-FM+RF, a fast 3D face recognition system that uses fast marching geodesic distance calculation with a Random Forest classifier. Key points are manually extracted from 3D facial meshes, geodesic distances are measured to create facial curves, and feature separability is achieved using Principal Component Analysis (PCA). The system's 99.11% recognition performance on the SHREC'08 database, which contains 427 images of 61 people, outperformed numerous current methods and showed its ability to robustly identify persons based on 3D facial scans.

Alkishri et al. [15] propose convolutional neural networks (CNNs) and Rotation Invariant Local Binary Patterns (RI-LBP) to detect fake faces with greater accuracy than RGB and HSV. It emphasizes color texture analysis in the YCbCr color space. The technique outperforms prior studies with a 3.2% Equal Error Rate (EER) using the MSU MFSD dataset. Selecting proper texture descriptors is crucial, and the authors urge future research to increase detection accuracy and efficiency in real-world applications, notably in digital media identity theft and fraud.

All the aforementioned papers primarily focus on methodologies in machine learning or deep learning. This study aims to explore the gaps between the two methodologies, particularly in the context of video injection detection.

1.jpg

Figure 1. The flowchart of the process in detecting the fake faces using deep learning

2.jpg

Figure 2. The flowchart of the process in detecting the fake faces using machine learning

Deep learning and machine learning have their own mechanism [16-18]. Deep learning is able to do classification without a need for the feature extraction process; thus, the process as shown in Figure 1 is used. While for machine learning, it needs the feature extraction process; thus, the process as shown in Figure 2 is used. The detail of each process is presented in the following subsections.

2.1 Dataset and processing

This research used the video dataset available on Kaggle [19]. The dataset was processed by changing the video into an image with extension (.png). The dataset consisted of six types of videos as shown in Figure 3. Pre-processing was done by involving the change of image format into PNG format and the adjustment of image size to make it consistent with the size of 224×224 pixel. In this study we used two classes of data, real and fake face.

3.jpg

Figure 3. Type of image in dataset [12]

2.2 Feature extraction and selection

The process of feature extraction in the image covered three types of main features. First, there was an extraction of texture features using Gray-Level Co-occurrence Matrix (GLCM) method [20]. This study used 7 GLCDM features: correlation, homogeneity, dissimilarity, contrast, energy, and angular second moment. The seven features were calculated at 0°, 45°, 90°, and 135°. If P_i,j is the GLCM probability, then the features used are calculated as in Eqs. (1)-(7):

$Correlation=\frac{\sum_i \sum_j(i-\bar{x})(j-\bar{y}) P(i, j)}{\sigma_x \sigma_y}$                  (1)

$A S M=\sum_i \sum_j P(i, j)^2$                 (2)

$Homogeneity=\sum_{i_1} \sum_{i_1} \frac{p\left(i_1, i_2\right)}{1+\left|i_1-i_2\right|}$                    (3)

$Dissimilarity=\sum_{i, j=0} P_{i, j}|i-j|$                  (4)

$Contrast=\sum_{i_1}\left(i_1-i_2\right)^2 p\left(i_1, i_2\right)$                 (5)

$Energy=\sum_{i_1} \sum_{i_1} p^2\left(i_1, i_2\right)$                 (6)

$A S M=\sum_i \sum_j P(i, j)^2$                   (7)

Second, the form feature was extracted by calculating the value of eccentricity of the image. The eccentricity of an image is presented in Eq. (8).

$e=\sqrt{1-\frac{b^2}{a^2}}$                   (8)

where, e=eccentricity, b=minor axis, and a=major axis.

Last, the feature of color was extracted using the HSV (Hue, Saturation, Value) color covering three main components: hue, saturation, and value [21]. This process could help to identify and select the dominant colors in the image. Overall, the process of feature extraction aimed to obtain the more level representation from the image to be used in any assignments of image analysis and image processing.

For the selection feature, several combinations are made between the texture, color, and shape features. Seven combinations are obtained for the entire selection feature: texture, color, shape, texture and color, texture and shape, color and shape, texture, color and shape.

2.3 Data splitting

The dataset was split into two parts for training data and data of model testing, each of which consisted of 80% for training process and 20% for training process of entire dataset, both for augmented and non-augmented data. Thus, the total data in the first scenario was 32 training data and 128 testing data. While the second scenario used 640 training data and 2560 testing data.

2.4 Classification

In this study, we did a comparison among convolutional neural networks (CNN), Artificial Neural Networks (ANN), and recurrent neural network (RNN) in the context of deep learning for image analysis. Furthermore, we considered the machine learning approach with K-Nearest Neighbors (KNN), SVM, and Random Forest as the comparison.

2.4.1 Convolutional neural networks (CNN)

CNN has been proven to have a good performance in any field, including in computer vision and machine learning [22, 23]. It consists of a number of layers including convolutional layer, pooling layer, and activation layer in which each layer in CNN has a special role. The convolutional layer acts to extract the low-level feature; pooling layer acts to reduce the dimension of feature maps while maintaining the important information for the classification process. The convolution process principally involves the operation of dot product in which its results will be forwarded into the activation layer to improve the non-linear feature in CNN [24]. Illustration of CNN network depicted in Figure 4.

4.jpg

Figure 4. Illustration of the architecture of CNN network

5.jpg

Figure 5. Illustration of the architecture of ANN multilayer feed forward network

2.4.2 Artificial Neural Networks (ANN)

ANN can be characterized as a model of information processing derived from the behavioral mechanisms of human brain networks in information processing [18]. However, what this ANN can only do is to imitate the basic function of the human brain. ANN consists of many processing units (neurons) that are connected to each other. Similar with humans, ANN also learns from any existing examples or experiences [14]. Each neuron is connected to each other via connection links, each of which is associated with a weight, containing information about the input signal. This information will later be used by the neuron network to solve certain problems [20]. Illustration of the ANN architecture is presented in Figure 5.

2.4.3 Recurrent neural network (RNN)

RNN is a neural network architecture developed for processing sequential data through feedback connections, enabling maintaining data over time steps. Unlike traditional neural networks, RNNs maintain a hidden state, which enables them to capture dependencies within sequences, making them ideal for tasks like time series prediction, language modeling, and speech recognition.

Specific configurations of RNN:

1) Vanilla RNN

2) Long Short-Term Memory (LSTM)

3) Gated Recurrent Unit (GRU)

These configurations allow RNNs to adapt to various sequential tasks with trade-offs between complexity, computation, and the ability to retain long-term dependencies.

2.4.4 K-Nearest Neighbors (KNN)

KNN is one of the methods in machine learning [25]. In KNN, the distance matrix such as Euclidean distance is used to measure the distance between data points. Euclidean distance [26] is measured by means of the Eq. (9).

$D(x, y)=\sqrt{\sum_{i=1}^n\left(x_i-y_i\right)^2}$               (9)

In KNN, there is a parameter called k, which is used to determine the number of neighbor’s to be considered when classifying or predicting the data points [27]. It is important to choose the appropriate k value as it can determine the performance of KNN model; thus, the k value should not be so high or so low.

2.4.5 Support Vector Machine (SVM)

SVM is a supervised learning technique utilized for classification and regression applications, while it is predominantly employed for problems related to classification. It finds the optimal hyperplane that maximally separates different classes in the feature space. For non-linearly separable data, SVM uses kernel functions (e.g., RBF, polynomial) to transform the input space into a higher-dimensional feature space where a linear separator can be found. The objective of SVM is to maximize the margin between the closest points (support vectors) from each class.

• Kernel Functions: Linear, Polynomial, RBF, Sigmoid
• Hyperparameters:

           - C: Regularization parameter (regulates the balance between margin size and classification error)

           - Gamma: Controls how far influence of a single data point reaches (for RBF or polynomial kernels)

2.4.6 Random Forest

Random Forest is an ensemble learning technique that constructs numerous decision trees during training and integrates their predictions to enhance accuracy and mitigate overfitting. Each tree is trained on a random subset of data (via bootstrap sampling) and uses a random subset of features to make splits, enhancing diversity in the model. The final prediction is made by majority voting for classification or averaging for regression.

• Key Hyperparameters:

           - n_estimators: Number of trees in the forest

           - max_features: Number of features considered for each split

           - max_depth: Maximum depth of each tree

• Advantage: Highly robust to overfitting, efficient in handling large datasets, and resistant to noise.

Both algorithms are widely used, with SVM excelling in smaller, well-separated datasets and Random Forest offering strong performance in complex, high-dimensional datasets.

2.5 System performance measurement

In evaluating the performance of the deep learning and machine learning model, confusion matrix as shown in Table 1 was used. Confusion matrix had a number of matrices including accuracy, precision, recall, and F1 score as shown in Eqs. (10)-(13). To calculate the matrices, four components; those are True Positives (TP), False Positives (FP), True Negative (TN), and False Negative (FN) were used. The matrices could help in predicting the real image and unreal one [28, 29].

Table 1. Confusion matrix