Online Transaction Fraud Detection Using Efficient Dimensionality Reduction and Machine Learning Techniques

In the last two decades [1-3] the digital economy has grown very fast. People who were earlier totally dependent on cash transactions have also started using online payments. The ease of use, hassle-free and smooth user experience and time-efficient use of e-commerce [4], online banking, e-governance [5, 6] etc. have facilitated the shift from cash-based transactions to online transactions. All this is aided by the advancements in technology that have given every household, whether urban or rural, access to the internet.

As everything is moving online, the number of online and cashless transactions is increasing. This shift was further aided by the pandemic. This increase along with great benefits poses some challenges as well. One of the major challenges is the number of frauds that happen in online transactions. As the quantity of online transactions grows, so does the number of online frauds. This calls for methods that could detect fraudulent transactions as soon as possible. This will allow the concerned authorities to take necessary measures to minimize the loss to public and private entities as well as people at large.

Machine learning algorithms [7] try to predict the outcome or provide us with the details about a data sample based on previous inferences. Based on the data available from prior transactions, the same can be used to forecast if a transaction is fraudulent or not. It is a supervised classification task [8, 9], wherein the aim is to detect and flag fraudulent transactions. As the system should work in a real environment and so would require fast responses ML techniques would be ideal as compared to deep learning techniques. This is so because the deep learning algorithm tends to take time while training as well as while giving inferences.

1.1 Contribution

Due to a large number of features and attributes, exploratory data analysis and feature selection were carried out to reduce the load and also gain an understanding of which features are more important.

Contribution of the paper is as follows:

1. In this, we proposed a dimensionality reduction technique before classification. Exploratory data analysis allowed converting the dataset into a usable format for the model. Detected dataset was used for training and testing the model.
2. We adopted XGBoost [10] which is known to perform well in classification tasks. This work tries to explore the use of XGBoost for online fraud detection on IEEE-CIS Fraud dataset.
3. Finally, the goal of this work is to create a model which is efficient enough and provides accurate results. This would allow the model to be used in real-world applications.

The researchers from the IEEE Computational Intelligence Society (IEEE-CIS) [21] wanted to prevent cases of online transaction frauds. This would save millions of dollars each year to the users who make online transactions. In an attempt to improve online transaction fraud detection, they partnered with Vesta Corporation, which is one of the world’s leading payment service companies. Together they provided a public dataset that comprises authentic transactions as well as fraud transactions. This dataset can be used to determine whether or not an online transaction is fraudulent, and this can be verified by making use of a binary target variable “isFraud”. Details of dataset are provided in Table 1 and Table 2.

Table 1. Identity dataset features

Table 2. Transaction dataset features

3.1 Proposed framework

In order to identity the fraud transactions in this work we initially perform the EDA techniques to find the relationship among the features avalible in the dataset represented in Table 1, Table 2, then the new Dimensionality reduction method identifies and remove unnecessary features from dataset and finally trained a model with train dataset and evaluated the performance of model using test dataset. The framework of our model is presented in Figure 1.

1.png

Figure 1. Framework model

3.2 Exploratory data analysis (EDA) and feature selection

We would initially conduct an Exploratory Data Analysis (EDA) [22, 23] on the “identity” and “transaction” datasets, to get a general understanding of the entire dataset and the relationship between them. Exploratory Data Analysis (EDA) can be performed using a variety of methods, such as histograms, heatmaps, correlation matrices, box plots, bar charts (grouped and ungrouped).

This allows us to reach the process of feature selection, in which finally we use only those variables that seem relevant to getting a prediction value. The inclusion of unnecessary and irrelevant variables can affect the performance of the machine learning model. It helps us to reduce the dimensionality of the final dataset to be used, which in turn reduces the computational cost and complexity of the machine learning model. It also prevents the overfitting of the model.

In the “identity” dataset, the variables “id_01” - “id_11” are continuous variables, while all the remaining variables are categorical. The data present inside this subset of the dataset represents information about the identity of the user and are related to the transactions. Now, coming over to the “transaction” dataset, there are about 20 categorical variables present in this subset of the dataset. Like the “identity” subset, critical information about the transactions is also masked to conceal their true meaning and usage.

Then for the categorical variables with low cardinality features are label encoded, while the higher cardinality features are encoded by making use of target encoding using the One Hot Encoding method.

3.3 Dimension reduction phase

In this subsection, we propose an efficient dimensionality reduction of features with the references [24, 25]. We have a corpus of d training documents, each of which is described by n features. Our main goal is to choose a small number l of features, where l ≤ n, such that, by using only those l features, we can obtain good classification quality, both in theory and in practice, when compared to using the full set of n features. The algorithmic steps are as follows:

Procedure (X, Y) –

1. Consider, X: Be set of conditional variables; Y: Be the decision variable
2. def Heuristic as, Y = [y₁] andX = [x₁, x₂ · · · x_m]

      $H(X \mid Y)=-\mathrm{E}[\log (p(x \mid y))]$

3. $L \leftarrow \phi$
4. do
5. $M\leftarrow L$
6. For all $x\in (X-L)$
7. if $H(L\cup \left\{ x \right\}|Y)<H(M|Y)$
8. $M\leftarrow (L\cup \left\{ x \right\})$
9. $L\leftarrow M$
10. Run up to $H(L|Y)<H(X|Y)$
11. Return L

The above dimensionality reduction phase works as, it initially takes’ m’ number of X variables and one Y decision variable. Then Line 2 is used to calculates conditional entropy of  X given Y. The Line 3 represents empty list which is used to store reduced features. Next Line 4 to Line 10 are repeated by calculating conditional entory value on each feature of X to identified feature to List L and finally it ends while the entopy of identified  features L value on Y exceeds the entropy value of X on given Y.Then it finally returns L, which is the list of features identified by our proposed dimentionality reduction method.

2.png

Figure 2. Framework of XGBoost model

3.4 XGBoost model

XGBoost (Extreme Gradient Boosting) is a machine learning algorithm that optimizes gradient descent boosting, by using gradient boosted decision trees. Figure 2 represents the evaluation of XGBoost model. It is an advanced machine learning algorithm that is capable of dealing with data that has higher degree of irregularity in it. XGBoost is an implementation of Gradient Boosted decision trees.

Eq. (1) represents the prediction scores of each individual decision tree mathematically as,

$\hat{y}=\sum_{i=1}^k f_k\left(x_i\right), f_k \in F$          (1)

where, k is the number of trees.

For the above step, Eq. (2) represents the objective function at n iterations as:

$\operatorname{obj}(\theta)=\sum_{i=1}^n l\left(y_i, \hat{y}_i\right)+\sum_{k=1}^n \Omega\left(f_k\right)$            (2)

where, first term l is the training loss function such as square loss or logistic loss computed with values of $({{y}_{i}},{{\overset{\hat{\ }}{\mathop{y}}\,}_{i}})$, and the second is the regularization parameter.

Eq. (3) represents the additive strategy applying process for minimization as,

$\hat{y}_i^{(t)}=\sum_{k=1}^t f_k\left(x_i\right)=\hat{y}_i^{(t-1)}+f_t\left(x_i\right)$               (3)

where,${{\overset{\hat{\ }}{\mathop{y}}\,}_{i}}^{(0)}=0$.

For the above step, Eq. (4) termed the objective function as

$\left.o b j^t=\sum_{i=1}^n\left[2{\hat{y_i}}^{(t-1)}-y_i\right) f_t\left(x_i\right)+f_t\left(x_i\right)^2\right]$$+\Omega\left(f_k\right)+C$           (4)

where, C is constant.

Now, Eq. (5) termed second order expansion after applying taylor series:

$o b j^t=\sum_{i=1}^n\left[l\left(y_i, \hat{y}_i^{(t-1)}\right)+g_i f_t\left(x_i\right)+\frac{1}{2} h_i f_t^2\left(x_i\right)\right]$$+\Omega\left(f_k\right)+C$         (5)

where, $\mathrm{g}_{\mathrm{i}}$ and $\mathrm{h}_{\mathrm{i}}$are defined in Eq. (6) and Eq. (7) as:

$g_i=\partial_{\hat{y}_i(t-1)} l\left(y_i, \hat{y}_i^{(t-1)}\right)$             (6)

$h_i=\partial_{\hat{y}_i^{(t-1)}}^2 l\left(y_i, \hat{y}_i^{(t-1)}\right)$      (7)

Eq. (8) termed simplifying form after removing the constant:

$\sum_{i=1}^n\left[g_i f_t\left(x_i\right)+\frac{1}{2} h_i f_t^2\left(x_i\right)\right]+\Omega\left(f_k\right)$             (8)

Eq. (9) termed the definition of the model:

$f\left(x_i\right)=w_{q(x)}, w \in R^T, R^d \rightarrow\{1,2, \ldots, T\}$          (9)

Here, w is the vector of scores.

The regularization term is then defined in Eq. (10) as:

$\Omega(f)=\gamma T+\frac{1}{2} \lambda \sum_{j=1}^T w_j{ }^2$       (10)

Now, our objective function showed in Eq. (11) and Eq. (12) as:

$o b j^t=\sum_{j=1}^T\left[\left(\sum_{i \in I_J} g_i\right) w_j+\frac{1}{2}\left(\sum_{i \in I_J} h_i+\lambda\right) w_j^2\right]+\gamma T$      (11)

Now, we simplify the above expression:

$o b j^t=\sum_{j=1}^T\left[G_j w_j+\frac{1}{2}\left(H_j+\lambda\right) w_j^2\right]+\gamma T$      (12)

where, ${{G}_{j}}=\sum\limits_{i\in {{I}_{J}}}{{{g}_{i}}}$ , ${{H}_{j}}=\sum\limits_{i\in {{I}_{J}}}{{{h}_{i}}}$.

Finally from below Eqns.(13)-(15), we get best objective reduction value from the best w_jfor a given structure q(x), where W_j ‘s are independent of each other,

$w_j^*=-\frac{G_j}{H_j+\lambda}$        (13)

$o b j *=-\frac{1}{2} \sum_{j=1}^T \frac{G_j^2}{H_j+\lambda}+\gamma T$        (14)

Gain $*=\frac{1}{2}\left[\frac{G_L^2}{H_L+\lambda}+\frac{G_R^2}{H_R+\lambda}-\frac{\left(G_L+G_R\right)^2}{H_L+H_R+\lambda}\right]-\gamma$         (15)

XGBoost is a classifier with a large number of hyper-parameters. To build a model with these parameter values are so crucial. Therefore, it is necessary to carefully select the parameter values. However, there is no theoretical method to guide the choice of parameters. So with the experience we selected some parameters as a crucial parameters with the following values to build our proposed model. Theyare:

1. subsample = 0.8

The default subsample value is 1. It is used to indicate the part of observations to be randomly selected samples of the decision tree model. Keeping a lower value of “subsample” prevents the model from overfitting, but making these values too small would lead to the underfitting of the model.

2. learning rate (learning_rate) = 0.02

Learning rate is the rate at which the gradient descent algorithm moves towards the minima. We generally want to keep the learning rate high enough that we can reach the minima at a reasonable passage of time, but at the same time we also want to keep it low enough that it doesn’t oscillate around the minima. It controls the speed at which the weights and biases of each network are updated during the training process.

3. maximum depth of decision tree (max_depth) = 12

The default maximum depth value is kept as 6. It is generally used to tackle the problem of over-fitting of the machine learning model. And it is not good as depth increases the decision trees often imply that the model has started to overfit on the dataset and it acts as a hard stop on the tree build process.

4. evaluation metric (eval_metric) = auc

It is the evaluation metric that the user wants to be used by the XGBoost model for the validation dataset. Some of the available values are root mean square error (rmse), negative log-likelihood (logloss), multiclass logloss (mlogloss), area under the curve (auc), mean absolute error (mae) and multiclass classification error rate (merror).