A Machine Learning Approach to Malware Detection Using Application Programming Interface Calls (MDAPI)

Thanks to its many capabilities, the Internet has become a concept that not only people but also all kinds of institutions and organizations in the public and private sector depend on. With the dependence of all kinds of institutions and organizations (including governments, armies and civilian companies and institutions) on the internet infrastructure, data production speed and capacity have increased significantly. Technology dependency has expanded tremendously and is likely to increase exponentially in the years to come.

While cyber space has increased its impact, cyber attackers have intensified their efforts accordingly. Whether for ransom, insider threats, politics, competition, cyber warfare, anger, or any other reason, cyber attackers find different ways to damage, stop, alter or monitor systems and devices. These methods are mostly implemented through malicious software that causes ever-increasing prices. As time passes, cyber security and dealing with malware will always be of significant importance in information systems.

While some losses can be expressed in billions of dollars, even these levels are insufficient to express some types of loss. As disrupting education systems leads to cessation of education, destroying a forensic data base leads to injustice and freeing criminals, altering sanitary information contributes to inaccurate examinations and possibly deaths; by infiltrating the military systems of a superpower nation, results similar to those that could be achieved by an all-out war will be easily and directly achieved.

After all these disaster scenarios, there are things to overcome them. The first step to protecting systems and ensuring proper cyber security requires effective malware protection. Malware can vary in size from only a few KB to GBs, as well as differ in characteristics, type, function and target. Malware can disable your computer, monitor your keyboard and mouse movements and clicks. It can also steal your private and vital information such as IBAN number, bank card details, and personal secrets, use your actions as part of big data and even make your device a zombie or crypto-currency mining bot, resulting in illegal use. As you can see, malware can harm any person, system, organization or institution, including armies, governments, and intelligence agencies.

Along with the increase and diversification of potential dangers in cyber space, there are many developments in cyber damages studies. As malicious black hat hackers find new ways to break into systems and good white hat hackers try new ways to eliminate and reveal their covert actions, this war will forever continue. Additionally, malicious software is examined, and it is found that it is largely automated. This is done by examining and reproducing previously written malicious software in study [1].

Today, a developer who knows any programming language, including cyber security libraries, regardless of their knowledge of cyber security issues, can work on code available on many code repository sites such as GitHub, GitBucket, BitBucket, and Launchpad. Even if this code was previously included in the antivirus's database and was considered malware, it can generate new malware by changing the names of a few variables or functions. This tweaked software basically does the same job as the previous software. However, since its content has undergone minor changes to deceive antivirus systems, it will not be recognized by antivirus and will be treated as another piece of software. Thus, new malware can be produced without much effort. This is why a novel approach is needed to provide an efficient and automated way to deal with all these dangers. The greatest requirement of this system is not only to prevent it by making inferences based on the information given to it, but also to determine the patterns of operations performed on the devices and to comment on situations that have not been reported to it before as a result of these patterns.

The research contributes to the literature by providing a solution that enables the detection of malicious software universally. This is done by eliminating the weaknesses of conventional malware combat methods, which are still insufficient. The main idea behind the research will be explained later. By explaining what traditional methods are, why they are insufficient today, and by discussing what could be the most efficient way to deal with malware while maintaining simplicity and speed, a solution proposal will be presented.

In the field of cyber security, machine learning methods such as K-Nearest Neighbor (KNN), Naive Bayes Classifier, Decision Tree and Support Vector Machine (SVM) have proven to be indispensable tools in revealing malicious intentions of heterogeneous structures [2]. There were many studies that produced different solutions to detect and eliminate malware [3-7].

Traditional methods such as manual analysis, antivirus software are insufficient to deal with malware today. Because of information technologies, malware development has come a long way and malicious software numbers have increased tremendously. There are also more resources than ever before for software and malware development. Thus, anyone can easily find a malware sample on the Internet and modify it more or less to produce new malware that cannot be detected through antivirus software. In other words, antivirus tools keep the hash values of harmful files in their database. They determine whether the file is harmful by looking at whether the file's hash value is registered as malicious in the database. However, the file hash value can be changed without affecting the software. In theory, there can be an infinite number of software that does the same job.

The biggest challenges are using features that can only be verified in a single dataset or machine method, such as deleting, adding, and modifying (heterogeneous) files without any time/design/hardware/cost effectiveness, tracking registry records, and running processes.

It is clear that to detect whether software is malicious, a stable and robust solution should be developed. This solution does not depend on some basic rules and does not decrease its effectiveness from case to case. Machine learning approaches deliver what we expect. They adhere to the rules, but also capture malware behavior patterns.

Using features that consist of only API calls (homogeneous) and that can be verified in many logically different ways, not just in a dataset or machine method, ensuring time/design/hardware/cost effectiveness is actually our proposed solution.

Table 1. The datasets used in the study

3.3 Preparation of dataset and feature extraction

For our purposes, some work needs to be done to get the first dataset ready for implementation in the first phase. Preparation and operations on an Ubuntu 18.04 LTS and 12GB RAM capable machine using Linux Bash scripts to process the data are as follows:

• Since data is presented in double-quoted strings, all starting and ending double quotes have been removed so that the data is clean strings to work on.

• Due to their indifference to the learning process, 17268 pieces of software, of which 14131 are malicious and 3137 benign were retained. In addition, 5878 unlabeled software was discarded.

• Two groups of API calls have been determined for the feature extraction phase. The first group is “API calls that occur in at least one malicious software but not in any benign software” (referred to as “Supposedly-Malicious API calls” in the rest of the study). The second group was “API calls that occurred in at least one benign software but were not involved in any malicious software” (which are to be referred to as Supposedly-Benign API calls in the rest of the study). But surprisingly, the first group, referred to as the Supposedly-Malicious group had 599 API calls, while the second group called the Supposedly-Benign group, had only 5 API calls. (The number of different API calls in the whole dataset is 1165). Considering the detailed study during the feature selection phase and the simplicity approach based on the research, 604 features (604=599 extracted from malware+4 extracted from benign software) were considered too many. In addition, it was evaluated that the selection of features only from malicious or only benign software would turn our machine learning-based work into a linear regression, and that this would be the result of a fixed algorithm, not machine learning, and as a result of these facts, other solutions were sought.

• As part of our search for a more suitable solution, we take a simpler approach. For this, it was thought to detect API calls dependent upon other API calls. The term “dependent” here means that an API call is included in the dataset only with another API call. This is not in instances where the other API call doesn’t happen. We thought we could simplify the features we extracted in this way. All API calls that have an impact on another API call have been detected. With this approach, the number of “Supposedly-Malicious API calls” was reduced from 599 to 387 and the number of “Supposedly-Benign API calls” from 5 to 4 (thus the total number of individual API calls was 391). However, the number 391 was considered too much for a simple and effective solution we needed.

• It is evaluated that some progress has been made, but there is still room for improvement. For “Supposedly-Malicious API calls” and “Unclear API calls” (API calls that cannot be grouped into the two previously mentioned groups-i.e., API calls that are involved in both at least one malware and at least one benign software), the most frequent number of occurrences should be considered. For “Supposedly-Benign API calls” nothing will be done since the number is 4 and it’s already low enough.

• “Supposedly-Malicious API calls” are listed in descending order of the number of occurrences in the software. API calls that occurred less than 85 (which is a heuristic criterion we determine based on the number of features we want to extract) in the entire dataset were finally identified as 44 “Supposedly-Malicious API calls”.

• “Uncertain API calls” are listed in descending order of the number of passes in the software examples. A total of 16 were determined for “Uncertain API calls”.

• Finally, the feature selection was completed with 44 Supposedly-Malicious, 4 Supposedly-Benign, and 16 Uncertain API calls, with a total of 64 features. In this way, a problem such as linearity mentioned in the previous articles was completely overcome. In this way, API calls (designed by us) were separated based on their characteristics and the most frequently used ones were selected, and steps were taken towards an effective and simple application.

• Considering the inequality of malicious and benign samples in the dataset, it was evaluated that the number of malicious samples (10994) should be equal to the number of benign samples (3137) to avoid an imbalance-bias problem. Although this will solve the imbalance-bias problem, it will cause the loss of more than half of the data in the dataset.

The accuracy rate obtained with 64 features selected as described above was 81.06%. When this success rate is examined in detail, it seems quite low compared to this detailed feature preparation process, which we can call the “Malicious/Benign Character API calls Approach”. Therefore, it was considered that another feature selection approach, which we can describe as the “Most Used API Calls Approach”, which deals with the number of API calls in the dataset, may be more useful. Accordingly, the number of API calls in the dataset was computed. All API calls were ranked in descending order of the total number of occurrences, which we consider their possible contribution to learning. And the numbers 10, 20, 40, 60, 80, 100, 200, 300, and 400 were chosen (intuitively) to see how much the first few "mostly occurring” API calls contributed to the accuracy rate. The accuracy rates, which we found insufficient in the “Malicious/Benign Character API Calls Approach”, increased significantly with the “Most Used API Calls Approach". When the simple feature extraction process was evaluated, it was more than satisfactory. Further evaluation, comparison and information on the “Most Used API Calls Approach" will be detailed in Section 4.

3.4 Representation of features

There are lots of researches that use API calls in various ways. Using API call frequencies [9, 17] using API call frequencies when similar API calls are grouped [11], using API calls executed in the first string of the software [18], the order in which API calls occur while running a software [5] and even the number of failures, successes and total API call frequency [8] studies have been conducted. However, our study will only deal with whether an API call is called and executed, keeping simplicity in mind.

A notation called “Bag of Words” will be used to represent the selected features, which seem most appropriate to the situation. To create the structure in Figure 1 for this demonstration, the following preparations will need to be performed.

Let’s call the input matrix “A”. If x^thsoftware contains y^th feature (API call), then A[x][y]=1. If x^th software does not contain y^th feature (API call), then A[x][y]=0. All the 0 and 1 values given in Figure 1 are randomly chosen examples based on this explanation. If we interpret the shape based on what we have explained here, 1st software includes 1^st API call (because A[0][0]=1), but does not include 2^nd API call (because A[0][1]=0).

A[6274][64] is a matrix with 6274 rows (derived from the total software count of 3137 malicious and 3137 benign examples) and 64 columns (extracted from API calls, designated as 44 Supposedly-Malicious, 4 Supposedly-Benign and 16 Uncertain API calls).

We will have another matrix for the output, called “B”. If x^th software is malicious, it will be expected to be B[x]=1, if x^th software is benign, then B[x]=0. The B[6274][1] matrix consists of 6274 rows (derived from the total number of software (3137 malicious and 3137 benign samples) and only 1 column (derived from a single binary result, which will determine whether the software is malicious or not).

19-1.png

Figure 1. Implementation of the “Bag of Words” used in our study

3.5 Implementation

The input size of the artificial neural network derived from the feature size was chosen as 64 and then, 10, 20, 40, 60, 80, 100, 200, 300 and 400. The reasons will be explained in more detail in Section 4, as they were explained in the previous sections. Derived from the binary result (the closer to 0 the more likely it is to be benign and the closer to 1 the more likely it is to be malicious), the network output size is only 1. Intuitively, the number of layers according to the number of features was calculated as given in Table 2. The hidden layer size and the number of neurons were determined intuitively by considering the rule of thumb “two-thirds majority”, which determines the size of a layer, at the ratio of 2/3 of the size of previous layer.

In addition to the varying input, output, hidden layer size and number of parameters, there are some parameters that we consider to give optimal results in all the specified architectures. When detailing these, learning rate is 0.001. In order to overcome the saturation problem and the output value to be stable and logically oscillating between 0 and 1, sigmoid was chosen as the final activation function used in the last 336 (output) layer. All activation functions of all layers except the last layer are ReLU (Rectified Linear Unit).

The formulas of these two functions are given in Eq. (1), Eq. (2).

$\operatorname{Sigmoid}(x)=1 /\left(1+e^{-x}\right)$          (1)

$\operatorname{ReLU}(x)=\max (\mathbf{0}, \mathbf{x})$            (2)

Afterwards, an intuitive batch size adjusted to 12, 20, and 50 epochs is sufficient for the network to learn. However, 12 is the best value in the study. The values are optimized using “Binary Cross Entropy” and finally the results were verified by shuffling and using 12-fold cross validation to obtain more accurate results.

When the 100-75-50-30-20-12-8-1 architecture given in Table 2 is evaluated as an example, it will be seen that we have 6 hidden layers and that there are that many neurons in these layers, respectively. In addition, it should be noted that in the studies carried out, it was observed that the number of layers, like the number of features, increases accuracy to a certain extent. However, after a certain point, it causes more hardware and time costs than the contribution to success rates. It should be noted that the values with optimal results in the time-utility dilemma are presented above. The most accurate accuracy value obtained was 90.34%.

Table 2. Layer sizes representing neuron sizes from input (Left) to output (Right)