Database Manipulation Security Risk Detection (DMSRD) Based on GSP and Markov Chain

Table 2. The user sequence of states is sorted by transaction date

For example, the ‘AA’ item is found in two sequence transactions SID #1 (<A,B,(AB),B,C>), and #2(<(AB), B, B,(AC), C>), while absent in SID #3 (<(AB), B, B, B, B>). The fifth step is to create all the possible three-item candidate nodes. The passed support criteria for two-item nodes will be used to generate the three-item candidate node. For example, the two items that passed the support criteria ‘AB’ will be separated into the first and second items as ‘A’ and ‘B’. After passing the support criteria node, the last item, ‘B’, will be considered to combine with the previous two items. From Table 6, for example, one node of the two items passed the support criteria, ‘AB’. Thus, the last item ‘B’ will combine with ‘BB’ to form the new two-item candidate node. The 2-item ‘BB’ is chosen to be put in the back of ‘B’ since it passed the support criteria, Table 6. Then, this new two-item candidate node will be preceded by the first letter ‘A’ to create the candidate three-item node, ‘ABB’. Each candidate's item node must be checked for validity (pruning). For example, ‘ABB’ is decomposed to ‘AB’, ‘BB’, and ‘AB’. These parts are measured to determine whether the support value is equal. In this case, the support values of ‘AB’=3, ‘BB’=3, and ‘AB’=3, thus all parts have equal support numbers. Therefore, this three-item candidate passed the validity test. If the validity check reveals that they are not equal, then this candidate will be pruned or purged. After that, the three-item candidate node will be checked against all the sequence transactions of all SIDs to determine whether the three-item candidate nodes can pass the support criteria, as shown in Table 8.

Table 8. The possible two-item node with a support number result

Table 9. The possible four items node with a support number result

The sixth step is to create all the possible four-item candidate nodes. Since three items support the passed criteria, ‘ABB’, ‘BBB’, and ‘(AB) B’, the fourth candidate node must be further defined. The solution for the four-item generating is to follow the same steps as step five. The difference between this step and step five is that the source three-item node must be separated from the last two items to be used as the predecessor of the three-item node generation. Then, these three generated nodes will be preceded by the first letter. The validity of each part of the four-item candidate node is checked. The pass validity checking four-item candidate node will be checked for its support number if it meets the support criteria. The result of the sixth step calculation is presented in Table 9. No four-item candidate node passes the support criteria, even though two of them pass the validity check, “ABBB” and “BBBB”. Therefore, there is no need to generate the five-item candidate node since no four-item nodes meet the support criteria.

In summary, Figure 1 illustrates an n-item node relationship structure that passed the support criteria tree.

1.png

Figure 1. The n-item node relationship tree of GSP analysis (support number criteria ≥3)

2.4 Markov chains and a research practical example

A) Markov chains concept

If Xt is the process as shown in Figure 2, for example, several states ‘i’ at time ‘t’.The process of state ‘i’ at time t, Xt, is presented as Xt=i. The process space {X0, X1,….Xn} represents the process Xt that varies from time t=0 to n, where ‘n’ is the size of the process space. If process Xt depends on only process Xt-1, but not on the past Xt-2,…Xt0, then these processes are called Markov Chains [10]. The transition state [11] space from time ‘t’ to time ‘t+1’, which processes x_t to x_t+1, is presented with its probability in the transition matrix. P=(p_ij), i and j =1,…, n is the transition matrix from each state at process Xt to all the states in Xt+1. The total probability of each state at row i to every state in columns 1, 2, … n has a total value of 1.0.

2.png

Figure 2. Markov chain concept

B) Markov chain - practical example

From the transaction of all sequence IDs in Table 7 and the sequence of items that occurred, Figure 1 is used to obtain the frequency of the item relationship. All the frequencies of item relationships are presented in Table 10.

Table 10. The frequency of item relationships from a particular row to all items in all columns

For example, frequency counting from item ‘A’ to ‘A’ can be obtained from the number of occurrences that start with ‘A’, then suddenly followed by item ‘A’. Since this situation did not occur, frequency A-A, thus f_A-A, is zero. In the following example, the frequency counting from item ‘BB’ to ‘B’ can be obtained from the number of occurrences that start with ‘B’, then suddenly followed by item ‘BB’. The result of counting is five given source and destination item patterns matching. A detailed explanation of the counting of matching item relationships is presented in Table 11.

Table 11. Example of item results counting

2.5 Likelihood of a hidden Markov chain on damage occurring notification (alert)

The frequency of item relationships from a particular row to all items in all columns in Table 10 is transformed into a transition matrix as illustrated in Table 12.

Table 12. Transition matrix of sample data

The transition matrix in the Markov chain is shown in Figure 3. The State or item sequence transaction in Table 7 showed the items or states of each user, which were categorized by day. During the phase of learning about security risks, the DB administrator (DBA) must monitor whether any undesirable incidents are occurring. On the day any incident occurs, DBA will collect all the transaction records for that date. This evidence defines its type of risk. The findings will be recorded to gain knowledge of security risk. The likelihood of the risk is determined based on the undesirable incident's effective items. For example, if there are two undesirable incidents occurred in date # 1, #2 and #3 with risk types ‘1’, and ‘2’, thus the significant item and likelihood which related to the incident ‘1’, ‘2’, and ‘1’ are (‘A’=0.33, ‘B’=0.33, ‘(AB)’=0.33), (‘B’=1.0), and(‘A’=0.33, ‘B’=0.33, ‘(AB)’=0.33), in respectively, as shown in Table 13, and Figure 3.

Table 13. The related item that occurred on the same date as the security incident

3.png

Figure 3. Hidden Markov chain of sample data

Only item ‘B’, which is simultaneously used by three users, causes the probability of risk occurring to be ‘2’.

The stable state vector is calculated from its transition matrix. This vector will provide information about the stability or steady likelihood. This vector provides the amount of each state at the stable state probability if the total state or item is known. Assume that there are 100 items in the scope of the calculation; Therefore, the amount of each state at the stable state is shown in Table 14.

Table 14. The amount of each item is based on the steady probability and the total amount. of state (e.g., ‘n’=100)

If DBA has defined that the red line of undesirable incident will do alert at eighty percent of the number of items occurring (N), therefore: Rule-1, and Rule-2 for risk type 1, 2 prevention are: Rule-1: If [number of item ‘A’ > (0.80*NA*SPA) .and. number of ‘B’> (0.80*NB*SPB). and. number of ‘(AB)’>0.80*NAB*SPAB)] then print “***Alert-Risk-1***”. Rule-2: If [amount of item ‘B’ > (0.80*NB*SPB)] then print “***Alert-Risk-2***”. Where, for example, NA is the number of item ‘A’ under the stable probability of item ‘A’, SPA.

2.6 Related research

A) ISO-27001

ISO-27001 [12] is a popular widely recognize standard that provides the framework for an information security management system. These standards are applied to enhance the security of the organization’s database. The ISO-27001 standard features cover security risk management, access control, data integrity, monitoring and logging, security awareness and training, etc.

B) Efficient storage and querying of sequential patterns in database systems

The sequential patterns generated from an original database become a massive collection of sequential patterns [13]. Many pattern discovery tools, such as DB-Miner, can generate an enormous amount of information. The various views of these patterns provide necessary details for capturing insights. The methods of storing these whole sequential patterns are proposed by effective pattern coding.

C) Trust factor-based analysis of user behavior using sequential pattern mining for detecting intrusive transactions in databases

Malicious access and modifications to databases by insider employees [14] are tough to detect and prevent. Some have attempted to escalate their privileges, while many have exploited SQL commands for database attacks. The pattern of the insider threat can be derived from the SQL commands used and database log examinations. The research results present the trust factor of the SQL command used by the user. Some users who deviate from the trust factor will be assumed to be an insider threat.

D) Database security and encryption

A Survey Study [15]. The user’s authorization always controls the integrity of database manipulation. Some intruders try to escalate their approval or secretly use another user's trustee rights. A simple solution to this problem is to add user authentication verification and database encryption. This method can discriminate between those who cause the database fault or failure.

E) Database security threats and how to mitigate them

A database security breach [16] can involve malicious access by an insider or an outsider organization. A database administrator takes responsibility for the database. He must have experience with database command vulnerabilities or dangers so that the failures or faults can be detected and corrected, ROLLBACK. Nevertheless, security attacks come from many sources. The manager must be concerned about the database security policy and tightly govern it to reduce and control vulnerabilities.

F) Advancing database security: A comprehensive systematic mapping

In total, 20 challenges related to database security [17] were identified. Research results show that ‘‘weak authorization system, ‘‘weak access control, ‘‘privacy issues/data leakage, ‘‘lack of NOP security, and ‘‘database attacks’’ are the most frequently cited critical challenges. Further analyses were performed to highlight the different difficulties associated with various phases of the software development lifecycle, venues of publications, types of database attacks, and active research institutes/universities researching database security. Organizations should implement adequate mitigation strategies to address the identified database challenges.

G) Sequential pattern analysis for event-based intrusion detection

Many events utilize computer resources, including databases, peripherals, and networks. Some events are intended to access the organization's database maliciously, etc. [18]. The behavior of user events (commands used) creates their processing patterns. The pattern with class attributes is composed to create a cluster of normal behavior users and distinguish them from unnatural users. The generalized sequential pattern (GSP) is used to seek SQL behavior patterns.

H) Database intrusion detection using role and user level sequential pattern mining and fuzzy clustering

The behavior of each database user [19] is examined to determine whether they are working correctly under the assigned rights. If he tries to work contrary to the assigned authority, this user will be flagged for careful observation. The use of SQL commands could cause an undesirable incident. Thus, the sequential pattern of his work with the data processing result examined will support the criteria for whether the user intends to access the data maliciously.

I) SQL injection hacking

SQL injection is a type of database intrusion that compromises sensitive information assets. There are some kinds of dangerous SQL commands [20]. These commands will be sent or injected to execute in the DBMS to retrieve some desirable information. The vulnerability of the DBMS will be detected before the intruder considers the capable SQL command to attack the DBMS.

J) Maintainability measurement

The system broke down due to an interruption in business operations. The clash system has to be repaired and restored to the normal state. The DBAs have to be concerned about at least two indexes. The first item is Mean time between failure, also known as MTBF [21]. The MTBF is used to measure the average time between system failures during a period. If MTBF is a small number of time (Tmtbf), then further failures shall occur on about T unit of time from the current time, e.g., T₀, as To+Tmtbf. The MTBF can be obtained from the formula: MTBF=Total operation time/ number of system breakdowns. The second item is Mean time to repair, aka MTTR [22]. The MMTR is used to measure the average repair time. With the large number of MTTR, the system will resume normal working, which will take a long time to repair, Tmttr. The formula to obtain the MTTR: MTTR(Tmttr)=Total number of downtimes/total number of system breakdowns. The DBAs must be concerned about the root cause of failures, problem-solving, gaining the best practice solution to prevent, and proactively preventing these possible causes of failures.

4.1 User’s sequence of database manipulation, sequential pattern analysis

1) Transition matrix

The summary of the n-item sequential pattern (Figure 5) and the SQL command used to sequence three organizational DB users (Table 13) were used to obtain the transition probability. The transition matrix of the n-item sequential pattern of the sample research pattern is presented in Table 19.

2) Stable transition vector

The transition matrix was calculated for the steady or stable state vector, as shown in Table 20. The stable transition vector will obtain the number of items that occur when the total number of used items is given from actual item counting or prediction.

4.2 Occurred undesirable incident data collection and Markov chain of database manipulation sequential pattern

A DB administrator takes responsibility for database performance and security preservation. In the event of an undesirable security incident, he must retrieve all the SQL commands that were used when the security loss occurred. For example, two undesirable security incidents occurred on day ‘5’, and ‘7’. From Table 24, date # ‘5’, Risk ‘1’ occurred while node ‘2*(CD)’ and ‘1*(DF)’ worked on day # ‘5’. Thus the probability of ‘(CD)’ was 0.66, while ‘(DF)’ was 0.33. On date #7, the Risk ‘2’ happened when n-items ‘E’, ‘DF’, and ‘CD’ were used simultaneously. Thus, the probability of ‘E’ was 0.33, ‘(DF)’ was 0.33, and ‘(CD)’ was 0.33.

Table 24. An undesirable security incident occurred in the sample research

	Day of Week 1-8
User ID	1	2	3	4	5	6	7	8
1	(AF)	B	B	C	(CD) (0.33)	D	E(0.33)	C
2	A	B	E	C	(CD)(0.33)	C	(DF)(0.33)	B
3	A	B	B	B	(DF)(0.33)	E	(CD)(0.33)	C
Occurred damage incident	-	-	-	-	1	-	2	-

1) Undesirable security incident

2) Hidden Markov chain of undesirable security incidents occurring in the sample research

The plain Markov chain was enhanced with the actuator nodes ‘1’ alert and ‘2’ alert. The hidden Markov chain was presented in Figure 6.

6.png

Figure 6. Hidden Markov chain of sample research

Note that some item nodes from the n-item sequential pattern in the GSP tree disappear in the hidden Markov chain because no events match the Markov chain rule of state (2.4), e.g., ‘ABDC’ and ‘ABDD’.

3) Rule of undesirable security incident occurring alert

The result of the Hidden Markov chain and stable transition matrix can be derived the rule as shown in pseudo code (1), and (2).

(1) Rule-1:

IF [number of item ‘DF’ >(0.80*N_DF*0.33) and the number of item ‘CD’ > (0.80*N_CD*0.66), then print “Alert undesirable security incident *1*”.

(2) Rule-2:

IF [number of item ‘DF’ > (0.80*N_DF*0.33) and the number of item ‘CD’ > (0.80*N_CD*0.33) and number of item ‘E’ > (0.80*N_E*.33), then print “Alert undesirable security incident *2*”.

7.png

The data on SQL commands used, from table B, were calculated for all n-item sequential patterns (step #5). The cumulated SID will be used to derive the current sequential pattern of n items (step #6), all of which will be kept in table B. The sequential pattern of n-items will be read from the ‘Damage incident management procedure’ (step #7). The transition probability matrix, stable transition probability vector, and Markov chain were generated. ‘Damage incident management’ procedure read table ‘A’, as shown in Table 26.

DMSRD protocol brief explanation: (two actors, DB users, and DBA)

1: DB users manipulate their authorized data in the organizational database through the application or a direct connection.

2: The DB users’ commands are sent to process their request in the DB. If the trustee rights checking is passed, the requesting command is permitted to run on the organizational DB.

3: The used SQL command and its details are kept in table A.

4: If there is any damage or undesirable security incident, the risk type number of the arising risk will be appended to the column ‘State of DB processing’, for example, ‘1’.

5: The User sequence item transactions (SID) are used to discover the n-item sequential pattern analysis.

6: The discovered n-item sequential patterns are kept in table B.

7: The discovered n-item sequential patterns are sent to ‘Damage incident management’ under the specified time duration. This procedure will derive the Markov chain and stable transition probability vector.

8: The undesirable security incident risk type number (if there are any) is mapped to all users’ n-item sequence transactions. The probability of the effective nodes is calculated. The stable transition matrix (7) and ‘N’ or total number of used items are used to derive the Hidden Markov chain and risk security alert rules.

9: The DBA will use the defined rules of the security incident alerts to detect potential security incidents.

10: The defined rules of security incident alerts are sent to the DBA.

11: If some not-appreciated evident possibilities could happen (10* notification), the DBA will configure some activity to the DB manipulation application, such as delaying or purging some user’s command from the ready queue, etc.

12: The configuration from the DBA will be managed by the DBA under the DBA’s role-based control.

13: The reaction message about the inconvenience of the user’s DB manipulation is sent to the current DB application and to particular DB users.

4.4 Protocol evaluation

The DMSRD protocol was validated for its accuracy in mitigating undesirable security risk incidents on eight additional days, while also demonstrating effectiveness over one-week period. Table 27 presents DBA’s define the risk type.

Table 27. The code of the risk type

The detection accuracy of risk type (‘1, 2, 3, 4, 5, 9’) was approximately 73.68%, as shown in Table 28.

Table 28. Accuracy of undesirable security incident detection

4.5 MTBF and MTTF

The DMSRD proposed protocol is for deployment in the organization for real database manipulation. The experiment was conducted over a four-month period. In the first month of DMSRD use, no defined sequential patterns, a Hidden Markov chain, and a rule for security risk alert. This month's number#1 was the duration of the item sequence, undesirable incident gathering. At the end of the first month of DMSRD use, the rule for an undesirable security incident alert will be created. This month, MTBF and MTTR were the worst numbers. DBAs will consider the combination of SQL commands that cause the system failure. The prevention guideline will be defined. These guidelines will be compiled if the rule generates an alert in the first month. DBAs will use this rule during the second month. At the end of the second month, other combinations of SQL commands may cause the database system to fail. These new changes will be an addition to the previous rule. This repetition training or learning of the latest cause of system failure will continue until there are no new causes of system failure, and the measuring of MTBF and MTTR meets the organizational criteria. The experimental research results for MTBF and MTTR calculations are presented in Table 29.

Table 29. Summary of the maintainability measuring of four months of experiment, seventy-two hours of working per month

The MTBF of the DMSRD protocol value is reduced from twenty hours in the first month to seventy-one hours in the fourth month. Therefore, the DMSRD protocol can well detect the cause of failure. DBAs could prevent frequent undesirable security risk incidents, thereby drastically increasing the mean time between failures. Moreover, the value of MTTR is reduced from four hours to one hour of system repair. The decrease in maintenance is about seventy-five percent.