Predicting Student Performance Using Mental Health and Linguistic Attributes with Deep Learning

Learning achievement is a crucial metric for assessing learning performance. Research has revealed that students who achieve poor grades are more likely to experience stress and hopelessness, and their probability of self-immolation [1] is significantly greater than that of students who perform well academically [2, 3]. Academies must recognize students at risk promptly to motivate their academic performance and provide early assistance [4, 5]. Teachers can recognize students who require extra classes, assignments, or encouragement to prevent undesirable behaviors [6, 7]. Educational data mining has been used to predict a student's learning performance [8].

Early identification of students with poor academic performance creates opportunities for research to improve those students' academic performance and guarantee their achievement [9]. High school students are the ideal target group for this research field due to their academic achievement having an impact on their future educational opportunities [10]. The dataset gathered from high school students contains socioeconomic and educational data about the students [11]. However, determining the academic achievement of all students is an extremely challenging task due to the large number of students and inadequate resources [12]. To solve this problem, numerous scholars have claimed in the literature that different types of ML models have been utilized to predict students' academic success [13-16].

Yağcı [17] presented a novel system based on ML models to predict students' final exam grades by considering their midterm exam grades as the source data. A Multi-Layer Perceptron Neural Network (MLPNN) was used by Hamadneh et al. [18] to predict students’ performance in blended learning and reduce academic failure rates. In the study [19], the authors aimed to predict a student's performance and provide support for academic guidance using a supervised ML.

Mostly, these studies demonstrated the strong correlation between different behavior factors and academic performance to offer guidance for teachers and enhance students’ academic achievements. But the manual extraction of attributes has difficulty while considering large-scale datasets. The quality and number of attributes directly influence the model training. They cannot completely define the temporal dependency of the time-series data of students. Also, a correlation among multi-source data needs to be further extracted. So, Li et al. [20] developed an end-to-end DNN model that automatically extracts attributes from students’ multi-source data to predict academic achievement. But they did not consider other factors such as the physiological factors of students to predict their academic achievements.

Hence, in this paper, the SADDL model is proposed by using diverse attributes of students, including physiological attributes (mental health and mood changes from the previous day), demographic attributes, and academic attributes. The main contributions of this study are the following:

• Initially, the students’ academic and demographic attributes are collected. Also, API services of Twitter, Facebook and Instagram are used to get the information shared by students. The collected information is pre-processed using NLP techniques.
• Then, the feature extraction is performed using the LDA scheme, which extracts the attributes regarding mental health and mood changes of students. These attributes, along with others, are learned by the SADDL model, which ensembles the LSTM, DCNN and MLP models.
• The LSTM network is used to extract the time-series attributes of all data variants. The temporal dependencies are converted into a feature tensor and fed to the multidimensional DCNN to learn the correlation between multiple attributes.
• Then, a unified attribute vector is obtained by fusing the temporal dependencies, correlation attributes and student demographic attributes.
• Further, the MLP classifier is used to predict students’ academic performance. Thus, this SADDL model enhances the efficiency of predicting students who are at risk in their academics.

The remaining article is prepared as follows: Section 2 presents the studies associated with the prediction of student learning achievement using ML models. Section 3 discusses the proposed algorithm and Section 4 shows its performance compared to the existing models. Section 5 concludes this study and suggests future enhancements.

This section explains the SADDL model for student performance prediction, which involves four major modules, as portrayed in Figure 1: data acquisition, pre-processing, feature extraction, and deploying a deep learner for prediction.

1.png

Figure 1. Block diagram of the proposed study

3.1 Data acquisition

In this study, the student records from the government and self-financed engineering colleges around Coimbatore, Tamil Nadu, are collected as a dataset. These records include academic and demographic attributes of the students who studied in the period from June 2022 to December 2022. The academic attributes include student number, course name, college category (i.e., government or self-financed), grades in each subject, study materials (i.e., lecture notes & book materials), way of teaching, the number of students in a course, an allowance of smartphones, etc. The demographic attributes are students’ name, age, sex, home place (i.e., rural, urban, or semi-urban), family category (i.e., nuclear or joint), career, academic skills of family aspirants, the guidance of parents for homework, the number of friends, time spent on watching TV, internet access at home, etc. This dataset contains 126 attributes (i.e., columns), 5000 instances (rows) and 1 class attribute (i.e., either distinction, pass, high-distinction, or fail). After obtaining this dataset, preprocessing is used to convert the textual/categorical data into numeric form using one-hot encoding. Using this dataset, the academic and demographic attributes of the students are directly extracted.

In addition to this dataset, the number of conversations and posts shared by the students about their academic experiences on Twitter, Facebook and Instagram are acquired as a students’ social media dataset. This dataset is preprocessed and used to extract the students’ physiological attributes, which are described in the following section.

3.2 Students’ physiological attribute extraction

3.2.1 Pre-processing

The post and conversation information in the students’ social media dataset is preprocessed using the following NLP techniques.

• Eliminate punctuation, emojis, unique typescripts and excess spacing.
• Remove duplicate characters from the string (e.g., “Gooood Morrrninggg” becomes “Good Morning”, etc.).
• Delete numeric values and translate words to lowercase.
• Enlarge contractions (e.g., “I’m” becomes “I am”, “I’ve” becomes “I have”, “can’t” becomes “cannot”, and so on).
• Eliminate stopwords (e.g., the, an, let, with, and so on) using the NLTK dictionary that includes 40 stop words.
• Lemmatize words using the WordNet to translate the words to their root form (e.g., “studied” becomes “study”, “drawn” becomes “draw”, and so on). This study uses lemmatization rather than stemming since it has higher accuracy for context analysis.

These preprocessing steps can reduce the total amount of information in the student’s social media dataset by removing redundant data. This preprocessed dataset is used to extract the students’ physiological attributes.

3.2.2 LDA feature extraction

LDA is the other feature extraction scheme utilized to create topic models by allocating a topic of each word after extracting it in a dataset. It is a probabilistic model that extracts latent topics from a group of documents. The major aim is that the student database is defined as an arbitrary combination of latent topics, which are represented by a distribution over words (e.g., physiological terms often used by the students). For a given dataset D containing M documents, with document d having N_d words (d∈1, …, M), the LDA models D based on the below generation procedure.

• Choose a multinomial distribution φ_t for the topic t, (t∈{1, …, T}) from a Dirichlet distribution with a variable β;
• Select a multinomial distribution ϕ_d for d, (d∈{1, …, M}) from a Dirichlet distribution with a variable α;
• For a word W_n, (n∈{1, …, N_d}) in d

            - Choose a topic Z_n from ϕ_d;

            - Select W_n from $\varphi_{Z_n}$;

According to this generation procedure, words in documents are only observed variables, whereas others are latent variables (φ, ϕ) and hyperparameters (α, β). To infer these variables, the probability of the observed data D is calculated and improved by

$P(D \mid \alpha, \beta)$$=\prod_{d=1}^M \int p\left(\phi_d \mid \alpha\right)\left(\prod_{n=1}^{N_d} \sum_{Z_{d n}} p\left(Z_{d n} \mid \phi_d\right) p\left(W_{d n} \mid Z_{d n}, \beta\right)\right) d \phi_d$        (1)

α parameters of the topic Dirichlet prior and the distribution of words over topics, which are obtained from the Dirichlet distribution and provided by β. Here, N denotes the size of vocabulary. The Dirichlet multinomial pair for dataset-level topic distributions (α, β) is taken. The Dirichlet multinomial pair for topic-word distributions is provided by β and φ. Variables ϕ_d are the document-level variables. Z_dn and W_dn are the word-level variables, which are samples for all words in all documents.

Figure 2 illustrates an example of LDA-based physiological feature extraction. It is noted that the LDA provides the words in the topics with their distribution using Dirichlet distribution. A topic is defined as a weighted list of words (i.e., students’ physiological attributes). Thus, the LDA scheme is performed to extract the physiological attributes related to the student’s mental health and mood changes from the student social media database.

2.png

Figure 2. Example of LDA for students’ physiological feature extraction

3.3 SADDL model for prediction

As illustrated in Figure 3, the SADDL-based prediction model comprises four major stages: (i) Input: different attributes of student physiological, academic, and demographic data are the input of the DDL model; (ii) Temporal analysis: the LSTM is applied to independently model each category of the attribute to learn their temporal dependency; (iii) Correlation analysis: a multidimensional DCNN is applied on the tensor converted from the temporal dependency of each data type to learn the correlation among multiple attributes; (iv) Output: the student’s physiological, academic, demographic, temporal, and correlation attributes are fused as the input of the MLP classifier for predicting learning achievements.

3.png

Figure 3. Structure of SADDL for student’s performance prediction

3.3.1 Input

This network model considers the different kinds of student information such as academic background, demographic data and physiological data. Each kind of student’s data is a time-series, i.e., all records have a timestamp, yet different students hold distinct attributes. Here, $X_i=\left(X_{i 1}, \ldots, X_{i j}, \ldots, X_{i N}\right)$ is the N categories of multi-source attributes of student i, $X_{i j}=\left[x_{i j}^1, \ldots, x_{i j}^t, \ldots, x_{i j}^{T_{i j}}\right]$  is the j^th attribute of student $i, x_{i j}^t\left(1 \leq t \leq T_{i j}\right)$  is the vector defining single event record information at period t, including single consumption record, single gateway login record, where T_ij is the length of the j^th attribute of student i.

3.3.2 Temporal analysis of attributes using LSTM network

LSTM is a kind of recurrent network, which is used to process a sequence of values. It can efficiently diminish the complexity of extracting long-term dependencies from long-range sequences. To extract temporal dependency in each attribute from the students’ data, the LSTM is applied.

The LSTM network layer comprises the input gate, forget gate and output gate. The forget gate is used to calculate a degree of forgetting a feature preceded by the current LSTM unit as Eq. (2):

$f_t=\sigma\left(W_f \cdot\left[h_{t-1}, x_t\right]+b_f\right)$         (2)

In Eq. (2), W_f, b_f are the weight vector and bias value of the forget layer, respectively. σ is the sigmoid activation function, x_t is the input feature in the input gate, f_t is the forget gate, and h_t-1 is the result of a previous hidden state.

The input gate is used to determine how much present feature is contained in the image as Eqs. (3), and (4):

$i_t=\sigma\left(W_i \cdot\left[h_{t-1}, x_t\right]+b_i\right)$        (3)

$\tilde{C}_t=\tanh \left(W_C \cdot\left[h_{t-1}, x_t\right]+b_C\right)$                 (4)

Here, W_i, W_c are the weight vector of the input gate and neuron condition vector, respectively. b_i, b_c are the bias values of the input gate and neuron condition vector, respectively. tanh is the hyperbolic tangent activation function, i_t, $\tilde{C}_t$ are the input gate, and the updated new cell state, respectively.

Once the features traverse via the input and forget gates, the LSTM fine-tunes their units to determine the outcome of the current LSTM unit and pass it to the consecutive LSTM unit as Eq. (5):

$C_t=f_t * C_{t-1}+i_t * \tilde{C}_t$       (5)

In Eq. (5), C_t is the current cell state, and C_t-1 is the old cell state. The output gate merges the present input and LSTM unit to compute the result of the present LSTM unit as:

$o_t=\sigma\left(W_o \cdot\left[h_{t-1}, x_t\right]+b_o\right)$      (6)

$h_t=o_t * \tanh \left(C_t\right)$      (7)

In Eqs. (6) and (7), h_t represent the hidden state that serves as the solution of the block over t, o_t is the output gate, W_o and b_o are the weight vector and bias value of the output gate, respectively.

3.3.3 Correlation analysis of attributes using DCNN

Since multi-source performance information is from a similar student, a relationship among various attributes should exist. To learn the correlation among multiple attributes, a tensor approach is applied to convert the temporal dependency vector of every attribute into a 2D DCNN is adopted to extract the correlation among multiple attributes. This DCNN is utilized to mine image characteristics, where an image is defined by a tensor (ω,h,c), where ω, h and c denote the width, height and number of channels, respectively. As depicted in Figure 2, the temporal dependency vectors of N kinds of attributes are converted into a 2D tensor, where ω*h=N, c=M, M is the dimension of the temporal dependency vector. Based on the tensor, the 2D DCNN is performed.

3.3.4 Output or Multi-Layer Perceptron (MLP) classifier

In this study, the learning performance prediction is modeled as a categorization process, the network result is $y \in$$\{0:$ Distinction, 1: Fail, 2: High Distinction, 3: Pass $\}$, and the grading process can be performed as follows:

A student’s academic performance is typically measured by Grade Point Average (GPA) of continuous numerical values. This work defines the prediction of academic performance as the classification task, i.e., estimating whether the student’s performance is a distinction, fail, high distinction, or pass. So, the GPA is split into discrete academic grades. First, each student in the database is ranked based on the GPA in descending manner, then the top k% of students’ achievements such as 85-100% are defined as high distinction and the bottom k% of students’ achievements such as 0-49% are defined as failure. The other student’s achievements such as 75-84% are defined as distinction, and 50-64% are defined as pass.

The MLP classifier is trained by fusing the students’ academic, demographic, physiological, temporal, correlation and demographic attributes to provide the learning grade. In the MLP classifier (as shown in Figure 4), the input layer has n neurons representing the number of attributes. The second layer is the hidden layer, in which the number of neurons should be less than twice the size of the input layer. The Rectified Linear Unit (ReLU) activation function is used for both input and hidden layers. The final layer is the output layer, which comprises the softmax activation for final prediction.

4.png

Figure 4. Structure of MLP classifier

The Softmax (P(y,b)) is a probability of attributes belonging to a given grade y. It is determined by Eq. (8).

$P(y, b)=\frac{P(b, y) \times P(y)}{\sum_{N=1}^y(N) \times P(b, N)}$        (8)

In Eq. (8), P(y) is grade probability, y is the total number of grades. Eq. (8) is rewritten as Eq. (9):

softmax $=P(y, b)=\frac{e^{\beta^y[b]}}{\sum_{N=1}^y e^{\beta^N[b]}}$                      (9)

Where $\beta^y[b]=\ln [P(b, y) \times P(y)]$                  (10)

Figure 5 presents a flow chart of the SADDL model for students’ performance prediction, which helps to understand how the model works.

5.png

Figure 5. Flow chart of SADDL model for students’ performance prediction

3.3.5 Model configuration

Table 1 presents an entire configuration of the SADDL model. The initial layer denotes the input containing N kinds of attributes, where T_i and F_i are the data sequence length and attribute number of i^th student. The second layer applies the LSTM on all kinds of attributes and provides a vector having 32 attributes. In the 3rd, 4th and 5th layers, the concatenation, reshape and permute layers are used to translate the temporal dependency of N types of attributes into a tensor of (ω,h,32). Because there are only some kinds of physiological behaviors, more 2D convolutional layers may tend to overfit; so, only 2 convolutional layers are configured in the 6th and 7th layers, with the kernel number of 32 and 64, respectively. The kernel sizes of (2,2) and the step sizes of (1,1), as well as, the 6th layer use filling mode to maintain the tensor size unchanged.

The tenth layer considers the students’ academic, demographic and physiological attributes as input. In the 11th layer, such attributes are concatenated with the temporal dependency and correlation attributes, where L₇, L₈ and L₁ denote the lengths of the output vectors in the 7th, 8th and first layers, respectively. The concatenated attribute vector is given as input to the MLP classifier in the 12th layer, where 4 denotes the grades of academic performance and the activation function is softmax. The dropout layer is assigned before the MLP layer to prevent overfitting and the dropout rate is set to 0.5.

3.3.6 Handling class imbalance problem using weighted cross-entropy error

To handle class imbalance problems, under-sampling, over-sampling and weighted loss functions are available. On the other hand, under-sampling may reduce the number of samples, which degrades the efficiency of SADDL training and over-sampling is computationally inefficient for high-dimensional datasets. Compared to these schemes, the weighted loss function needs fewer computing resources to handle class imbalance problems. As a result, the weighted cross-entropy error is considered as the loss function in this paper. The weighted cross-entropy error is defined as follows:

loss $=\frac{1}{N} \sum_{k=1}^N \sum_{c=1}^M w_c y_c^k \log \left(p_c^k\right)$       (11)

Where $w_C=\frac{N}{M * N_C}$        (12)

In Eqs. (11) and (12), w_c is the weight of label c, N indicates the total amount of student data instances, N_c denotes the amount of student data instances belonging to c, M denotes the amount of class labels, $y_c^k$  indicates the actual grade of k^th data instances belonging to c, and $p_i^k$ defines the estimated grade probability.

3.3.7 Preventing overfitting issue

A huge gap between the number of model parameters and the number of student data made the SADDL model prone to overfitting. To prevent overfitting, this study applies early stipping and dropout during the model training. If the error value is less than 0.001 for 40 consecutive epochs, the training is terminated and the parameter setting of dropout is given in Table 1.

Table 1. Entire configuration of SADDL model