Hybrid Approach for Automated Answer Scoring Using Semantic Analysis in Long Hindi Text

Using computers to evaluate content that would otherwise be assessed by humans is known as "automated scoring." Modern automated scoring systems owe a great deal to the research conducted in the area of artificial intelligence scoring. AES is the process of assigning grades to student essays written using state-of-the-art computers in a classroom setting. In addition to its use as an NLP tool, it may also be seen as a kind of educational assessment. The principle aim of the research is to develop a novel approach to simplify and automate scoring of Hindi literary works through the use of semantic analysis. Several machine learning methods are acknowledged as promising candidates for achieving this objective. It has been demonstrated that in order to increase accuracy and efficiency, filtering the dataset with an optimizer is crucial. A machine learning model must be trained using the best available data in order to guarantee optimal learning. Therefore, the suggested work has given a creative and high-performance approach to finding extended text in Hindi. The results of this research might be used to the analysis of texts containing educational statistics published in Hindi. The team's efforts are concentrated on lengthy texts written in Hindi. In order to achieve this goal, scientists are investigating several machine learning approaches. In order to get the most out of an optimizer, you need to filter the dataset. The best training data may be used if your machine learning configuration is optimized. Therefore, the suggested research has developed a very effective and perceptive method of discovering substantial Hindi literature. Research of this kind has the potential to radically transform information retrieval practices in schools and libraries that make use of Hindi-language materials.

1.1 Hindi text analysis

Hindi NLP is used in the branch of AI known as text analytics to convert the free-form text present in documents and databases into more traditional, structured data suitable for analysis or for powering ML algorithms. Examples of this include contact centre transcripts, online testimonials, surveys, and focus group notes. These raw Hindi text files contain a wealth of useful information that has yet to be extracted. These underutilized data sources may now be put to use with the help of text mining and analytics. Hindi text analysis is the process of preprocessing texts to extract information that a machine can understand. The purpose of Hindi Text Analysis is to provide structure to previously unorganized data. It's similar to attempting to make sense of mountains of paperwork by breaking it down into manageable chunks.

1.2 Automated essay scoring

Automated scoring involves using computers to evaluate tasks previously done by humans. AI techniques play a crucial role in this process, particularly in automated essay scoring (AES) where computers assign grades to written assignments [1]. This approach is valuable in education, providing quicker feedback to students to enhance their writing skills. However, questions involving answer choices, file uploads, or diagrams still require manual grading, as there may not be a single correct solution. Subjective questions often need human graders to assess free-form responses. Automated scoring is mainly associated with using technology for tasks traditionally done by people. Automatic scoring, especially in the field of Natural Language Processing (NLP), has gained prominence [2]. It assesses students' work based on logical and semantic connections with the correct answer. Technology tends to provide more consistent ratings than human graders, whose assessments may vary based on the grader's subjectivity. Automatic short answer scoring methods are presented in Figure 1.

1.png

Figure 1. Methods of Automatic short answer scoring

Various studies have been offered for the purpose of data categorization and Hindi textual analysis. Some researchers have employed a supervised machine learning technique, while others have used an unsupervised one. Some studies used a multi-model approach to sentiment analysis, while others used a hybrid approach. Polynomial computing and load-balanced scheduling has also been the subject of studies. Some studies looked at using deep learning to automatically grade essays. It takes a long time for researchers to find a remedy. It has also been noted that machine learning systems' accuracy and performance might need some work. The scalability must be improved by including an optimization method. Therefore, the objective of this research is to propose a hybrid machine learning technique that combines automatic essay scoring algorithms based on natural language processing (NLP) with optimisation mechanisms to produce a more dependable, flexible, and scalable solution for Long Text Hindi categorization. The Hybrid PSO-Roberta technique has also been studied by applying a deep learning approach [3].

In automated answer scoring using semantic analysis on long Hindi text, a series of essential tasks are performed to optimize the data.

These tasks encompass cleaning the text by eliminating non-alphanumeric characters and specific Hindi symbols, as well as tokenizing it, considering compound words and intricate sentence structures. Lowercasing ensures uniformity, while the removal of common Hindi stop words minimizes noise. Additionally, lemmatization or stemming captures word base forms, spelling checks enhance precision, and sentence splitting accounts for complex sentence structures. Furthermore, part-of-speech tagging assigns grammatical categories, entity recognition identifies named entities, and text normalization standardizes numerical values and dates in Hindi. Employing LSTM (Long Short-Term Memory) architecture as part of this preprocessing stage enables the model to recognize temporal dependencies and patterns within the text data, enhancing the semantic analysis's accuracy and understanding. The inclusion of Hindi word embeddings and specialized language models ensures the readiness of the text for comprehensive semantic assessment.

LSTM model: The decision to choose LSTM (Long Short-Term Memory) over other deep learning algorithms in the context of automated answer scoring and semantic analysis is based on the unique strengths of LSTMs for handling sequential data, especially when dealing with long texts [14]. LSTMs are a type of recurrent neural network (RNN) architecture that excels in capturing temporal dependencies and patterns within sequences, making them highly suitable for tasks like natural language processing, text analysis, and scoring. LSTMs can capture long-range dependencies in the data, which is crucial in understanding the context and meaning of text in longer responses [15]. This makes them effective for maintaining context and tracking semantic relationships over extended sequences. It can also handle variable-length sequences and also Preventing Vanishing Gradient problem encountered in traditional RNNs, enabling them to effectively model sequences with a more extensive context. LSTMs have a memory cell that can store and retrieve information over long periods, allowing them to capture not only the relationships between words but also the continuity and flow of ideas in a response. Different steps used for LSTM based detection Model are given in Figure 2.

ROBERTA: Similar to BERT, RoBERTa is a transformer-based language that uses self-attention to evaluate input sequences and construct phrase-level contextual representations. Different steps used for RoBERTa Model are given in Figure 3. Since RoBERTa was trained on a considerably bigger dataset than BERT, it is more effective. RoBERTa is functionally comparable to BERT but is designed differently since it employs a byte-level BPE tokenizer (like GPT-2) and a new pretraining approach. In contrast to the 16GB dataset used to train BERT, RoBERTa is trained on almost 160GB of uncompressed text.

2.png

Figure 2. Flow chart for planned work

3.png

Figure 3. Roberta Model for proposed model

Flow chart for planned work is given in Figure 4. In this work, the step where the trained model's accuracy is verified is called testing. The purpose of the sample dataset is to evaluate the model's accuracy. To make predictions, a variety of datasets are processed via the network model that was trained on the prior dataset. Model dependability is shown on the testing face. A trained network is taken into consideration and supervised during the testing phase. Next, the testing dataset is obtained and a trained network is then used to process the test in order to determine accuracy, precision, and f1-score while taking new test values into account. To begin our investigation, we will train the model on 70% of the data and test it on the remaining 30%.

4.png

Figure 4. Flow chart for planned work

In the testing step, the accuracy of the trained model is examined so that improvements may be made if necessary. The accuracy of the model is evaluated based on the sample dataset that is collected. When performing predictions, the network model that was first trained using the prior dataset is then processed using a variety of datasets. The testing side of the model is where the dependability of it is shown. During the phase of testing known as supervision of the trained network is considered. After that, the testing dataset is taken into consideration. Following this, the data that will be tested is processed using a trained network in order to determine accuracy, f-score, and precision while taking into account fresh test values. For the sake of our study, let's train the model on the initial seventy percent of the data, and then test it on the remaining thirty percent.

Multi-model, hybrid-model, and polynomial-function support are only a few of the models that have been implemented. The issues and concerns raised by previous studies during automated scoring have been taken into account. Issues with scalability, performance, and precision were the root of the problem. The proposed model takes a creative approach to addressing problems throughout its implementation. The proposed method uses optimization, natural language processing (NLP), and classifiers in conjunction with deep learning to score essays automatically in the case of lengthy Hindi texts while minimizing performance and accuracy difficulties. After the suggested model has been constructed, its performance and accuracy will be compared. The first step in deploying a hybrid PSO- Machine learning model based on LSTM for classification is extracting answers from the dataset. After the hybrid model's parameters have been set, a training and testing set is created from the data. During the validation phase, a classification operation is carried out, and the accuracy is determined by calculating the confusion matrix. The accuracy of the suggested Hybrid method is then compared to that of the standard model.

Correlation diagram between different scoring methods: The two separate AMT systems that provide 88-note output and chrome onset output are shown on the left. Using DTW, the results are combined and synchronized to the MIDI score. We used the state-of-the-art AMT system as our starting point, rather than developing our own, since our focus was on developing a neural network-based system that generates features for automated response scoring. Nonetheless, Figure 5 shows how we made some minor adjustments to the training set.

The system's efficacy is measured by contrasting the output it produces with the outcome supplied by human raters. The results of the correlation coefficient show that the scores given by the system and the human raters are strongly associated with one another, with a value close to one indicating a positive correlation. Score agreement between a human rater and the system is quite near to the value reached between human raters. The table below displays the system-generated rankings for a sample of 50 students. The ultimate score of 50 student answers and the score provided by 5 human raters are graphically shown in Figure 6. The chart indicates that the system's output is consistent with that of human evaluators. As proof of this system's precision, a negligible outlier may be disregarded with ease. The system-generated score falls within the range of human raters' scores, demonstrating the system's accuracy and dependability.

5.png

Figure 5. Correlation among various scoring techniques

6.png

Figure 6. Comparison between 5 human raters and the system score of 50 student responses