A Suitable Technique for Enhancing Arabic-Language Consumer Sentiment Analysis Using Natural Language Processing and Stacking Machine Learning Model

Sentiment analysis in Arabic language involves using natural language processing (NLP) techniques to analyze and classify the emotional tone of text written in Arabic [1]. The goal is to identify the sentiment of the text as negative, neutral, or positive. There are several challenges to performing sentiment analysis in Arabic, including the complexity of the language, the lack of standardized spelling and grammar rules, and the presence of dialects and regional variations. However, several approaches can be used to address these challenges, including machine learning techniques, rule-based systems, and lexicon-based methods [2].

Consumer sentiment analysis (CSA) is a vital aspect of business intelligence, marketing, and customer experience management. It involves the extraction of opinions, attitudes, and emotions expressed by consumers in various forms of communication, such as social media, reviews, and customer support interactions [3]. With the growth of e-commerce and social media, companies are increasingly turning to sentiment analysis to gain insights into their customers' behavior and preferences. However, analyzing large volumes of unstructured text data is a complex and time-consuming task that requires sophisticated machine learning (ML) approaches and NLP [4]. CSA is a vital instrument for companies to use to comprehend their customers' thoughts and opinions regarding the services or products that the company provides. By analyzing customer feedback, businesses can gain insights into areas of improvement, identify customer pain points, and make informed decisions to improve customer experience. This, in turn, can lead to increased customer loyalty, customer retention, and improved bottom lines [5].

One typical way to sentiment analysis in Arabic is to utilize ML techniques such as neural networks (NN) or SVM to divide the text into different sentiment categories. These algorithms are trained on labeled datasets of Arabic text to learn the features and patterns that are associated with different sentiment categories [6].

Recent research [3, 4] demonstrated that the application of sentiment analysis has grown to incorporate text and visual data. This problem is related to the recognition of emotions within the field of research known as affective computing [7]. Affective computing and sentiment analysis are two areas that are extremely important to Artificial Intelligence development [8] technology and have a great deal of untapped potential in a variety of different fields. Classification issues arise when ascertaining whether a piece of writing conveys an excellent or negative mood [9]. Yet, sentiment analysis is a complex procedure; instead, it requires analyzing various natural language processing (NLP) subtasks, such as recognizing subjectivity and sarcasm [10]. In addition, the writing might not have a structure, and it has the potential to have errors and colloquialisms [11].

Researchers, businesses, governments, and other organizations have all acknowledged the significance of sentiment analysis [1]. Online resources such as blogs, message boards, social networks, and wikis have emerged as significant sources of information in recent years as the number of people who use the internet has continued to rise. Because the perspectives and viewpoints presented in these online resources are highly pertinent to our day-to-day lives, it is vital to analyze this data utilizing automated public opinion monitoring to facilitate decision-making [12]. For instance, posts made on Twitter have been studied to ascertain the results of elections [13].

As a direct consequence of this, the topic of sentiment analysis has received a considerable amount of interest within the scientific world over the previous fifteen years. Since 2004, the discipline of SA has established itself with the greatest rate of expansion and level of activity. In the past few years, there has been a discernible spike in articles focusing on opinion mining and sentiment analysis [13], which proves this is the case. Google Trends indicates that there has been a rise in the number of people interested in sentiment analysis in recent years.

This research aims to analyze the efficiency of a variety of ML approaches in terms of interpreting the sentiments communicated in Arabic. In addition, the TF-IDF technique was utilized to extract features from the dataset. As a direct consequence of this, the techniques of RF, DT, LDA, KNN, SVM, QDA, LR, GBRT, and SGD Classifier were utilized in the process of sentiment analysis (SA) in the three different Arabic datasets used, which are the Hotel Arabic-Reviews Dataset (A), the Books Reviews in Arabic Dataset (B), and the Arabic Reviews dataset (C). A stacked-based ML model was built, which, when compared to other ML simple classifiers mentioned in this article, performed significantly better, and we used LR and SVM as meta-classifiers. Compared to existing machine learning simple classifiers, our stacked model with 10-fold cross-validation shows a higher accuracy, precision, Cohen's Kappa, recall, and F1-score.

In the following section, we will discuss some material that is pertinent to the backdrop. Section 3 describes our strategy. The results of the system's evaluation are covered in Section 4, which may be found here. In the final section, both a summary and some recommendations for the future are presented.

1.1 Problem statement and research questions

The main challenge of sentiment analysis is accurately categorizing sentiments expressed in a text, especially in the case of consumer sentiment analysis where opinions and attitudes are often nuanced and context-dependent. This research aims to address this challenge by developing a stacking machine learning model for consumer sentiment analysis using NLP techniques.

1.2 Contribution and significance of the study

The proposed model aims to enhance the efficiency and accuracy of sentiment analysis in consumer data by combining the advantages of different ML algorithms. The study contributes to the field of sentiment analysis by providing a comprehensive approach that incorporates NLP techniques and stacking machine learning models. The results of the study can benefit businesses, marketers, and customer experience managers in gaining valuable insights into their customers' attitudes and preferences.

In the following sentences, we will go into our ML models, which will include the datasets, feature extraction, and dataset processing. Nevertheless, before we get to that point, we will begin with the primary structure of our research, which is illustrated in Figure 1.

1.png

Figure 1. Global architecture of our model

3.1 Dataset used

For this paper, the following datasets were used to determine how well our model performed:

The Hotel Arabic-Reviews Dataset (A) [24] is made up of 490587 hotel reviews that were gathered from the Booking website throughout June and July of 2016. The evaluations are composed in dialectal Arabic (DA) and Modern Standard Arabic (MSA) and scored on a scale ranging from one to 10. The HARD dataset, which consisted of both positive and negative evaluations, was utilized in our analysis. The reviews were categorized as either positive (with scores between 4 and 5) or negative (with scores ranging from 1 to 3). The review types represented in this dataset include 93700 total reviews, with an equal number of negative (46850) and positive (46850) categories.

The Books Reviews in Arabic Dataset (B) [25] comprises 510,600 reviews of Arabic books obtained from GoodReads.com between June and July of 2016. The reviews are primarily written in DA and MSA style. We utilized the BRAD dataset, which includes equal good and negative testimonials and ratings. Only positive (four and five stars) and negative research were found in the reviews (one and two stars). The collection includes slightly more than 156,000 different reviews.

The Arabic Reviews dataset (C) [26] contains feedback from more than one hundred thousand customers on hotels, books, movies, and various other things, in addition to specific airlines. The reviewers' ratings are classified into negative, positive, or mixed. More than three reviewers gave it a positive rating, while fewer than three gave it an unfavorable rating. There is now a text and a label connected with each row, and the text in each row has been cleansed of non-Arabic characters and Arabic diacritics. There are only a few duplicate reviews in this collection.

3.2 Pre‑processing

Pre-processing of datasets involves the processes performed to prepare raw data for machine learning tasks. These steps typically consist of data cleaning, data transformation, and data partitioning into testing and training sets. The specific pre-processing steps required vary depending on the form of data utilized and the objectives of the machine-learning task.

Some common pre-processing steps include:

• Tokenization, or segmentation, prepares a document for subsequent processing by separating it into a list of tokens consisting of numbers, words, and other special characters. This can be done by splitting documents, such as crawling reviews.

• The process of normalization converts all word tokens in a document to either all lowercase letters or all capital letters to normalize them. This is done because most assessments contain both uppercase letters and lowercase. With this method, one can improve the accuracy of their forecasts.

• The removal of typical stop words, such as prepositions, unnecessary words, memorable characters, and ASCII code, as well as excessive white spaces, new lines, and other elements, improves the functionality of the feature selection technique. This is accomplished through the use of the stop word removal technique.

• The stemming process involves converting all tokens into the root form, also known as the stem. This method is speedy and uncomplicated, and it makes the process of extracting characteristics much simpler.

3.3 Feature extraction

To accomplish our objectives in natural language processing, we make use of a variety of machine-learning strategies. During the training process for our model, we extract each phrase's features based on two primary criteria. To put this strategy into action, we use a method known as tokenization, which involves deconstructing phrases into the individual words that make them up, irrespective of how frequent or uncommon the individual terms may be. In addition, we use a numerical figure that determines the significance of a phrase in a text. This technique is TF-IDF. Because of the method's track record of success for various languages, we have decided to embrace it. As a direct consequence, our machine learning algorithms have accomplished remarkable things.

3.4 Machine Learning (ML) models: An overview

ML uses various sequences to learn from data and assign it to categories. A categorization model is formed utilizing the training set's knowledge to categorize input data as either positive or negative [27]. The training set contains the names and categories of the input feature vectors. The extracted feature sets are used in training the classifier, which determines whether the analysis of the dataset is favorable or unfavorable based on the results. The objective of the stacking techniques in machine learning is to produce a more accurate prediction model by integrating several base models.

3.4.1 Support Vector Machine (SVM)

SVM is a popular supervised ML algorithm for regression and classification analysis. In the Support vector machine, the objective is to identify the hyperplane that most effectively separates the data into distinct classes. The hyperplane is chosen to maximise the margin between the different classes. The points closest to the hyperplane are known as support vectors, which play a critical role in determining the hyperplane. SVM can be used for both non-linearly and linearly separable data and can handle high-dimensional data. A powerful algorithm can provide accurate results with relatively small datasets. SVM can also handle imbalanced datasets, where the number of examples in one class is much larger than the other [28].

3.4.2 Logistic regression (LR)

LR is a prominent statistical learning approach used to predict a binary output variable from one or more input variables or predictors. The outcome variable is typically coded as 0 or 1, indicating the presence or absence of a certain event or condition [29].

In LR, the result is modeled using a logistic function or sigmoid function, which maps any real-valued input to the range [0,1]. The logistic function calculates the probability of the positive class given the input variables, and the negative class probability is simply the complement of the positive class probability. The logistic regression algorithm estimates the model parameters by maximizing the likelihood function of the data, which is the probability of observing the data given the model parameters.

3.4.3 Decision Tree (DT)

DT is a model that may be applied to classification and regression problems. The data is divided into subsets using a recursive process that is based on the value of one of the features. This process continues until the subsets only include one class or until a set of specified stopping criteria is satisfied. Overfitting is a risk associated with decision trees; however, this issue can be mitigated by employing strategies such as pruning and establishing a limited depth [30].

3.4.4 Random Forest (RF)

The RF is an ensemble learning technique comprising several different decision trees. The algorithm chooses a subset of features for each tree based on a random selection, and then it aggregates the predictions from all of the trees to develop the overall forecast. Random forests are frequently utilized when compared to a single decision tree because they are better at preventing overfitting and improving accuracy [31].

3.4.5 K-Nearest Neighbors (KNN)

KNN is a non-parametric technique applicable to regression and classification applications. Finding the k data points that are geographically closest to a new data point is the first step in the algorithm's process. Then, the classes of those data points are utilized to guess the nature of the unique data point. The value of k can be determined based on the cross-validation results; however, choosing the best number can be time-consuming [32].

3.4.6 Linear Discriminant Analysis (LDA)

LDA is an example of a supervised ML approach that may be used for classification. This type of algorithm searches for a linear combination of features that can best separate the classes. It assumes that the data follow a normal distribution and that the covariances of the different types are equivalent. Because it projects the data onto a lower-dimensional space while preserving the information that allows for maximum discrimination, LDA is frequently utilized in the dimensionality reduction process in feature extraction [33].

3.4.7 Quadratic Discriminant Analysis (QDA)

QDA stands for quadratic discriminant analysis. It is comparable to LDA, except it enables individualized covariances for each class. This results in QDA being a more flexible model but requiring additional data to estimate the covariance matrices accurately. When the underlying distribution of the data is nonlinear, QDA can provide more accurate results than LDA [34].

3.4.8 Gradient Boosting Regression Trees (GBRT)

GBRT [35] is an approach for supervised learning used for regression and classification. It is an ensemble method that makes a final forecast by combining the outcomes of multiple DTs. The GBRT sequentially creates the trees, with each new tree attempting to repair the mistakes that were introduced by the trees that came before it. The ultimate forecast is derived from a weighted sum of the predictions made by each tree. The GBRT algorithm is reliable and capable of dealing with nonlinear connections between the characteristics and the target variable. Nevertheless, if the parameters are incorrectly calibrated, it may overfit the data.

3.4.9 Stochastic Gradient Descent (SGD) Classifier

SGD Classifier stands for stochastic gradient descent, and it is an example of a supervised machine learning approach that is used for categorization. An optimization method will make changes to the model parameters in an iterative manner. These changes will be based on a batch, a randomly selected chunk of the training data. SGD can be sensitive to the initial learning rate and the batch size, even though it is computationally efficient and can manage big datasets. The SGD classifier can solve linear and nonlinear problems and is compatible with a wide variety of loss functions, including logistic regression and support vector machines [36].

3.4.10 Ensemble stacking ML model

Stacking involves using heterogeneous weak classifiers trained in parallel and integrated by a meta-learner to give a forecast based on the projections of the individual susceptible learners. As can be seen in Figure 2, the meta-learner utilizes the predictions as input features and the goal as ground truth values. It then learns how to optimally combine these input predictions to generate a more accurate output prediction. This method is efficient because the various models can pick up knowledge that complements what they already know, ultimately enhancing performance. Stacking is a beneficial technique when working with datasets that are not evenly distributed because it lowers the prediction variance. It is crucial, however, to carefully calibrate both the separate models and how they are integrated. This is because, when compared to other machine learning techniques, it is a sophisticated approach.

This research used DT, RF, KNN, LDA, QDA, GBRT, and SGD as base classifiers. SVM and LR as meta-classifiers.

2.png

Figure 2. The stacking model

3.5 Performance measures

In this section, we will discuss the assessment measures we employed to evaluate the performance of our model.

For this discussion, the terms "false positives," "true positives," "false negatives," and "true negatives," respectively, are abbreviated as Fp, Tp, Fn, and Tn, as shown in Table 1 [2, 31-33].

Table 1. Summary of sentiment analysis evaluation metrics