A Graph-Based Feature Representation Approach for Cyberbullying Detection Using PageRank and Supervised Machine Learning

Cyberbullying detection is a complex supervised classification task. Various techniques have been proposed in the literature to mitigate its adverse effects, particularly for vulnerable populations such as adolescents and young adults. Prominent approaches include ML and deep learning (DL) algorithms, whose effectiveness and robustness are typically assessed using a variety of performance metrics. However, effective cyberbullying mitigation requires more than simple detection; it requires understanding the context and dynamics of online interactions.

The disclosure of personal information significantly increases the likelihood of becoming a victim of cyberbullying. A survey of 1,555 internet users from Nigeria and the United States (US) revealed a direct correlation between the availability of personal information—whether through direct disclosure or via social media platforms—and the frequency of cyberbullying incidents. Cyberbullying typically involves not only the "perpetrator" and the "victim" but also a third party, referred to as the "discloser," who contributes to the dissemination of sensitive information. Both strangers and acquaintances may be involved, making such attacks particularly challenging [3].

The findings of this study carry important implications for social media literacy programs and policy frameworks. In the United States, nearly 40% of adults have reported experiencing cyberbullying, while at least 36% of adolescents in middle and high school have encountered similar behavior. The consequences include stress, anxiety, depression, and reduced self-esteem. As personal information becomes more accessible, its role in fueling cyberbullying intensifies. For instance, sharing private family details or intimate images in online conversations can be exploited by malicious individuals, resulting in victimization. Many users rely on standard platform features such as reporting mechanisms, user blocking, and content filtering to prevent and mitigate cyberbullying. In the United States, users are generally familiar with these tools; however, the awareness and adoption in Nigeria are much lower, contributing to the persistence of the issue. Effective mitigation requires limiting the disclosure of personal information. Future technological solutions, such as watermarking, may offer a means of tracking and restricting the transmission of personal data [3].

Aggressive behavior in online environments is often based on factors including gender, nationality, religion, race, and ethnicity. The control of hate speech and abusive behavior on social networks has proven difficult, particularly because manual monitoring is not scalable. Intelligent systems represent a promising solution for addressing this issue. Among social media platforms, Twitter is one of the most widely used; its users are frequently exposed to offensive content. However, detecting cyber-attacks and offensive language on Twitter is particularly challenging due to the platform’s use of short messages and the linguistic diversity of its user base. Furthermore, cyberbullying and aggression may be expressed indirectly through sarcasm or trolling, which complicates automated detection. Current techniques in natural language processing (NLP), text mining, and ML face significant challenges in accurately classifying such content, limiting their effectiveness in addressing these nuanced forms of online aggression [4].

Twitter is a valuable resource for analyzing collective opinion, as users frequently share their views on various issues. This study aims to analyze public sentiment regarding COVID-19 vaccination on social media, with a particular focus on Twitter. To combat the pandemic, public healthcare agencies are working to increase vaccination rates, and understanding public opinion on social media is central to this objective. By promoting positive messages and countering negative sentiment, authorities can shape public perception and encourage more individuals to get vaccinated. Tracking public sentiment on social media is essential for effective planning and decision-making [5], particularly in healthcare campaigns. Platforms such as Twitter offer real-time insight into public attitudes, making them invaluable tools for public health strategy.

The definition of cyberbullying varies across studies, as each investigation offers its own perspective, leading to different interpretations. Broadly, cyberbullying refers to harmful behaviors conducted online, often through social media platforms or text messages. It is typically characterized by repeated aggressive actions carried out by an individual or a group against a victim who is unable to defend themselves, with the attacks facilitated via electronic devices. Some scholars classify cyberbullying as a form of hate and harassment, constituting a criminal act. The anonymity afforded by social media, where users can adopt pseudonyms rather than real names, enables perpetrators to engage in harmful behaviors without being easily identified. Cyberbullying is recognized as a significant ethical issue, with the number of affected individuals reaching substantial levels. It involves the use of online networks as channels for aggressive actions directed at victims who often cannot respond effectively. Additionally, cyberbullying presents a challenge for text classification due to the dynamic nature of language, which makes it difficult to maintain a static list of offensive terms [6].

Cyberbullying is considered a serious public health concern, with numerous studies documenting its adverse effects on individuals' physical and mental well-being, as well as on academic performance. In severe cases, the associated psychological distress has been linked to suicidal ideation. Given its pervasive nature, occurring at any time and in any setting, developing automated detection models for cyberbullying is critical to enhance human security. Social media platforms such as Twitter, where users can broadcast content to a large audience while remaining anonymous, amplify the potential risk of harmful messages spreading widely. The anonymity intensifies the seriousness of the transmitted content, making social platforms conducive to cyberbullying. In the era of big data, traditional detection methods are insufficient. This has created a demand for automated models capable of detecting cyberbullying more efficiently [7].

Researchers typically define cyberbullying as the use of electronic devices to disseminate abusive communications aimed at individuals. It can take various forms, including posting hateful messages, targeting victims with threats, or leaving aggressive remarks publicly. A particularly harmful characteristic of cyberbullying is that it can spread rapidly without direct confrontation between perpetrator and victim, which complicates efforts to curb it. Therefore, developing predictive models capable of effectively handling both textual and non-textual content, such as images and videos, is critical to addressing this problem [7].

ML, a subfield of AI, involves the ability of a system to learn from historical data to build predictive models and solve specific tasks. DL, a subset of ML based on artificial neural networks (ANNs), has demonstrated promising performance in various domains. In contrast, AI encompasses a broader range of techniques that enable computers to mimic human behavior and make decisions with minimal human intervention. The complexity of human language and the challenge of codifying all human knowledge present several limitations, particularly when dealing with high-dimensional data for tasks such as classification, clustering, or regression. Different learning models are required for specific problems and datasets. The three main types of ML are supervised, unsupervised, and reinforcement learning.

Tweets were classified into cyberbullying and non-cyberbullying categories using convolutional neural networks (CNNs). Following data collection and cleaning, annotations were performed using CrowdFlower. Word similarities and semantics were computed using Global Vectors for Word Representation (GloVe). The results showed that varying CNN parameters, including the number of epochs, neurons per layer, and network depth, influenced classification accuracy and that metaheuristic optimization algorithms provided the most effective means of improving it [6].

Beyond this specific approach, the broader body of research on cyberbullying detection relies on data collected from social media platforms, with feature engineering playing a central role in model performance. While many feature selection algorithms are used to extract features for classifier training, they often do not explicitly determine the significance of specific features. Commonly utilized algorithms include Information Gain, Pearson Correlation, and Chi-Square Tests. Supervised ML algorithms frequently employed for cyberbullying detection include Support Vector Machine (SVM), Naïve Bayes, Random Forest, Decision Tree, k-nearest neighbors (KNN), and Logistic Regression. Various evaluation metrics, such as accuracy, precision, recall, F1-score, and area under the curve (AUC), are used to measure the models' effectiveness. Given that datasets are often imbalanced, Synthetic Minority Over-sampling Technique (SMOTE) is frequently applied to mitigate this imbalance and reduce the risk of overfitting [7].

A proposed solution for detecting cyberbullying on social media networks leverages supervised ML and NLP techniques. The researchers used TF-IDF to assess word importance within a document and evaluated whether the text contained profanity in conjunction with a pronoun. The Affective Norms for English Words (AFINN) lexicon was used to determine polarity at the sentence level, while SVM was chosen as the classifier due to its effectiveness in binary classification tasks. The hybrid model, integrating TF-IDF with sentiment analysis, achieved the best performance metrics, with accuracy, precision, recall, and F1-score all reaching 75% [8].

The authors utilized an annotated cyber-trolls dataset and implemented preprocessing techniques using various Python libraries. These techniques included the removal of stop words, punctuation, and numbers, lowercase conversion, and tokenization. For feature extraction, TF-IDF was applied in both unigram and bigram formats, and a feature selection module was employed to avoid overfitting. The proposed method was based on a deep neural network (DNN) architecture with fully connected layers of a Multilayer Perceptron (MLP). For comparison, they employed CNNs with Long Short-Term Memory (LSTM) and Bidirectional Long Short-Term Memory (BiLSTM), which are commonly used in text classification. MLP was also used as a classification module to assign labels to each input. Evaluation metrics, including average accuracy, precision, recall, and F1-score, demonstrated that the MLP model using TF-IDF outperformed other models, achieving an accuracy of 92% [4].

The authors outlined a framework for cyberbullying detection combining NLP and ML. The NLP component included various preprocessing steps, such as removing stop words, punctuation, numbers, tokenization, and stemming. For feature extraction, methods such as Bag-of-Words and TF-IDF were employed, with ML algorithms such as Decision Tree, Naïve Bayes, Random Forest, and SVM applied for classification. The findings indicated that TF-IDF yielded higher accuracy compared to Bag-of-Words, and SVM outperformed the other algorithms [9].

In another study, an annotated public dataset from Twitter and Formspring was used. The preprocessing steps included removing special characters and encoding the text into numerical form using a label encoder. Classifiers such as SVM, Naïve Bayes, Random Forest, and an ensemble approach combining these algorithms were employed for classification. The research addressed tasks such as role labeling, text classification, and sentiment analysis, with SVM achieving the highest accuracy score of 81.6%. The researchers also investigated DL algorithms, such as BiLSTM and CNN, with F1-score, recall, and precision metrics indicating that CNN achieved the highest performance. The study highlighted the challenge of detecting sarcasm in textual features [10].

A model for assessing the severity of cyberbullying on Twitter was tested using five classifiers—Naïve Bayes, KNN, Random Forest, SVM, and Decision Tree—on two data representations: Bag-of-Words and Word2Vec. Feature selection techniques, including Information Gain, Chi-Square, and Correlation, were applied in various combinations to evaluate both feature performance and classifier performance. The researchers used Pointwise Mutual Information (PMI) to calculate the semantic orientation of words in the corpus, finding that integrating PMI with predicted features improved classifier performance by 20%. SMOTE was also employed to address dataset imbalance.

The PMI between two words, word1 and word2, is defined by Eq. (1):

$P M I($word1, word2$)=\log 2\left[\frac{p(\text {word} 1 \& \text {word} 2)}{p(\text {word} 1) p(\text {word} 2)}\right]$   (1)

Researchers employed Arabic Natural Language Processing (ANLP) for cyberbullying detection, with preprocessing steps such as cleaning, normalization, tokenization, and stemming. Data representation was conducted using GloVe through the `glove_python` library. A comparison of GloVe and TF-IDF showed that GloVe achieved higher classification performance in this context. Classification was performed using a CNN optimized with a Genetic Algorithm (GA). This hybrid GA-CNN model achieved high classification accuracy on the Saudi Newspapers Articles Dataset (SNAD) and Moroccan Newspapers Articles Dataset (MNAD) [11].

Another study aimed to detect aggression in Spanish-language texts by combining lexicon-based approaches with ML algorithms to analyze emotions expressed in Twitter comments. Five lexicon-based approaches were proposed, combining Lexicon, word embedding (WE), and TF-IDF features in different configurations: Lexicon, WE_Lexicon, TF_IDF_Lexicon, WE_Lexicon_TF-IDF, and an ensemble approach. GridSearchCV was utilized to optimize the models' parameters, with the hybrid approach incorporating WordEmbedding, Lexicons, and ML classifiers outperforming the basic models. The researchers also developed a web application to test the proposed methodology's effectiveness in classifying tweets [12].

Using a custom dataset of hotel reviews, this study evaluated the polarity of Arabic texts as either positive or negative. Steps included data collection, cleaning, annotation, preprocessing, feature extraction, and classifier training and testing. Classifiers used were Logistic Regression, Naïve Bayes, Support Vector Classification (SVC), Ridge Classifier, Decision Tree, and Random Forest. The results indicated that the SVC yielded the highest accuracy, reaching a score of 95.62%. Hyperparameter tuning techniques, such as Bayesian Optimization, Grid Search, Random Search, GA, and Particle Swarm Optimization, significantly enhanced performance, particularly for Naïve Bayes [13].

Another study investigated the detection of irony and sarcasm on Twitter using a combination of ML and NLP techniques, including DL methods and CNN. The study began by clearly defining the terms "irony" and "sarcasm" to establish a consistent conceptual framework for the analysis. The first experiment compared different classification techniques and found that DL methods yielded superior results. However, a noted limitation of DL methods is their requirement for large datasets to achieve high performance. The second experiment evaluated the impact of different data preprocessing techniques when applying the CNN algorithm. The findings emphasized the importance of carefully implementing preprocessing steps to avoid removing data critical for detecting sarcasm or irony [13].

In the third experiment, the study compared irony and sarcasm detection by training models on both datasets in a cross-evaluation setup. The results showed a similarity score of 0.94, indicating a strong correlation between sarcasm and irony. The final experiment involved comparing sarcasm and cyberbullying detection by training models on sarcasm data and testing them on cyberbullying datasets. This yielded an F1-score of 0.889, suggesting that enhanced detection of irony and sarcasm can improve the accuracy of cyberbullying detection [14].

Other researchers proposed a hybrid model combining an ANN with Deep Reinforcement Learning (DRL) for the classification of cyberbullying content from raw datasets. They developed multiple frameworks capable of extracting cyberbullying instances from a comprehensive dataset, which included user comments, psychological traits, and contextual information. The ANN-DRL model improves with each iteration by leveraging the feedback mechanism of DRL. The classification results, measured in terms of accuracy, demonstrated that the ANN-DRL model achieved the highest accuracy score of 80.69%, compared to ANN alone (77.40%), Random Forest (75.55%), Logistic Regression (75.10%), Naïve Bayes (75.19%), and SVM (75.44%). The ANN-DRL model's iterative feedback process contributed significantly to its superior performance, highlighting its potential for enhancing cyberbullying detection accuracy [15].

The authors investigated whether Bidirectional Encoder Representations from Transformers (BERT)’s attention mechanisms explain why it makes certain predictions. Previous studies showed that BERT is effective at detecting cyberbullying, but the interpretability of its internal decision-making process remained unexamined. The study fine-tuned BERT (base, uncased) on five datasets (Twitter-Racism, Twitter-Sexism, Kaggle-Insults, WTP-Toxicity, and WTP-Aggression), which together cover a range of abusive language types. The fine-tuned BERT achieved the best performance against traditional DL models with an F1-score of 0.786 on the WTP-Toxicity dataset. The study demonstrated that while fine-tuned BERT outperforms LSTM and Bi-LSTM models, its attention mechanisms do not correlate with gradient-based feature importance scores, indicating that attention weights do not offer faithful explanations for model predictions. A key finding from the study is that BERT depends heavily on syntactic elements such as determiners and auxiliaries rather than on bullying-related lexical content. This means that the model relies on superficial linguistic markers rather than genuine semantic understanding. The authors recommend using gradient-based explanation techniques and training on more diverse datasets that are syntactically diverse. This combined strategy would improve interpretability, generalization, and fairness [16].

Another study proposed a semantic-enhanced marginalized denoising auto-encoder (smSDA) for text-based cyberbullying detection, addressing the challenge of learning robust representations from short and noisy social media messages. The method extends the marginalized stacked denoising auto-encoder (mSDA) with two innovations: first, semantic dropout noise assigning a higher corruption probability to bullying features, and second, sparsity constraints via L1 regularization to capture meaningful word correlations. Bullying features were automatically constructed using a predefined insulting word list (350 words) expanded via word embeddings (Word2Vec on 400 million tweets) with a cosine similarity threshold of 0.8. Subsequent layers updated bullying features using the Fisher score. Evaluated on Twitter (7,321 instances, 4,413 features) and MySpace (1,539 instances, 3,240 features) with small training sets (800 and 400 samples), smSDA outperformed BoW, LSA, LDA, and mSDA. On Twitter, smSDA achieved 84.9% accuracy and 71.9% F1-score compared to mSDA (84.1% accuracy, 70.4% F1-score). On MySpace, it achieved 89.7% accuracy and 77.6% F1-score compared to mSDA (88.0% accuracy, 76.0% F1-score). By focusing exclusively on text and ignoring word order, the model may fail to capture important syntactic patterns. Future work should incorporate sequential information to improve performance [17].

Other researchers introduced an automated DL framework called SBiGRU-BCO for cyberbullying detection on SM platforms. The model combined Stacked Bidirectional Gated Recurrent Unit (SBi-GRU) with an attention mechanism, BERT embedding, and Binary Chimp Optimization (BCO)-based feature selection. This methodology begins with data preprocessing, including stop word removal and text normalization. The model achieved strong performance with an accuracy of 99.12% and an F1-score of 93.91%. The datasets were sourced from three SM platforms: Formspring with 10,000 samples, Instagram with 12,000 samples, and MySpace with 8,500 samples. The datasets are public and are binary-annotated as cyberbullying and non-cyberbullying. The authors deployed BERT to categorize aggressive content and used attention mechanisms to improve the sequential semantic representations in textual cyberbullying detection. The authors proposed the use of Feature Density (FD) calculations to quantify dataset complexity. The feature selection process employs Binary Multi-Objective Chimp Optimization (BMCO) for identifying optimal feature subsets to improve classification performance. Based on a comparative analysis, the proposed approach significantly outperformed existing methods such as BiGRU and CNN. The research presents practical solutions enabling real-time intervention in cyberbullying incidents for online safety measures [18].

The authors introduced ELECTRA_POS as a pretraining methodology for transformer models, integrating Part-of-Speech (POS) information through a Greek-letter substitution approach. The key innovation involves replacing standard POS tags with Greek letters (e.g., 'VERB' becomes 'δ') and appending them to each word before processing with the SentencePiece Unigram tokenizer. This methodology enhances the model’s comprehension of linguistic structure and contextual nuances. The model was evaluated against a retrained baseline (ELECTRA_Vanilla) on the GLUE benchmark and a specific cyberbullying detection task. The ELECTRA_POS model demonstrated a notable gain in recall compared to ELECTRA_Vanilla, achieving 0.6209 compared to 0.5878. This indicates a superior ability to identify true positives (TPs) of harmful online behavior. The research confirms that integrating grammatical features, such as POS tags, into transformer architectures benefits both cyberbullying detection and general NLP tasks. However, the fusion of POS tags presents challenges, including increased computational load from longer sequences and a lack of standardized tagging across datasets, indicating a need for further optimization. In summary, this study demonstrates that the fusion of grammatical information can improve a model’s contextual understanding; future work should focus on hyperparameter tuning, learning rate optimization, and comparison with other transformers [19].

Researchers introduced an interpretable hybrid model for multiclass cyberbullying detection. The novel approach employs a hybrid ensemble framework integrating Bidirectional Encoder Representations from Transformers (BERT) to generate contextual sentence embeddings from social media text and SVM with grid-search optimization for discriminative multiclass classification. The model classifies tweets into five categories: Cybertalking, Doxing, Revenge Porn, Sexual Harassment, and Slut Shaming. The proposed model combines BERT’s strength in semantic understanding with SVM’s efficiency and discriminative power in high-dimensional spaces. The model demonstrated high efficacy, achieving a 90% accuracy rate on test data, outperforming individual baseline models including BERT alone (87%), BiLSTM (85%), and traditional ML baselines. The authors also deployed the Shapley Additive exPlanations (SHAP) framework to enhance model transparency by explaining the reasons behind the proposed model’s predictions. The primary contribution is a validated hybrid architecture; however, the model’s validation on a small balanced dataset (2,140 tweets) may not fully represent the noise and imbalance of real-world social media streams, which remains an open question for future research [20].

Authors explored the use of graph convolutional networks (GCNs) for the categorization of online harassment on Twitter by modeling tweets as nodes in a semantic similarity graph rather than using sequence-based DL models and analyzed the relational structure between tweets based on their semantic content. The dataset consisted of 10,622 English Twitter posts labeled for harassment across three categories: “indirect”, “sexual,” and “physical” harassment. Each tweet was transformed into a node in a single undirected graph. Edges were weighted by the cosine similarity of the tweet’s Sentence-BERT (SBERT) embeddings, connecting each node to its five most semantically similar neighbors. Nine classical machine-learning classifiers, including Logistic Regression, Random Forest, Naïve Bayes, Decision Tree, SVMs, and ensemble methods, were evaluated as baselines using TF-IDF and Word2Vec features, while the proposed two-layer GCN model used SBERT embeddings as node features, with evaluation performed via 10-fold cross-validation. In the binary classification task (harassment vs. non-harassment), the GCN model outperformed all classical models, achieving the highest accuracy. For multi-class classification into the three harassment categories, Random Forest obtained the best performance with an average accuracy of 93.5%; GCN still outperformed most other classical approaches. The study also found that TF-IDF representations were more effective than Word2Vec embeddings for this task, as the pre-trained word vectors failed to capture the informal and abusive language characteristic of tweets. An additional experiment using only 50% of the training data showed that the GCN’s accuracy decreased by just 3%, demonstrating its robustness in learning from limited labeled data. The study concludes that GCNs are an efficient and powerful tool for harassment detection, capable of learning effective representations from both text semantics and graph structure. The main limitations identified were the imbalanced class distribution and the computational cost of graph construction. Future research directions include exploring alternative graph neural architectures on larger multilingual datasets and integrating the temporal dynamics of conversations [21].

Researchers explored the use of a GCN named SOSNet for fine-grained multiclass cyberbullying detection on Twitter by building a global graph of semantically related tweets and analyzing the conceptual connections between tweets to construct a global semantic similarity graph for classification purposes. The authors created and used a novel balanced dataset of 69,767 English Twitter posts labeled for cyberbullying targeting five victim attributes: age, ethnicity, gender, religion, and other. Each tweet was represented as a node in a single undirected graph, with edges created between tweets based on a threshold cosine similarity of their embeddings. Eight tweet embedding methods, including BOW, TF-IDF, word2vec, GloVe, fastText, BERT, DistilBERT, and SBERT, were evaluated in combination with seven classifiers: Logistic Regression, Naïve Bayes, KNN, SVM, XGBoost, Multi-Layer Perceptron, and the proposed SOSNet GCN, with evaluation performed using 5-fold stratified cross-validation. The proposed two-layer SOSNet model used a similarity threshold to filter significant semantic connections when constructing the graph adjacency matrix. On the full 40,000-tweet dataset, BOW+XGBoost and TF-IDF+XGBoost achieved the highest accuracy and F1 scores, indicating that lexical features are highly discriminative for this fine-grained task. However, on a smaller 4,000-tweet dataset, SBERT+SOSNet achieved the highest accuracy and F1-score, matching or exceeding traditional classifiers and demonstrating its effectiveness with limited data. The study also found that SBERT emerged as the most robust embedding across classifiers, while Dynamic Query Expansion (DQE) successfully addressed severe class imbalance as a semi-supervised method that collects and balances natural data, unlike synthetic methods such as ADASYN or SMOTE. The study confirmed that GCNs are a powerful method for fine-grained detection, effectively leveraging semantic graph structure among tweets, especially when data is limited, and establishes DQE as a superior alternative for creating balanced social media datasets. This study’s future work should explore other GNN architectures, apply DQE to other social media data mining tasks, and test the models on multilingual and larger-scale social media datasets [22].

AI is a field of computer science aimed at building systems capable of performing tasks that typically require human intelligence, such as understanding human communication, problem-solving, and decision-making. Achieving this is complicated by the dynamic nature of real-world data, which affects the quality of the datasets used to train such systems.

Within this context, classification refers to the task of assigning a predefined category to a given document, a task whose complexity is governed by two factors: the accuracy of the classifier and the high dimensionality of the feature space. Feature selection addresses this issue by identifying a subset of the most pertinent features from the original dataset [27, 28].

4.1 Data collection

The dataset used in this study is a public dataset available on Kaggle [29]. It contains 60,000 tweets labeled for suspicious content related to cyberbullying, stored in a single CSV file (“suspicious tweets.csv”, 4.7 MB). The dataset contains two columns: Message, containing the tweet text, and Label, a binary indicator where 0 denotes “bullying” and 1 denotes “non-bullying”. The dataset is intended for supervised binary classification and consists of raw tweet text that may include usernames and URLs requiring preprocessing. As is typical of real-world social media data, the dataset is imbalanced. Following a balance check, bullying tweets (label 0) represent 10% of the corpus while non-bullying tweets (label 1) represent the remaining 90%.

4.2 Python libraries

The preprocessing and modeling pipeline was implemented in Python, using its standard library alongside NLTK for natural language processing and NumPy, Pandas, Matplotlib, and Scikit-learn for data manipulation, visualization, and ML.

4.3 Data pre-processing

The choice of text representation and feature selection is critical to cyberbullying detection performance. Unfortunately, multiple cyberbullying studies focus primarily on feature selection while ignoring the way the text is represented during preprocessing [30]. The following subsections describe the preprocessing pipeline applied to prepare the raw tweet data for modeling.

4.3.1 Data cleaning

Remove null values: null values impact the reliability, accuracy, and quality of data analysis, making it appropriate to handle this issue in the data preprocessing pipeline before ML modeling or statistical analysis. For our specific dataset, we utilized the dropna() function from the Pandas library in Python. This function provides an effective method for dealing with missing values within Pandas DataFrames by removing any rows containing at least one null value. We selected this approach because it is easy to implement and computationally efficient. It should be noted that after applying dropna() and inspecting the resulting DataFrame, we confirmed that there were no missing values present in our dataset. Consequently, no rows were removed, and the dataset remains intact.

Remove duplicates: duplicate entries have exact or partial identical values across the columns in a dataset. Duplicates can influence database operations, data analysis, and reporting, and they may lead to inefficiencies and inaccuracies. To address this challenge in our work, we applied the drop_duplicates() method from the Pandas library to the DataFrame. This method operates by identifying and removing rows that have identical values across all columns. By default, drop_duplicates() retains the first occurrence of each duplicated row and discards all subsequent duplicates (parameter settings: keep = ’first’). The cleaned DataFrame was then stored as a new object for downstream processing.

Fix contractions: expanding contractions is part of text standardization and language consistency. This step restores shortened forms to their original full form. To implement contraction expansion in our preprocessing pipeline, we utilized the contractions library, a specialized Python package designed for this purpose. The library’s primary function, contractions.fix(), accepts a text string as input and returns a modified string with all recognized contractions expanded to their full forms. In our specific implementation, we imported the contractions library at the beginning of our preprocessing script using the standard import contractions statement. We then applied the contractions.fix() function to the text content of each tweet in the Message column.

Remove single letters: depending on the objectives of the analysis and the context of the data, single-letter tokens may or may not need to be removed. The reasons behind this pre-processing step are several, including noise reduction, vocabulary size reduction, preventing sparsity, and enhancing readability. In our specific preprocessing pipeline, we decided to remove single-letter tokens because the majority of single-letter occurrences were typographical fragments or noise characters rather than meaningful semantic units. To implement single-letter removal, we employed regular expressions (Regex), a powerful pattern-matching tool for text manipulation. After applying this step, we verified the transformation by inspecting random samples of tweets. The resulting cleaned text was passed to subsequent preprocessing stages.

Remove URLs: removing URLs from text is an essential preprocessing step for improving text quality and reducing noise. URLs carry limited semantic value for most downstream tasks and provide limited benefit to a classification model. To implement URL removal in our preprocessing pipeline, we employed a combination of two techniques: regular expressions (Regex) and string manipulation. In addition to the Regex pattern, which can capture a wide variety of URL formats with a single pattern, we incorporated string manipulation techniques to clean the resulting text. We defined a comprehensive URL pattern as follows:

urls = re.finditer(r'http[\w]*:\/\/[\w]*\.?[\w-]+\.+[\w]+[\/\w]+', urlfree)

For the Twitter dataset analyzed in this study, we found that the vast majority of URLs followed the standard formats covered by our pattern.

Remove hashtags (#) and mentions (@): removing hashtags (denoted by the “#” symbol) and mentions (denoted by the “@” symbol) is a common preprocessing step in NLP for social media text. Both elements were removed in our preprocessing pipeline to improve text quality, reduce noise, and standardize the input for downstream modeling. To implement hashtag and mention removal, we employed a filter() function in Python combined with a custom condition to exclude tokens starting with “#” or “@”. The conceptual approach involves splitting the text string into individual tokens (words), filtering out any token that begins with either the “#” or “@” symbol, and rejoining the remaining tokens into a single string with spaces.

Remove punctuation: this step involves eliminating punctuation marks from the text. Punctuation marks are symbols such as question marks, exclamation marks, parentheses, and commas. Their role is to give structure to a sentence, indicate pauses, or express a specific tone while reading. To implement punctuation removal in our preprocessing pipeline, we utilized the string punctuation constant provided by Python’s built-in string module. This constant contains a predefined set of punctuation characters commonly used in English text. The complete content of string punctuation is:

'!"#$%&\'()*+,-./:;< = >?@[\\]^_`{|}~'

Remove digits: removing digits is a standard text preprocessing technique in many NLP pipelines. Digits are the numerical characters 0 through 9 appearing in various forms within text, including isolated numbers and multi-digit sequences. To implement digit removal in our preprocessing pipeline, we adopted a string-manipulation approach using Python’s built-in isdigit() method, which returns True if all characters in a given string are digits (0-9) and False otherwise. The conceptual approach involves splitting the text into individual tokens and checking whether each token is entirely digits; tokens that are purely numerical are removed, while tokens containing a mix of digits and other characters are preserved.

Remove repeated characters: repeated characters occur when a writer intentionally or unintentionally duplicates a character multiple times in succession, such as “Helooooooo”. Such repetitions are used in informal digital communication to convey excitement or frustration and therefore pose several challenges for NLP systems, justifying their normalization or removal. To implement repeated character removal in our preprocessing pipeline, we employed Regular Expressions (Regex). The core idea is to match any character followed by one or more consecutive repetitions of the same character and replace the entire sequence with a single instance of that character.

4.3.2 Data tokenization

Tokenization is the process of decomposing a sentence or corpus into smaller linguistic units called tokens. The tokens could be words or phrases, and tokenization is an essential step in the NLP and text analysis pipeline. Tokenization was performed using the n-gram technique encompassing both unigrams and bigrams as sequences of tokens. A unigram is the simplest type of n-gram output, consisting of a single individual token or word in a text. To implement tokenization in our preprocessing pipeline, we employed a combination of Regular Expressions (Regex) and string manipulation techniques. The Regex pattern \W+ matches non-alphanumeric characters and underscores, automatically excludes punctuation, and splits text on any non-word character. After tokenization, we converted all tokens to lowercase using Python’s built-in x.lower() method. Lowercasing ensures consistency and prevents the word appearing in different cases from being treated as distinct tokens.

4.3.3 Remove stopwords

Stopwords are frequently occurring function words that carry limited semantic content in a given language. These words add no significant meaning to the context in text analysis tasks or NLP tasks, and best practice is to remove them during the preprocessing pipeline in order to focus on the remaining text content. The list of stopwords can vary according to the language and also depends on the text’s context or domain. Stopwords are characterized by high frequency and low information content, and removing them can reduce noise, improve analysis, and increase efficiency. For our cyberbullying detection task, we determined that stopword removal was appropriate because the presence or absence of function words such as “the” was unlikely to be the primary differentiator between bullying and non-bullying tweets. The core abusive content is instead carried by lexical words such as “kill”, “hate,” or “suicide”. To implement stopword removal in our preprocessing pipeline, we utilized two Python libraries: NLTK and Scikit-learn. The decision to use both libraries stemmed from the observation that when using only one stopword list, certain English stopwords remained in the corpus after removal, requiring a complementary list. From NLTK, we retrieved the standard English stopword list using nltk.corpus.stopwords.words('english'), and from scikit-learn we used sklearn.feature_extraction.text.ENGLISH_STOP_WORDS. We then performed a manual review of the combined stopword list to customize it for our specific task.

4.3.4 Stemming

Stemming is the process of reducing words to their morphological root form by removing prefixes and/or suffixes, allowing many words with the same underlying meaning but different forms to be treated as identical. Because morphology varies widely across languages, it is necessary to select a stemming algorithm suited to the language’s specific morphological structure. According to the literature [14, 31], the Porter Stemmer from the NLTK library is one of the most widely used stemming algorithms. In our case, we instantiated the Porter stemming algorithm using NLTK.PorterStemmer() to reduce words to their root form. We selected the classic Porter stemmer for our pipeline because of its widespread use and computational efficiency. After implementing stemming, we performed validation checks to confirm the stemmer was functioning as expected, manually inspecting a random sample of 100 stemmed tokens to verify the results. Stemming was applied after stopword removal to ensure stopwords were removed before stemming, an ordering that is more efficient and prevents stopwords from being mapped to non-standard stems that might no longer be recognized as stopwords.

4.3.5 Untokenization

At this stage, the dataset has been cleaned of noise, and the next step is feature transformation or feature engineering. Directly applying this transformation to tokenized data yields poor results, with features being transformed incorrectly. The solution is to transform tokens back into sentences; that is, each set of tokens is joined to reconstruct a sentence. Untokenization refers to the process of rebuilding a sequence of tokens into its original text, so that the recombined units reconstitute a sentence with the same word order but without the noisy words or characters removed earlier. Untokenization is frequently essential after tokenizing text for NLP tasks such as summarization, machine translation, or text generation since preprocessed text may need to be restored to its original form for specific analyses. It is important to note that there is no dedicated built-in function or universal method for untokenization in standard Python or NLP libraries. In our case, a combination of basic Regular Expressions (Regex) and string manipulation methods, specifically the join() and strip() methods, was sufficient to perform the untokenization task since all tokens were already cleaned and contained no internal spaces or special characters.

4.3.6 Class imbalance

A critical challenge in developing an ML model for cyberbullying detection is the class imbalance present in the Twitter dataset. After performing a balance check on the 60,000 tweets, we observed that bullying tweets (label = 0) constitute only 10% of the dataset, while non-bullying tweets (label = 1) represent the majority at 90%. This class ratio is a characteristic feature of real-world social media data, where harmful content is far less than benign content. To address this imbalance, we employed SMOTE, a technique recognized for its effectiveness in handling imbalanced data, applying it to the training data after TF-IDF vectorization. SMOTE operates in feature space rather than in the original input space, following these steps:

•For each minority-class instance, the KNN are identified among other minority class instances (k = 5).

•One of these KNN is selected randomly.

•The difference vector between the current instance and its selected neighbor is computed.

•The difference vector is multiplied by a random number between 0 and 1.

•A new synthetic instance is generated accordingly.

SMOTE was applied to the training data following TF-IDF vectorization, ensuring that oversampling occurred in the transformed feature space rather than on raw text. SMOTE addressed the class imbalance present in this Twitter cyberbullying detection dataset.

4.4 Feature engineering/feature extraction and data representation

Feature engineering is a key factor and central task in data preparation for the success of any ML model. This process involves selecting, manipulating, and transforming raw data into features. In both supervised and unsupervised ML pipelines, features have a substantial impact on model performance. A simple transformation function can convert features from one representation to another [7, 14].

Is feature transformation, therefore, essential? Yes. ML algorithms require the feature space to be numeric. The input must be a matrix with features in columns and instances in rows. A common approach to applying a machine-learning model to text or sentiment analysis is to transform documents into vectors; this operation is called vectorization. In this study, to perform feature transformation from cleaned text strings to numerical vectors, we imported the TfidfVectorizer class from the sklearn.feature_extraction.text module. After vectorization, each word is represented as a column in the feature matrix. The resulting feature matrix can be used directly as input to any scikit-learn classifier. To verify that the transformation was performed correctly, we called the tfidf_vectorizer.get_feature_names_out() method in scikit-learn to inspect the vocabulary and confirm that the vectorization process captured the expected features.

This verification step confirmed that:

•The vocabulary size was appropriate given the corpus size and preprocessing.

•Common stop words had been successfully excluded.

•Stemming had collapsed morphological variants as expected

•The TF-IDF weights were properly computed.

In summary, preparing a raw dataset is a crucial prerequisite for successful classification and evaluation. The essential steps in our methodology are summarized in Figure 4, which outlines the overall process of the proposed approach.

image027.png

Figure 4. Our actual approach

qq20260922-161433.png

Figure 5. Data pre-processing

image029.png

Figure 6. Data cleaning

The data preprocessing stage, the second step in our approach, is further divided into five distinct steps, as illustrated in Figure 5. This segmentation ensures a structured and systematic handling of the data, enhancing the quality and reliability of the subsequent analysis.

Data cleaning, a fundamental component of data preprocessing, involves several essential tasks aimed at improving data quality. The specific procedures and techniques employed are detailed in Figure 6, which highlights the critical role of this step in the preprocessing workflow.

4.5 Feature selection and feature weighting

Feature selection, also known as attribute selection, is a technique used to reduce the dimensionality of the feature space by identifying and retaining the most informative features, while eliminating irrelevant and redundant ones from the original dataset. The primary objectives of feature selection are to enhance the classification model's speed and accuracy, and to improve its overall performance. Achieving these objectives involves meeting three key goals: improving classification accuracy, accelerating the training and testing processes, and increasing the model's interpretability [32].

Feature selection methods can be broadly categorized into two main types: attribute evaluation algorithms and subset evaluation algorithms. In attribute evaluation algorithms, features are assessed individually, with each feature assigned a weight based on its relevance to the target variable. In subset evaluation algorithms, however, groups of features are selected based on evaluation criteria applied to feature subsets.

Three widely recognized categories of feature selection techniques are filter methods, wrapper methods, and embedded methods. Filter methods involve ranking features based on specific criteria, assigning a score to each feature. These methods are generally efficient and help prevent overfitting [27].

Given the social network structure, which can be represented as a graph, we propose exploring graph-based algorithms for feature ranking. One well-known technique in web information retrieval is the PageRank algorithm. However, while PageRank is effective for ranking web pages, its application to feature ranking for classification may not yield the same level of performance.

This work aims to develop a novel and comprehensive feature ranking strategy that can deliver improved classification results in sentiment analysis. Achieving this goal will involve considering factors such as the method used for graph construction, the nature of the dataset, and the traditional classification techniques employed. To validate the effectiveness of the new strategy, we plan to conduct real-world experiments using our dataset. A comparative analysis with the traditional TF-IDF filter method will be performed to evaluate the improvements in classification performance.

A more detailed explanation of the traditional TF-IDF method and the proposed approach based on the PageRank algorithm is provided in the next section.

4.5.1 First method: Term Frequency-Inverse Document Frequency

TF-IDF is commonly employed for feature extraction, text preprocessing, and tasks such as sentiment analysis and document classification. However, in classification tasks, a higher dimensionality of the feature matrix can increase the problem's complexity, leading to longer computational times and potentially hindering model performance. To address this, the `max_features` parameter can be used to limit the dimensionality of the TF-IDF matrix, thereby improving memory efficiency and model performance. This approach is especially beneficial for handling large vocabularies. In this work, we computed TF-IDF weights for each feature. By setting the `max_features` to 5,000, we limited the vocabulary to the 5,000 most frequent terms for subsequent training and testing with a ML classifier. When creating a `TfidfVectorizer` object with `max_features` set to 5,000, the vectorizer analyzes the text corpus and constructs a vocabulary of the 5,000 most frequent terms based on document frequency. During the transformation process, the TF-IDF matrix is generated using only these top 5,000 terms, while any terms outside this vocabulary are excluded. This strategy ensures that the model focuses on the most significant features, optimizing computational efficiency while retaining the most pertinent information for classification. Setting the default value of max_features from “None” to a fixed number (5,000 in our case) allows us to control the dataset size based on our main constraints. Figure 7 outlines the classification process using TF-IDF as the first method.

image030.png

Figure 7. The first classification method using Term Frequency-Inverse Document Frequency (TF-IDF)

4.5.2 Second method: Proposed approach based on PageRank algorithm

The typical representation of a social media network involves modeling users as nodes and their interactions as edges connecting those nodes. This representation is displayed as a graph. In this work, we consider words as nodes and the relationships between them as edges.

Overall, representing a social media network as a graph affords a strong framework for identifying influential nodes or communities, analyzing sentiment dynamics within the network, and detecting the most valuable nodes. Many social media network analysis tasks can be addressed through this model, such as content moderation, recommendation systems, and community detection.

In a traditional machine-learning context, PageRank is not directly applicable to feature selection or feature weighting; however, its capabilities in structural network analysis and propagation can inspire new methods for feature weighting, especially when the data is represented as a graph. In this work, we were inspired by PageRank to address the problem of information retrieval. However, the strong performance of PageRank in web page ranking does not guarantee similar behavior in sentiment analysis. Its effectiveness depends on many factors; therefore, experimentation and validation on our dataset are essential to justify the choice of using a PageRank-inspired method for information retrieval in sentiment analysis or classification. It is also necessary to compare this inspired method to another common feature selection method; in this work, this baseline was TF-IDF as a filter method. In this work, we leveraged a modified PageRank algorithm by incorporating weights, aiming to enhance feature ranking for text classification tasks. The process began with preprocessing the dataset to extract a list of bigram tokens. This list was then imported into Neo4j as a CSV file, where we used the Cypher query language to construct a graph representing features as nodes and their relationships as edges. The size and complexity of the graph increased with the number of nodes and relationships, which in our case consisted of 34,061 nodes and 257,547 edges.

After creating the graph, various properties were assigned to each node and edge to facilitate centrality analysis. We aimed to assign a centrality score to each node using algorithms integrated into Neo4j, including PageRank. Our approach extended the traditional PageRank algorithm by weighting each token's relevance based on its connections within the graph. The PageRank score for each token was computed and ranked using Cypher queries, providing an ordered list of nodes based on their centrality scores.

Figure 8 outlines the classification process using PageRank as a second method.

image031.png

Figure 8. The second proposed method based on PageRank

To maintain consistency with the approach used in the TF-IDF method, we restricted the final feature set to the top 5,000 ranked tokens. The scores and corresponding node identifiers were exported from Neo4j to a Jupyter environment for further processing. Using Python, we filtered and formatted the data, ensuring that only the top 5,000 nodes were included in the final CSV file, while the remaining data was discarded. This selection process ensured that the most relevant features were retained for subsequent ML tasks.

4.5.3 Term Frequency-Inverse Document Frequency versus PageRank

The core idea behind PageRank is that the importance of a node is determined not only by the number of links pointing to it (in-degree) but also by the importance of the nodes that provide those links. In other words, a link from a highly authoritative page contributes more to the target’s rank than a link from a low-authority page.

Traditional text classification methods such as TF-IDF + NB/SVM operate under the independence assumption. This assumption implies that each document is treated as an isolated vector in a high-dimensional space, and each term is treated independently of others. While this assumption works well in mathematical contexts, it fails to reflect linguistic reality. PageRank directly addresses these limitations by modeling the text corpus as an interconnected graph rather than a collection of independent vectors. Moreover, PageRank can be interpreted as a random walk on the graph-based terms. This probabilistic interpretation provides an intuitive understanding of why PageRank is effective for text classification. Consider a random walker who moves on the graph according to the following rules:

•With probability d (typically 0.85), the walker follows an outgoing edge from the current node.

•With probability 1-d, the walker teleports to a seed node.

The PageRank score of a node is the stationary probability that the random walker visits that node after many steps. The random walk interpretation provides a probabilistic justification for why PageRank propagates influence through the graph. A new token that shares terms with known cyberbullying tweets will have a shorter hitting time to the cyberbullying seed, resulting in a high PageRank score. In other words, term nodes receive high scores if they are frequently visited, which occurs when they co-occur with other high-importance terms.

4.6 Training and testing classifiers

The choice of classifier depends on many factors such as problem complexity, the nature of the data, and computational resources. Among the wide range of available classifiers, in this work we focus on four main ones: Logistic Regression, Naïve Bayes, Random Forest, and Support Vector Machine (SVM). The dataset was divided into 80% for training and 20% for testing. Furthermore, while experimenting with our models, we encountered issues related to the algorithmic diversity among the classifiers. Investigating these differences in depth yielded notable results.

4.7 Evaluation metrics: Accuracy, precision, recall and F1-score

In the reviewed literature, the typical evaluation metrics for assessing the performance of cyberbullying detection models are accuracy, F1-score, precision, recall (also known as true positive rate or sensitivity), and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC). These metrics are calculated based on the four outcomes (TP, TN, FP, FN) that characterize a binary classification task. In contrast, only a few studies use the error rate or mean squared error (MSE) [2].

The accuracy score is one of the most common metrics used for classification. However, it is not suitable when the dataset is imbalanced, as it is generally dominated by the majority class. Indeed, some studies employing DL algorithms did not report accuracy. Conversely, F1-score is recommended as a more reliable metric due to its ability to balance recall and precision [2].

Additionally, accuracy can be misleading when the dataset is imbalanced, contains outliers, or experiences overfitting. In such cases, accuracy alone cannot fully assess model performance or reflect its true effectiveness. Therefore, additional metrics such as recall, precision, and F1-score are adopted as they capture aspects that accuracy alone may overlook.