Machine Learning-Based Recommendation Support for Personalized Marketing Information Systems: A Comparative Study

Personalized marketing strategies aim to deliver innovation and relevance to each customer based on their data and behavior. With the help of AI, companies can efficiently analyze large amounts of data, understand individual preferences, and design marketing campaigns tailored to the needs and desires of each customer [1]. The application of artificial intelligence (AI) to customised marketing strategies is essential. AI-powered personalised marketing techniques can improve client acquisition and retention [2, 3] Customer conversion refers to the desired actions of customers, such as purchasing products or services, while customer retention refers to a company's ability to keep customers over time. Both aspects are vital for business growth and sustainability [4]. This study highlights the importance of personalization in modern marketing, as well as the objectives and benefits expected from this research. Additionally, the structure and methodology of the research used to achieve these objectives will be outlined [5]. By understanding and implementing AI-based personalized marketing strategies, companies are expected to achieve higher levels of customer conversion and retention, ultimately increasing business revenue and profit [6].

A subfield of computer science called AI concentrates on developing systems or machines that are able to mimic human abilities in thinking and performing tasks. In this context, AI is applied to marketing strategies to enhance customer loyalty [7]. AI's role enables assisting humans in voice recognition, for example, recommending products to customers. By utilizing machine learning (ML) algorithms, AI can examine customers' purchase history and browsing behavior to suggest the most relevant products, thereby enhancing the likelihood of customer purchases [8]. AI can also perform other tasks, working through algorithms and mathematical models designed to learn from data and experience, allowing machines to improve their performance over time. With AI, we can develop advanced applications such as virtual assistants, autonomous vehicles, recommendation systems, and various other innovations that make human life easier [9].

Sentiment analysis employs natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and analyze emotional states and subjective data. This method is employed to comprehend and categorize emotions (positive, negative, and neutral) in text through text analysis techniques, frequently utilized on online platforms such as social media, e-commerce, and websites [10]. The alterations in customer purchasing behaviors throughout the pandemic have profoundly influenced the market; yet, employing sentiment analysis techniques can enhance sales for Micro, Small, and Medium Enterprises (MSMEs). This study seeks to analyze consumer reactions to the acquisition of MSME products in Karanganyar Regency. This technique operates by clustering existing data and processing it to generate the desired result to assist MSMEs in Karanganyar Regency [11].

Although artificial intelligence and ML have been widely applied to personalized marketing and recommendation systems, most existing studies focus on algorithm implementation or performance reporting in isolated scenarios rather than systematically comparing multiple classification models using real-world datasets to determine the relatively more suitable approach for personalized product recommendations. Additionally, previous research has often prioritized sentiment analysis, hybrid recommendation strategies, or collaborative filtering techniques, while limited attention has been given to evaluating classifier-based content-filtering models using measurable performance indicators such as precision, recall, F1-score, Area Under the Curve (AUC), and error rate [3, 12]. The absence of comparable empirical evidence generates ambiguity concerning whether ML approach provides the most dependability and practical applicability for automated personalization in e-commerce settings. This study seeks to address the research gap by assessing and contrasting the efficacy of Decision Tree, Random Forest, K-Nearest Neighbors (KNN), and Naïve Bayes classification models utilizing a real dataset, with the goal of determining the most precise and reliable algorithm to enhance personalized marketing decision-making and improve recommendation quality [13, 14].

Beyond algorithm comparison, this study positions the evaluated classification models as the predictive core of a recommendation-support module within a personalized marketing information system. In this framing, the recommendation output is not treated merely as a static classification result, but as decision-support information that can assist product targeting, prioritization, and operational personalization processes. This study enhances model evaluation and informs the design of intelligent decision-support components for AI-driven marketing workflows by integrating classification-based recommendations within a comprehensive information-system framework.

This research employs the Content-Based Filtering method, which does not require data from other users as a reference for recommendations [15]. Content-Based Filtering uses existing information within several items or data to determine appropriate recommendations for users [16, 17]. Content-Based Filtering determines recommendations based on the similarity level of one item to other items in the data. This method selects and ranks items based on the similarity of item attributes. This method's benefit is that users get info on items that matter to them, as each item's content can be known from its representation. Item attributes and descriptions are crucial in the filtering process of this method. Items with the highest similarity level to other candidate items will be recommended to users as recommendations [18].

In its implementation, content-based filtering uses vector calculations, Term Frequency-Inverse Document Frequency (TF-IDF), and cosine similarity, which essentially convert data into vector form. The development of recommendation systems aims to reduce excessive information by selecting the most relevant information from large data and providing a list of items that match the user's interests or preferences [19]. Recommendation systems can predict user preferences by analyzing the user's history or other users' history, which is a key feature of this system to generate recommendations that match each user's preferences [20]. Therefore, the findings of this study are expected to provide empirical guidance for both researchers and practitioners in selecting ML models that align effectively with data structure and operational needs in AI-driven marketing systems.

This study adopts a comparative experimental approach to evaluate the performance of four ML classification algorithms, namely Decision Tree, Random Forest, KNN, and Naïve Bayes, implemented using the Python programming environment. The objective is to identify the relatively more suitable model for predicting book recommendation categories based on user-related data [50]. The steps involved in data processing are as follows:

Selection Stage. This stage involves preparing the dataset for model development by selecting relevant data and partitioning it into training and testing sets. The dataset was divided using an 80:20 ratio, where 80% of the data were used for training and 20% for testing to evaluate model performance [51, 52].

Research Pre-Processing. The pre-processing stage began with checking the dataset for missing values. For the Goodreads-books dataset, no missing values were identified. Within this pre-processing stage, there are steps to create labels/classes and convert string attributes to numeric [53].

Within the content-based filtering framework, transactional item attributes such as quantity and total_price were used as structured input features to represent item content relevant to the recommendation process. These features underwent vector-based representation and similarity-oriented content processing to identify relevant item patterns, which subsequently served as input to the classification models (Decision Tree, Random Forest, KNN, and Naïve Bayes). In this workflow, the classification models function as the predictive layer that assigns items into recommendation categories, while the predicted categories constitute the recommendation output used to support decision-making in personalized marketing contexts.

Model Selection Stages. Decision Tree Algorithm originates from the Concept Learning System (CLS) and Iterative Dichotomiser 3 (ID3) algorithms. Decision Trees based on the C4.5 algorithm are a commonly used classification technique to extract relevant relationships in data [54]. The C4.5 algorithm is a program that constructs a decision tree based on labeled input data. Its advantage lies in its easily interpretable model that can handle both continuous and discrete attribute values. C4.5 algorithm divides training data using information gain. Attributes with high frequencies are considered to separate data based on available information in the dataset. Before calculating the gain value, first determine the entropy value using the following equation:

${Entropy}(S)=\sum_{i=1}^n p i * \log _2 p i$

${Gain}(S, A)={Entropy}(S)-\sum_{i=1}^n \frac{|S i|}{|S|} * {Entropy}(S i)$

The Decision Tree algorithm was implemented as one of the classification models in this study, utilizing a standard tree-based learning approach to identify patterns within the dataset. In addition to Decision Tree, three other classification algorithms—Random Forest, KNN, and Naïve Bayes—were also employed to enable a comprehensive comparative evaluation of model performance [55].

Table 2. Parameter settings used in the comparative classification models

Model comparisons were conducted using parameter settings selected to ensure a fair and consistent evaluation across all classification algorithms. The parameter configuration used in each model is presented in Table 2.

Stages of evaluation and validation of results. In this stage, all classification models were evaluated to assess their predictive performance and enable comparative analysis. The dataset was partitioned using a stratified 80:20 train–test split, where 80% of the data were used for training and 20% for testing. This approach ensures that class distributions are preserved across both subsets, supporting a more reliable evaluation of model performance. The dataset was partitioned into training and testing sets using an 80:20 ratio, comprising 8,684 records for training and 2,171 records for testing. All classification models were trained on the training set and evaluated on the testing set to ensure an unbiased assessment of performance, with no overlap between training and testing data. Model evaluation was conducted using standard classification metrics, including accuracy, precision, recall, F1-score, and ROC/AUC, derived from the confusion matrix. To demonstrate the performance of the algorithm used, the equation is as follows:

$Accuracy$ $=\frac{T P+T N}{T P+F P+T N+F N} \times 100 \%$

The accuracy produced is calculated based on the confusion matrix. The calculations in the confusion matrix are computed according to true positive predictions (True Positive), false positive predictions (False Positive), true negative predictions (True Negative), and false negative predictions (False Negative). The higher the accuracy value obtained, the better the resulting method. Precision is the ratio of true positive predictions (TP) compared to the overall positive predictions.

$Precision$ $=\frac{T P}{T P+F P} \times 100 \%$

Recall is the ratio of TP compared to the total actual positives.

$Recall$ $=\frac{T P}{T P+F N} \times 100 \%$

F1-score is a weighted average of precision and recall.

$F 1-$$Score$ $=2 \times \frac{{Precision} \text { × } {Recall}}{({Precision} \text { + } {Recall})}$

Model performance was evaluated using standard classification metrics, including accuracy, precision, recall, F1-score, and ROC/AUC, derived from the confusion matrix. These metrics were used to assess the predictive capability of each model and to enable consistent and fair comparison across all evaluated algorithms, namely

• Accuracy of 0.90 – 1.00 = Excellent classification
• Accuracy of 0.80 – 0.90 = Good classification
• Accuracy of 0.70 – 0.80 = Fair classification
• Accuracy of 0.60 – 0.70 = Poor classification
• Accuracy of 0.50 – 0.60 = Failure

At this stage it will be displayed in the form of tables and graphs for comparison of classification algorithms.

The results of this study reinforce existing literature that highlights the growing influence of artificial intelligence and ML in improving personalized marketing and recommendation accuracy. Prior research indicates that AI-driven personalization enables companies to deliver more relevant product suggestions and increase engagement metrics such as click-through and conversion rates [56, 57], and the findings here are consistent with studies showing that targeted machine learning models outperform generic approaches when personalization is grounded in behavioral and preference-based data [58].

The notably high performance demonstrated in this study must be understood in the context of the dataset attributes and problem definition. The organized and comparatively low-dimensional feature space, along with the conversion of continuous rating values into distinctly defined categorical groups, probably led to well-defined decision boundaries that benefit tree-based algorithms. To assure the validity of these results, the target variable (average_rating) was solely utilized for label generation and excluded from the input features during model training, hence reducing the chance of information leaking. These findings indicate that although high predictive accuracy is attainable in structured datasets, the efficacy of the model is significantly contingent upon the congruence between data attributes and algorithmic design, necessitating caution when extrapolating these results to more intricate real-world recommendation contexts.

Similarly, Chabane et al. [59] emphasized that tailored algorithm selection significantly improves predictive performance in recommendation systems, a trend also reflected in this study where Decision Tree and Random Forest showed the strongest classification stability. The broader literature, including the bibliometric review by Bawack et al. [60], further supports the growing interdisciplinary relevance of AI-powered recommender systems within e-commerce, integrating insights from marketing, data science, and consumer psychology.

However, some contrast appears when compared with studies advocating alternative approaches such as hybrid recommendation strategies that combine collaborative filtering and content-based filtering to enhance prediction reliability and personalization depth [61]. Although these studies indicate robust performance from probabilistic and hybrid models, the current findings demonstrate that Naïve Bayes and KNN were significantly less effective, presumably due to the dataset's dependence structure and the relationships among categorical variables that contravene fundamental model assumptions. This divergence suggests that the effectiveness of recommendation models is not universal but dependent on the alignment between algorithm logic and dataset characteristics. Thus, this research advances prior work by demonstrating through empirical evidence that tree-based approaches, rather than probabilistic or similarity-driven methods, are better suited for structured categorical product-rating environments in personalized marketing contexts, highlighting model–data compatibility as a critical determinant of recommender system performance.

The findings of this study provide meaningful implications both theoretically and practically, reinforcing the growing body of evidence that ML models play a critical role in enhancing personalization accuracy within modern recommendation systems driven by artificial intelligence [56, 57]. Theoretically, the strong performance of Decision Tree contributes to ongoing discussions in AI-based marketing by demonstrating that algorithm suitability is highly dependent on dataset structure and feature dependency, supporting recent claims that tailored model selection often yields superior predictive outcomes compared to generalized or deep learning approaches applied without contextual alignment [58, 59]. Practically, the findings indicate that firms utilizing personalized marketing engines should prioritize interpretable and computationally efficient models, especially when dealing with datasets comprising categorical product features and rating categories. This study methodologically emphasizes the importance of empirical model comparison and multi-metric evaluation, consistent with recent studies advocating for more transparent benchmarking frameworks in the development of recommender systems [9, 60]. However, several limitations should be acknowledged, including reliance on a single dataset, the absence of hybrid or deep learning–based recommendation models such as collaborative filtering or neural recommenders, and the use of a single train–test validation split rather than multi-fold cross-validation. These limitations indicate that the comparative findings should be interpreted within the present experimental setting and should not be implicitly generalized beyond the structured recommendation context examined in this study. Additional research across diverse datasets, model variants, and validation strategies is needed to strengthen robustness, transferability, and broader empirical validation of model performance in recommendation contexts [16, 62]. Overall, this study contributes empirical evidence demonstrating that model–data alignment is a key determinant of effectiveness in AI-driven personalized marketing systems. By showing that tree-based models outperform probabilistic and similarity-based algorithms in structured categorical environments, the findings reinforce ongoing shifts toward context-aware ML in recommender system research and practice.

However, the exceptionally high predictive performance reported in this study should be interpreted in light of several characteristics of the experimental setting. The structured and relatively low-dimensional feature space, together with clearly defined categorical labels, may have contributed to simpler decision boundaries that favor tree-based models and support near-perfect classification outcomes. While these conditions strengthen performance within the present dataset, they may also limit generalizability to more complex real-world recommendation environments characterized by dynamic user behavior, noisier feature interactions, and less deterministic data structures. Therefore, the findings should be understood as strong evidence within a controlled structured-data context, while broader applicability requires further validation in more diverse and operational settings.