Aspect-Based Sentiment Analysis of Multilingual Hotel Reviews in Jakarta Using Multilingual Transformer Models: A Comparative Study of Multilingual BERT and Cross-Lingual RoBERTa

The theoretical justification for using ABSA with multilingual transformer models and sentence-pair classification, as well as the rationale for model selection, is provided in this section.

4.1 Theoretical framework

This research is anchored in several foundational theories that explain the influence of online reviews in the hospitality industry and provide the conceptual basis for the analytical approach:

•Social Proof Theory posits that individuals look to the behavior and opinions of others-such as online reviewers-especially in uncertain situations, to guide their own decisions. In the context of hotel selection, positive reviews serve as social validation, shaping consumer trust and booking intentions [15, 16].

•The Information Adoption Model explains how consumers process and utilize information from online reviews. This model suggests that the credibility, relevance, and perceived expertise of the reviewer influence how potential guests interpret and act upon review content [16].

•Expectation Confirmation Theory provides a framework for understanding customer satisfaction, emphasizing the gap between pre-consumption expectations and post-consumption experiences as articulated in reviews [16].

These theories collectively justify the focus on sentiment analysis of online hotel reviews as a method for uncovering actionable insights into guest satisfaction, preferences, and service quality. They also inform the selection of analytical techniques that can capture nuanced, aspect-specific sentiment from diverse and multilingual user-generated content.

4.2 Aspect-based sentiment analysis and model rationale

ABSA extends traditional sentiment analysis by identifying not only whether a review is positive or negative overall, but also which specific aspects of the service (such as cleanliness, service, or price) are associated with those sentiments [8, 12]. This level of granularity is particularly important in the hospitality sector, where guest satisfaction depends on multiple dimensions of the hotel experience and management needs to know precisely which aspects drive positive or negative perceptions [12, 13]. ABSA therefore enables hotels to move beyond aggregate ratings and extract targeted, actionable insights from large volumes of unstructured review text [13].

Recent advances in transformer-based language models make ABSA especially effective in multilingual and code-mixed settings [17]. Multilingual models such as mBERT and XLM-RoBERTa are pretrained on large corpora covering many languages, allowing them to capture shared semantic structures across Indonesian and English reviews [14]. Their deep contextual representations help resolve challenges such as informal language, spelling variations, and domain-specific expressions that are common in hotel reviews, providing a robust foundation for aspect-level sentiment classification on bilingual guest feedback [12].

The methodological innovation in this study lies in adapting a sentence-pair classification strategy for ABSA. Instead of treating each review as a single sequence, the task is reformulated so that each review is paired with an auxiliary sentence representing a candidate aspect–sentiment statement, and the model predicts whether the sentiment expressed in the review supports that statement [17]. This formulation is inspired by prior work that recasts ABSA and targeted ABSA as Natural Language Inference–like problems and demonstrates strong performance gains for BERT-based models compared with single-sentence classification [17]. By leveraging transformer models’ ability to reason over sentence pairs, this approach improves the detection of aspect-specific sentiment, particularly in reviews that mention multiple aspects or use implicit, context-dependent expressions [12, 17].

4.3 Integration of theory and analytical methods

The integration of these theoretical perspectives with advanced analytical methods ensures that the research not only addresses the technical challenge of extracting nuanced sentiment from multilingual reviews but also aligns with established frameworks on consumer behavior and satisfaction in hospitality. The use of ABSA, supported by transformer-based models, operationalizes the theoretical constructs, such as Social Proof and Expectation Confirmation, by systematically mapping review content to specific aspects and sentiments. This alignment allows for the extraction of actionable insights that are directly relevant to hotel management and industry stakeholders, transforming qualitative guest feedback into quantitative performance metrics.

4.4 Methodological considerations and novelty

•Model Selection Justification: The choice of mBERT and XLM-RoBERTa is grounded in their demonstrated effectiveness for cross-lingual and domain-adaptive sentiment analysis tasks. While mBERT provides a stable baseline for multilingual understanding, XLM-RoBERTa offers potential performance advantages due to its larger training corpus and capacity to handle code-switching, a critical feature for Jakarta's bilingual review landscape.

•Sentence-Pairing Approach: By leveraging auxiliary sentences and sentence-pair classification, the study addresses the limitations of traditional single-sentence ABSA approaches, such as difficulty in handling code-switching and domain-specific vocabulary [3].

•Novelty in Application: This research represents one of the first comparative applications of the sentence-pair ABSA strategy specifically for the Indonesian hospitality sector using these two transformer architectures. By benchmarking their performance on a real-world, code-mixed dataset of 5-star hotel reviews, the study provides empirical evidence on the efficacy of transfer learning for operational reputation management.

4.5 Limitations

Potential limitations include reliance on automated sentiment labeling, which may introduce labeling noise; the absence of manual validation for all aspect-sentiment pairs; and possible dataset bias toward guests more likely to leave online reviews.

Table 3. Validation metrics of the mBERT model

Comparing the baseline and sentence-pair configurations across Tables 6–9 shows that sentence pairing yields consistent gains in F1-score for both architectures without destabilizing validation metrics, indicating that the reformulation to a sentence-pair NLI-style task adds discriminative information rather than simply increasing model capacity. This pattern supports the interpretation that explicit aspect–sentiment pairing helps both mBERT and XLM-RoBERTa focus on fine-grained opinion cues in code-mixed reviews, rather than memorizing surface-level correlations.

Heatmap analysis generated after running all the data through the fine-tuned XLM-RoBERTa model, identified cleanliness as the most positively perceived aspect (99.4% positive sentiment), while pricing generated the highest negativity (11.2% negative sentiment). Facilities and service quality dominated review frequency, accounting for 79,713 and 78,773 mentions, respectively. Notably, 79% of negative pricing reviews coexisted with positive remarks on other aspects, reflecting guests’ willingness to critique specific elements while maintaining overall satisfaction.

Figure 1. Heatmap distribution of sentiment

As visualized in Figure 1, positive sentiment dominates almost all aspects, with cleanliness and facilities showing the highest concentration of strong positive polarity, while price exhibits a visibly thicker band of negative values than other aspects. This pattern confirms that Jakarta’s five-star hotels are consistently praised for tangible quality attributes such as cleanliness, facilities, and service, whereas perceived value for money remains a relative weakness. The heatmap also reveals that negative opinions on price often co-occur with positive sentiment on other aspects, indicating that guests are generally satisfied with the overall experience but remain sensitive to pricing and cost-related issues.

Based on the results of running the entire dataset through the fine-tuned XLM-RoBERTa model, the aspect-based sentiment classification produced clear trends across all major hotel service dimensions. As summarized in the results table, the model identified overwhelmingly positive sentiment for most aspects: facilities (79,713 positive), services (78,773 positive), cleanliness (33,324 positive), rooms (41,359 positive), location (64,574 positive), and food (29,608 positive). Negative sentiment was most frequently associated with location (2,712) and services (2,690), while the price aspect showed the highest proportion of negative (791) and neutral (624) classifications relative to its total mentions. Cleanliness stood out with the lowest number of negative (100) and neutral (114) cases, and no errors, indicating exceptionally consistent guest satisfaction in this area. Across all aspects, the number of ambiguous or error cases detected by the model remained minimal (e.g., 24 for location, 20 for facilities, and 101 errors in total), demonstrating both the robustness of the classification and the clarity of guest feedback in the dataset. These results highlight that, for five-star hotels in Jakarta, facilities, services, and cleanliness are the most consistently praised aspects, while price remains the most sensitive and divisive factor among guests.

Table 10 summarizes the distribution of sentiment predictions across the seven key aspects in the test set, including counts for positive, negative, and neutral classifications, as well as errors. The results highlight exceptionally high positive sentiment for cleanliness, facilities, and service, reinforcing these as core strengths for Jakarta’s five-star hotels. In contrast, the aspect of price records notably higher negative sentiment, indicating ongoing guest dissatisfaction in this area. The relatively low error counts across all aspects further illustrate the reliability and robustness of the classification pipeline. This table provides crucial insights for hotel management, offering a data-driven basis for targeting improvements and sustaining high service standards.

Table 10. Distribution of sentiment classification results per aspect

From a managerial perspective, Table 10 highlights that interventions should prioritize pricing policies and communication, since price is the only aspect where negative and neutral sentiments form a substantial share of all mentions, while operational efforts in cleanliness, facilities, and service can focus on maintaining already strong performance. The very low error counts across aspects also indicate that these patterns are not artifacts of model uncertainty, but reflect consistent signals in guest feedback that can guide targeted improvements.

6.2 Discussion

6.2.1 Addressing the research questions

The results of this study reveal important insights about the application of multilingual language models for sentiment analysis of hotel reviews. The findings demonstrate that both mBERT and XLM-RoBERTa models with sentence pairing techniques can effectively classify sentiment in multilingual hotel reviews, with XLM-RoBERTa showing slightly higher performance metrics but requiring more computational resources. The analysis also provides valuable insights into guest experiences at 5-star hotels in Jakarta, identifying cleanliness, facilities, and service as highly appreciated aspects, while price emerges as a significant challenge.

6.2.2 Success of the methodology

The sentence pairing technique proved highly effective for sentiment analysis in this domain, significantly improving performance for both models compared to their respective baselines. For mBERT, the implementation of sentence pairing increased the F1-score by 1.12%, from 95.56% to 96.66% on test data, while precision improved by 1.41%, from 95.41% to 96.82%. Similarly, XLM-RoBERTa showed improvements with sentence pairing, though the magnitude was somewhat smaller with a 0.71% increase in F1-score and a 0.22% increase in precision. These findings suggest that the sentence-pairing approach may be particularly beneficial for mBERT, potentially due to its architecture's greater responsiveness to contextual information provided through paired sentences.

The zero-shot classification approach for aspect extraction also proved successful, allowing for efficient identification of multiple aspects within single reviews without requiring extensive labeled data. This addresses a common challenge in ABSA, particularly for languages with limited resources, such as Indonesian. The comprehensive evaluation metrics, including Matthews Correlation Coefficient (MCC) and Area Under the ROC Curve (AUC), further validate the robustness of the models, with both achieving AUC values exceeding 0.99.

6.2.3 Performance comparison with previous studies

The findings align with and extend previous research in several key ways. The superior performance of XLM-RoBERTa compared to mBERT is consistent with observations done by research [2], who found that XLM-R outperformed mBERT in text classification tasks for the Indonesian language. Their study attributed this advantage to XLM-R's larger architecture and more extensive, balanced training dataset. However, our research adds nuance to this finding by demonstrating that while XLM-RoBERTa achieves higher peak performance, mBERT offers significantly better stability across cross-validation folds, with 5.3 times lower standard deviation in accuracy.

The effectiveness of sentence pairing for ABSA confirms and builds upon the findings from research [3], who introduced this approach for Indonesian hotel reviews using mBERT. Our results extend their work by demonstrating the applicability of this technique to XLM-RoBERTa and providing a direct performance comparison between the two models. The observed improvements in F1-scores are comparable to their reported enhancements, though our study reveals interesting differences in how each model responds to the sentence pairing intervention.

Our findings regarding the importance of specific hotel aspects align with research [18], who also found that aspects related to service and facilities significantly influenced guest satisfaction in luxury hotels. However, our results add the insight that cleanliness emerged as the most consistently positively rated aspect (99.4% positive sentiment), while price represented the greatest challenge (11.2% negative sentiment), which may reflect the specific context of Jakarta's luxury hotel market.

6.2.4 Understanding guest experiences

The comprehensive sentiment analysis across seven key aspects provides valuable insights for hotel management. The extremely high positive sentiment for cleanliness (99.4%) suggests that 5-star hotels in Jakarta are exceeding guest expectations in this domain. Similarly, the strong positive sentiment towards facilities (95.8%) and service (96.0%) indicates these are core strengths that should be maintained. These findings align with the regulatory requirements outlined in Peraturan Kementerian Pariwisata dan Ekonomi Kreatif Nomor 4 Tahun 2021, which emphasizes standards for facilities, personnel, and service management.

Conversely, the relatively high negative sentiment regarding price (11.2%) suggests a potential misalignment between guest expectations and perceived value. This presents an opportunity for hotel management to either adjust pricing strategies or enhance the communication of value propositions to guests. The keyword analysis, which highlighted terms like "staff," "kamar," and "breakfast" across sentiment categories, provides further granularity in understanding specific elements that contribute to guest satisfaction or dissatisfaction.

6.2.5 Methodological implications

From a methodological perspective, the findings of this study provide several important insights for the design and deployment of multilingual ABSA models in the hospitality domain. The first concerns the trade-off between peak performance and stability when comparing XLM-RoBERTa and mBERT. XLM-RoBERTa consistently achieved higher accuracy, F1-score, and lower loss on the test set than mBERT, particularly for minority sentiment classes, demonstrating a clear advantage when maximum predictive performance is the primary objective. At the same time, cross-validation results showed that XLM-RoBERTa exhibited greater variability across folds, whereas mBERT produced more stable and predictable scores. This pattern suggests that XLM-RoBERTa is better suited for one-off, high-precision analytical tasks, while mBERT may be preferable in operational settings that require regular retraining and highly consistent behaviour over time.

The architectural differences between the two models help explain this behaviour. XLM-RoBERTa is pretrained on the CommonCrawl-based CC100 corpus, which is substantially larger and more balanced across 100 languages than the Wikipedia-based corpus used for multilingual BERT (mBERT) [1, 18]. This broader and more heterogeneous pretraining data makes XLM-RoBERTa more robust to noisy, user-generated content and better able to model low-resource languages such as Indonesian, especially when mixed with English in code-switched reviews. By contrast, mBERT’s reliance on Wikipedia text makes it more sensitive to domain shift when exposed to informal hotel reviews containing slang, spelling variations, and non-standard expressions [19, 20]. These pretraining differences are reflected in the empirical results of this study, where XLM-RoBERTa more effectively captures nuanced aspect–sentiment relations, particularly for negative and neutral classes that are under-represented in the dataset, in line with previous comparative evaluations of the two models on multilingual benchmarks [1].

Tokenizer design further contributes to XLM-RoBERTa’s advantage. XLM-RoBERTa employs a SentencePiece tokenizer with a language-agnostic subword vocabulary, which is well suited to handling morphologically rich words and mixed-language tokens that frequently appear in Indonesian–English hotel reviews [1, 20]. This configuration allows rare, misspelled, or code-mixed terms to be represented with relatively few, semantically meaningful subwords. In contrast, mBERT’s WordPiece tokenizer, as introduced in the original BERT architecture, tends to segment such tokens into longer subword sequences [19]. This increases sequence length and can dilute important contextual cues. In practical terms, XLM-RoBERTa can form more coherent representations for domain-specific expressions such as “upgrade kamar”, “late checkout”, or mixed Indonesian–English phrases, which supports its higher peak performance on the ABSA task observed in this study.

The results also clarify why the sentence-pairing strategy benefits the two models to different degrees. mBERT was pretrained not only with masked language modeling but also with a next-sentence prediction (NSP) objective [19]. Reformulating ABSA as a sentence-pair classification problem—where each review is paired with an auxiliary aspect–sentiment sentence—closely aligns with this NSP-style pretraining and directly leverages mBERT’s capacity to model relationships between two textual segments. Prior work has shown that such sentence-pair formulations can substantially improve ABSA performance for mBERT in Indonesian settings [3, 21, 22], and the present study confirms this pattern: introducing sentence pairing increased mBERT’s F1-score by 1.12 points and precision by 1.41 points on the test set compared to its single-sentence baseline. In effect, sentence pairing provides mBERT with additional task structure that sharpens aspect–sentiment boundaries, particularly in reviews containing multiple aspects or implicit sentiment.

XLM-RoBERTa, in contrast, is pretrained solely with a masked language modeling objective and does not include an explicit NSP component [1]. Its strong single-sequence contextual representations, learned from a much larger multilingual corpus, already capture many aspect–sentiment relationships without requiring an auxiliary sentence. As a result, reframing ABSA as a binary sentence-pair classification task yields a smaller but still positive improvement: in this study, sentence pairing increased XLM-RoBERTa’s F1-score by 0.71 points and precision by 0.22 points relative to its baseline. Intuitively, sentence pairing refines an already strong decision boundary for XLM-RoBERTa, while it acts as a stronger performance multiplier for mBERT by aligning more directly with its pretraining objectives and compensating for its more limited pretraining corpus.

These observations have direct methodological implications for model and task design. For applications where computational resources are constrained or where stability and reproducibility across training runs are critical, mBERT with sentence pairing offers an attractive balance between accuracy and consistency. For use cases that prioritize maximum discriminatory power on noisy, code-mixed data and can tolerate somewhat greater variance across runs, XLM-RoBERTa with sentence pairing is a preferable choice. More broadly, the results suggest that sentence-pair formulations are particularly valuable for models whose pretraining includes sentence-level relationship objectives, while models trained solely with masked language modeling may derive more limited incremental benefit from the same task reformulation.

Finally, the study demonstrates that combining zero-shot classification for aspect and sentiment labeling with sentence-pair fine-tuning is a practical and effective strategy in low-resource settings. Zero-shot labeling made it possible to construct a large, aspect-specific training set without manual annotation, while the subsequent sentence-pair models achieved high AUC values (above 0.99) and strong Matthews Correlation Coefficients despite pronounced class imbalance. This pipeline shows that domain practitioners can obtain robust, aspect-level sentiment models for multilingual hotel reviews even when labeled data are scarce, provided that careful attention is paid to model choice, tokenizer behaviour, and the alignment between pretraining objectives and the downstream task formulation.

8.1 Current limitations

This research, while providing valuable insights into ABSA for five-star hotel reviews in Jakarta, is subject to several limitations:

•Assumptions: The study assumes that online reviews from TripAdvisor and Google Reviews accurately represent guest experiences at five-star hotels in Jakarta. It also presumes that the multilingual models (mBERT and XLM-RoBERTa) can handle mixed Indonesian-English reviews without significant bias toward either language.

•Number of variables: The analysis was limited to seven main aspects (room, location, services, price, food, cleanliness, and facilities) extracted using zero-shot classification. Other potentially relevant aspects, such as technology, security, or unique guest experiences, were not included in this study.

•Constraints:

-Data limitations: Only 129,519 reviews in Indonesian and English from 2022 onwards were analyzed. Reviews in other languages or from earlier periods were excluded, which may limit the generalizability of findings.

-Sampling: Of the total aspect-level data generated (over 350,000 entries), only 20,000 samples were used for model training and evaluation to ensure computational efficiency. This means a broader range of data variability was not fully captured.

-Model and methodology: The study focused solely on comparing two multilingual transformer models (mBERT and XLM-RoBERTa) using the sentence-pairing approach. Other models or ensemble techniques that could potentially enhance performance were not explored.

•Aspect Detection Pipeline: The current models are unable to autonomously identify specific aspects from raw reviews, necessitating a separate zero-shot learning step for aspect extraction prior to sentiment classification. This two-stage approach may limit end-to-end efficiency and integration.

8.2 Recommendations for future research

To address these limitations and advance the field, several directions are recommended for future studies:

•Evaluate sentence-pair ABSA across architectures: Future research should systematically test the sentence-pair classification strategy on additional multilingual encoders such as DeBERTa-v3, mT5, or LLaMA-based models, to determine whether the performance gains observed for mBERT and XLM-RoBERTa in this study generalize across different pretraining objectives and architectures.

•Develop joint aspect sentiment models: Instead of separating zero-shot aspect extraction and supervised sentiment classification, future studies could design multi-task or end-to-end architectures that simultaneously detect aspects and predict their sentiment, thereby reducing error propagation between stages and simplifying deployment in production hotel review analytics pipelines.

•Broaden variables and aspects: Future work may include additional aspects such as technology features, security, or sustainability-related attributes to capture a more comprehensive picture of guest expectations and to test whether the current seven-aspect schema remains sufficient for luxury hotel analysis in evolving market conditions.

•Widen data sources, timeframes, and regions: Collecting data from more review platforms, extending the temporal range beyond the current post-pandemic window, and incorporating reviews from different hotel categories or cities (for example, other major Indonesian or Southeast Asian destinations) would enable cross-city or cross-country comparisons and help assess the robustness of the proposed approach under domain shift.

•Improve handling of class imbalance and rare patterns: Subsequent studies should explore more advanced techniques for dealing with highly skewed sentiment distributions, such as focal loss, curriculum sampling, or contrastive learning, to further enhance performance on negative and neutral classes and on aspects with fewer mentions, such as price-related complaints identified in this work.

By addressing these limitations and pursuing the suggested directions, future research can provide deeper and more comprehensive insights into ABSA in the hospitality industry, particularly in multilingual and dynamic urban contexts like Jakarta.