An End-to-End BERT–LSTM Framework for Aspect-Based Sentiment Analysis of Animated Movie Reviews

Animated films have become a dominant force in the global entertainment industry, with studios such as Pixar and DreamWorks producing content that attracts millions of viewers across age groups. These films are no longer limited to children but appeal to a broader demographic due to their complex narratives and artistic visuals. As reported in the global animation industry reached a market value of USD 373.23 billion in 2024 and is projected to grow to USD 393.3 billion by 2025 [1]. This rapid development has heightened audience expectations, leading to a surge in user-generated reviews that express opinions on specific aspects such as plot, character design, animation quality, and moral messages.

Digital platforms like Internet Movie Database (IMDb), Rotten Tomatoes, and Letterboxd have enabled viewers to provide not only ratings but also narrative reviews that reflect detailed viewing experiences. According to the study by Ha [2], audience engagement has evolved from mere consumption to active participation through opinion sharing. For instance, “Kung Fu Panda 4” received praise for maintaining visual consistency [3], yet was also criticized for its underwhelming action scenes [4]. Similarly, “Elemental” was commended for its visual quality [5], while the plot was deemed too predictable, raising suspicions of AI-generated writing [6]. These varied perspectives highlight the need to analyze audience sentiment at a more granular, aspect-specific level.

Although a significant number of user reviews are available, scientific methods for extracting meaningful insights from such data remain limited. Traditional sentiment analysis techniques, including lexicon-based approaches and classical machine learning models, often fail to capture complex language nuances. For example, the Support Vector Machine (SVM)-based analysis [7] on “Squid Game” reviews achieved high accuracy but lacked generalization across domains. Similarly, lexicon-based approaches such as those in the study by Wang et al. [8] could not effectively detect implicit sentiments like sarcasm. Transformer-based methods like HauBERT [9] show promise but still face challenges related to imbalanced data and domain-specific adaptation. In Indonesia, IndoBERT was applied in the study by Hendriyani and Budayawan [10] to classify Instagram comments, achieving an accuracy of 91.57%, although the analysis was limited to sentence-level polarity rather than aspect-specific sentiment.

Aspect-Based Sentiment Analysis (ABSA) addresses these limitations by extracting opinions related to specific components within a text [11]. In the context of animated films, ABSA enables producers to understand how audiences perceive particular features such as animation quality or storyline depth, thereby facilitating more targeted improvements. However, despite the abundance of rich user-generated content, the use of ABSA for analyzing animated film reviews remains underexplored.

ABSA comprises two main tasks: aspect extraction and sentiment classification [12]. Aspect extraction aims to identify specific topics or features mentioned in a text [13], and recent advances in deep learning—especially with Bidirectional Encoder Representations from Transformers (BERT)—have shown strong performance due to contextual understanding capabilities. Studies [14] achieved up to 98% accuracy in multi-domain ABSA using fine-tuned BERT models, while Chauhan et al. [15] reported F1-score improvements of 2.5% to 5% over Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) baselines. Joint ABSA modeling approaches—where aspect extraction and sentiment classification are performed simultaneously—have also been explored, such as the attention-based graph convolutional model in the study by Liang et al. [12], which leverages affective knowledge to improve joint prediction. While joint approaches can reduce error propagation across pipeline stages, their architectural complexity and domain-specific tuning requirements may limit their applicability in practical deployments. In contrast, conventional models such as SVM and Term Frequency-Inverse Document Frequency (TF-IDF) are less effective in capturing both syntactic and semantic features [16].

Sentiment classification, which determines the polarity of user opinions toward identified aspects [17], has also benefited from deep learning approaches. Long Short-Term Memory (LSTM) models, in particular, have proven effective in capturing sequential patterns in textual data. In the study by Cahyaningtyas et al. [18], an LSTM-based model achieved 89.6% accuracy in classifying Indonesian film reviews, while Mutmainah et al. [19] reported 92.5% accuracy using a similar architecture. These results outperformed traditional classifiers like Naive Bayes and SVM, which often struggle with sparse or imbalanced feature spaces [20]. In this study, LSTM is employed for the sentiment classification stage because it captures sequential dependencies in text and pairs naturally with the aspect context via a dual-input architecture. While fine-tuning BERT for the classification stage would represent an alternative approach, the dual-model pipeline was adopted to allow modular evaluation of each component and to reduce computational overhead in the deployed Streamlit system.

Motivated by these findings, this study proposes a pipeline-based ABSA system that combines BERT for aspect extraction and LSTM for sentiment classification. The system is applied to user reviews of animated films produced by Pixar and DreamWorks between 2022 and 2024 as a domain-specific application study. The objectives of this study are: (1) to evaluate the effectiveness of the BERT-LSTM pipeline in identifying dominant aspects and their associated sentiments; (2) to compare the pipeline against a baseline model to contextualize performance; and (3) to implement a Streamlit-based web interface with a reproducible data processing architecture that enables interactive exploration of the analysis results. This research contributes to a deeper understanding of audience sentiment and provides a practical system architecture applicable to domain-specific opinion mining tasks.

This section presents the results of the ABSA system, covering dataset characteristics, model development, performance evaluation, baseline comparison, and application implementation.

3.1 Dataset and data preparation

This study utilized user reviews of animated films produced by DreamWorks and Pixar released between 2022 and 2024. A total of 11 films were selected, comprising seven titles from DreamWorks and four from Pixar. Reviews were collected from IMDb.com via Selenium-based web scraping, resulting in 6,453 raw reviews. The detailed distribution of reviews is presented in Table 2.

Table 2. Distribution of reviews by studio, year, and title

Table 3. Sample format for Bidirectional Encoder Representations from Transformers (BERT)-based aspect extraction

Following collection, reviews underwent the preprocessing pipeline described in Section 2.1, producing 6,987 annotated sentences. For the aspect extraction task, sentences were tokenized using the BERT-base-uncased tokenizer and annotated with BIO tags. Table 3 illustrates the input format used for BERT training.

For the sentiment classification task, 4,075 sentence–aspect pairs were created and labeled with binary polarity (positive = 1, negative = 0). Table 4 presents the dataset structure used for LSTM training.

Table 4. Sample format for Long Short-Term Memory (LSTM)-based sentiment classification

3.2 Bidirectional Encoder Representations from Transformers model performance for aspect extraction

Following BIO annotation of 6,987 sentences, the BERT-base-uncased model was fine-tuned for token classification. Training was conducted for three epochs using the AdamW optimizer. The full training configuration is summarized in Table 5.

Evaluation on the validation set yielded the results shown in Table 6. The primary performance metric is the span-level F1-score of 83.06%, which measures the model's ability to correctly identify complete aspect spans (not merely individual tokens). Token-level accuracy of 99.20% is reported for completeness; however, it is not the primary metric because the BIO tag distribution is heavily skewed toward the "O" label, making token accuracy an overly optimistic measure of true aspect recognition performance.

BERT's bidirectional contextual understanding proved especially advantageous for long sentences containing multiple opinions. In many cases, the model successfully extracted more than one aspect from a single sentence, even when aspects were separated by complex grammatical structures. For example, in the sentence "The animation was beautiful, but the story was boring," the model correctly identified both animation and story as distinct aspects.

Table 5. Training parameters for the Bidirectional Encoder Representations from Transformers (BERT) model

Table 6. Bidirectional Encoder Representations from Transformers (BERT) model performance

However, several limitations were observed. For multi-word aspect terms (e.g., "moral message"), the model occasionally misclassified tokens as separate entities rather than a single span (labeling as B-ASP instead of B-ASP I-ASP). Error analysis revealed that span boundary errors accounted for approximately 58% of misclassifications, while false negatives (missed aspects) comprised approximately 29%, and false positives (incorrectly predicted aspects) approximately 13%. These patterns are consistent with difficulties in recognizing infrequent multi-word phrases and implicit or metaphorical aspect references.

3.3 Long Short-Term Memory model performance for sentiment classification

The sentiment classification stage used the 4,075 manually labeled sentence–aspect pairs divided into 3,260 training and 815 test instances via stratified sampling. Embeddings were initialized randomly; the use of randomly initialized rather than pretrained embeddings represents a design trade-off: it avoids vocabulary mismatch with the general-purpose GloVe corpus but may limit representational richness for domain-specific terms. The LSTM architecture is described in Section 2.2. Model evaluation on the 20% held-out test set is presented in Table 7.

Table 7. Classification report for Long Short-Term Memory (LSTM) sentiment classifier

Evaluation: The model achieved 82% accuracy on the test set, with balanced precision and recall across both classes, indicating no significant bias toward either sentiment category. The confusion matrix (Figure 5) showed 335 true negatives and 405 true positives, with 102 false positives and 73 false negatives.

5.png

Figure 5. Confusion matrix of the Long Short-Term Memory (LSTM) model

The complete dual-input LSTM architecture is illustrated in Table 8.

Table 8. Long Short-Term Memory (LSTM) model architecture

Misclassifications were most frequent in sentences conveying indirect or implicit sentiment, particularly those involving metaphorical language, polarity-reversing phrases (e.g., "the dark tone of the ending was refreshing"), or domain-specific references. Quantitatively, implicit/metaphorical sentiment errors accounted for approximately 47% of total misclassifications, while ambiguous or sarcastic expressions accounted for approximately 31%, and domain-specific idioms accounted for approximately 22%. The absence of an attention mechanism may further limit the model's ability to assign appropriate semantic weight to sentiment-bearing phrases within long inputs.

3.4 Baseline comparison

To contextualize the pipeline's performance, a TF-IDF + Logistic Regression baseline was trained and evaluated on the same dataset and split. Results are summarized in Table 9.

Table 9. Performance comparison: BERT-LSTM pipeline vs baseline

The BERT–LSTM pipeline outperforms the baseline by approximately 21.6 percentage points in accuracy and approximately 21.8 points in macro F1-score, demonstrating the benefit of deep contextual representation over sparse feature-based methods for this domain-specific ABSA task. These results confirm that the computational cost of the BERT–LSTM approach yields meaningful performance gains for fine-grained sentiment analysis.

3.5 Integrated Aspect-Based Sentiment Analysis pipeline performance

The integrated ABSA pipeline—comprising preprocessing, BERT-based aspect extraction, and LSTM sentiment classification—was evaluated end-to-end using the 4,075 manually annotated sentence–aspect pairs. Pipeline accuracy is computed at the pair level: a prediction is counted as correct only when both the extracted aspect and its predicted sentiment match the ground-truth annotation. Results are presented in Table 10.

Table 10. Classification report for the Aspect-Based Sentiment Analysis (ABSA) pipeline

The pipeline achieved an end-to-end accuracy of 91.8%, with macro-averaged precision, recall, and F1-score all at 0.918, indicating balanced and stable performance across both sentiment classes. The confusion matrix (Figure 6) confirmed 1,848 true negatives and 1,892 true positives, with 187 false positives and 148 false negatives. The relatively even distribution of errors across classes reinforces the absence of systematic class bias.

6.png

Figure 6. Confusion matrix of Aspect-Based Sentiment Analysis (ABSA) pipeline

These results confirm that the BERT–LSTM ABSA pipeline operates effectively as an integrated system. However, final performance remains bounded by the limitations of each component: BERT's difficulty with implicit multi-word aspects may introduce suboptimal inputs to the LSTM classifier, and LSTM's sensitivity to sarcastic or polarity-reversing language can reduce end-to-end accuracy. These observations are consistent with the known trade-offs of sequential pipeline architectures compared to joint modeling approaches. It should also be noted that the dataset is limited to English-language reviews of animated films from two studios (2022–2024); cross-domain generalizability has not been evaluated, and performance may differ for other genres, languages, or time periods.

3.6 Dominant aspects and sentiment trends

The fine-tuned BERT model was employed to extract aspect terms from user reviews of animated films. Frequency analysis revealed the ten most frequently mentioned aspects: character (6,141), story (4,980), animation (3,164), plot (1,588), scene (1,264), voice (1,067), message (965), theme (958), villain (938), and humor (671), as illustrated in Figure 7. These findings indicate that audiences predominantly focus on narrative and visual elements, particularly characters, storylines, and animation quality.

A complementary word cloud visualization (Figure 8) confirmed these results, with additional aspects such as theme, music, visuals, and style emerging, indicating user interest in aesthetic qualities.

7.png

Figure 7. Ten highest frequency aspects

8.png

Figure 8. Word cloud of aspects in animated film reviews

9.png

Figure 9. Sentiment distribution for popular aspects

Sentiment analysis across the ten most frequent aspects (Figure 9) revealed that animation received consistently positive feedback, with viewers praising visual quality and artistic style. Character reviews were predominantly positive, commending emotional depth and strong main character arcs, though some noted underdeveloped supporting roles. The story was generally well-received for its thematic depth and emotional resonance, with some criticism directed at overly simple structures and rushed pacing.

Conversely, plot and villain were associated with higher proportions of negative sentiment. Plot-related criticism cited predictability and lack of originality, while villain reviews highlighted antagonists perceived as clichéd or lacking motivational depth. These observations highlight areas for creative improvement in animated storytelling.

3.7 Streamlit interface system and computational performance

The ABSA system was deployed as a web-based interface using Streamlit Community Cloud, accessible at https://animated-movie-absa.streamlit.app. The interface supports English-language animated film review input and returns extracted aspects with their sentiment classifications. The system follows the modular architecture described in Section 2.5.

System performance was measured across 50 sample inputs of varying lengths. For single-sentence inputs (1–2 sentences), average response latency was 0.8 seconds. For paragraph-level inputs (5–10 sentences), average latency was 1.6 seconds. Peak throughput on the Streamlit Community Cloud deployment was observed at approximately 30 concurrent requests before response time degraded beyond 3 seconds. These measurements indicate that the current deployment is suitable for demonstration and small-scale exploratory use but would require infrastructure scaling (e.g., dedicated GPU hosting, model batching) for high-volume production deployment.

The main interface (Figure 10) consists of a text input field, a "Sentiment Analysis" trigger button, a completion status indicator, a results table displaying segmented sentences, detected aspects, and predicted sentiment, as well as bar and pie charts illustrating sentiment distribution and aspect frequency.

10.png

Figure 10. Streamlit web interface for Aspect-Based Sentiment Analysis (ABSA) system