Implementation of BERT-BiLSTM Model for Sentiment Classification of Movie Reviews on Rotten Tomatoes

Abstract

This study aims to analyze movie review sentiment using a Bidirectional Encoder Representations from Transformers-Bidirectional Long Short-Term Memory (BERT-BiLSTM) hybrid model designed to capture global contextual information and local sequential dependencies in complex texts. The dataset is obtained from Rotten Tomatoes Critic Reviews, consisting of 1,048,576 entries, of which 10,000 samples are randomly selected due to computational limitations. The research methodology includes text preprocessing, tokenization with the BERT tokenizer, model training, and performance evaluation using accuracy, precision, recall, and F1-score. The implementation uses the AdamW optimizer with a learning rate of 2×10⁻⁵ for three epochs on an AMD Ryzen 3 5300U CPU. For comparison, the hybrid model is evaluated alongside a single BERT model and a single BiLSTM model. Experimental results show that the BERT-BiLSTM model achieves the highest performance, obtaining an accuracy, precision, recall, and F1-score of 84%. In contrast, the single BERT model achieved 66% accuracy and an F1 score of 57%, while the single BiLSTM model achieved 63% accuracy and an F1 score of 48%. These results indicate that combining transformer-based contextual representation with bidirectional sequence modeling improves the effectiveness of sentiment classification compared to the single-model approach. Therefore, the BERT-BiLSTM hybrid method proves effective for analyzing complex textual sentiment and has potential applications across various review domains that require deep contextual understanding.