Hybrid Framework Addressing Imbalanced Data in Indonesian E-commerce Multimodal Sentiment Analysis

Shabrina Rasyid Munthe; Samsir; Rina Asriani Levianti

doi:10.34148/teknika.v14i3.1332

Authors

Shabrina Rasyid Munthe Informatics Department, Faculty of Computer Science, Universitas Al Washliyah Labuhanbatu, Rantauprapat, Indonesia
Samsir Informatics Department, Faculty of Computer Science, Universitas Al Washliyah Labuhanbatu, Rantauprapat, Indonesia
Rina Asriani Levianti Management Department, Faculty of Economics, Universitas Al Washliyah Labuhanbatu, Rantauprapat, Indonesia

DOI:

https://doi.org/10.34148/teknika.v14i3.1332

Keywords:

Sentiment Analysis, E-Commerce, Class Imbalance, SMOTEENN, Feature Extraction

Abstract

Severe class imbalance in Indonesian language e-commerce reviews hampers the detection of customer grievances, because about 88 percent of testimonials are positive whereas under one percent are negative. This study designs a simple yet effective framework to mitigate that bias. The data 2,611 authentic reviews covering electronics, fashion, and household goods and reflecting online shopping patterns were cleaned, structured, and represented using a combination of TF-IDF weighting and Word2Vec embeddings to condense meaning. Imbalance was addressed with the Synthetic Minority Oversampling Technique integrated with Edited Nearest Neighbour, equalising the proportions of positive, neutral, and negative classes. A Support Vector Machine was trained with five-fold cross-validation and benchmarked against Multinomial Naïve Bayes and a Decision Tree classifier. Experiments yielded 94.6 percent accuracy and a 94.8 percent F1-score, while precision for the negative class reached 100 percent, outperforming conventional approaches by up to twelve percent. The contribution of this research is the demonstration that minimal pre-processing, lightweight feature extraction, and targeted data balancing can markedly enhance grievance detection without resorting to complex model architectures. Practically, the framework enables online merchants to monitor service quality and respond to customer issues in real time, and it can be integrated into recommendation engines and business dashboards to accelerate data-driven decision making and strengthen transparency for stakeholders. Academically, the findings open avenues for integrating large language models, temporal analysis, and cross-domain adaptation to enrich the global e-commerce ecosystem.

Downloads

Download data is not yet available.

References

[1] W. Xiong, Y. Zuo, M. Zhang, C. Zhang, dan C. Guo, “Research on Sentiment Analysis of E-commerce Live Comments based on Text Mining,” Front. Comput. Intell. Syst., vol. 6, no. 3, hal. 34–36, 2024, doi: 10.54097/c2wofcb2.

[2] F. Pang, X. Chen, L. Li, X. Xin, dan Z. Xing, “F_MixBERT: Sentiment Analysis Model Using Focal Loss for Imbalanced E-Commerce Reviews,” Ksii Trans. Internet Inf. Syst., vol. 18, no. 2, 2024, doi: 10.3837/tiis.2024.02.001.

[3] S. A. Elzeheiry, W. A. Gab-Allah, N. Mekky, dan M. Elmogy, “Sentiment Analysis for E-Commerce Product Reviews: Current Trends and Future Directions,” 2023, doi: 10.20944/preprints202305.1649.v1.

[4] E. Nazaruddin, C. Caroline, Andrijanni, dan U. S. Sulistyawati, “Analyzing Customers in E-Commerce Using Dempster-Shafer Method,” Int. J. Softw. Eng. Comput. Sci., vol. 3, no. 2, hal. 174–183, 2023, doi: 10.35870/ijsecs.v3i2.1497.

[5] S. Gupta dan A. Kumar, “Sentiment-Aware Enhancements of PageRank-based Citation Metric, Impact Factor, and H-Index for Ranking the Authors of Scholarly Articles,” Comput. Sci., vol. 25, no. 2, 2024, doi: 10.7494/csci.2024.25.2.6042.

[6] A. Bourchouq dan P. Wang, “Design of a Model in Machine Learning for Credit Card Fraud Detection,” Ceis, 2022, doi: 10.7176/ceis/13-2-06.

[7] I. G. B. A. Budaya dan I. K. P. Suniantara, “Comparison of Sentiment Analysis Algorithms with SMOTE Oversampling and TF-IDF Implementation on Google Reviews for Public Health Centers,” MALCOM Indones. J. Mach. Learn. Comput. Sci., vol. 4, no. 3, hal. 1077–1086, Jul 2024, doi: 10.57152/malcom.v4i3.1459.

[8] R. S. Abdulsadig dan E. Rodriguez-Villegas, “A comparative study in class imbalance mitigation when working with physiological signals,” Front. Digit. Heal., vol. 6, no. March, hal. 1–11, 2024, doi: 10.3389/fdgth.2024.1377165.

[9] J. Tang et al., “Joint modeling strategy for using electronic medical records data to build machine learning models: an example of intracerebral hemorrhage,” BMC Med. Inform. Decis. Mak., vol. 22, no. 1, 2022, doi: 10.1186/s12911-022-02018-x.

[10] A. Gandhi, K. Adhvaryu, S. Poria, E. Cambria, dan A. Hussain, “Multimodal Sentiment Analysis: A Systematic Review of History, Datasets, Multimodal Fusion Methods, Applications, Challenges and Future Directions,” Inf. Fusion, vol. 91, hal. 424–444, 2023, doi: 10.1016/j.inffus.2022.09.025.

[11] R. A. Putranto, M. D. Purbolaksono, dan W. Astuti, “Sentiment Analysis of Practo Application Reviews Using Naïve Bayes and TF-IDF Methods,” J. Media Inform. Budidarma, vol. 7, no. 3, hal. 1070, 2023, doi: 10.30865/mib.v7i3.6311.

[12] M. E. Purbaya, D. P. Rakhmadani, M. P. Arum, dan L. Z. Nasifah, “Implementation of n-gram Methodology to Analyze Sentiment Reviews for Indonesian Chips Purchases in Shopee E-Marketplace,” J. RESTI, vol. 7, no. 3, hal. 609–617, 2023, doi: 10.29207/resti.v7i3.4726.

[13] A. Onan, “Sentiment analysis on product reviews based on weighted word embeddings and deep neural networks,” Concurr. Comput. Pract. Exp., vol. 33, no. 23, hal. 1–12, 2021, doi: 10.1002/cpe.5909.

[14] G. Kim dan C. D. Yoo, “Blending Query Strategy of Active Learning for Imbalanced Data,” Ieee Access, vol. 10, hal. 79526–79542, 2022, doi: 10.1109/access.2022.3194068.

[15] R. Obiedat et al., “Sentiment Analysis of Customers’ Reviews Using a Hybrid Evolutionary SVM-Based Approach in an Imbalanced Data Distribution,” IEEE Access, vol. 10, hal. 22260 – 22273, 2022, doi: 10.1109/ACCESS.2022.3149482.

[16] K. Ghag dan K. Shah, “Conceptual Sentiment Analysis Model,” Int. J. Electr. Comput. Eng., vol. 8, no. 4, hal. 2358, 2018, doi: 10.11591/ijece.v8i4.pp2358-2366.

[17] J. P. Munggaran, A. A. Alhafidz, M. Taqy, D. A. R. Agustini, dan M. Munawir, “Sentiment Analysis of Twitter Users’ Opinion Data Regarding the Use of ChatGPT in Education,” J. Comput. Eng. Electron. Inf. Technol., vol. 2, no. 2, hal. 75–88, 2023, doi: 10.17509/coelite.v2i2.59645.

[18] A. W. Pradana dan M. Hayaty, “The Effect of Stemming and Removal of Stopwords on the Accuracy of Sentiment Analysis on Indonesian-Language Texts,” Kinet. Game Technol. Inf. Syst. Comput. Netw. Comput. Electron. Control, hal. 375–380, 2019, doi: 10.22219/kinetik.v4i4.912.

[19] J. Kavanagh, K. Greenhow, dan A. Jordanous, “Assessing the Effects of Lemmatisation and Spell Checking on Sentiment Analysis of Online Reviews,” Proc. - 17th IEEE Int. Conf. Semant. Comput. ICSC 2023, hal. 235–238, 2023, doi: 10.1109/ICSC56153.2023.00046.

[20] R. Kosasih dan A. Alberto, “Sentiment Analysis of Game Product on Shopee Using the TF-IDF Method and Naive Bayes Classifier,” Ilk. J. Ilm., vol. 13, no. 2, hal. 101–109, 2021, doi: 10.33096/ilkom.v13i2.721.101-109.

[21] S. A. Ryanto, D. Richasdy, dan W. Astuti, “Partner Sentiment Analysis for Telkom University on Twitter Social Media Using Decision Tree (CART) Algorithm,” J. Media Inform. Budidarma, vol. 6, no. 4, hal. 1940, 2022, doi: 10.30865/mib.v6i4.4533.

[22] J. Zhang, “Weibo Text Sentiment Classification Model Based on FastText-BERT-Attention,” vol. 2, no. 1, hal. 29–35, 2024, doi: 10.62517/jbdc.202401104.

[23] X. Zou, Y. Feng, H. Li, dan S. Jiang, “SROT: Sparse Representation-Based Over-Sampling Technique for Classification of Imbalanced Dataset,” Iop Conf. Ser. Earth Environ. Sci., vol. 81, hal. 12201, 2017, doi: 10.1088/1755-1315/81/1/012201.

[24] S. S. Pangastuti, K. Fithriasari, N. Iriawan, dan W. Suryaningtyas, “Classification Boosting in Imbalanced Data,” Malaysian J. Sci., vol. 38, hal. 36–45, 2019, doi: 10.22452/mjs.sp2019no2.4.

[25] M. E. Purbaya, D. P. Rakhmadani, M. P. Arum, dan L. Z. Nasifah, “Implementation of N-Gram Methodology to Analyze Sentiment Reviews for Indonesian Chips Purchases in Shopee E-Marketplace,” J. Resti (Rekayasa Sist. Dan Teknol. Informasi), vol. 7, no. 3, hal. 609–617, 2023, doi: 10.29207/resti.v7i3.4726.

[26] P. Vuttipittayamongkol, E. Elyan, dan A. Petrovski, “On the class overlap problem in imbalanced data classification,” Knowledge-Based Syst., vol. 212, hal. 106631, 2021, doi: 10.1016/j.knosys.2020.106631.

[27] N. K. Gondhi, Chaahat, E. Sharma, A. H. Alharbi, R. Verma, dan M. A. Shah, “Efficient Long Short-Term Memory-Based Sentiment Analysis of E-Commerce Reviews,” Comput. Intell. Neurosci., vol. 2022, 2022, doi: 10.1155/2022/3464524.

[28] A. Serlina, A. Rahim, dan Arbansyah, “Comparative Analysis of Naïve Bayes Algorithm Performance in English and Indonesian Text Sentiment Classification on Duolingo Application in Playstore,” Teknika, vol. 14, no. 1, hal. 165–171, 2025, doi: 10.34148/teknika.v14i1.1207.

[29] H. Li, Y. Lu, H. Zhu, dan Y. Ma, “A Novel AB-CNN Model for Multi-Classification Sentiment Analysis of e-Commerce Comments,” Electron., vol. 12, no. 8, 2023, doi: 10.3390/electronics12081880.

[30] E. Aljohani, “Enhancing Arabic Fake News Detection: Evaluating Data Balancing Techniques Across Multiple Machine Learning Models,” Eng. Technol. Appl. Sci. Res., vol. 14, no. 4, hal. 15947 – 15956, 2024, doi: 10.48084/etasr.8019.

[31] M. H. Jamal, N. Naz, M. A. K. Khattak, F. Saeed, S. N. Altamimi, dan S. N. Qasem, “A Comparison of Re-Sampling Techniques for Detection of Multi-Step Attacks on Deep Learning Models,” IEEE Access, vol. 11, hal. 127446 – 127457, 2023, doi: 10.1109/ACCESS.2023.3332512.