Food Portion Weight Prediction and Nutritional Estimation from Images Using YOLOv8 Segmentation and XGBoost Regression

Ery Setiyawan Jullev Atmadji; Freda Adi Ferdana; Husin; Aji Seto Arifianto

doi:10.34148/teknika.v15i2.1474

Authors

Ery Setiyawan Jullev Atmadji Information Technology Department, Politeknik Negeri Jember, Jember, East Java, Indonesia
Freda Adi Ferdana Information Technology Department, Politeknik Negeri Jember, Jember, East Java, Indonesia
Husin Information Technology Department, Politeknik Negeri Jember, Jember, East Java, Indonesia
Aji Seto Arifianto Information Technology Department, Politeknik Negeri Jember, Jember, East Java, Indonesia

DOI:

https://doi.org/10.34148/teknika.v15i2.1474

Keywords:

Food Detection, YOLOv8, XGBoost, Nutrition Estimation, Computer Vision

Abstract

Understanding the nutritional content of food is essential for maintaining balanced dietary habits. However, most existing nutrition information sources rely on fixed portion sizes and do not reflect the actual amount of food consumed. This study proposes an image-based system for estimating food nutrition dynamically using computer vision and machine learning. The system integrates YOLOv8m-seg for food object detection and segmentation with XGBoost regression for food weight prediction. Images uploaded by users through a Telegram chatbot are processed to detect food containers and segment individual food objects. Features extracted from segmentation results, such as object area and dimensions, are then used to estimate the weight of each food item. Nutritional values including calories, fat, carbohydrates, and protein are calculated based on the predicted weights. Experimental results show that the segmentation model achieved an average accuracy of 87.68%, with several food categories reaching 97–98% accuracy. The weight prediction model obtained an MAE of 21.138 g, RMSE of 50.10 g, and R² of 0.7838, indicating reasonable predictive performance. The developed system demonstrates the potential of combining object detection, segmentation, and regression models to provide automated nutritional estimation through an accessible chatbot interface, supporting more practical and personalized dietary assessment.

Downloads

Download data is not yet available.

References

[1] R. Aslam, S. R. Sharma, J. Kaur, A. S. Panayampadan, and O. I. Dar, “A systematic account of food adulteration and recent trends in the non-destructive analysis of food fraud detection,” J. Food Meas. Charact., vol. 17, no. 3, pp. 3094–3114, Jun. 2023, doi: 10.1007/s11694-023-01846-3.

[2] G. C. Utami, C. R. Widiawati, and P. Subarkah, “Detection of Indonesian Food to Estimate Nutritional Information Using YOLOv5,” Teknika, vol. 12, no. 2, pp. 158–165, Jun. 2023, doi: 10.34148/teknika.v12i2.636.

[3] G. I. Nugraha et al., “The urgency in proposing the optimal obesity cutoff value in Indonesian population: A narrative review,” Medicine (Baltimore), vol. 101, no. 49, p. e32256, Dec. 2022, doi: 10.1097/MD.0000000000032256.

[4] S. Oktaviani, M. Mizutani, R. Nishide, and S. Tanimura, “Factors associated with overweight/obesity of children aged 6–12 years in Indonesia,” BMC Pediatr., vol. 23, no. 1, p. 484, Sep. 2023, doi: 10.1186/s12887-023-04321-6.

[5] “Enhancing Nutritional Predictive Models: Addressing Class Imbalance with Machine Learning,” J. Technol. Humanit., vol. 6, no. 1, pp. 22–32, Jun. 2025, doi: 10.53797/jthkks.v6i1.3a.2025.

[6] R. Ramadhansyah, S. Simatupang, and R. Abdillah, “Development of a YOLO-Based Artificial Intelligence (AI) System for Early Detection of Stunting Risk in Children in 3T Regions of North Sumatra,” J. Comput. Netw. Archit. High Perform. Comput..

[7] A. P. Saputri, A. Taqwa, and S. Soim, “Analisis Deteksi Objek Citra Digital Menggunakan Algoritma YOLO Dan Cnn Dengan Arsitektur Repvgg Pada Sistem Pendeteksian dan Pengenalan Ekspresi Wajah”.

[8] P. Nani, S. Das, and S. Dey, “Enhancing object recognition: a comprehensive analysis of CNN based deep learning models considering lighting conditions and perspectives,” Evol. Intell., vol. 18, no. 4, p. 72, Aug. 2025, doi: 10.1007/s12065-025-01061-7.

[9] R. Afrinanda, W. Tawa Bagus, and L. Efrizoni, “Comparison of Machine Learning Algorithm Models in Bitcoin Price Sentiment Analysis,” Indones. J. Comput. Sci., vol. 12, no. 2, pp. 502–513, Apr. 2023, doi: 10.33022/ijcs.v12i2.3180.

[10] N. Sulistianingsih and G. H. Martono, “Analysis of the Effectiveness of Traditional and Ensemble Machine Learning Models for Mushroom Classification”.

[11] Asrul Abdullah, M. Iwan, and S. Rama Dani, “Applying Tree Based Model for Crop Recommendation System Based on Soil Parameters And Weather Conditions,” JITK J. Ilmu Pengetah. Dan Teknol. Komput., vol. 11, no. 1, pp. 228–235, Aug. 2025, doi: 10.33480/jitk.v11i1.6476.

[12] M. Sakmar, N. T. Kadir, P. A. Shofo, and A. Darmawan, “Efektivitas Xgboost Lightgbm dan Catboost Pada Dataset Imbalanced Predictive Maintenance,” vol. 3, 2026.

[13] H. Wu, Q. Liu, and X. Liu, “A Review on Deep Learning Approaches to Image Classification and Object Segmentation,” Comput. Mater. Contin., vol. 60, no. 2, pp. 575–597, 2019, doi: 10.32604/cmc.2019.03595.

[14] M. Farooqui et al., “Food Classification Using Deep Learning: Presenting a New Food Segmentation Dataset,” Math. Model. Eng. Probl., vol. 10, no. 3, pp. 1017–1024, Jun. 2023, doi: 10.18280/mmep.100336.

[15] Usha Ruby Dr.A, “Binary cross entropy with deep learning technique for Image classification,” Int. J. Adv. Trends Comput. Sci. Eng., vol. 9, no. 4, pp. 5393–5397, Aug. 2020, doi: 10.30534/ijatcse/2020/175942020.