Evaluating the Performance of Machine Learning Classifiers for Network Intrusion Detection: A Comparative Study Using the UNSW-NB15 Dataset

Iwan Handoyo Putro

doi:10.34148/teknika.v14i2.1276

Authors

Iwan Handoyo Putro Electrical Engineering Department, Petra Christian University, Surabaya, East Java, Indonesia

DOI:

https://doi.org/10.34148/teknika.v14i2.1276

Keywords:

Machine Learning, Intrusion Detection Systems, Feature Selection, UNSW-NB15, Models Comparison

Abstract

Network security has become a critical concern in digital data transmission. It is because of their growing adoption and complexity of cyber-attacks. Therefore, protecting network infrastructures and identifying malicious behavior becomes a necessity. This paper gives a comparative performance analysis of multiple machine learning (ML) classifiers for intrusion detection systems (IDS) by using the UNSW-NB15 dataset. To gain better insight into the IDS performances, several ML classifiers are being assessed. This includes the Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), k-Nearest Neighbors (k-NN), Naïve Bayes (NB), and Gradient Boosting (XGB). The performance matrix in the analysis comprises the training score, accuracy, precision, recall, F1-score, training time, and AUC-ROC. The results reveal that DT and RF scored the highest training marks of 99.77%. Regarding accuracy, the RF model achieves the highest percentage at 95.05%. In terms of the computational time, k-NN displayed the lowest training time at 0.01 seconds. These analysis results provide guidance to the selection of appropriate ML-based cyberattack classification. It also provides insights for further research in ML-based cybersecurity systems to support the development of intelligent and efficient IDS solutions.

Downloads

Download data is not yet available.

References

[1] G. Kocher and G. Kumar, "Machine learning and deep learning methods for intrusion detection systems: recent developments and challenges," Soft Computing, vol. 25, no. 15, pp. 9731-9763, 2021.

[2] I. Kabanov and S. E. Madnick, "A Systematic Study of The Control Failures in the Equifax Cybersecurity Incident," SSRN, vol. 2020-19, pp. 1-25, 2020.

[3] A. B. Nassif, M. A. Talib, Q. Nasir, and F. M. Dakalbab, "Machine Learning for Anomaly Detection: A Systematic Review," IEEE Access, vol. 9, pp. 78658-78700, 2021.

[4] A. Alshammari and A. Aldribi, "Apply machine learning techniques to detect malicious network traffic in cloud computing," Journal of Big Data, vol. 8, no. 1, 2021.

[5] A. Thakkar and R. Lohiya, "A survey on intrusion detection system: feature selection, model, performance measures, application perspective, challenges, and future research directions," Artificial Intelligence Review, vol. 55, no. 1, pp. 453-563, 2022.

[6] A. Heidari and M. A. Jabraeil Jamali, "Internet of Things intrusion detection systems: a comprehensive review and future directions," Cluster Computing, vol. 26, no. 6, pp. 3753-3780, 2023/12/01 2023.

[7] N. Moustafa and J. Slay, "UNSW-NB15: A Comprehensive Data set for Network Intrusion Detection systems (UNSW-NB15 Network Data Set)," in Military Communications and Information Systems Conference (MilCIS), Canberra, ACT, Australia, 2015, pp. 1-6.

[8] Q. Liu, V. Hagenmeyer, and H. B. Keller, "A Review of Rule Learning-Based Intrusion Detection Systems and Their Prospects in Smart Grids," IEEE Access, vol. 9, pp. 57542-57564, 2021.

[9] R. Ahmad, I. Alsmadi, W. Alhamdani, and L. a. Tawalbeh, "Zero-day attack detection: a systematic literature review," Artificial Intelligence Review, vol. 56, no. 10, pp. 10733-10811, 2023/10/01 2023.

[10] S. Varalakshmi, S. Kottur, R. Sasikumar, K. Saravanan, T. Kanungo, and B. Annamalai, "AI-Powered Anomaly Detection to Strengthen Internet of Things Security and Forestall Cyber Attacks in Networked Device Environments," presented at the 2025 3rd International Conference on Advancement in Computation & Computer Technologies (InCACCT), Punjab, India, 17-18 April 2025, 2025.

[11] B. F. Azevedo, A. M. A. C. Rocha, and A. I. Pereira, "Hybrid approaches to optimization and machine learning methods: a systematic literature review," Machine Learning, vol. 113, no. 7, pp. 4055-4097, 2024.

[12] N. Moustafa and J. Slay, "The evaluation of Network Anomaly Detection Systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set," Information Security Journal: A Global Perspective, vol. 25, no. 1-3, pp. 18-31, 2016.

[13] S. M. Kasongo and Y. Sun, "Performance Analysis of Intrusion Detection Systems Using a Feature Selection Method on the UNSW-NB15 Dataset," Journal of Big Data, vol. 7, no. 1, 2020.

[14] M. Ahmad, Q. Riaz, M. Zeeshan, H. Tahir, S. A. Haider, and M. S. Khan, "Intrusion detection in internet of things using supervised machine learning based on application and transport layer features using UNSW-NB15 data-set," EURASIP Journal on Wireless Communications and Networking, vol. 2021, no. 10, pp. 1-23, 2021.

[15] R. A. Disha and S. Waheed, "Performance analysis of machine learning models for intrusion detection system using Gini Impurity-based Weighted Random Forest (GIWRF) feature selection technique," Cybersecurity, vol. 5, no. 1, 2022.

[16] M. M. Ahsan, M. A. P. Mahmud, P. K. Saha, K. D. Gupta, and Z. Siddique, "Effect of Data Scaling Methods on Machine Learning Algorithms and Model Performance," Technologies, vol. 9, no. 52, pp. 1-17, 2021.

[17] G. Karatas, "The Effects of Normalization and Standardization an Internet of Things Attack Detection," European Journal of Science and Technology, vol. 29, pp. 187-12, 201.

[18] A. Y. Hussein, P. Falcarin, and A. T. Sadiq, "Enhancement performance of random forest algorithm via one hot encoding for IoT IDS," Periodicals of Engineering and Natural Sciences, vol. 9, no. 3, pp. 579-571, 2021.

[19] H. Chen, H. Zhang, S. Si, Y. Li, D. Boning, and C.-J. Hsieh, "Robustness verification of tree-based models," in The 33rd International Conference on Neural Information Processing Systems, Vancouver, Canada, 2019, pp. 12327 - 12337: Curran Associates Inc.

[20] P. Cunningham and S. J. Delany, "k-Nearest Neighbour Classifiers - A Tutorial," ACM Computing Surveys, vol. 54, no. 6, pp. 1-25, 2021.

[21] R. Blanquero, E. Carrizosa, P. Ramírez-Cobo, and M. R. Sillero-Denamiel, "Variable selection for Naïve Bayes classification," Computers and Operations Research, vol. 135, 2021.

[22] B. S. Bhati, G. Chugh, F. Al-Turjman, and N. S. Bhati, "An improved ensemble based intrusion detection technique using XGBoost," Trans Emerging Tel Tech., vol. 32, no. 6, 2020.

[23] M. B. Musthafa, S. Huda, M. A. Ali, Y. Kodera, and Y. Nogami, "Evaluation of IDS model by improving accuracy and reducing overfitting using stacking LSTM," presented at the 2024 IEEE International Conference on Consumer Electronics (ICCE), 2024.

[24] T. Kim and J.-S. Lee, "Exponential Loss Minimization for Learning Weighted Naive Bayes Classifiers," IEEE Access, vol. 10, pp. 22724-22736, 2022.

[25] M. Verkerken, L. D’hooge, T. Wauters, B. Volckaert, and F. De Turck, "Towards Model Generalization for Intrusion Detection: Unsupervised Machine Learning Techniques," Journal of Network and Systems Management, vol. 30, no. 1, p. 12, 2021/10/17 2021.