Comparative Analysis of YOLOv5, YOLOv8, and YOLOv11 for Military Aircraft Detection on Aerial Imagery Supporting Airspace Surveillance

Fajar Sidik Suganda; Asep Adang Supriyadi

doi:10.34148/teknika.v15i2.1483

Authors

Fajar Sidik Suganda Sensing Technology Study Program, Republic of Indonesia Defense University, Sentul, West Java, Indonesia
Asep Adang Supriyadi Sensing Technology Study Program, Republic of Indonesia Defense University, Sentul, West Java, Indonesia

DOI:

https://doi.org/10.34148/teknika.v15i2.1483

Keywords:

YOLO, Object Detection, Military Aircraft, Aerial Imagery, Airspace Surveillance

Abstract

Automated military aircraft detection from aerial imagery is a key enabler for modern Intelligence, Surveillance, and Reconnaissance (ISR) operations, particularly for nations with vast airspace such as Indonesia. While the You Only Look Once (YOLO) family has become a de facto standard for such tasks, the literature lacks a controlled, head-to-head comparison of its recent generations on a standard military aircraft benchmark, which leaves architecture selection for ISR systems largely guided by version recency rather than empirical evidence. This study addresses that gap by benchmarking three YOLO generations YOLOv5s, YOLOv8s, and YOLOv11s on the MAR20 dataset (3,842 images, 20 classes, 22,341 instances) under identical training conditions, and by analyzing the results along three axes: aggregate accuracy, per-class behavior, and accuracy–parameter trade-off. Counter to the assumption that newer is necessarily better, the oldest architecture in the comparison, YOLOv5s, achieves the highest Precision (0.985) and F1-Score (0.978) with the smallest parameter count (7.2M), while YOLOv11s, the most recent iteration, leads only on Recall (0.972) and YOLOv8s offers the most balanced profile; all three exceed [email protected] of 0.988. Per-class analysis further shows that detection difficulty is driven by inter-class visual similarity rather than class imbalance, with visually similar fighter aircraft (F-22, F-16, F-15) forming a distinct hard-class cluster (mean AP 0.705–0.788) despite adequate training samples. Based on these findings we propose an operationally-mapped model-selection guide YOLOv5s as a strong candidate for edge-constrained ISR platforms, YOLOv8s for balanced ground-station workloads, and YOLOv11s for recall-critical wide-area surveillance while explicitly noting that the recommendations rest on accuracy–parameter evidence rather than on-device latency measurements, and that the single-seed, small-variant setting bounds the generality of the ranking. Validation through repeated runs with statistical testing, profiling on representative edge hardware, and evaluation on operational imagery remain important directions for future work.

Downloads

Download data is not yet available.

References

[1] Z. F. Tiwery, I. H. Hanafi, and W. A. Riry, “Pelanggaran Wilayah Udara Indonesia Oleh Pesawat Asing Menurut Hukum Internasional,” Balobe Law Journal, vol. 4, no. 1, p. 35, Apr. 2024, doi: 10.47268/balobe.v4i1.2046.

[2] K. M. Elgamily, M. A. Mohamed, A. M. Abou-Taleb, and M. M. Ata, “Enhanced object detection in remote sensing images by applying metaheuristic and hybrid metaheuristic optimizers to YOLOv7 and YOLOv8,” Sci. Rep., vol. 15, no. 1, Dec. 2025, doi: 10.1038/s41598-025-89124-8.

[3] S. Gui, S. Song, R. Qin, and Y. Tang, “Remote Sensing Object Detection in the Deep Learning Era A Review,” Jan. 01, 2024, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/rs16020327.

[4] J. Terven, D. M. Córdova-Esparza, and J. A. Romero-González, “A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS,” Dec. 01, 2023, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/make5040083.

[5] H. Juwiantho and J. E. Saputra, “Robust Balinese Script Detection for Transliteration Using YOLOv8,” Teknika, vol. 15, no. 1, pp. 178–185, Mar. 2026, doi: 10.34148/teknika.v15i1.1466.

[6] M. Mao and M. Hong, “YOLO Object Detection for Real-Time Fabric Defect Inspection in the Textile Industry: A Review of YOLOv1 to YOLOv11,” Apr. 01, 2025, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/s25072270.

[7] G. Cinarer, Y. S. Taspinar, and M. Zeybek, “Detection of Aerial Vehicles Using Satellite Imagery: Comparative Analysis of U-Net Segmentation Model and YOLO Object Detection Model,” International Journal of Aerospace Engineering, vol. 2025, no. 1, 2025, doi: 10.1155/ijae/5599522.

[8] R. H. Pratama, Ernawati, and W. O. KZ, “Modification of YOLOv11 Architecture with Convolutional Block Attention Module Integration for Batik Besurek Motif Detection,” Teknika, vol. 15, no. 1, pp. 103–112, Mar. 2026, doi: 10.34148/teknika.v15i1.1449.

[9] X. V. Tandianto, D. H. Bangkalang, and N. Setiyawati, “Comparison and Implementation of CNN Facial Emotion Recognition Model with Hyperparameter Analysis on Multiple Datasets,” Teknika, vol. 15, no. 1, pp. 75–83, Mar. 2026, doi: 10.34148/teknika.v15i1.1445.

[10] G. Yao, S. Zhu, L. Zhang, and M. Qi, “HP-YOLOv8: High-Precision Small Object Detection Algorithm for Remote Sensing Images,” Sensors, vol. 24, no. 15, Aug. 2024, doi: 10.3390/s24154858.

[11] B. Chen, Z. Ma, and X. Li, “Boosting Detection Accuracy: An Enhanced YOLOv8 for Small Target Detection in Remote Sensing,” International Journal of Computational Intelligence Systems, vol. 18, no. 1, Dec. 2025, doi: 10.1007/s44196-025-00895-4.

[12] W. Yu et al., “MAR20：A benchmark for military aircraft recognition in remote sensing images,” National Remote Sensing Bulletin, vol. 27, no. 12, pp. 2688–2696, 2023, doi: 10.11834/jrs.20222139.

[13] X. Cao, Y. Zhang, S. Lang, and Y. Gong, “Swin-Transformer-Based YOLOv5 for Small-Object Detection in Remote Sensing Images,” Sensors, vol. 23, no. 7, Apr. 2023, doi: 10.3390/s23073634.

[14] S. Rattanaamporn, A. Perera, A. Nguyen, T. B. Ngo, and J. Chahl, “Drone Imaging and Sensors for Situational Awareness in Hazardous Environments: A Systematic Review,” Oct. 01, 2025, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/jsan14050098.

[15] C. Bai, X. Bai, and K. Wu, “A Review: Remote Sensing Image Object Detection Algorithm Based on Deep Learning,” Dec. 01, 2023, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/electronics12244902.

[16] B. Zhao, Y. Zhou, R. Song, L. Yu, X. Zhang, and J. Liu, “Modular YOLOv8 optimization for real-time UAV maritime rescue object detection,” Sci. Rep., vol. 14, no. 1, Dec. 2024, doi: 10.1038/s41598-024-75807-1.

[17] M. Hussain, “YOLOv1 to v8: Unveiling Each Variant-A Comprehensive Review of YOLO,” IEEE Access, vol. 12, pp. 42816–42833, 2024, doi: 10.1109/ACCESS.2024.3378568.

[18] F. Chen, Y. Zhang, L. Fu, H. Xie, Q. Zhang, and S. Bi, “From YOLOv5 to YOLOv8: Structural Innovations and Performance Improvements,” International Journal of Computer Science and Information Technology, vol. 6, no. 1, pp. 93–100, May 2025, doi: 10.62051/ijcsit.v6n1.12.

[19] Y. Zhao, F. Sun, and X. Wu, “FEB-YOLOv8: A multi-scale lightweight detection model for underwater object detection,” PLoS One, vol. 19, no. 9 September, Sep. 2024, doi: 10.1371/journal.pone.0311173.

[20] P. Lu, X. Hao, W. Li, C. Yi, R. Kong, and T. Wang, “ECF-YOLO: An enhanced YOLOv8 algorithm for ship detection in SAR images,” Electronic Research Archive, vol. 33, no. 5, pp. 3394–3409, 2025, doi: 10.3934/era.2025150.

[21] H. Huelss, “Transcending the fog of war? US military ‘AI’, vision, and the emergent post-scopic regime,” European Journal of International Security, vol. 10, no. 2, pp. 190–210, May 2025, doi: 10.1017/eis.2024.21.

[22] K. Helkala, G. Østby, G. Reichberg, and M. Metcalf, “The PRC considers military AI ethics: Can autonomy be trusted?,” Oct. 2022. doi: 10.3389/fdata.2022.991392.

[23] A. Downey, “The alibi of AI: algorithmic models of automated killing,” Digital War, vol. 6, no. 1, Dec. 2025, doi: 10.1057/s42984-025-00105-7.

[24] X. Chen et al., “DET-YOLO: An Innovative High-Performance Model for Detecting Military Aircraft in Remote Sensing Images,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 17, pp. 17753–17771, 2024, doi: 10.1109/JSTARS.2024.3462745.

[25] S. El Ghazouali, A. Gucciardi, F. Venturini, N. Venturi, M. Rueegsegger, and U. Michelucci, “FlightScope: An Experimental Comparative Review of Aircraft Detection Algorithms in Satellite Imagery,” Dec. 01, 2024, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/rs16244715.

[26] Z. Li et al., “RSI-YOLO: Object Detection Method for Remote Sensing Images Based on Improved YOLO,” Sensors, vol. 23, no. 14, Jul. 2023, doi: 10.3390/s23146414.

[27] J. Wang, Z. Bai, X. Zhang, and Y. Qiu, “A Lightweight Remote Sensing Aircraft Object Detection Network Based on Improved YOLOv5n,” Remote Sens. (Basel)., vol. 16, no. 5, Mar. 2024, doi: 10.3390/rs16050857.

[28] T. Adli, D. Bujaković, B. Bondžulić, M. Laidouni, and M. Andrić, “Comparative analysis of YOLO algorithms for aircraft detection in remote sensing images,” Centre for Evaluation in Education and Science (CEON/CEES), Oct. 2024, pp. 326–331. doi: 10.5937/oteh24059a.

[29] I. E. Tampu, A. Eklund, and N. Haj-Hosseini, “Inflation of test accuracy due to data leakage in deep learning-based classification of OCT images,” Sci. Data, vol. 9, no. 1, Dec. 2022, doi: 10.1038/s41597-022-01618-6.

[30] J. Chen, S. Pan, Y. Chan, Y. Ni, and D. Ye, “A new method based on YOLOv5 and multiscale data augmentation for visual inspection in substation,” Sci. Rep., vol. 14, no. 1, Dec. 2024, doi: 10.1038/s41598-024-60126-2.

[31] K. Nakamura, B. Derbel, K. J. Won, and B. W. Hong, “Learning-rate annealing methods for deep neural networks,” Electronics (Switzerland), vol. 10, no. 16, Aug. 2021, doi: 10.3390/electronics10162029.

[32] R. Padilla, W. L. Passos, T. L. B. Dias, S. L. Netto, and E. A. B. Da Silva, “A comparative analysis of object detection metrics with a companion open-source toolkit,” Electronics (Switzerland), vol. 10, no. 3, pp. 1–28, Feb. 2021, doi: 10.3390/electronics10030279.

[33] R. Khanam, T. Asghar, and M. Hussain, “Comparative Performance Evaluation of YOLOv5, YOLOv8, and YOLOv11 for Solar Panel Defect Detection,” Solar, vol. 5, no. 1, Mar. 2025, doi: 10.3390/solar5010006.

[34] R. Sapkota et al., “YOLO advances to its genesis: a decadal and comprehensive review of the You Only Look Once (YOLO) series,” Artif. Intell. Rev., vol. 58, no. 9, Sep. 2025, doi: 10.1007/s10462-025-11253-3.

[35] N. Habash, A. A. Alqumsan, and T. Zhou, “Recent Real-Time Aerial Object Detection Approaches, Performance, Optimization, and Efficient Design Trends for Onboard Performance: A Survey,” Dec. 01, 2025, Multidisciplinary Digital Publishing Institute (MDPI). doi: 10.3390/s25247563.