Evaluation and Optimization of Machine Learning Models in Intrusion Detection Systems Using PCA and ICA Dimensionality Reduction Methods

Document Type : Original Article

Authors

1 Bachelor's degree, Tabriz University of Technology, Tabriz, Iran

2 Assistant Professor-University of Tabriz, Tabriz, Iran

Abstract

Intrusion Detection Systems (IDS) play a crucial role in protecting modern computer networks from diverse cyberattacks. However, the high dimensionality and complexity of network traffic data often degrade the accuracy and efficiency of machine learning–based IDS models. This paper proposes a comprehensive comparative framework that adaptively integrates two dimensionality reduction methods—Principal Component Analysis (PCA) and Independent Component Analysis (ICA)—to enhance IDS performance. Six widely adopted machine learning algorithms—K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Random Forest, Decision Tree, Logistic Regression, and XGBoost—are evaluated using the modern UNSW-NB15 dataset. The performance of each model is assessed based on classical evaluation metrics (Accuracy, Precision, Recall, and F1-Score) as well as operational efficiency indicators (training and prediction time). Experimental results demonstrate that ICA generally outperforms PCA, achieving a better balance between detection accuracy and computational cost. The findings provide valuable insights for designing practical, efficient, and high-performance IDS solutions for real-world applications.

Keywords

Main Subjects

References

[1]     Zhang, S.; Cheng, D.; Deng, Z.; Zong, M.; Deng, X. “A Novel kNN Algorithm with Data-Driven k Parameter Computation”; Pattern Recognit. Lett. 2018, 109, 44–54. DOI: 10.1016/j.patrec.2017.09.036.
[2]     Zahiri, M.; Shirini, K.; Samadi Gharehveran, S. “Network Traffic Analysis with Machine Learning for Faster Detection of Distributed Denial of Service Attack”; J. Adv. Def. Sci. Technol. 2024, 14, 273–282. (In Persian) DOR: 20.1001. 1.26762935.1402.14.4.6.2.
[3]     Kazemitabar, J.; Taheri, R.; Kheradmandian, G. “A Novel Technique for Improvement of Intrusion Detection via Combining Random Forest and Genetic Algorithm”; J. Adv. Def. Sci. Technol. 2019, 10, 287–296. (In Persian) DOR: 20.1001.1.26762935.1398.10.3.9.5
[4]     Samadi Gharehveran, S.; Ghassemzadeh, S.; Rostami, N. “Two-Stage Resilience-Constrained Planning of Coupled Multi-Energy Microgrids in the Presence of Battery Energy Storages”; Sustain. Cities Soc. 2022, 83, 103952. DOI: 10.1016/j.scs.2022.103952
[5]     Asadi, M.; Zarei, B. “Detection of Denial of Service Attacks by Ensemble Learning Method”; J. Adv. Def. Sci. Technol. 2023, 14, 51–68. (In Persian) DOR: 20.1001.1.26762935. 1402.14.1.5.5
[6]     Alharthi, A.; Alaryani, M.; Kaddoura, S. “A Comparative Study of Machine Learning and Deep Learning Models in Binary and Multiclass Classification for Intrusion Detection Systems”; Array 2025, 100406. DOI: 10.1016/j.array. 2025.100406.
[7]     Badrinarayanan, V.; Kendall, A.; Cipolla, R. “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation”; IEEE Trans. Pattern Anal. 2017, 39, 2481–2495. DOI: 10.1109/TPAMI.2016.2644615.
[8]     Samadi Gharehveran, S.; Nasiri, M. “Resilient Planning Against Disturbances and Optimal Location Determination for Mobile Energy Storage Systems in Smart Microgrids”; Passive Defense 2025, 16, 69–80. (In Persian) DOR: 20.1001.1.20086849.1404.16.2.6.2.
[9]     Moustafa, N.; Slay, J. “UNSW-NB15: A Comprehensive Data Set for Network Intrusion Detection Systems”; Mil. Commun. Inf. Syst. Conf. 2015. DOI: 10.1109/MilCIS. 2015.7348942.
[10] Corea, P. M.; Liu, Y.; Wang, J.; Niu, S.; Song, H. “Explainable AI for Comparative Analysis of Intrusion Detection Models”; IEEE Int. Mediterr. Conf. Commun. Netw. 2024, 585–590. IEEE. DOI: 10.1109/MeditCom 61057.2024.10621339.
[11] Malele, V.; Mathonsi, T. E. “Testing the Performance of Multi-Class IDS Public Dataset Using Supervised Machine Learning Algorithms”; arXiv.2302.14374, 2023. DOI: 10.48550/arXiv.2302.14374.
[12] Zoghi, Z.; Serpen, G. “Ensemble Classifier Design Tuned to Dataset Characteristics for Network Intrusion Detection”; arXiv.2205.06177, 2022. DOI: 10.48550/ arXiv.2205.06177
[13] Samadi Gharehveran, S.; G. Zadeh, S.; Rostami, N. “Resilience-Oriented Planning and Pre-Positioning of Vehicle-Mounted Energy Storage Facilities in Community Microgrids”; J. Energy Storage 2023, 72, 108263. DOI: 10.1016/j.est.2023.108263.
[14] Ajagbe, S. A.; Bamidele, A. J.; Florez, H. “Intrusion Detection: A Comparison Study of Machine Learning Models Using Unbalanced Dataset”; SN Comput. Sci. 2024, 5, 1028. DOI: 10.1007/ s42979-024-03369-0.
[15] Rastgoo, M.; Jalali, M. “Detection of Cybercrimes in Online Connections by the Data Mining Approach”; Passive Defense 2020, 11, 63–70. (In Persian) DOR: 20.1001. 1.20086849.1399.11.1.6.5.
[16] Saheed, Y. K.; Misra, S. “A Voting Gray Wolf Optimizer-Based Ensemble Learning Models for Intrusion Detection in the Internet of Things”; Int. J. Inf. Secur. 2024, 23, 1557-1581. DOI: 10.1007/s10207-023-00803-x.
[17]Abbasi, R.; Javadzade, M. A. “Predicting Public Unrest Using Social Networks, Based on Machine Learning in the Natural Language Processing”; Passive Defense 2022, 13, 45-56. (In Persian) DOR: 20.1001.1.20086849.1401.13. 3.5.2.
[18]Nemati, R.; Shirini, K.; Samadi Gharehveran, S.  “FER-HA: A Hybrid Attention Model for Facial Emotion Recognition”; J. Supercomput. 2025, 81, 1485. DOI: 10.1007/s11227-025-07983-4.
[19]Saeedi, N.; Baharvand, D.; Shirini, K.; Samadi Gharehveran, S.  “Prediction of Electrical Energy Consumption Using Principal Component Analysis and Independent Components Analysis”; J. Supercomput. 2025, 81, 1072. DOI: 10.1007/s11227-025-07505-2
[20]Samadi Gharehveran, S.; Shirini, K.; Khavar, S. C.; Abdollahi, A. “Optimizing Day-Ahead Power Scheduling: A Novel MIQCP Approach for Enhanced SCUC with Renewable Integration”; e-Prime-Adv. Electr. Eng., Electron. Energy 2025, 101022. DOI: 10.1016/j.prime. 2025.101022
[21]Samadi Gharehveran, S.; Shirini, K.; Abdolahi, A. “Optimizing Energy Storage Solutions for Grid Resilience: A Comprehensive Overview”; intechopen 2025. DOI: 10.5772/ intechopen.1006499
[22]Taherihajivand, A.; Shirini, K.; Samadi Gharehveran, S. “An Overview of Product Performance Prediction Using Artificial Algorithms”; J. Agric. Mechanization 2024, 9, 1–14. DOI: 10.22034/jam.2024.61899.1276.
[23]Zaki Dizaji, H.; Shirini, K.; Taherihajivand, A.; Monjezi, N. “Modelling Variables Affecting the Yield of Sugarcane Fields Using Deep Recurrent Neural Network”; Iran. J. Biosyst. Eng. 2024, 55, 93–108. DOI: 10.22059/ijbse.2025. 378958.665557.
[24]Sattari, M. T.; Bagheri, R.; Shirini, K.; Allahverdipour, P. “Modeling Daily and Monthly Rainfall in Tabriz Using Ensemble Learning Models and Decision Tree Regression”; Clim. Change Res. 2024, 5, 31–48. DOI: 10.30488/ccr.2024. 433394.1192.
[25]Sattari, M. T.; Shirini, K.; Javidan, S. “Evaluating the Efficiency of Dimensionality Reduction Methods in Improving the Accuracy of Water Quality Index Modeling in Qizil-Uzen River Using Machine Learning Algorithms”; Water Soil Manag. Model. 2024, 4, 89–104. DOI: 10.22098/ mmws.2023.12434.1241.
[26]Taherihajivand, A.; Shirini, K.; Samadi Gharehveran, S. “Weed Detection in Fields Using Convolutional Neural Network Based on Deep Learning”; Agric. Eng. 2024, 47, 129–142. DOI: 10.22055/agen.2024.45327.1688.
[27]Zahiri, M.; Shirini, K.; Samadi Gharehveran, S. “Network Traffic Analysis with Machine Learning for Faster Detection of Distributed Denial of Service Attack”; J. Adv. Def. Sci. Technol. 2024, 14, 273–282. DOR: 20.1001.1.26762935. 1402.14.4.6.2. 
[28]Shirini, K.; Taherihajivand, A.; Samadi Gharehveran, S. “A Review of Algorithms for Solving the Project Scheduling Problem with Resource Constraints Considering Agricultural Problems”; J. Agric. Mechanization 2023, 8, 1–14. DOI: 10.22034/jam.2023.55751.1227.
[29]Tan, J.; Radhi, R. M.; Shirini, K.; Samadi Gharehveran, S.; Parisooz, Z.; Khosravi M.; Azarinfar, H. “Innovative Framework for Fault Detection and System Resilience in Hydropower Operations Using Digital Twins and Deep Learning”; Sci. Rep. 2025, 15, 15669. DOI: 10.1038/s41598-025-98235-1
[30]Jin, K.; Banizaman, H.; Samadi Gharehveran, S.; Jokar, M. R.; Mohamadi Amidi, A. R.; Yu, J.; Oleiwi Shami; H. “Robust Power Management Capabilities of Integrated Energy Systems in the Smart Distribution Network Including Linear and Non-Linear Loads”; Sci. Rep. 2025, 15, 6615. DOI: 10.1038/s41598-025-89817-0.
[31] Shirini, K.; Aghdasi, H. S.; Saeedvand, S. “Modified Imperialist Competitive Algorithm for Aircraft Landing Scheduling Problem”; J. Supercomput. 2024, 80(10). DOI: 10.1007/s11227-024-05999-w. 
 

Files

History

  • Receive Date: 30 December 2024
  • Revise Date: 25 January 2025
  • Accept Date: 12 February 2025
  • Publish Date: 21 February 2025

Share

How to cite

Statistics