Abstract: Class imbalance is a common issue in real-world machine learning datasets. This problem is more obvious in intrusion detection since many attack types only have very few samples. Ignoring the imbalance issue or constructing the machine learning classifier on partial classes will lead to bias in the model performance. Motivated by a recent study that addressing the real-world class imbalance problem in dermatology, we explore the effectiveness of different techniques in handling class imbalance in a Network-based Intrusion Detection System (NIDS). Experiments on the NSL-KDD dataset show that downsampling + upsampling + SMOTE (DUS) is the best re-sampling technique in imbalanced data. In addition, compared to other machine learning classifiers, the Ensemble model with DUS achieves the highest performance. We also design experiments to validate how the number of classes affects the NIDS model performance, finding that more imbalance classes will negatively impact the model performance. Our experiment demonstrates that many of the existing machine learning-based NIDS systems which yield very high performances might be misleading. The results in the article provide insights on the effect of class imbalance on the machine learning performance in NIDS and guide researchers on how to improve the NIDS performance in real-world imbalanced data.

Key words: network-based intrusion detection, data quality, machine learning, class imbalance