Individual Packet Features are a Risk to Model Generalisation in ML-Based Intrusion Detection

Machine learning is increasingly used for intrusion detection in IoT networks. This paper explores the effectiveness of using individual packet features (IPF), which are attributes extracted from a single network packet, such as timing, size, and source-destination information. Through literature review and experiments, we identify the limitations of IPF, showing they can produce misleadingly high detection rates. Our findings emphasize the need for approaches that consider packet interactions for robust intrusion detection. Additionally, we demonstrate that models based on IPF often fail to generalize across datasets, compromising their reliability in diverse IoT environments.

💡 Research Summary

The paper critically examines the widespread use of Individual Packet Features (IPF) – attributes extracted from a single network packet such as timing, size, source/destination IP or MAC – in machine‑learning‑based intrusion detection for IoT environments. While many recent studies (10 out of 68 surveyed between 2019 and 2023) report near‑perfect detection rates (often 95‑100 % accuracy, recall, and F1) using only IPF, the authors argue that these results are largely artefacts of dataset design rather than evidence of genuine model capability.

A systematic literature review reveals that most IPF‑based works rely on publicly available CSV files that contain raw packet records for each attack type. Because each attack is stored in a single file, the usual train‑test split inadvertently places packets from the same flow or session in both training and testing sets. Consequently, identifying features such as IP addresses, MAC addresses, ports, sequence numbers, and ACK numbers become “shortcuts” that the classifier can exploit. This information leakage inflates performance metrics; the model learns to recognize the identifiers rather than the underlying malicious behavior.

To substantiate this claim, the authors conduct experiments on the IoT‑NID dataset, which provides multiple distinct sessions for each attack. They merge the first two sessions, then perform a 10‑times 10‑fold cross‑validation (CV) while using a single session‑based identifier (e.g., destination port) as the only feature. In the CV setting, the identifier yields very high accuracy because packets from the same session appear in both folds. However, when the first session is used for training and the second for testing, the same identifier’s performance collapses, demonstrating severe over‑fitting caused by leakage.

The paper also highlights the problem of low data complexity. Certain attacks (e.g., HTTP Flood) generate highly uniform packet sizes across datasets. When only packet size is used as a feature, the classifier can achieve perfect detection on the same dataset, but such a model would fail on any dataset where the attack’s size distribution varies. This illustrates that simplistic IPF can appear powerful when the dataset is overly homogeneous.

Based on these findings, the authors advocate for richer feature sets that capture packet interactions: flow‑based features (statistics over a packet flow such as byte counts, inter‑arrival times, flow duration) and window‑based features (temporal aggregates within fixed time windows). These representations are less dependent on identifiers and better reflect the dynamics of attacks, leading to models that generalize across different IoT deployments.

The paper contributes several practical recommendations: (1) remove all identifying fields (IP, MAC, ports, checksums) before model training; (2) split data at the flow or session level to avoid mixing related packets between train and test; (3) employ cross‑validation schemes that respect flow boundaries; (4) prefer flow‑ or window‑based features, or hybrid combinations, for robust detection. The authors also release all preprocessing scripts and experimental code to promote reproducibility.

In conclusion, the study demonstrates that reliance on individual packet features can give a false sense of security, as models trained on such features are prone to information leakage and fail to generalize to unseen network environments. By shifting focus toward interaction‑aware features and rigorous evaluation protocols, future IoT intrusion detection systems can achieve more reliable and transferable performance.