Abstract
Abstract Because of the high dimensionality, heterogeneity, and resource constraints of IoT environments, protecting data from intrusions is a critical challenge in the ever-changing field of sensor networks. In this research, the authors use the NSL-KDD dataset to examine how to best implement and optimize TabNet, a deep learning model designed for organized tabular data, for detecting intrusions in networks. Despite a minimal loss of 0.05087 and a validation accuracy of 0.93893 achieved after 33 epochs with the baseline TabNet setup, it was time-consuming to train and didn’t converge optimally. Our innovative optimization framework combines Optuna-based hyperparameter tuning with Automatic Mixed Precision (AMP) training to overcome these limitations, improving performance while simultaneously making computation more efficient. Trial 10 outperformed the other twenty-four Optuna trials, showing early convergence and better generalization with a validation accuracy of 0.98531 at epoch 6. Multiple further trials demonstrated an accuracy level higher than 0.98, indicating that the improved setups were quite reliable. The confusion matrix analysis provided additional evidence of this enhancement, with results showing a 99.6% overall accuracy, a 99.26% precision, and a 99.67% recall for detecting malicious traffic. This study is groundbreaking because it incorporates TabNet, Optuna, and AMP into a single optimization pipeline for the first time. As a result, the pipeline converges quicker, is easier to understand, and produces better results, and it can be scaled up to use in real-world Internet of Things systems. For future intrusion detection designs in contexts with limited resources, this framework lays the groundwork.