Abstract
The rapid growth of data in various domains has heightened the need for accurate and efficient predictive models, particularly for tabular data. While deep learning has revolutionized fields like computer vision and natural language processing, its effectiveness in tabular data classification remains a topic of debate. This study conducts a comprehensive comparison between deep learning models (NODE, SAINT, TabNet, Tab Transformers) and tree-based models (Random Forest, XGBoost, LightGBM, CatBoost) to determine whether deep learning can outperform traditional methods in this context. The results indicate that tree-based models, particularly LightGBM and CatBoost, consistently achieve the highest accuracy and F1 scores, coupled with efficient execution times, making them more suitable for real-world applications that require quick and accurate predictions. In contrast, deep learning models show varied performance. Although SAINT sometimes achieves comparable accuracy, its processing time makes it less practical. The findings suggest that despite the potential of deep learning, tree-based models remain superior for tabular data classification tasks, particularly when considering a balance of accuracy, speed, and robustness. This study contributes to the ongoing discussion on the role of deep learning in tabular data and highlights the conditions under which traditional models may still be preferred.