Abstract
The transition toward Industry 4.0 has established predictive maintenance (PdM) as a critical strategy for optimizing operational efficiency and reducing unexpected downtime through real-time sensor data analytics. However, the practical implementation of PdM is frequently hindered by the extreme class imbalance inherent in industrial datasets, where equipment failure events are significantly rarer than normal operating hours. This study presents a comprehensive comparative evaluation of five machine learning algorithms—Random Forest (RF), Decision Tree (DT), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Logistic Regression (LR)—utilizing the AI4I 2020 Predictive Maintenance Dataset. By implementing a rigorous preprocessing pipeline that employs Z-score normalization for feature scaling and the Synthetic Minority Over-sampling Technique (SMOTE) to mitigate majority-class bias within the training partition. The models were evaluated on a hold-out test set of 2,000 instances using metrics including accuracy, precision, recall, and the F1-score. Results indicate a pronounced “Accuracy Paradox” in linear and distance-based models; while KNN and Logistic Regression achieved deceptively high accuracies exceeding 96%, they failed to reliably detect actual failure signatures. In contrast, Random Forest emerged as the superior architecture, achieving an F1-score of 97.10% and a recall of 98.53%, correctly identifying 67 out of 68 failure instances. This research concludes that F1-score and Recall are more vital indicators of industrial reliability than simple accuracy. The findings provide a standardized framework for selecting ensemble-based classifiers to support scalable, data-driven maintenance strategies in modern smart manufacturing environments.