Abstract
Dimensionality reduction is essential in industrial analytics, as high-dimensional process data often contain redundant and correlated variables that increase computational burden and model complexity. This study proposes a weighted composite evaluation framework (Swc) that jointly considers predictive performance, computational execution time, and representation dimensionality when comparing feature selection (FS) and feature extraction (FE) methods. The framework was evaluated on the Tennessee Eastman Process benchmark using seven fault classes, 52 process variables, a common XGBoost classifier, and 10 independently seeded stratified repetitions. Nine FS methods and four FE methods were benchmarked across four operational weighting scenarios: Balanced, Predictive Focus, Efficiency Focus, and Compactness Focus. Among the FE techniques, linear discriminant analysis (LDA) consistently achieved the highest composite score across all scenarios (mean rank = 1.00) due to its high-class separability and minimal computational overhead. For FS methods, forward selection and XGBoost-based selection jointly achieved the leading overall average rank (mean rank = 1.50), with XGBoost leading under predictive- and compactness-focused scenarios, while forward selection led under Balanced and Efficiency Focus constraints. Nonparametric statistical analysis indicated significant overall differences among methods based on the Friedman test (p < 0.05), while no individual pairwise comparison remained significant after Holm correction. The results demonstrate that evaluating methods solely by predictive performance provides an incomplete assessment. The proposed framework serves as a transparent multicriteria decision support mechanism for selecting dimensionality reduction strategies aligned with specific Industry 4.0 operational priorities.