Abstract
Vocational school accreditation is a high-stakes institutional classification, yet the geographic transferability and probability reliability of models based on routinely collected administrative data remain underexplored. This study developed a reproducible, leakage-controlled machine-learning framework for classifying the recorded accreditation classes C, B, and A among Indonesian vocational schools. Its main contribution is an integrated evaluation design that combines province-disjoint validation, probability calibration, cluster-aware uncertainty, and subgroup auditing to test reliability beyond conventional random-validation performance. A cross-sectional dataset containing 149,376 program-level records was deterministically aggregated into 14,391 schools, including 14,134 with valid labels. Eight provinces comprising 1,536 schools were locked before feature selection, model comparison, tuning, and calibration; 12,598 schools from 31 provinces supported development. The selected CatBoost model used 35 structural and vocational features with grouped out-of-fold temperature scaling. On the locked holdout, the calibrated model achieved balanced accuracy of 0.586, Macro F1 of 0.533, quadratic weighted kappa of 0.499, Macro ROC-AUC of 0.757, and expected calibration error of 0.049. School size, teacher resources, vocational staffing, program diversity, and status were most in-fluential. Class B had the lowest recall, and performance varied across school groups and provinces. The framework supports calibrated, auditable preliminary screening, but not automated accreditation or replacement of professional assessors.