Benchmarking AI prompt performance of large language models across chemical process systems engineering domains using multi-judge evaluation

Process systems engineering (PSE) integrates process modeling, optimization, machine learning, and process design — the quantitative foundation for chemical process development. Large language models (LLMs) are increasingly used as engineering consultation tools, yet their reliability varies across knowledge domains, and confidently presented but incomplete responses pose risks in safety-critical applications. This study evaluates five commercial LLMs (DeepSeek-V3, Claude Sonnet 4, Gemini 2.5 Flash, GPT-4o, and Llama 3.3 70B) on 200 open-ended questions across four domains — process modeling and simulation (MOD), process optimization (OPT), machine learning for chemical processes (ML), and process design and systems engineering (DES) — scored by five independent AI judges against 7-element rubrics. The Friedman test confirmed significant differences among models (χ²(4) = 491.4, p ML > OPT > MOD held across all models. Human validation by three domain experts on a stratified 60-response subset supported the broad performance tiering (Spearman rs = 0.416, p < 0.001; inter-expert ICC = 0.793) and revealed a systematic AI leniency bias of +0.85 points, although the exact ordering of middle-tier models was less stable. Sensitivity analysis demonstrated perfectly stable rankings under alternative scoring schemes. These findings indicate that current LLMs can support procedural PSE tasks but require expert verification for process modeling and optimization tasks, where critical technical elements are frequently omitted from AI responses. PSE-Bench provides a standardized framework for tracking LLM capabilities and calibrating trust across PSE domains.