Abstract
Source term estimation provides release rate and thermodynamic state inputs for consequence analysis, which is one stage of Quantitative Risk Assessment (QRA). Event frequency estimation and subsequent dispersion, fire, explosion, and toxic effect calculations are outside the scope of this study. Large Language Models (LLMs) can extract parameters from unstructured accident descriptions, but direct use of autoregressive generation for numerical source term calculations raises reliability concerns. In the evaluated cases, native LLMs produced operator omissions, incompatible equation combinations, and variable results across repeated runs. This study proposes STEM-Agent (Source Term Estimation Model Agent), a LangGraph based framework that separates probabilistic semantic parsing from deterministic calculation. The LLM is restricted to parameter extraction under a Pydantic data contract, while a coded physics engine selects and evaluates source term equations through explicit rules. Five model configurations were evaluated across 15 leakage scenarios based on TNO and CCPS guidance, with five repeated runs per configuration. The Overall Mean Absolute Errors were 363.00% for ChatGPT, 18.61% for DeepSeek, 1261.04% for Qwen, and 8.16% for Gemini. STEM-Agent, using the same Qwen API as its perception layer, achieved an Overall Mean Absolute Error of 0.75% and identical final outputs across the five repeated runs for each scenario. The findings indicate deterministic solver execution as a reliability control for the evaluated source term tasks. They do not demonstrate autonomous use across the wider QRA workflow, and uncertainties in inputs, parameters, and model form remain to be quantified.
•Five systematic failure modes of LLMs in source term estimation are identified and classified.•A decoupled agentic architecture separates semantic extraction from deterministic computation.•Pydantic schema enforcement eliminates numerical hallucinations in safety-critical calculations.•STEM-Agent achieves zero computational variance across 15 industrial leakage scenarios.•The hybrid framework enables reliable QRA input without domain expertise requirements.