Abstract
Introduction
ChatGPT's conversational interface invites anthropomorphic interpretations that conflict with its underlying probabilistic architecture. Prior research has identified folk theories and conceptions of generative AI among technical laypersons, but has largely examined these mental concepts in isolation or at the group level. Less is known about how conceptualizations of ChatGPT as a deterministic, probabilistic, anthropomorphic or dynamic AI system coexist, interact, and contradict one another within technical laypersons. This study examined (RQ1) recurring qualitative mental model configurations of ChatGPT among participants and (RQ2) whether technical AI terminology was supported by mechanism-specific understanding or coexisted with misconceptions concerning the same or another functional mechanism.
Methods
We conducted semi-structured interviews with technical laypersons (
N
= 21) to ask them about their understanding of how ChatGPT works. Statements were deductively coded into four mental concepts (deterministic, probabilistic, anthropomorphic, and dynamic). Mental models were constructed for each participant. Statements containing technical terminology were additionally classified according to whether the term was contradicted by the participant's explanation of the same mechanism (“Chauffeur Knowledge”) or co-occurred with a misconception in another mechanism.
Results
Three recurring mental model configurations emerged: (i) Predominantly Probabilistic Thinking Patterns (
n
= 8); (ii) Anthropomorphic and Dynamic Thinking Patterns (
n
= 4), characterized by limited probabilistic reasoning; and (iii) Predominantly Deterministic Retrieval-based Thinking Patterns (
n
= 9), characterized mainly by deterministic conceptions alongside limited probabilistic awareness. All participants showed at least one misconception about ChatGPT and among 10 participants, who used technical terminology in their explanations, seven showed chauffeur knowledge, seven demonstrated correct or partially correct understanding of one mechanism alongside a misconception about another, and four participants met both criteria.
Discussion
Participants' mental models of ChatGPT are fragmented and often internally contradictory. Technical terminology may mask misunderstanding of the mechanism it describes or coexist with correct understanding in one domain and misconceptions in another. These findings challenge AI literacy assessments that rely predominantly on knowledge of technical terminology. Identifying recurring qualitative configurations provides a basis for developing measures that capture the complexity of mental models and may improve predictions of over-reliance on ChatGPT output.