Less stress, better scores, same learning: The dissociation of performance and learning in AI-supported programming education

Texto completo

Introduction: Generative AI is reshaping programming education, yet its effects on conceptual learning, intrinsic motivation, and cognitive load remain unclear. This study tests whether assistance deepens understanding or primarily boosts task completion, and how scaffolded versus answer-giving designs matter. Objectives: This study compares performance, learning, cognitive load, frustration, and motivation across three AI support types, and examines students’ perceptions. Methods: A three-arm randomized controlled trial was conducted in an introductory programming (CS1) course at TUM (N=275). Participants completed a 90-minute exercise on concurrency, implementing a parallel sum with threading in one of three conditions: (1) Iris, a scaffolded tutor providing calibrated hints while withholding full solutions; (2) ChatGPT, unrestricted assistance that can provide complete solutions; (3) no-AI control using traditional web resources. Pre- and post-knowledge tests and a code comprehension task measured learning, while auto-graded test coverage measured performance. Validated scales captured intrinsic, germane, and extraneous cognitive load, frustration, and intrinsic motivation. Results: Both AI groups achieved substantially higher exercise scores than the control group, with distinct distributions: ChatGPT users clustered at high scores, control participants at low scores, and Iris users spread across the full range. Despite these performance gains, neither AI condition produced greater pre–post knowledge gains or code-comprehension advantages. Both AI groups reported lower frustration and reduced extraneous and germane load than the control group, while intrinsic load did not differ. Only Iris increased intrinsic motivation. Students rated ChatGPT as easier to use and more helpful. Conclusion: In this setting, generative AI acted primarily as a performance aid rather than a learning enhancer. Scaffolded, hint-first design preserved motivational benefits, whereas AI providing unrestricted solutions encouraged a “comfort trap” where students’ preferences misaligned with pedagogical effectiveness. These findings motivate scaffolded AI integration and assessment designs resilient to environments where performance no longer reliably tracks understanding.