modelos de lenguaje (LLM)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación grandes modelos de lenguaje

Temas IA: IA y evaluación modelos de lenguaje (LLM) estudio empírico

Resumen:

Identifying logical errors in complex, incomplete or even contradictory and overall heterogeneous data like students’ experimentation protocols is challenging. Recognizing the limitations of current evaluation methods, we investigate the potential of Large Language Models (LLMs) for automatically identifying student errors and streamlining teacher assessments. Our aim is to provide a foundation for productive, personalized feedback. Using a dataset of 65 student protocols, an Artificial Intelligence (AI) system based on the GPT-3.5 and GPT-4 series was developed and tested against human raters. Our results indicate varying levels of accuracy in error detection between the AI system and...

LLMs do not grade essays like humans

Jerin George Mathew, Sumayya Taher, Anindita Kundu (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial expresión escrita evaluación

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y evaluación modelos de lenguaje (LLM)

Resumen:

Large language models have recently been proposed as tools for automated essay scoring, but their agreement with human grading remains unclear. In this work, we evaluate how LLM-generated scores compare with human grades and analyze the grading behavior of several models from the GPT and Llama families in an out-of-the-box setting, without task-specific training. Our results show that agreement between LLM and human scores remains relatively weak and varies with essay characteristics. In particular, compared to human raters, LLMs tend to assign higher scores to short or underdeveloped essays, while assigning lower scores to longer essays that contain minor grammatical or spelling errors. We...

Taming the black box: Design principles for rule-integrated LLM tutoring systems in primary school mathematical problem solving

Chee-Kit Looi, Zhaolong Liu, Daner Sun (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje andamiaje/scaffolding

Temas IA: modelos de lenguaje (LLM) prompts IA y aprendizaje

Resumen:

The integration of large language models (LLMs) into intelligent tutoring systems promises personalized learning, yet procedural domains such as primary mathematics remain challenging due to inconsistency and pedagogical opacity. LLM tutors may give unsolicited answers or show unstable scaffolding when students vary arithmetic, units, or rounding. This design science research develops and evaluates a rule-guided tutoring system for mathematical word problems in primary schools. We formalize a distinction between rule-guided scaffolding, in which tutoring is governed by a three-layer architecture (diagnosis → intent selection → constrained response generation), and ad-hoc scaffolding, where...

Enhancing data analysis and programming skills through structured prompt training: The impact of generative AI in engineering education

Ashish Garg, K. Nisumba Soodhani, Ramkumar Rajendran (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial pensamiento computacional educación superior

Temas IA: prompts IA y aprendizaje modelos de lenguaje (LLM)

Resumen:

The advent of Generative Artificial Intelligence (GenAI) and large language models like LLama, Palm2, GPT, Gemini, and Claude has revolutionized education by generating human-like text and contextually relevant responses. Our research investigates the impact of structured prompt training on students' learning in data analysis and programming. We experimented with 157 first-year engineering students divided into three groups: a control group (internet access, no GenAI), an experimental group 1 (internet and GenAI without prompt training), and an experimental group 2 (internet and GenAI with prompt training). The prompt training session included techniques like few-shot prompting, chain...

Evaluating the performance of ChatGPT and GPT-4o in coding classroom discourse data: A study of synchronous online mathematics instruction

Simin Xu, Xiaowei Huang, Chung Kwan Lo (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial ChatGPT grandes modelos de lenguaje

Temas IA: análisis de producción de IA chatGPT modelos de lenguaje (LLM)

Resumen:

High-quality instruction is essential to facilitating student learning, prompting many professional development (PD) programmes for teachers to focus on improving classroom dialogue. However, during PD programmes, analysing discourse data is time-consuming, delaying feedback on teachers' performance and potentially impairing the programmes' effectiveness. We therefore explored the use of ChatGPT (a fine-tuned GPT-3.5 series model) and GPT-4o to automate the coding of classroom discourse data. We equipped these AI tools with a codebook designed for mathematics discourse and academically productive talk. Our dataset consisted of over 400 authentic talk turns in Chinese from synchronous online...

LLM sentiment quantification reveals selective alignment with human course-evaluation raters

Joyce W. Lacy, Chi Nnoka, Zachary Jock (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje procesamiento del lenguaje natural

Temas IA: modelos de lenguaje (LLM) análisis de producción de IA estudio empírico

Resumen:

Student course evaluations contain rich qualitative feedback in the form of comments written in response to open-ended questions. However, this qualitative data, which may be more nuanced and detailed than quantitative ratings, is often unexamined in both administrative and research settings due to the labor-intensive nature of manual analysis. We investigate whether large language models (LLMs), including BERT, RoBERTa, and OpenAI model variants, can accurately replicate human judgments of sentiment in these comments. We compare masked and generative language models, using both naïve and fine-tuned approaches, to analyze a curated dataset of 1000 de-identified course evaluation responses....

Level-specific feedback generation for scene descriptions via fine-tuning multimodal large language models

ZhiWei Xie, Tse-Tin Chan, Philip L.H. Yu (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial expresión escrita feedback/retroalimentación

Temas IA: IA y enseñanza-aprendizaje de lenguas modelos de lenguaje (LLM) IA y evaluación

Resumen:

Scene description tasks effectively enhance students' English writing skills in contextual settings, facilitating the establishment of authentic situational connections. However, evaluating descriptive quality and providing accurate, level-appropriate feedback present significant challenges. Although Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities in vision-language tasks, their generated feedback for scene description tasks often remains generic. It fails to account for students' educational stages. To address this limitation, we construct a novel level-specific feedback dataset for scene description tasks. This dataset is constructed using GPT-4o with...

AI literacy and its implications for prompt engineering strategies

Nils Knoth, Antonia Tolzin, Andreas Janson (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial alfabetización en IA grandes modelos de lenguaje

Temas IA: prompts modelos de lenguaje (LLM) IA y educación

Resumen:

Artificial intelligence technologies are rapidly advancing. As part of this development, large language models (LLMs) are increasingly being used when humans interact with systems based on artificial intelligence (AI), posing both new opportunities and challenges. When interacting with LLM-based AI system in a goal-directed manner, prompt engineering has evolved as a skill of formulating precise and well-structured instructions to elicit desired responses or information from the LLM, optimizing the effectiveness of the interaction. However, research on the perspectives of non-experts using LLM-based AI systems through prompt engineering and on how AI literacy affects prompting behavior is...

AI-based learning content generation and learning pathway augmentation to increase learner engagement

Chaitali Diwan, Srinath Srinivasa, Gandharv Suri (2023)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial tecnología educativa creación de materiales

Temas IA: IA y creación de materiales IA y aprendizaje modelos de lenguaje (LLM)

Resumen:

Texto completo en HTML

Retaining learner engagement is a major challenge in online learning environments, which is even more intensified with learning spaces increasingly built by combining resources from multiple independent sources. Narrative-centric learning experience has been found to improve learner engagement by several researchers. Towards this end, we propose an AI-based approach that generates auxiliary learning content called narrative fragments which are interspersed into the learning pathways to create interactive learning narratives. The proposed approach consists of the automatic generation of two types of narrative fragments– overviews of the learning pathway segments and reflection...

A scoping review of generative AI-powered agentic AI in education: Research landscape, agentic capabilities, and insights from the frontier agent paradigm, exemplified by OpenClaw

Ningxia Wang, Di Zou, Haoran Xie (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: revisión bibliográfica

Temas: inteligencia artificial grandes modelos de lenguaje tecnología educativa

Temas IA: modelos de lenguaje (LLM) IA y educación revisión de bibliografía

Resumen:

Generative AI-powered agentic systems are increasingly proposed for educational applications. However, the research landscape remains fragmented, and the relationship between technical capabilities and pedagogical design remains poorly understood. To address this gap, this scoping review systematically mapped 474 studies published between January 2020 and May 2026. Guided by three research questions, we analysed publication characteristics, study designs, agent roles, AI models and architectures, six dimensions of agentic capability, and the extent to which educational theory was incorporated. The results show that the field has expanded rapidly, particularly since 2025. At the same time,...