Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

Identifying logical errors in complex, incomplete or even contradictory and overall heterogeneous data like students’ experimentation protocols is challenging. Recognizing the limitations of current evaluation methods, we investigate the potential of Large Language Models (LLMs) for automatically identifying student errors and streamlining teacher assessments. Our aim is to provide a foundation for productive, personalized feedback. Using a dataset of 65 student protocols, an Artificial Intelligence (AI) system based on the GPT-3.5 and GPT-4 series was developed and tested against human raters. Our results indicate varying levels of accuracy in error detection between the AI system and...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

This study examines the reliability and accuracy of large language models (LLMs) for automated classroom observation using the World Bank's TEACH Primary framework. As education systems increasingly explore AI-based tools to scale teacher feedback and professional development, empirical validation of these systems is critical. Using a corpus of 12 primary classroom videos, we compared 8618 AI-generated evaluations from eight LLM endpoints against consensus-based ratings from certified TEACH experts. To account for model stochasticity, each model produced 10 independent evaluations per video–element pair. Reliability was assessed using variability and inter-rater consistency indicators,...

Ver ficha completa

Tipo: artículo

Metodología: revisión bibliográfica

Temas: inteligencia artificial

Resumen:

Texto completo

Advancements in large language models (LLMs) have transformed AI-driven education, enabling innovative applications across various learning and teaching domains. However, LLMs still face several challenges, including hallucination and static internal knowledge, which hinder their reliability in educational settings. Retrieval-Augmented Generation (RAG) enhances LLMs by retrieving relevant information from an external knowledge base and incorporating it into the LLM's generation process. This approach improves factual accuracy and enables dynamic knowledge updates, making LLMs particularly suitable for educational applications. In this paper, we comprehensively review existing research that...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

Large language models (LLMs) that power chatbots such as ChatGPT have capabilities across numerous domains. Teachers and students have been increasingly using chatbots in science, technology, engineering, and mathematics (STEM) subjects in various ways, including for assessment purposes. However, there has been a lack of systematic investigation into LLMs’ capabilities and limitations in automatically generating items for STEM subject assessments, especially given that LLMs can hallucinate and may risk promoting misconceptions and hindering conceptual understanding. To address this, we systematically investigated LLMs' conceptual understanding and quality of working in generating question...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

Student engagement, widely recognised as a key determinant of learning outcomes, warrants greater attention to its antecedent mechanisms for effective learning design. As LLM-based chatbots (hereafter chatbot) are increasingly used to support student learning, understanding what drives student engagement in such environments becomes especially important. Guided by appraisal theory, this study proposes a theoretically grounded model of students' engagement in chatbot-supported learning. Unlike prior studies that focus on isolated antecedents, the model integrates learners’ capacity appraisals, including technology-related (GenAI competency) and task-related (academic self-efficacy)...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

This paper introduces the design and development of a framework that integrates a large language model (LLM) with a retrieval-augmented generation (RAG) approach leveraging both a knowledge graph and user interaction history. The framework is incorporated into a previously developed adaptive learning support system to assess learners’ code, generate formative feedback, and recommend exercises. Moreover, this study examines learner preferences across three instructional modes: adaptive, Generative AI (GenAI), and hybrid GenAI–adaptive. An experimental study was conducted to compare the learning performance and perception of the learners, and the effectiveness of these three modes using four...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

Researchers have sought for decades to automate holistic essay scoring. Over the years, these programs have improved significantly. However, accuracy requires significant amounts of training on human-scored texts—reducing the expediency and usefulness of such programs for routine uses by teachers across the nation on non-standardized prompts. This study analyzes the output of multiple versions of ChatGPT scoring of secondary student essays from three extant corpora and compares it to quality human ratings. We find that the current iteration of ChatGPT scoring is not statistically significantly different from human scoring; substantial agreement with humans is achievable and may be...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

To address the complexity and high costs of developing listening tests for test-takers of varying proficiency levels, this study investigates the capabilities of an OpenAI's large language model, ChatGPT 4, in developing listening assessments. Employing prompt engineering and fine-tuning of prompts, the study specifically focuses on creating listening scripts and test items using ChatGPT 4 for test-takers across a spectrum of proficiency levels (academic, low, intermediate, and advanced). For comparability, the 24 topics of these scripts were selected from topics found in academic listening tests. We conducted two types of analyses to evaluate the quality of the output. First, we performed...

Ver ficha completa

Tipo: artículo

Metodología: revisión bibliográfica

Temas: inteligencia artificial

Resumen:

Texto completo

The adoption of Generative AI (GenAI) and Large Language Models (LLMs) in healthcare education has accelerated rapidly since 2023, yet the evidence base for their use in Scenario-Based Learning (SBL) remains fragmented. This systematic review synthesises empirical research on GenAI applications across scenario-based, case-based, problem-based, and simulation-based learning in healthcare education. Following the PRISMA 2020 guidelines, five databases were searched on 9 November 2025 for peer-reviewed studies published from January 2023 onwards. Of 1151 initial records, 23 studies met the inclusion criteria. Quality was assessed using the Mixed Methods Appraisal Tool (MMAT). Thematic...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

This research assesses the effectiveness of state-of-the-art large language models (LLMs), including ChatGPT, Llama, Aya, Jais, and ACEGPT, in the task of Arabic automated essay scoring (AES) using the AR-AES dataset. It explores various evaluation methodologies, including zero-shot, few-shot in context learning, and fine-tuning, and examines the influence of instruction-following capabilities through the inclusion of marking guidelines within the prompts. A mixed-language prompting strategy, integrating English prompts with Arabic content, was implemented to improve model comprehension and performance. Among the models tested, ACEGPT demonstrated the strongest performance across the...

Ver ficha completa