modelos de lenguaje (LLM)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación grandes modelos de lenguaje

Temas IA: IA y evaluación modelos de lenguaje (LLM) estudio empírico

Resumen:

Semi open-ended multipart questions consist of multiple sub questions within a single question, requiring students to provide certain factual information while allowing them to express their opinion within a defined context. Human grading of such questions can be tedious, constrained by the marking scheme and susceptible to the subjective judgement of instructors. The emergence of large language models (LLMs) such as ChatGPT has significantly advanced the prospect of automatic grading in educational settings. This paper introduces a topic-based grading approach that harnesses LLM capabilities alongside a refined marking scheme to ensure fair and explainable assessment processes. The...

Large language models for education: An open-source paradigm for automated Q&A in the graduate classroom

Ryann M. Perez, Marie Shimogawa, Yanan Chang (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje andamiaje/scaffolding

Temas IA: modelos de lenguaje (LLM) IA y aprendizaje estudio empírico

Resumen:

Large Language Models (LLMs) offer scalable educational support, but face barriers regarding accuracy, cost, and learning depth. To interrogate these limitations, we developed the Teaching Assistant for Specialized Knowledge (TAsk), a retrieval-augmented generation enabled and educator curated pipeline. In this nine-week pilot study (N = 33 participants), we deployed TAsk in a graduate-level biological chemistry course. We compared TAsk against human expert teaching assistants (TAs) using blinded review process and analyzed inquiry depth. We observed three major findings related to potential pedagogical decisions and educational theory. First, TAsk delivered effective feedback that was...

From knowledge gaps to learning opportunities: Leveraging student questions and dual use of generative AI to support student learning at scale

Stanislav Pozdniakov, Jonathan Brazil, Oleksandra Poquet (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje feedback/retroalimentación

Temas IA: modelos de lenguaje (LLM) IA y creación de materiales IA y aprendizaje

Resumen:

University courses with hundreds of students have become common, particularly during early years of university studies. The sheer scale of these courses limits traditional instruction, shifting it towards a one-to-many mode of delivery. This shift reduces student–instructor interaction and tailored instructor feedback which are crucial for student success. Automated feedback systems allow scaling feedback, but they often reduce instructor contributions to student learning. This paper investigates how emerging technologies can support, rather than replace, instructors in tailoring their teaching and feedback to identify and correct student knowledge gaps at scale. To address this challenge,...

Large language models and automated essay scoring of English language learner writing: Insights into validity and reliability

Austin Pack, Alex Barrett, Juan Escalante (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial expresión escrita evaluación

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y evaluación modelos de lenguaje (LLM)

Resumen:

Advancements in generative AI, such as large language models (LLMs), may serve as a potential solution to the burdensome task of essay grading often faced by language education teachers. Yet, the validity and reliability of leveraging LLMs for automatic essay scoring (AES) in language education is not well understood. To address this, we evaluated the cross-sectional and longitudinal validity and reliability of four prominent LLMs, Google's PaLM 2, Anthropic's Claude 2, and OpenAI's GPT-3.5 and GPT-4, for the AES of English language learners' writing. 119 essays taken from an English language placement test were assessed twice by each LLM, on two separate occasions, as well as by a pair of...

Can large language models write reflectively

Yuheng Li, Lele Sha, Lixiang Yan (2023)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje expresión escrita

Temas IA: análisis de producción de IA modelos de lenguaje (LLM) chatGPT

Resumen:

Texto completo en HTML

Generative Large Language Models (LLMs) demonstrate impressive results in different writing tasks and have already attracted much attention from researchers and practitioners. However, there is limited research to investigate the capability of generative LLMs for reflective writing. To this end, in the present study, we have extensively reviewed the existing literature and selected 9 representative prompting strategies for ChatGPT – the chatbot based on state-of-art generative LLMs to generate a diverse set of reflective responses, which are combined with student-written reflections. Next, those responses were evaluated by experienced teaching staff following a theory-aligned...

Standardized assessment of LLM English proficiency

Shaonan Wang, Shangchao Min, Hui Wang (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje evaluación

Temas IA: IA y enseñanza-aprendizaje de lenguas modelos de lenguaje (LLM) análisis de producción de IA

Resumen:

Large language models (LLMs) are increasingly used in language learning and assessment, yet their English proficiency is seldom reported against interpretable proficiency standards. We introduce the China’s Standards of English Language Ability (CSE) framework to assess the proficiency levels and subskills of LLMs. The test is referred to as the CSEBench and comprises 624 expert-annotated multiple-choice items across CSE Levels 2–7. Each item is accompanied by metadata, including difficulty level and subskill labels covering vocabulary, syntax, phonology, and cohesion/discourse. Critically, the dataset includes test responses from 2,050 middle school and and sophomore college students who...

Enhancing traditional ITS architectures with large language models for generating motivational feedback

Angelo Gaeta, Francesco Orciuoli, Antonella Pascuzzo (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje feedback/retroalimentación

Temas IA: modelos de lenguaje (LLM) IA y aprendizaje estudio empírico

Resumen:

Large Language Models (LLMs) have emerged as powerful computational tools with significant potential across various fields, particularly education. Previous research demonstrated that LLMs can be effectively integrated into the Tutoring Model of Intelligent Tutoring Systems (ITS) to generate motivational feedback within learning environments. To further explore this area, the authors define an ITS architecture that integrates an LLM (referred to as the ITS-LLM architecture) and validates this architecture by implementing a prototype using Llama 3.2 model. Moreover, the prototype aims to evaluate the impact of LLM-generated motivational feedback on student motivation and their learning...

Scaffolding critical thinking with generative AI: Design principles for integrating large language models in higher education

Mireia Vendrell, Samantha-Kaye Johnston (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: trabajo teórico

Temas: inteligencia artificial pensamiento crítico andamiaje/scaffolding

Temas IA: modelos de lenguaje (LLM) IA y aprendizaje

Resumen:

The rapid adoption of Large Language Models (LLMs) such as GPT-4 and DeepSeek R1 is transforming learning in higher education, yet unstructured use can weaken critical thinking by encouraging cognitive offloading, metacognitive disengagement, and reduced epistemic agency. This paper presents a conceptual and normative analysis that synthesises research from cognitive psychology, educational theory, and AI ethics to develop a design-oriented pedagogical framework for integrating LLMs in ways that strengthen, rather than displace, higher-order thinking. Grounded in design-based research principles, the framework identifies six essential processes that underpin critical engagement: conceptual...

Assessing how accurately large language models encode and apply the common European framework of reference for languages

Luca Benedetto, Gabrielle Gaudeau, Andrew Caines (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial MCER grandes modelos de lenguaje

Temas IA: IA y enseñanza-aprendizaje de lenguas modelos de lenguaje (LLM) análisis de producción de IA

Resumen:

Large Language Models (LLMs) can have a transformative effect on a variety of domains, including education, and it is therefore pressing to understand whether these models have knowledge of – or, in other words, how they have encoded – the specific pedagogical requirements of different educational domains, and whether they use this when performing educational tasks. In this work, we propose an approach to evaluate the knowledge – or encoding – that the LLMs have of the Common European Framework of Reference for Languages (CEFR), and use it to evaluate five modern LLMs. Our study shows that the suite of tasks we propose is quite challenging for all the LLMs, and they often provide results...

LLaVA-docent: Instruction tuning with multimodal large language model to support art appreciation education

Unggi Lee, Minji Jeon, Yunseo Lee (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje chatbots

Temas IA: modelos de lenguaje (LLM) chatbots IA y aprendizaje

Resumen:

Despite the development of various AI systems to support learning in various domains, AI assistance for art appreciation education has not been extensively explored. Art appreciation, often perceived as an unfamiliar and challenging endeavor for most students, can be more accessible with a generative AI enabled conversation partner that provides tailored questions and encourages the audience to deeply appreciate artwork. This study explores the application of multimodal large language models (MLLMs) in art appreciation education, with a focus on developing LLaVA-Docent, a model designed to serve as a personal tutor for art appreciation. Our approach involved design and development research...