IA y evaluación

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial feedback/retroalimentación andamiaje/scaffolding

Temas IA: IA y enseñanza-aprendizaje de lenguas chatGPT IA y evaluación

Resumen:

Teacher corrective feedback (TCF) plays a vital role in second language (L2) learning. Recent studies have examined feedback provided by both human teachers and large language models (LLMs). However, little is known about how students' trust differs toward scaffolded corrective feedback (SCF)—that is, feedback that incrementally progresses from indirect to direct during interaction—when it is provided by an LLM such as ChatGPT versus a language teacher. To address this gap, this study compared the effects of SCF, delivered by language teachers and ChatGPT, on L2 learning outcomes and student trust. Using a mixed-methods design, 40 lower-intermediate Iranian learners of English as a foreign...

The rapid rise of generative AI and its implications for academic integrity: Students’ perceptions and use of chatbots for assistance with assessments

Jan Henrik Gruenhagen, Peter M. Sinclair, Julie-Anne Carroll (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial integridad académica ChatGPT

Temas IA: chatGPT ética de la IA IA y evaluación

Resumen:

The rapid adoption of generative AI tools such as ChatGPT by students has the potential to disrupt the higher education sector, with concerns being raised by academics about potential threats to academic integrity. This paper contributes to the pressing discussion about responses to AI tools by examining students' perceptions and the use of generative AI to assist them with assessments. Based on a survey among 337 Australian university students, this study found that more than a third of students have used a chatbot for assistance with an assessment, and do not necessarily perceive this as a breach of academic integrity. The study further investigated to what extent different psychosocial...

A Bayesian active learning approach to comparative judgement within education assessment

Andy Gray, Alma Rahat, Tom Crick (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación

Temas IA: IA y evaluación estudio empírico

Resumen:

Assessment is a crucial part of education. Traditional marking is a source of inconsistencies and unconscious bias, placing a high cognitive load on the assessors. One approach to address these issues is comparative judgement (CJ). In CJ, the assessor is presented with a pair of items of work, and asked to select the better one. Following a series of comparisons, a rank for any item may be derived using a ranking model, for example, the Bradley-Terry model, based on the pairwise comparisons. While CJ is considered to be a reliable method for conducting marking, there are concerns surrounding its transparency, and the ideal number of pairwise comparisons to generate a reliable estimation of...

AI proctoring for offline examinations with 2-Longitudinal-Stream Convolutional Neural Networks

Jason Tong Liu (2023)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación tecnología educativa

Temas IA: IA y evaluación IA y educación

Resumen:

Texto completo en HTML

In recent years, Artificial Intelligence (AI)-assisted proctoring techniques have developed rapidly, but most are for online exams. On the contrary, this study aims to explore the AI-based visual proctoring approach through surveillance cameras in real-world offline examination rooms (usually classrooms) to assist proctoring, with contributions from three perspectives: First, to proctor in the offline examination rooms, a visual assessment system is introduced that can be applied in the majority of various classrooms for proctoring after corresponding model training. Second, the new proposed 2-Longitudinal-Stream Convolutional Neural Networks model can adequately adapt to the large-...

Enhancing language learning through generative AI feedback on picture-cued writing tasks

Yipeng Zhuang, Ruibin Zhao, ZhiWei Xie (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial expresión escrita feedback/retroalimentación

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y evaluación modelos de lenguaje (LLM)

Resumen:

Generative AI (GAI) is transforming education, despite its widespread use, its role in supporting student learning remains underexplored, and how to effectively leverage GAI for educational purposes is still unclear. This study focuses on enhancing language learning through GAI, specifically in picture-cued writing tasks, where students describe life scenarios depicted in pictures through text. A Generative AI-assisted language learning system was developed, powered by fine-tuned multimodal Large Language Models (LLMs), designed to evaluate students' textual descriptions in relation to corresponding images and provide adaptive feedback. A lot of middle school students participated in the...

Beyond accuracy: Multimodal modeling of structured speaking skill indices in young adolescents

Candy Olivia Mawalim, Chee Wee Leong, Guy Sivan (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial expresión oral evaluación

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y evaluación estudio empírico

Resumen:

This study introduces a novel method for explainable speaking skill assessment that utilizes a unique dataset featuring video recordings of conversational interviews for high-stakes outcomes (i.e., admission to high schools and universities). Unlike traditional automated speaking assessments that prioritize accuracy at the expense of interpretability, our approach employs a new multimodal dataset that integrates acoustic and linguistic features, visual cues, turn-taking patterns, and expert-derived scores quantifying various speaking skill aspects observed during interviews with young adolescents. This dataset is distinguished by its open-ended question format, which allows for varied...

Investigating the affordances of OpenAI's large language model in developing listening assessments

Vahid Aryadoust, Azrifah Zakaria, Yichen Jia (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial comprensión oral evaluación

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y evaluación IA y creación de materiales

Resumen:

To address the complexity and high costs of developing listening tests for test-takers of varying proficiency levels, this study investigates the capabilities of an OpenAI's large language model, ChatGPT 4, in developing listening assessments. Employing prompt engineering and fine-tuning of prompts, the study specifically focuses on creating listening scripts and test items using ChatGPT 4 for test-takers across a spectrum of proficiency levels (academic, low, intermediate, and advanced). For comparability, the 24 topics of these scripts were selected from topics found in academic listening tests. We conducted two types of analyses to evaluate the quality of the output. First, we performed...

LLMs do not grade essays like humans

Jerin George Mathew, Sumayya Taher, Anindita Kundu (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial expresión escrita evaluación

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y evaluación modelos de lenguaje (LLM)

Resumen:

Large language models have recently been proposed as tools for automated essay scoring, but their agreement with human grading remains unclear. In this work, we evaluate how LLM-generated scores compare with human grades and analyze the grading behavior of several models from the GPT and Llama families in an out-of-the-box setting, without task-specific training. Our results show that agreement between LLM and human scores remains relatively weak and varies with essay characteristics. In particular, compared to human raters, LLMs tend to assign higher scores to short or underdeveloped essays, while assigning lower scores to longer essays that contain minor grammatical or spelling errors. We...

The development and implementation of a computer adaptive progress test across European countries

Neil Rice, José Miguel Pêgo, Carlos Fernando Collares (2022)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación educación superior

Temas IA: IA y evaluación estudio empírico

Resumen:

Longitudinal progress testing promotes self-directed deep learning across a full spectrum of knowledge, enabling early detection of underperformance and opportunities for remediation. Computer adaptive testing (CAT), where the difficulty of a test dynamically adjusts according to a test taker's ability, has benefits in a progress testing context, but significant resource and experience is required to develop appropriate test materials. This study describes how a transnational consortium from eight medical schools in five countries across Europe was formed to develop a computer adaptive progress test applicable across international curricula. 1,212 students from more than 40 nationalities...

Optimizing automated scoring in ILSAs with prompt compression

Ji Yoon Jung, Ummugul Bezirhan, Matthias von Davier (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación grandes modelos de lenguaje

Temas IA: IA y evaluación modelos de lenguaje (LLM) prompts

Resumen:

Automated scoring (AS) has become increasingly prevalent in educational measurement. However, applying it to international reading assessments remains challenging, particularly due to the length and complexity of the required prompting, driven by the need to include lengthy reading passages and detailed scoring guides. Processing these lengthy inputs results in high computational costs and may impede the performance of large language models (LLMs). This study explored the potential of optimizing AS with prompt compression using OpenAI's LLM, GPT-4o. Our results show that prompt compression significantly reduces the length of reading passages and scoring guides while maintaining their...