Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial juegos evaluación

Temas IA: IA y evaluación estudio empírico

Resumen:

Texto completo

Highlights:

• The analysis of gameplay data from the Blood Transfusion Serious Game (BTSG) revealed crucial insights into the engagement, performance, and competency levels of participating nurses.
• The study highlighted the efficiency of the adaptive assessment algorithm in determining competency indicators through gameplay activities.
• Notable discrepancies existed between the game-embedded result and the adaptive assessment result, suggesting potential biases in the game result, which tended to overestimate the demonstrated competence.

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación grandes modelos de lenguaje

Temas IA: IA y evaluación modelos de lenguaje (LLM) estudio empírico

Resumen:

Texto completo

Identifying logical errors in complex, incomplete or even contradictory and overall heterogeneous data like students’ experimentation protocols is challenging. Recognizing the limitations of current evaluation methods, we investigate the potential of Large Language Models (LLMs) for automatically identifying student errors and streamlining teacher assessments. Our aim is to provide a foundation for productive, personalized feedback. Using a dataset of 65 student protocols, an Artificial Intelligence (AI) system based on the GPT-3.5 and GPT-4 series was developed and tested against human raters. Our results indicate varying levels of accuracy in error detection between the AI system and...

Ver ficha completa

Tipo: artículo

Metodología: trabajo teórico

Temas: inteligencia artificial integridad académica evaluación

Temas IA: IA y evaluación ética de la IA

Resumen:

Texto completo

We consider how a future of pervasive human/AI coauthorship challenges current notions of validity and academic integrity. Specifically, we address widespread concerns that in non-proctored contexts, students are using generative artificial intelligence (GenAI) to submit texts they do not understand. Adopting an assessment validity lens, we show how GenAI undermines the integrity of multiple forms of validity evidence, leading us to propose Coauthorship Integrity as a new conceptual source of validity evidence for addressing these threats. Coauthorship Integrity is violated when students submit AI-generated content that they do not understand. To hold students accountable in this regard, a...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial feedback/retroalimentación procesamiento del lenguaje natural

Temas IA: IA y evaluación estudio empírico

Resumen:

Texto completo

Feedback is a crucial element of a student's learning process. It enables students to identify weaknesses and improve self-regulation. However, studies show this to be an area of great dissatisfaction in higher education. With ever-growing course participation numbers, delivering effective feedback is becoming an increasingly challenging task. The efficacy of feedback will depend on four levels of feedback; namely, feedback about the self, task, process or self-regulation. Hence, this paper explores the use of automated content analysis to examine feedback provided by instructors for feedback practices measured on self, task, process, and self-regulation levels. For this purpose, four...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial feedback/retroalimentación educación superior

Temas IA: IA y evaluación estudio empírico

Resumen:

Texto completo

While feedback is essential for guiding student learning, providing timely and personalized guidance in large-scale educational settings remains a significant challenge. Generative AI offers a scalable solution, yet little is known about students’ perceptions of AI-generated feedback. In this paper, we aim to investigate how the identity of the feedback provider (human vs. AI) affects students’ ability to assess feedback quality and whether their judgments are biased. We propose a comprehensive rubric for assessing the pedagogical quality of formative feedback. We use it to compare the objective quality of AI-generated and human-crafted feedback (N = 979). Next, using data collected from...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación ChatGPT

Temas IA: IA y evaluación IA y creación de materiales análisis de producción de IA

Resumen:

Texto completo

Not much is known about how LLM-generated questions compare to gold-standard, traditional formative assessments concerning their difficulty and discrimination parameters, which are valued properties in the psychometric measurement field. We follow a rigorous measurement methodology to compare a set of ChatGPT-generated questions, produced from one lesson summary in a textbook, to existing questions from a published Creative Commons textbook. To do this, we collected and analyzed responses from 207 test respondents who answered questions from both item pools and used a linking methodology to compare IRT properties between the two pools. We find that neither the difficulty nor discrimination...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial juegos evaluación

Temas IA: IA y evaluación estudio empírico

Resumen:

Texto completo

This study explores the use of serious games combined with machine learning techniques to evaluate student efficiency and determine appropriate degree programs. The main research question addressed is: How can predictive machine learning algorithms, applied to soft skills data collected through serious games, identify the most suitable study path for each student, thereby improving academic orientation and enhancing the likelihood of educational success? The research involved 211 university students from a single university in Italy, focusing on the application of predictive algorithms based on participants' soft skills and academic performance (GPA). The study employs logistic regression...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial ChatGPT educación superior

Temas IA: chatGPT IA y evaluación estudio empírico

Resumen:

Texto completo

ChatGPT stands out among other AI technology tools to provide assessment support in the higher education context. This exploratory study investigates the motivators and barriers which influence the use intention of ChatGPT for assessment support among undergraduates in Hong Kong. We tested the adapted and extended UTUAT model using the structural equation modeling approach. Through self-report online questionnaire, we received usable responses from 483 undergraduates of eight Hong Kong universities.In a highly competitive Chinese context, assessment results hold immense significance in determining the value and merit of students. The findings reveal that trust acts as the strongest...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación educación primaria y secundaria

Temas IA: IA y evaluación IA y educación ética de la IA

Resumen:

Texto completo en HTML

The COVID-19 pandemic has created significant challenges for UK schools, but a time of cancelled exams and uncertainty around future examinations can provide opportunities to explore novel assessment methods. Hence, the 2020 proposal of the Ofqual algorithm which combines teachers' estimated grades and schools' historical performance seemed timely. However, the algorithmically calculated grades resulted in a public backlash and withdrawal of the proposal. While the failed Ofqual algorithm could be considered an example of AI, we do not yet have a thorough understanding of its numerical accuracy and how it performs in comparison to other AI models. This paper investigates this novel...

Ver ficha completa

Tipo: artículo

Metodología: revisión bibliográfica

Temas: inteligencia artificial evaluación procesamiento del lenguaje natural

Temas IA: IA y evaluación revisión de bibliografía

Resumen:

Texto completo

Text-based open-ended questions in academic formative and summative assessments help students become deep learners and prepare them to understand concepts for a subsequent conceptual assessment. However, grading text-based questions, especially in large (>50 enrolled students) courses, is tedious and time-consuming for instructors. Text processing models continue progressing with the rapid development of Artificial Intelligence (AI) tools and Natural Language Processing (NLP) algorithms. Especially after breakthroughs in Large Language Models (LLM), there is immense potential to automate rapid assessment and feedback of text-based responses in education. This systematic review adopts a...

Ver ficha completa