estudio empírico

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación ChatGPT

Temas IA: IA y evaluación IA y creación de materiales análisis de producción de IA

Resumen:

Not much is known about how LLM-generated questions compare to gold-standard, traditional formative assessments concerning their difficulty and discrimination parameters, which are valued properties in the psychometric measurement field. We follow a rigorous measurement methodology to compare a set of ChatGPT-generated questions, produced from one lesson summary in a textbook, to existing questions from a published Creative Commons textbook. To do this, we collected and analyzed responses from 207 test respondents who answered questions from both item pools and used a linking methodology to compare IRT properties between the two pools. We find that neither the difficulty nor discrimination...

The critical role of trust in adopting AI-powered educational technology for learning: An instrument for measuring student perceptions

Tanya Nazaretsky, Paola Mejia-Domenzain, Vinitra Swamy (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial creencias y actitudes de los estudiantes educación superior

Temas IA: IA y educación estudio empírico

Resumen:

In recent decades, we have witnessed the democratization of AI-powered Educational Technology (AI-EdTech). However, despite the increased accessibility and evolving technological capabilities, its adoption is accompanied by significant challenges, predominantly rooted in social and psychological aspects. At the same time, limited research has been conducted on human factors, especially trust, influencing students' readiness and willingness to adopt AI-EdTech. This study aims to bridge this gap by addressing the multidimensional nature of trust and developing a new instrument for measuring students' perceptions of adopting AI-EdTech. With 665 student responses, we employ Exploratory and...

How does students' perception of ChatGPT shape online learning engagement and performance?

Mahboobeh Mehrvarz, Ghasem Salimi, Samaneh Abdoli (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial ChatGPT enseñanza en línea e híbrida

Temas IA: chatGPT estudio empírico

Resumen:

Artiicial intelligence technologies, such as ChatGPT, Bert, LaMDA, and T5 have become deeply integrated into universities, influencing numerous functions within higher education and various aspects of students' lives. This article examined how students' perceptions of ChatGPT influence their engagement and performance in online learning, focusing on the mediating role of engagement. The perception of ChatGPT involves a student's understanding of, willingness to utilize, and concerns regarding the technology. These interconnected factors can influence students' online learning engagement with the software and, consequently, affect their academic performance. The study used self-reported data...

Effects of integrating an open learner model with AI-enabled visualization on students' self-regulation strategies usage and behavioral patterns in an online research ethics course

Jerry Chih-Yuan Sun, Hsueh-Er Tsai, Wai Ki Rebecca Cheng (2023)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial tecnología educativa enseñanza en línea e híbrida

Temas IA: IA y aprendizaje estudio empírico

Resumen:

Texto completo en HTML

This study seeks to understand the effects of a personalized learning platform, which applied an open learner model with self-regulation and AI-enabled data visualization features, on students' self-regulated learning strategies used and behavioral patterns during their self-directed online learning process. Results were based on a self-regulated learning scale and a prior knowledge test from 182 university students and supplemented with students' system logs to understand their behavioral patterns while interacting with the personalized learning platform. It showed that the combined self-regulation and data visualization features (dual features) improve student performance via self...

Stretching AI's reach: Assessing an AI-driven feedback system for extended academic writing

Jim Lo, Christy Wong, Agnes Ng (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial escritura académica feedback/retroalimentación

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y evaluación modelos de lenguaje (LLM)

Resumen:

Advances in large language models (LLMs) enable timely and scalable writing evaluation. Previous research has shown that LLM-driven conversational systems, such as ChatGPT, can provide feedback on short essays. However, it is unclear whether AI can effectively evaluate more demanding genres. This study investigates a custom-built writing feedback system developed at a Hong Kong university that uses OpenAI's GPT-4 Turbo (0125-preview) to provide rubric-based feedback on a 1500-word academic report. Guided by a detailed, rubric-aligned prompt, the system generated 333 feedback items from 37 undergraduates, which were analysed for accuracy, tone, and inclusion of examples. The analysis showed...

Automatic proficiency scoring for early-stage writing

Michael Riis Andersen, Kristine Kabel, Jesper Bremholm (2023)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial expresión escrita evaluación

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y evaluación estudio empírico

Resumen:

In this work, we study the feasibility of using machine learning and natural language processing methods for assessing writing proficiency in Danish with respect to text construction, sentence construction, and use of modifiers. Our work is based on the analytical framework for scoring early writing proposed by Kabel et al. (2022), where each text is first annotated by a human expert according to a predefined coding scheme and subsequently scored using statistical Rasch modeling (Rasch, 1960). We investigate two different strategies for estimating these scores automatically: 1) we propose a system for identifying the central linguistic features automatically mimicking the role of the human...

The impact of item-writing flaws on difficulty and discrimination in item response theory

Robin Schmucker, Steven Moore (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación

Temas IA: IA y evaluación estudio empírico

Resumen:

High-quality test items are essential for educational assessments, particularly within Item Response Theory (IRT). Traditional validation methods rely on resource-intensive pilot testing to estimate item difficulty and discrimination. More recently, Item-Writing Flaw (IWF) rubrics emerged as a domain-general approach for evaluating test items based on textual features. This method offers a scalable, pre-deployment evaluation without requiring student data, but its predictive validity concerning empirical IRT parameters is underexplored. To address this gap, we conducted a study involving 7,126 multiple-choice questions across various STEM subjects (physical science, mathematics, and life/...

Generative AI in higher education: Seeing ChatGPT through universities' policies, resources, and guidelines

Hui Wang, Anh Dang, Zihao Wu (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial educación superior integridad académica

Temas IA: IA y educación chatGPT ética de la IA

Resumen:

The advancements in Generative Artificial Intelligence (GenAI) can provide opportunities for enriching educational experiences, but at the same time raise concerns regarding academic integrity. Many educators have expressed anxiety and hesitation when it comes to integrating GenAI in their teaching practices. Thus, recommendations and guidance from institutions are needed to support instructors in this new and emerging GenAI era. In response to this need, this study explores different U.S. universities' academic policies and guidelines regarding the use of GenAI tools (e.g., ChatGPT) for teaching and learning, and from there, gains understanding of how these universities respond and adapt...

Assessing reading fluency in elementary grades: A machine learning approach

Gabriel Candido da Silva, Rodrigo Lins Rodrigues, Américo N. Amorim (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial comprensión lectora alfabetización

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y evaluación estudio empírico

Resumen:

This study compares eleven widely used machine learning algorithms to identify the most accurate and comprehensive method for assessing children's reading fluency. Our fluency framework integrates three key dimensions: accuracy in word decoding, reading speed (words per minute), and prosody, which captures appropriate pausing and intonation during reading. Audio recordings from 2nd and 3rd grade students across 144 Brazilian schools were transcribed using the advanced Whisper ASR system, enabling automated extraction of fluency features. The research objective was to determine which algorithm best predicts fluency, considering diverse evaluation setups including binary classification (...

A study of developing administrative Staff's conceptual understanding of generative artificial intelligence through professional Development: Evaluation of a course using tests, surveys and thematic analysis of reflective writings

Siu-Cheung Kong, Wenxi Hu (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial alfabetización en IA diseño curricular

Temas IA: IA y educación estudio empírico

Resumen:

Emerging research underscores the growing importance of artificial intelligence (AI) literacy for administrative staff as AI technologies continue to reshape the labor market. However, there remains a limited study about the curriculum components of AI literacy for administrative staff. This study aims to bridge this gap by designing and evaluating a professional development programme for administrative staff grounded in a four-dimensional AI literacy framework. A total of 62 administrative staff from diverse professional backgrounds participated in a 30-h course on understanding Generative Artificial Intelligence (GenAI) and its applications. Pre- and post-course conceptual tests and...