estudio empírico

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial escritura académica expresión escrita

Temas IA: IA y enseñanza-aprendizaje de lenguas chatGPT estudio empírico

Resumen:

This study examines the impact of structured instruction on artificial intelligence (AI) use for improving academic writing in a non-English context. Participants included first-year students enrolled in a teacher education program, including some with disabilities. Three writing tasks were designed: typed essay, AI-based essay, and a critical AI use essay. The repeated measures analyses revealed that targeted AI integration supported overall writing quality, as indicated by improvements in language use, structure, and content. Additionally, students felt more confident when familiarizing themselves with effective prompt strategies. Students with learning difficulties exhibited the most...

Analysis of LLMs for educational question classification and generation

Said Al Faraby, Ade Romadhony, Adiwijaya (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje ChatGPT

Temas IA: IA y creación de materiales modelos de lenguaje (LLM) análisis de producción de IA

Resumen:

Large language models (LLMs) like ChatGPT have shown promise in generating educational content, including questions. This study evaluates the effectiveness of LLMs in classifying and generating educational-type questions. We assessed ChatGPT's performance using a dataset of 4,959 user-generated questions labeled into ten categories, employing various prompting techniques and aggregating results with a voting method to enhance robustness. Additionally, we evaluated ChatGPT's accuracy in generating type-specific questions from 100 reading sections sourced from five online textbooks, which were manually reviewed by human evaluators. We also generated questions based on learning objectives and...

The mediating role of academic stress, critical thinking and performance expectations in the influence of academic self-efficacy on AI dependence: Case study in college students

Benicio Gonzalo Acosta-Enriquez, Marco Agustín Arbulú Ballesteros, Maria de los Angeles Guzman Valle (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial educación superior pensamiento crítico

Temas IA: IA y educación estudio empírico

Resumen:

This study investigated the mediating roles of academic stress, critical thinking, and performance expectations in the relationship between academic self-efficacy and AI dependency among university students. Data were collected via validated instruments and analyzed via structural equation modeling (PLS-SEM) in a cross-sectional study that included 676 students from six universities in northern Peru. The findings indicated that the relationship between academic self-efficacy and AI dependency was substantially mediated by academic stress (β = 0.398, p < 0.001). Furthermore, this relationship is serially mediated by academic stress and performance expectations (β = 0.325, p < 0.001)....

How do LLMs perform in the context of MCQs across different levels of thinking skills in a business education course at higher education? A comparison of ChatGPT, Gemini, and Copilot

Laurens Goorts, Ryan Hollevoet, Vanessa Xia (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje ChatGPT

Temas IA: análisis de producción de IA modelos de lenguaje (LLM) chatGPT

Resumen:

This exploratory study investigates the performance of three widely-used and freely available large language models (LLMs)— ChatGPT (GPT-3.5 Turbo), Gemini, and Copilot—in answering multiple-choice questions (MCQs) categorized by cognitive complexity based on the revised Bloom's Taxonomy. Although MCQs offer a structured, efficient, and scalable method for evaluation, a gap exists in the literature on how LLMs handle varying cognitive levels, particularly in a business course at higher education. Understanding LLM performance in this context is crucial, as students increasingly use LLMs for searching answers but also to receive tailored and scaffolded responses for an interactive and...

From procrastination to engagement? An experimental exploration of the effects of an adaptive virtual assistant on self-regulation in online learning

Eduard Pogorskiy, Jens F. Beckmann (2023)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial tecnología educativa enseñanza en línea e híbrida

Temas IA: IA y aprendizaje estudio empírico

Resumen:

Texto completo en HTML

Compared to traditional classroom learning, success in online learning tends to depend more on the learner’s skill to self-regulate. Self-regulation is a complex meta-cognitive skill set that can be acquired. This study explores the effectiveness of a virtual learning assistant in terms of (a) developmental, (b) general compensatory, and (c) differential compensatory effects on learners’ self-regulatory skills in a sample of N = 157 online learners using an experimental intervention-control group design. Methods employed include behavioural trace data as well as self-reporting measures. Participants provided demographic information and responded to a 24-item self-regulation...

Machine learning role playing game: Instructional design of AI education for age-appropriate in K-12 and beyond

Yusuke Kajiwara, Ayano Matsuoka, Fumina Shinbo (2023)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial alfabetización en IA juegos

Temas IA: IA y educación estudio empírico

Resumen:

The study on K-12 artificial intelligence (AI) education poses three challenges: ''(i) What topics in the machine learning (ML) process should be hidden from K-12 students?” '' (ii) How do you teach K-12 students the mathematical model behind the ML process?” and '' (iii) How does AI education influence the acceptance of AI technology?”. This paper addresses challenges (i) to (iii). We developed a machine learning role-playing game (ML-RPG) and had 166 participants from lower grades of elementary school to the elderly use it. Participants used ML-RPG to role-play the ML process of acquiring data, representing data with graphs, inferencing based on if-then rules, and optimizing parameters...

EvalYaks: Instruction tuning datasets and LoRA fine-tuned models for automated scoring of CEFR B2 speaking assessment transcripts

Nicy Scaria, Silvester John Joseph Kennedy, Thomas Latinovich (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial expresión oral evaluación

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y evaluación modelos de lenguaje (LLM)

Resumen:

Relying on human experts to evaluate the Common European Framework of Reference for Languages (CEFR) speaking assessments in an e-learning environment creates scalability challenges, as it limits how quickly and widely assessments can be conducted. We aim to automate the evaluation of CEFR B2 English speaking assessments in e-learning environments from conversation transcripts. First, we evaluate the capability of leading open source and commercial Large Language Models (LLMs) to score a candidate’s performance across various criteria in the CEFR B2 speaking exam in both global and India-specific contexts. Next, we create a new expert-validated, CEFR-aligned synthetic conversational dataset...

Comparing human and LLM ordered coding of qualitative data: How coding differences cascade through temporal analysis

Kamila Misiejuk, Sonsoles López-Pernas, Eduardo Araujo Oliveira (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial metodología de investigación grandes modelos de lenguaje

Temas IA: modelos de lenguaje (LLM) análisis de producción de IA estudio empírico

Resumen:

Automating the process of qualitatively coding text data from learners has been a long-standing ambition of learning analytics researchers since it represents an essential step toward delivering timely and scalable feedback. Automating this process is especially challenging in the case of ordered coding schemes —necessary for temporal analytical methods— where one text utterance can be assigned more than one qualitative code and the assignment order matters. This problem goes beyond multi-class and multi-label classification and, therefore, cannot be easily tackled using classic language models such as BERT. Recent advances in generative artificial intelligence, especially with the advent...

AI advocates and cautious critics: How AI attitudes, AI interest, use of AI, and AI literacy build university students' AI self-efficacy

Arne Bewersdorff, Marie Hornberger, Claudia Nerdel (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial alfabetización en IA educación superior

Temas IA: IA y educación estudio empírico

Resumen:

This study investigates how cognitive, affective, and behavioral variables related to artificial intelligence (AI) build AI self-efficacy among university students. Based on these variables, we identify three meaningful student groups, which can guide educational initiatives. We recruited 1465 undergraduate and graduate students from the United States, the United Kingdom, and Germany and measured their AI self-efficacy, AI literacy, interest in AI, attitudes towards AI, and AI use. Using a path model, we examine the correlations and paths among these variables. Results reveal that AI usage and positive AI attitudes significantly predict interest in AI, which in turn and together with AI...

Evaluating the capability of large language models in characterising relational feedback: A comparative analysis of prompting strategies

Wei Dai, Yixin Cheng, Ahmad Ari Aldino (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial feedback/retroalimentación grandes modelos de lenguaje

Temas IA: modelos de lenguaje (LLM) prompts IA y evaluación

Resumen:

Relational feedback is increasingly recognised for its crucial role in enhancing student-instructor relationships and promoting the assimilation of feedback. Despite its significance, no studies have tried to develop automated methods to analyse written feedback for properties of relational feedback to promote its use at scale and assist feedback providers with their relational feedback practices. This automated analysis of relational feedback can be performed as a classification task. However, traditional machine and deep learning methods for text classification typically require extensive human labelling and pose a significant challenge for educators and researchers lacking machine...