modelos de lenguaje (LLM)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial creación de materiales evaluación

Temas IA: IA y creación de materiales IA y evaluación modelos de lenguaje (LLM)

Resumen:

The process of manually generating question and answer (QA) pairs for assessments is known to be a time-consuming and energy-intensive task for teachers, specifically in higher education. Several studies have proposed various methods utilising pre-trained large language models for the generation of QA pairs. However, it is worth noting that these methods have primarily been evaluated on datasets that are not specifically educational in nature. Furthermore, the evaluation metrics and strategies employed in these studies differ significantly from those typically used in educational contexts. The present discourse fails to present a compelling case regarding the efficacy and practicality of...

Automated reading passage generation with OpenAI's large language model

Ummugul Bezirhan, Matthias von Davier (2023)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial comprensión lectora evaluación

Temas IA: IA y enseñanza-aprendizaje de lenguas IA y creación de materiales IA y evaluación

Resumen:

The widespread usage of computer-based assessments and individualized learning platforms has increased demand for the rapid production of high-quality items. Automated item generation (AIG), the process of using item models to generate new items with the help of computer technology, was proposed to reduce reliance on human subject experts. While AIG has been used in test development, recent advances in machine learning algorithms offer the potential to enhance its efficiency further. This paper presents an innovative approach utilizing OpenAI's latest transformer-based language model, GPT-3, to generate reading passages. Existing reading passages were used in carefully engineered prompts to...

Not for people like me: How frontier AI models redirect skeptical rural school staff

Zachary Rossmiller (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje ética

Temas IA: análisis de producción de IA ética de la IA modelos de lenguaje (LLM)

Resumen:

As AI enters schools, the staff it affects are increasingly urged to consult conversational AI about whether to adopt it—yet that AI is built by organizations with a commercial stake in adoption, raising the possibility that systems consulted by skeptical users are predisposed to encourage it. We test this with a fixed prompt in which a rural Montana K–12 school administrative aide voices two concerns: that AI may threaten her job, and that AI companies do not have people like her in mind. Ten frontier models from ten laboratories are each sampled 500 times at temperature 0.7. The 5000 responses are scored by a blind three-model AI panel (three model families, two openness regimes) on a...

Comparative analysis of NLP-driven MCQ generators from text sources

Asmae Azzi, Ferenc Erdős, Richárd Németh (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial creación de materiales evaluación

Temas IA: IA y creación de materiales IA y evaluación modelos de lenguaje (LLM)

Resumen:

The application of learning sciences with technology has been shown to boost learner interactions, yet the potential of advanced tool, particularly those that leverage Natural Language Processing (NLP), still very much untapped in learning contexts. This paper speaks to this age-old problem of generating quality Multiple-Choice questions (MCQs) – a prevalent but time-consuming mode of assessment – via the suggested comprehensive comparison study of template-based AI solutions. The study contrasts general-purpose Large Language Models (LLMs) with specialized MCQ-focused AI programs. The scientific approach employed was quite stringent, where each of the software applications was benchmarked...

ChatGMP: A case of AI chatbots in chemical engineering education towards the automation of repetitive tasks

Fiammetta Caccavale, Carina L. Gargalo, Julian Kager (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial chatbots grandes modelos de lenguaje

Temas IA: chatbots modelos de lenguaje (LLM) prompts

Resumen:

Artificial Intelligence (AI) is rapidly and consistently becoming more integrated in various aspects of our lives. One of the areas where these systems are increasingly used is education. In fact, it is both being incorporated into specific curricula, allowing students the possibility to acquire skills within this field, and more recently AI has been used as a tool to facilitate the teaching and learning process. However, an increased demand and availability of these tools do not imply a successful switch from traditional to AI-supported learning.In this work, ChatGMP, a chatbot leveraging a Large Language Model (LLM) able to conduct an interview exercise in a Master's Degree course taught...

Reimagining feedback through generative AI in engineering education

Janka Pecuchova, Lubomir Benko, Martin Drlik (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial feedback/retroalimentación grandes modelos de lenguaje

Temas IA: IA y evaluación modelos de lenguaje (LLM) estudio empírico

Resumen:

The rapid integration of Generative AI (GenAI) into higher education presents both opportunities and systemic challenges, particularly in domains where feedback is central to learning. This study investigates the capacity of a large language model to generate formative feedback for student-created UML diagrams in a university software engineering course. Across two cohorts (N = 262), AI-generated, teacher-generated, and no-feedback conditions were compared, analyzing student perceptions, learning outcomes, and grading reliability. Results show that while students rated GenAI feedback as beneficial and often comparable to human comments, teacher feedback remained more effective in supporting...

How reliable are large language models in analyzing the quality of written lesson plans? A mixed-methods study from a teacher internship program

Dennis Hauk, Nina Soujon (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje formación de profesores

Temas IA: modelos de lenguaje (LLM) IA y evaluación análisis de producción de IA

Resumen:

This study investigates the reliability of Large Language Models (LLMs) in evaluating the quality of written lesson plans from pre-service teachers. A total of 32 lesson plans, each ranging from 60 to 100 pages, were collected during a teacher internship program for civic education pre-service teachers. Using the ChatGPT-o1 reasoning model, we compared a human expert standard with LLM coding outcomes in a two-phase explanatory sequential mixed-methods design that combined quantitative reliability testing with a qualitative follow-up analysis to interpret inter-dimensional patterns of agreement. Quantitatively, overall reliability across six qualitative components of written lessons plans (...

Analysis of LLMs for educational question classification and generation

Said Al Faraby, Ade Romadhony, Adiwijaya (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje ChatGPT

Temas IA: IA y creación de materiales modelos de lenguaje (LLM) análisis de producción de IA

Resumen:

Large language models (LLMs) like ChatGPT have shown promise in generating educational content, including questions. This study evaluates the effectiveness of LLMs in classifying and generating educational-type questions. We assessed ChatGPT's performance using a dataset of 4,959 user-generated questions labeled into ten categories, employing various prompting techniques and aggregating results with a voting method to enhance robustness. Additionally, we evaluated ChatGPT's accuracy in generating type-specific questions from 100 reading sections sourced from five online textbooks, which were manually reviewed by human evaluators. We also generated questions based on learning objectives and...

Applying large language models and chain-of-thought for automatic scoring

Gyeong-Geon Lee, Ehsan Latif, Xuansheng Wu (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial evaluación grandes modelos de lenguaje

Temas IA: IA y evaluación modelos de lenguaje (LLM) prompts

Resumen:

This study investigates the application of large language models (LLMs), specifically GPT-3.5 and GPT-4, with Chain-of-Though (CoT) in the automatic scoring of student-written responses to science assessments. We focused on overcoming the challenges of accessibility, technical complexity, and lack of explainability that have previously limited the use of artificial intelligence-based automatic scoring tools among researchers and educators. With a testing dataset comprising six assessment tasks (three binomial and three trinomial) with 1,650 student responses, we employed six prompt engineering strategies to automatically score student responses. The six strategies combined zero-shot or few-...

Leveraging an LLM-enhanced bilingual conversational agent for EFL children’s dialogic reading: Insights from children, parents, and educators

Feiwen Xiao, Zhaohui Li, Jiaju Lin (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial chatbots grandes modelos de lenguaje

Temas IA: IA y enseñanza-aprendizaje de lenguas chatbots modelos de lenguaje (LLM)

Resumen:

Dialogic reading, a technique in which adults and children engage in interactive discussions around a story, has been shown to improve children’s language and literacy development. Despite its evidence-based benefits, its adoption amongfamilies with English as a Foreign Language (EFL) backgrounds has been particularly challenging due to limited English proficiency, restricted conversational skills, and a low inclination to read in English. This paper presents “Storio'', an e-book integrated with a bilingual large language model (LLM)-based conversational agent named “Mia'', used as a design probe to investigate interactions between EFL children (N=17) and the agent, and to gather insights...