análisis de producción de IA

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje chatbots

Temas IA: análisis de producción de IA modelos de lenguaje (LLM) estudio empírico

Resumen:

Generative Artificial Intelligence (GenAI) chatbots continue to amaze users worldwide with their rapid improvements. These tools possess vast general knowledge and can thus be used in various fields, including education. However, before rolling out these models in pedagogical applications, it is fundamental to understand whether the information provided is reliable and if any of the currently available chatbots are best suited for domain-specific tasks. The objective of this study is to thoroughly investigate these aspects in a specific domain, fermentation, with the overarching goal of providing guidelines to students and teachers to select the best GenAI assistant. To achieve this goal,...

Assessing how accurately large language models encode and apply the common European framework of reference for languages

Luca Benedetto, Gabrielle Gaudeau, Andrew Caines (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial MCER grandes modelos de lenguaje

Temas IA: IA y enseñanza-aprendizaje de lenguas modelos de lenguaje (LLM) análisis de producción de IA

Resumen:

Large Language Models (LLMs) can have a transformative effect on a variety of domains, including education, and it is therefore pressing to understand whether these models have knowledge of – or, in other words, how they have encoded – the specific pedagogical requirements of different educational domains, and whether they use this when performing educational tasks. In this work, we propose an approach to evaluate the knowledge – or encoding – that the LLMs have of the Common European Framework of Reference for Languages (CEFR), and use it to evaluate five modern LLMs. Our study shows that the suite of tasks we propose is quite challenging for all the LLMs, and they often provide results...

Assessing the quality of automatic-generated short answers using GPT-4

Luiz Rodrigues, Filipe Dwan Pereira, Luciano Cabral (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje evaluación

Temas IA: análisis de producción de IA modelos de lenguaje (LLM) IA y evaluación

Resumen:

Open-ended assessments play a pivotal role in enabling instructors to evaluate student knowledge acquisition and provide constructive feedback. Integrating large language models (LLMs) such as GPT-4 in educational settings presents a transformative opportunity for assessment methodologies. However, existing literature on LLMs addressing open-ended questions lacks breadth, relying on limited data or overlooking question difficulty levels. This study evaluates GPT-4's proficiency in responding to open-ended questions spanning diverse topics and cognitive complexities in comparison to human responses. To facilitate this assessment, we generated a dataset of 738 open-ended questions across...

Not for people like me: How frontier AI models redirect skeptical rural school staff

Zachary Rossmiller (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje ética

Temas IA: análisis de producción de IA ética de la IA modelos de lenguaje (LLM)

Resumen:

As AI enters schools, the staff it affects are increasingly urged to consult conversational AI about whether to adopt it—yet that AI is built by organizations with a commercial stake in adoption, raising the possibility that systems consulted by skeptical users are predisposed to encourage it. We test this with a fixed prompt in which a rural Montana K–12 school administrative aide voices two concerns: that AI may threaten her job, and that AI companies do not have people like her in mind. Ten frontier models from ten laboratories are each sampled 500 times at temperature 0.7. The 5000 responses are scored by a blind three-model AI panel (three model families, two openness regimes) on a...

Evaluating the potential of ChatGPT-reformulated essays as written feedback in L2 writing

Yingzhao Chen (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial ChatGPT expresión escrita

Temas IA: IA y enseñanza-aprendizaje de lenguas chatGPT análisis de producción de IA

Resumen:

Reformulation is a form of written corrective feedback to help second language (L2) learners improve their writing. This study examined whether ChatGPT could produce reformulations that (1) retain the meanings of the original essays and (2) are linguistically more developed than learners’ original essays. In addition, three types of ChatGPT prompts were compared to see which type yielded better reformulations. One thousand two hundred argumentative essays written for the TOEFL iBT® independent writing task were submitted to ChatGPT. ROUGE-L scores, used as a proxy for meaning retention, showed that ChatGPT reformulations largely retained the meaning of the original essays. A qualitative...

Student and AI responses to physics problems examined through the lenses of sensemaking and mechanistic reasoning

Amogh Sirnoorkar, Dean Zollman, James T. Laverty (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial grandes modelos de lenguaje cognición y procesamiento

Temas IA: análisis de producción de IA estudio empírico

Resumen:

Several reports in education have called for transforming physics learning environments by promoting sensemaking of real-world scenarios in light of curricular ideas. Recent advancements in Generative-Artificial Intelligence have garnered increasing traction in educators' community by virtue of its potential to transform STEM learning. In this exploratory study, we adopt a mixed-methods approach in comparatively examining student- and AI-generated responses to two different formats of a physics problem through the theoretical lenses of sensemaking and mechanistic reasoning. The student data is derived from think-aloud interviews of introductory students and the AI data comes from ChatGPT's...

Performance of ChatGPT on the US fundamentals of engineering exam: Comprehensive assessment of proficiency and potential implications for professional environmental engineering practice

Vinay Pursnani, Yusuf Sermet, Musa Kurt (2023)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial ChatGPT grandes modelos de lenguaje

Temas IA: análisis de producción de IA chatGPT prompts

Resumen:

In recent years, advancements in artificial intelligence (AI) have led to the development of large language models like GPT-4, demonstrating potential applications in various fields, including education. This study investigates the feasibility and effectiveness of using ChatGPT, a GPT-4 based model, in achieving satisfactory performance on the Fundamentals of Engineering (FE) Environmental Exam. This study further shows a significant improvement in the model's accuracy when answering FE exam questions through noninvasive prompt modifications, substantiating the utility of prompt modification as a viable approach to enhance AI performance in educational contexts. Furthermore, the findings...

Bias and representation in AI generated text-to-image in education: A systematic review

Lilach Alon, Dorit Hadar Shoval, Inbar Levkovich (2026)

Detalles Cerrar ✕

Tipo: artículo

Metodología: revisión bibliográfica

Temas: inteligencia artificial ética inclusión y equidad

Temas IA: ética de la IA análisis de producción de IA revisión de bibliografía

Resumen:

AI generated text-to-image tools are rapidly entering educational settings, yet little is known about how their visual biases intersect with goals of equity, inclusion, and critical AI literacy. This systematic literature review maps and analyzes empirical studies that examine bias and representation in educational uses of AI generated text-to-image. Following PRISMA guidelines, we identified 31 peer reviewed studies published between 2023 and 2025 across K-12, higher education, and professional learning contexts. We used a six part analytic framework (gender; race, ethnicity, and socioeconomic status; culture and religion; age; body and (dis)ability; and content) to code how studies...

What percentage of secondary school students do their homework with the help of artificial intelligence? - A survey of attitudes towards artificial intelligence

Mátyás Turós, Róbert Nagy, Zoltán Szűts (2025)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial educación primaria y secundaria integridad académica

Temas IA: análisis de producción de IA IA y educación estudio empírico

Resumen:

Based on practical experiences, news articles, and subjective reports, a significant proportion of high school students complete their writing assignments (essays) using ChatGPT or other text-generating software. However, to the best of our knowledge, there is a very limited amount of work on this topic. Therefore, our study has two objectives. First, through human coding, we determine whether student essays submitted as homework were created by humans or software, and second, we examine these essays' sentiment and authorship, as well as factors determining the use of artificial intelligence in learning. Data collection was conducted in April and May 2024 across thirty classes from twenty...

Teachers' and students' perceptions of AI-generated concept explanations: Implications for integrating generative AI in computer science education

Soohwan Lee, Ki-Sang Song (2024)

Detalles Cerrar ✕

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial ChatGPT pensamiento computacional

Temas IA: análisis de producción de IA IA y creación de materiales chatGPT

Resumen:

The educational application of Generative AI (GAI) has garnered significant interest, sparking discussions about the pedagogical value of GAI-generated content. This study investigates the perceived effectiveness of concept explanations produced by GAI compared to those created by human teachers, focusing on programming concepts of sequence, selection, and iteration. The research also explores teachers' and students' ability to discern the source of these explanations. Participants included 11 teachers and 70 sixth-grade students who were presented with concept explanations created or generated by teachers and ChatGPT. They were asked to evaluate the helpfulness of the explanations and...