IA: Bibliografía: análisis de producción de IA
Nº de publicaciones: 54Detalles Cerrar β
Tipo: artículo
Metodología: estudio empírico
Temas: inteligencia artificial grandes modelos de lenguaje diseño curricular
Temas IA: modelos de lenguaje (LLM) análisis de producción de IA estudio empírico
Resumen:
Learning outcomes are clear and concise statements that describe what students should be able to do or know at the end of a particular course. These statements are crucial in instructional planning, curriculum development, and assessment of student progress and learning. Although there is no universal guidance on how to develop learning outcomes, Bloom's taxonomy is one widely used framework that helps instructors develop outcomes that reflect different levels of thinking, from basic remembering to creative problem-solving. This study investigates the potential of generative AI, specifically GPT-4, in classifying course learning outcomes according to their respective cognitive levels within...
Detalles Cerrar β
Tipo: artículo
Metodología: estudio empírico
Temas: inteligencia artificial evaluación ChatGPT
Temas IA: IA y evaluación IA y creación de materiales análisis de producción de IA
Resumen:
Not much is known about how LLM-generated questions compare to gold-standard, traditional formative assessments concerning their difficulty and discrimination parameters, which are valued properties in the psychometric measurement field. We follow a rigorous measurement methodology to compare a set of ChatGPT-generated questions, produced from one lesson summary in a textbook, to existing questions from a published Creative Commons textbook. To do this, we collected and analyzed responses from 207 test respondents who answered questions from both item pools and used a linking methodology to compare IRT properties between the two pools. We find that neither the difficulty nor discrimination...
Detalles Cerrar β
Tipo: artículo
Metodología: estudio empírico
Temas: inteligencia artificial integridad académica evaluación
Temas IA: IA y evaluación análisis de producción de IA estudio empírico
Resumen:
The growing use of generative AI (GenAI) tools like ChatGPT raises serious concerns about academic integrity, especially in the context of assessments. While detection efforts have focused on open-ended responses, multiple-choice questions (MCQs), which are common in high-stakes testing, remain largely overlooked, partly due to their perceived detection difficulty. The present work establishes a theoretical and empirical foundation for applications of psychometric theory to separate GenAI and human responses. Specifically, it investigates whether person-fit statistics (PFS), a class of methods within Item Response Theory (IRT) used to evaluate how well an examinee’s response pattern fits...
Detalles Cerrar β
Tipo: artículo
Metodología: estudio empírico
Temas: inteligencia artificial grandes modelos de lenguaje expresión escrita
Temas IA: análisis de producción de IA modelos de lenguaje (LLM) chatGPT
Resumen:
Generative Large Language Models (LLMs) demonstrate impressive results in different writing tasks and have already attracted much attention from researchers and practitioners. However, there is limited research to investigate the capability of generative LLMs for reflective writing. To this end, in the present study, we have extensively reviewed the existing literature and selected 9 representative prompting strategies for ChatGPT – the chatbot based on state-of-art generative LLMs to generate a diverse set of reflective responses, which are combined with student-written reflections. Next, those responses were evaluated by experienced teaching staff following a theory-aligned...
Detalles Cerrar β
Tipo: artículo
Metodología: estudio empírico
Temas: inteligencia artificial grandes modelos de lenguaje formación de profesores
Temas IA: modelos de lenguaje (LLM) IA y evaluación análisis de producción de IA
Resumen:
This study investigates the reliability of Large Language Models (LLMs) in evaluating the quality of written lesson plans from pre-service teachers. A total of 32 lesson plans, each ranging from 60 to 100 pages, were collected during a teacher internship program for civic education pre-service teachers. Using the ChatGPT-o1 reasoning model, we compared a human expert standard with LLM coding outcomes in a two-phase explanatory sequential mixed-methods design that combined quantitative reliability testing with a qualitative follow-up analysis to interpret inter-dimensional patterns of agreement. Quantitatively, overall reliability across six qualitative components of written lessons plans (...
Detalles Cerrar β
Tipo: artículo
Metodología: estudio empírico
Temas: inteligencia artificial escritura académica ChatGPT
Temas IA: IA y enseñanza-aprendizaje de lenguas chatGPT análisis de producción de IA
Resumen:
The rise of generative AI has been a major disruptive force in academia. Academics are concerned about its impact on student learning. Students can use generative AI technologies, such as ChatGPT, to complete many academic tasks on their behalf. This could lead to poor academic outcomes as students use ChatGPT to complete assessments, rather than engaging with the learning material. One particularly vulnerable academic activity is academic writing. This paper reports the results of an active learning intervention where ChatGPT was used by students to write an academic paper. The resultant papers were then analysed and critiqued by students to highlight the weaknesses of such AI-produced...
Detalles Cerrar β
Tipo: artículo
Metodología: estudio empírico
Temas: inteligencia artificial competencia intercultural creación de materiales
Temas IA: IA y enseñanza-aprendizaje de lenguas IA y creación de materiales chatGPT
Resumen:
Classroom teachers are usually responsible for creating materials to meet students' needs and course requirements. The arrival of generative AI, such as ChatGPT, offers EFL teachers an opportunity to regularly collaborate with AI chatbots to create new source materials. The writing experiments in this study considered how to effectively and ethically work with generative AI to produce culturally appropriate EFL teaching materials. The experiments involved co-producing moral dilemmatic stories with ChatGPT to support a new Chinese EFL curriculum unit on “Morals and Virtues”. We draw on Lo's (2023a, 2023b) CLEAR framework for prompt engineering to generate appropriate stimulus materials. We...
Detalles Cerrar β
Tipo: artículo
Metodología: estudio empírico
Temas: inteligencia artificial metodología de investigación grandes modelos de lenguaje
Temas IA: modelos de lenguaje (LLM) análisis de producción de IA estudio empírico
Resumen:
Automating the process of qualitatively coding text data from learners has been a long-standing ambition of learning analytics researchers since it represents an essential step toward delivering timely and scalable feedback. Automating this process is especially challenging in the case of ordered coding schemes —necessary for temporal analytical methods— where one text utterance can be assigned more than one qualitative code and the assignment order matters. This problem goes beyond multi-class and multi-label classification and, therefore, cannot be easily tackled using classic language models such as BERT. Recent advances in generative artificial intelligence, especially with the advent...
Detalles Cerrar β
Tipo: artículo
Metodología: estudio empírico
Temas: inteligencia artificial grandes modelos de lenguaje ChatGPT
Temas IA: análisis de producción de IA modelos de lenguaje (LLM) chatGPT
Resumen:
This exploratory study investigates the performance of three widely-used and freely available large language models (LLMs)— ChatGPT (GPT-3.5 Turbo), Gemini, and Copilot—in answering multiple-choice questions (MCQs) categorized by cognitive complexity based on the revised Bloom's Taxonomy. Although MCQs offer a structured, efficient, and scalable method for evaluation, a gap exists in the literature on how LLMs handle varying cognitive levels, particularly in a business course at higher education. Understanding LLM performance in this context is crucial, as students increasingly use LLMs for searching answers but also to receive tailored and scaffolded responses for an interactive and...
Detalles Cerrar β
Tipo: artículo
Metodología: estudio empírico
Temas: inteligencia artificial grandes modelos de lenguaje ChatGPT
Temas IA: IA y creación de materiales modelos de lenguaje (LLM) análisis de producción de IA
Resumen:
Large language models (LLMs) like ChatGPT have shown promise in generating educational content, including questions. This study evaluates the effectiveness of LLMs in classifying and generating educational-type questions. We assessed ChatGPT's performance using a dataset of 4,959 user-generated questions labeled into ten categories, employing various prompting techniques and aggregating results with a voting method to enhance robustness. Additionally, we evaluated ChatGPT's accuracy in generating type-specific questions from 100 reading sections sourced from five online textbooks, which were manually reviewed by human evaluators. We also generated questions based on learning objectives and...
Etiquetas
- análisis de producción de IA (54)
- catálogo de herramientas (15)
- chatbots (71)
- chatGPT (156)
- conceptos básicos (39)
- estudio empírico (491)
- ética de la IA (126)
- guía (29)
- IA y aprendizaje (165)
- IA y creación de materiales (66)
- IA y educación (367)
- IA y ELE (31)
- IA y enseñanza-aprendizaje de lenguas (212)
- IA y evaluación (143)
- IA y variedades del español (1)
- modelos de lenguaje (LLM) (87)
- prompts (65)
- revisión de bibliografía (136)