Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

The widespread usage of computer-based assessments and individualized learning platforms has increased demand for the rapid production of high-quality items. Automated item generation (AIG), the process of using item models to generate new items with the help of computer technology, was proposed to reduce reliance on human subject experts. While AIG has been used in test development, recent advances in machine learning algorithms offer the potential to enhance its efficiency further. This paper presents an innovative approach utilizing OpenAI's latest transformer-based language model, GPT-3, to generate reading passages. Existing reading passages were used in carefully engineered prompts to...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

International large-scale assessments (ILSAs) have used graphical response-based items to measure student ability for decades, but they have yet to implement automated scoring of these responses and instead rely on human scoring alone. To investigate how scores provided by machine algorithms compare to those provided by human raters, we applied convolutional neural networks (CNNs) to classify image-based responses from eight Timss 2019 items. Our results show that the most accurate CNN models classified over 99% of the image responses into the appropriate scoring category for dichotomous items and almost 98% for one trichotomous item. Additionally, during the modeling process, the CNNs...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

Even before the age of artificial intelligence, automated scoring received considerable attention in educational measurement. However, its application to constructed response (CR) items in international large-scale assessments (ILSAs) has remained a challenge, primarily due to the difficulty of handling multilingual responses spanning many languages. This study addresses this challenge by investigating two machine learning approaches — supervised and unsupervised learning — for scoring multilingual responses. We explored various scoring methods to assess three science CR items from TIMSS 2023 across all participating countries and 42 languages. The results showed that the supervised...

Ver ficha completa

Tipo: artículo

Metodología: estudio empírico

Temas: inteligencia artificial

Resumen:

Texto completo

Automated scoring (AS) has become increasingly prevalent in educational measurement. However, applying it to international reading assessments remains challenging, particularly due to the length and complexity of the required prompting, driven by the need to include lengthy reading passages and detailed scoring guides. Processing these lengthy inputs results in high computational costs and may impede the performance of large language models (LLMs). This study explored the potential of optimizing AS with prompt compression using OpenAI's LLM, GPT-4o. Our results show that prompt compression significantly reduces the length of reading passages and scoring guides while maintaining their...

Ver ficha completa