Assessing AI-generated (GPT-4) Versus Human Created MCQs In Mathematics Education: A Comparative Inquiry into Vector Topics

Abstract

Using multiple-choice questions as learning and assessment tools is standard at all levels of education. However, when discussing the positive and negative aspects of their use, the time and complexity involved in producing plausible distractor options emerge as a disadvantage that offsets the time savings in relation to feedback. The article attempts to understand whether, with the AI conquests on the educational landscape, we can now remove this aspect from the list of drawbacks. This paper aims to determine the suitability of GPT-4 for generating questions and answer options for multiple-choice questions using prompts in Estonian on topics related to vectors and their similarities and differences compared to questions and answers created by a human expert. The results show that GPT-4 can generate multiple-choice questions and answer options based on given learning objectives, theory, and sample problems. However, the suggested correct answer option often requires correction and is not yet linguistically at such a level that teachers can use the questions without editing. Verifying the generated tasks still becomes labour-intensive for teachers. Nonetheless, it is more crucial that the AI tool accurately determines the correct answer than that some of the generated distractors are not plausible.