Year

2023

Season

Fall

Paper Type

Master's Thesis

College

College of Computing, Engineering & Construction

Degree Name

Master of Science in Computer and Information Sciences (MS)

Department

Computing

NACO controlled Corporate Body

University of North Florida. School of Computing

First Advisor

Dr. Indika Kahanda

Second Advisor

Dr. Xudong Liu

Third Advisor

Dr. Zornitza Prodanoff

Department Chair

Dr. Asai Asaithambi

College Dean

Dr. William Klostermeyer

Abstract

As education technology continues to evolve, the domains of Automatic Short-Answer Grading (ASAG) and Automated Misconception Detection (AMD) stand at the forefront of innovative approaches to educational assessment. We explore the transformative potential of Large Language Models (LLMs) in revolutionizing these critical areas. Leveraging the remarkable capabilities of LLMs in semantic inference, contextual understanding, and transfer learning, we embark on a comprehensive journey to enhance both ASAG and AMD. On ASAG, we illuminate the efficacy of transfer learning by fine-tuning RoBERTa Large, a state-of-the-art LLM, on task-related corpora, e.g. the Multi-Genre Natural Language Inference (MNLI) corpus. The model's adaptability across unseen questions and domains on the minority class, coupled with its narrowed performance gap from unseen answers, highlights the profound impact of transfer learning on grading diverse student responses. In the emerging realm of AMD, we pioneer a dataset and methodology that inaugurates a new era in misconception detection. Framing the task as Recognizing Textual Entailment (RTE), our approach with RoBERTa Large MNLI captures nuanced misconceptions, unveiling the untapped potential of LLMs in unraveling the intricate landscape of automated misconception detection. The synergy between these endeavors presents a holistic view of the transformative role anticipated for LLMs in automated educational assessment. Our research, spanning adaptability in short-answer grading and groundbreaking advancements in misconception detection, establishes a foundation for a future where LLMs excel not only in understanding nuanced student responses but also in pinpointing and rectifying misconceptions with unparalleled precision. These insights contribute significantly to the dynamic field of educational technology, heralding a new era wherein the full potential of LLMs is utilized to shape the trajectory of educational assessment.

Share

COinS