Investigating the use of vector analysis to assess students’ understanding

Andrea Goncher, Wageeh W. Boles

Research output: Other contribution to conferenceOther

Abstract

CONTEXT Many methods have been proposed in the study of linguistics for the representation of words and sentences. Most classical methods are symbolic and consist in things like dictionaries, thesauri, ontologies and syntax trees. Another approach is to represent words and sentences via the use of high dimensional vectors, which capture the distributional statistics of words and sentences. One application of representing words as vectors is to automatically evaluate text, which can further be applied to the assessment of students’ text-based answers. PURPOSE This study investigated approaches to automatically analyse student responses to questions in the signal processing domain. APPROACH We investigated vector analysis approaches to capture various semantic and syntactic features of words, such that these representations can be compared and scored in a graded fashion, as distinct to simply true/false or same/different. The approaches used in this study can be trained in a semi- supervised fashion, where minimal human input is typically required. RESULTS The data investigated in this study consisted of student responses to short-answer questions in text form with associated metadata indicating the correctness for answers. Difficulties encountered when automatically assessing student short answers, either for correctness or knowledge gaps, were a) variations in vocabulary b) variations in grammatical structures c) precisely determining when specific concepts occur and don't occur, and d) relevant concept modifiers that may alter the assessment of the short answer. One element—important for addressing these difficulties— is how words and sentences are represented in short-answer question responses. CONCLUSIONS The study described in this paper focused on vector space representations for text. We recommend the development an agile methodology to be employed so that regular outputs be produced and sent for comment, which can then be used to inform further work. We suggest the best approach is to make use of a combination of methods including the many classical Natural Language Processing (NLP) techniques such as part of speech (POS) tagging, and phrase chunking.
Original languageEnglish
Publication statusPublished - 2016
Event27th Annual Conference of the Australasian Association for Engineering Education: AAEE 2016 - Novotel Pacific Bay Resort, Coffs Harbour, Australia
Duration: 04 Dec 201607 Dec 2016
https://search.informit.com.au/browsePublication;isbn=9780994152039;res=IELENG (conference publications)

Conference

Conference27th Annual Conference of the Australasian Association for Engineering Education
Abbreviated titleThe Changing Role of the Engineering Educator for Developing the Future Engineer
CountryAustralia
CityCoffs Harbour
Period04/12/1607/12/16
Internet address

Fingerprint Dive into the research topics of 'Investigating the use of vector analysis to assess students’ understanding'. Together they form a unique fingerprint.

  • Cite this

    Goncher, A., & Boles, W. W. (2016). Investigating the use of vector analysis to assess students’ understanding. 27th Annual Conference of the Australasian Association for Engineering Education, Coffs Harbour, Australia. https://eprints.qut.edu.au/105497/