CONTEXT Many methods have been proposed in the study of linguistics for the representation of words and sentences. Most classical methods are symbolic and consist in things like dictionaries, thesauri, ontologies and syntax trees. Another approach is to represent words and sentences via the use of high dimensional vectors, which capture the distributional statistics of words and sentences. One application of representing words as vectors is to automatically evaluate text, which can further be applied to the assessment of students’ text-based answers. PURPOSE This study investigated approaches to automatically analyse student responses to questions in the signal processing domain. APPROACH We investigated vector analysis approaches to capture various semantic and syntactic features of words, such that these representations can be compared and scored in a graded fashion, as distinct to simply true/false or same/different. The approaches used in this study can be trained in a semi- supervised fashion, where minimal human input is typically required. RESULTS The data investigated in this study consisted of student responses to short-answer questions in text form with associated metadata indicating the correctness for answers. Difficulties encountered when automatically assessing student short answers, either for correctness or knowledge gaps, were a) variations in vocabulary b) variations in grammatical structures c) precisely determining when specific concepts occur and don't occur, and d) relevant concept modifiers that may alter the assessment of the short answer. One element—important for addressing these difficulties— is how words and sentences are represented in short-answer question responses. CONCLUSIONS The study described in this paper focused on vector space representations for text. We recommend the development an agile methodology to be employed so that regular outputs be produced and sent for comment, which can then be used to inform further work. We suggest the best approach is to make use of a combination of methods including the many classical Natural Language Processing (NLP) techniques such as part of speech (POS) tagging, and phrase chunking.
|Publication status||Published - 2016|
|Event||27th Annual Conference of the Australasian Association for Engineering Education: AAEE 2016 - Novotel Pacific Bay Resort, Coffs Harbour, Australia|
Duration: 04 Dec 2016 → 07 Dec 2016
https://aaee.net.au/aaee-2016/ (Conference website)
https://web.archive.org/web/20160723044325/http://scu.edu.au/aaee2016/index.php (Host location conference website)
https://web.archive.org/web/20160628213408/http://scu.edu.au/aaee2016/index.php/11/ (Conference program)
|Conference||27th Annual Conference of the Australasian Association for Engineering Education|
|Abbreviated title||The Changing Role of the Engineering Educator for Developing the Future Engineer|
|Period||04/12/16 → 07/12/16|
|Other||Since its first offering in 1989, the AAEE conference series has grown into a well-respected and well-known forum aimed to promote scholarship in engineering education and to facilitate the dissemination of best practices throughout the engineering education community.|
Historically, AAEE has generally themed conference events and for 2016 the theme is 'The Changing Role of the Engineering Educator for developing the Future Engineer'. This theme touches upon multiple contemporary aspects that can be explored during the conference.