Rasch analysis of three first year approaches to benchmark Chemistry

Rasch Analysis, a statistical and probabilistic analysis tool, is being used to examine chemistry exams across subjects and years; the analysis transforms the raw performance of students and exam questions separately - representing “ability” and “difficulty” respectively - into a common scale of measurement called “logit”. It therefore allows the academic to compare the performance of students versus that of the questions using a single understanding of “performance”. The Rasch approach differs strongly from other approaches - as opposed to other statistical approaches Rasch analysis is suitable to measure a single “construct” or constructs that are likely to be correlated. Unlike its counterparts, once validated, this approach is no longer sensitive to cohort size. As to date there have been few applications of Rasch analysis to the examination of university chemistry assessments. This talk will report on how this method has been used to validate exams, examine the coverage/difficulty across and between four different first year chemistry classes, and examine student's performance from a demographic perspective such as course and gender. These results and insights serve as the first step in bridging to work with exams from other Australian universities to benchmark exams, exam questions, subjects (and perhaps student abilities) between universities.