Rasch analysis 2: Application to first year Chemistry exams

Do you use Multiple Choice tests? How do you know if test is balanced in difficulty and coverage of material? Do male students perform better than their female counterparts? Is the exam valid for DE and Internal students? Can the same exam be provided to any course cohort? This presentation reports on a Rasch analysis (Rasch, 1960) performed on two final exams for first year, service taught, chemistry subjects. These subjects targeted students of vastly different educational backgrounds, but the exams were designed and proofed by the same team of academics.Using this approach, we were able to assess the performance of students as well as that of the exam questions (Bond and Fox, 2007). We identified misfitting of students and questions in one exam, and none in the other.We were able to ascertain that one of the exams would benefit from removing several questions and that the other would benefit from integrating one or several 'harder' questions. Similarly, we were able to review the performance of students in view of their gender, compare students from different courses, and in one subject,their mode and location of study. This information will be presented in a graphical and non- mathematical manner. Importantly, we found that for one exam, the performance of students correlated well with their grades, but in the other, a poor correlation was observed. We observed for example that students with a low understanding of chemistry were able to achieve a good grade on that occasion.We suggest that the use of Rasch analyses yield important information with regards to the performance of exams and students. In particular, we propose that this method can contribute towards banking validated exam questions for the purpose of randomly populated online exams. We suggest however, that these data be used in combination with other data analytics, such as CSU Interact reports, to enrich the nature of the subject coordinator's reflections.