Capturing cognition using the Critical Decision Method

A number of methods have been developed to decompose the processes underlying skilled performance by capturing a snapshot of cognition (i.e., what people are attending to and thinking about during complex tasks). Cognitive Task Analysis (CTA) is an approach that investigates the processes required to complete cognitively complex tasks, which are typically characterised by high consequence, high information content, and occur within the constraints of real-world operation. The most common form of CTA involves the use of retrospective interview procedures, which yield interviewee reports of relatively ‘deep’ levels of processing, while mitigating the risks associated with disruption to real-world operations that might be associated with concurrent-style techniques (e.g., think aloud). The Critical Decision Method (CDM; Klein, Calderwood, & MacGregor, 1989) involves the interviewer using specially designed probes to extract information pertaining to the interviewee’s cognitive processes during a critical incident, and, more specifically, the goals, strategies, and cues used. The CDM attempts to disambiguate processes that are usually associated with the operator’s limited opportunity for conscious deliberation and is, therefore, commonly used to investigate aspects of expertise, which may inform training and system design. This workshop seeks to examine CTA methods, providing users with explicit instruction on the use of the three critical stages of the CDM – data capture, analysis, and representation. The 2-hour workshop will utilise a combination of slideshow presentation, instructor-led demonstrations, and interactive practical exercises that simulate the application of CDM procedures. By the end of the workshop attendees will be able to: perform a semi-structured interview using CDM probing techniques; identify the cognitive elements that occur in an exemplar incident; recall a number of methods for representing analysis outcomes; appreciate the use of mixed methods approaches in CTA; and describe ways in which CDM outcomes may inform training and system design. Participants will be provided with a copy of the slideshow presentation, which will include an overview of common CTA techniques, CDM procedure and analysis instructions, and examples of common analysis representation practices. Participants will also take home their completed exercise worksheets, which will include exemplar interview schedules and transcripts, and their completed attempts at the three stages of CDM. The instructor is a registered psychologist and human factors researcher who has conducted many forms of CTA across a range of domains (e.g., criminal investigation, aviation, sport), and has extensive experience in the validation and application of CTA outcomes.