A cross-linguistic investigation of reliability of measurement of stuttering severity

Stuttering measurement is a vital component of assessment and treatment. It is essential for speech-language pathologists to reliably measure stuttering, to enable efficient tracking of clients’ progress and outcomes. Stuttering severity rating scales are widely used in research and clinical practice, and are recommended for measuring severity of stuttering in different languages. With the popularity of severity rating scales worldwide, investigation into the reliability of use of the scale is critical, regardless of whether it is being used for clinical or research purposes. Thus, the present research investigates speech-language pathologists’ reliability of measuring severity of stuttering of adult-speakers who stutter, across languages, using a 9-point severity rating scale.
Previous research has examined speech-language pathologists’ abilities to reliably measure severity of stuttering using a severity rating scale and has shown conflicting results. However, these studies used different analyses and varying durations of speech samples, which could have contributed to the observed differences. Study 1 presents the first study, which examined whether the duration of speech samples influenced the reliability of measurement of severity of stuttering. Results from the study determined the duration of speech samples used for the remainder of studies in the program of research. Specialist speech-language pathologists rated audio, English samples of three different durations (1, 3, and 5-minutes) of adults who stutter, using a 9-point severity rating scale. Judges showed greater variability (poorer reliability) when rating 1-minute speech samples. The 3- and 5-minute samples produced similar results when using the scale to measure stuttering severity.
Studies 2 and 3 used 3-minute speech samples to investigate the reliability of use of the scale across languages. With a rise in globalisation, speech-language pathologists are increasingly tasked with assessing and treating stuttering in languages that may differ from their own. Study 2 expanded existing knowledge of measuring severity of stuttering in a foreign language that is commonly spoken worldwide: Vietnamese. The aim of Study 2 was to determine whether English-speaking speech-language pathologists could reliably measure severity of stuttering in English (familiar language) and Vietnamese (unfamiliar language); while Study 3 provides the first stuttering measurement data for Vietnamese-speaking speech-language pathologists in Vietnam and investigated whether Vietnamese-speaking speech-language pathologists can reliably measure severity of stuttering in Vietnamese (familiar language) and English (other language, a common second language for Vietnamese-speakers). Results of Studies 2 and 3 exhibited consistent findings: (i) judges performed better measuring severity of stuttering in a familiar language than an unfamiliar/other language; (ii) judges produced greater intrajudge than interjudge agreement, for both languages; and (iii) judges showed better use of the scale for mild and severe stuttering across both languages, with the moderate range of the scale showing greater variability.
The main purpose of Study 4 was to present a re-analysis of data from Hoffman, Wilson, Copley, Hewat, and Lim (2014) to determine whether the consistent findings across the present studies were robust. The re-analysis investigated Australian-English speech-language pathologists’ reliability of use of the scale in Australian English (familiar language) and Mandarin (unfamiliar language). The re-analysis replicated the analyses of Studies 2 and 3. Results of the re-analyses corroborated those of Studies 2 and 3, resulting in three robust findings. The findings of the studies combined are synthesised, with a discussion of the contributions of the research and suggestions for future directions.