The reliability and validity of the perceive, recall, plan and perform assessment in children with a mitochondrial disorder

Purpose
To investigate the reliability and validity of the Perceive, Recall, Plan and Perform System of Task Analysis (PRPP-Assessment) by parent-provided videos of children with mitochondrial disorders.
Methods
Videos of 13 children performing 3–7 activities each were the subject of study, resulting in 65 activities. These were scored using the PRPP-Assessment by 11 raters. Internal consistency was calculated with Cronbach’s alpha. Intra-rater reliability was evaluated by Bland-Altman Plots on 44 PRPP-Assessment scores. Inter-rater reliability was calculated with ICCAgreement on 128 PRPP-Assessment scores. Construct validity was assessed by comparing the PRPP-Assessment scores to the Canadian Occupational Performance Measure using Cohen’s Kappa. PRPP-Assessments scores were evaluated with a multi-faceted Rasch Analysis.
Results
Internal consistency was high (0.92). Intra-rater reliability was sufficient to be good (92–96% within the 95%-Limits of the Agreement). The ICCAgreement for stage 1 Mastery Score showed acceptable inter-rater reliability (0.646). Stage 2 of the PRPP-Assessment showed low ICCs due to a lack of variability within the sample. Four out of six hypotheses on validity were accepted. Rasch's analysis demonstrated sound goodness-of-fit, and supported the validity of the PRPP-Assessment.
Conclusion
The PRPP-Assessment by parent-provided videos in this heterogenic group showed sufficient to good psychometric properties. In practice, careful task selection and formulating criterion is recommended.