Objective: Self-evaluation is an essential skill throughout a physician's career, and reflection is thought to be a necessary mechanism for effective self-evaluation. The aim of our study was to establish the reliability and validity of structured assessments of critical self-reflection.
Study design: Thirty-two residents completed 6 exercises that were scored from 0 (no description of event) to 6 (deep reflection). We calculated interrater and internal consistency reliability for the exercises and compared scores by postgraduate year and with other competency assessments.
Results: Residents completed 183 reflections. Interrater reliability was 0.89. Surgical skill reflections scored highest (score, 3.2 +/- 0.91 [SD]). Five exercises had adequate internal consistency reliability (0.62). Senior residents received higher reflection scores than junior residents; the magnitude of difference was similar for other competency measures and not statistically significant. Reflection scores were correlated with professionalism and communication skill assessments (score, 0.36-0.37; P < .01) but not with medical knowledge.
Conclusion: Self-reflection can be assessed reliably with scored exercises that demonstrate concurrent validity with other assessments. We encourage further research that should include multiple training programs to further evaluate our approach for the assessment of reflection in postgraduate education.