What were you Thinking? A Comparison of Rater Coding and word Counts for Content Analysis of Thought Samples in Depression

This study examined the convergence between two methods of thought content analysis, manual coding by trained raters and computer-generated word counts, in a sample of clinically depressed participants assessed before and after treatment. Automated word count programs have traditionally used longer narrative texts so their utility for shorter thought samples is uncertain. Aims were to evaluate their direct correspondence and to determine whether the two methods yield similar results in assessing change from pre- to post-treatment. Thirty participants recorded in-the-moment thoughts during random phone-based signaling. Thought samples were analyzed for presence of negative emotion (NE), positive emotion (PE), and self-focus (SF), using hand coded ratings and automated word counts. Correlations between ratings and word counts for each of the three content categories were significant for all but post-treatment NE. Thought samples rated as showing the presence of NE, PE, or SF showed significantly higher NE, PE, and SF (respectively) word counts than those without. Comparisons of pre/post data showed significant decreases in NE and no differences in PE across both methods; increases in SF emerged only for ratings. While limited by a small sample size, these findings suggest that word count analyses may be a reasonable replacement for more laborious hand coding in thought sampling data, but there may be important differences across content categories. These results contribute to knowledge about the methodology of thought sampling analysis in clinical samples.