Harnessing AI to Optimize Thought Records and Facilitate Cognitive Restructuring in Smartphone CBT: An Exploratory Study

Cognitive-behavior therapy (CBT) is the most researched and established psychotherapy for depression and other emotional disorders (Cuijpers et al., 2021; Hofmann et al., 2012). The growing awareness of the impact of common mental disorders across the world is now calling for psychotherapies that can be delivered at scale (Herrman et al., 2022). Various delivery methods, believed to be more efficient than the traditional face-to-face individual format, have been tried for CBT. Of these, the self-help CBT appears most promising (Cuijpers et al., 2019), and with the rapid popularization of the internet we are witnessing a growing number of internet CBT (iCBT) applications (Karyotaki et al., 2021).

While CBT now encompasses a range of cognitive and behavioral intervention skills (Furukawa et al., 2021), cognitive restructuring (CR) remains the central technique deeply rooted in the cognitive model of emotional disorders (Beck et al., 1979). The theory posits that it is not the stressful situations themselves but rather their appraisal which leads to negative emotions. In CR, the patients are encouraged to monitor their experienced emotions and their thoughts amid the stressful situations, and then prompted to challenge their initial, automatic and often dysfunctional thoughts. The pre-requisite of CR then is the accuracy of the so-called thought record, in which patients record their emotions and thoughts: Unless the patients can accurately identify the thoughts that underlie their emotions, i.e. that directly lead to their emotions, challenging the thoughts would not easily lead to a reduction in their negative emotions. In the traditional face-to-face CBT sessions, psychotherapists achieve such refinement in identified automatic thoughts through Socratic questionings such as the downward-arrow technique.

In the self-help applications, however, the accuracy of the thought record is largely left to the acumen of the users and no work-up of automatic thoughts has been possible. This difficulty may partly explain the recent findings from the component network meta-analysis of iCBT in which CR was not found to be particularly helpful, in contrast with more directive approaches such as behavioral activation or problem solving (Furukawa et al., 2021). In order to augment the efficacy of CR in iCBT, we need to find ways to help users of iCBT in identifying the underlying automatic thoughts that correspond with their experienced emotions.

Recent advances in the natural language processing (NLP), mainly spurred by the emergence of Large Language Models (LLMs), may bring about a sea-level change in this regard. LLMs learn universal language representations from large volumes of text data using self-supervised learning and transfer this knowledge to more specific applications through fine-tuning (Subramanyam Kalyan et al., 2021; Wang et al., 2022). In this study we used thought records from two previous randomized controlled trials of iCBT and examined if the Text-to-Text Transfer Transformer (T5), one of the most advanced LLMs, can help identify thought records in which users may be having difficulties in accurately identifying the automatic thoughts that underlie their emotions. First we trained the T5 to predict the emotions associated with each automatic thought. Next we tested the validity of these NLP-based predictions by comparing them with the human experts’ judgements. We finally tested the validity of our assumption that better matched thought-emotion records enable more effective CR by comparing the results of CR of matched vs non-matched thought records.

If the T5 can identify thoughts that may not correspond with the experienced emotions, the smartphone CBT app can issue prompts to reconsider or refine the thoughts recorded, which would then enable more effective cognitive restructuring and ultimately lead to a greater reduction in depression through the therapy.

We applied the T5, the most advanced NLP using the transformer-based pre-trained language model, to 7182 thought records (thought-feeling pairs) provided by people with major or subthreshold depression. In the threefold internal validation, the T5 was able to correctly predict the self-reported feeling based on the recorded thought in 73.5% of the thought records. Thought-feeling pairs which the T5 correctly matched showed greater agreement with the gold standard rating as identified by human cognitive-behavior therapists than those pairs which the T5 showed to be discordant. Moreover, when submitted to cognitive restructuring, the former records of thought-feeling pairs led to greater reduction in negative emotions than the latter.

The artificial intelligence (AI) in general and the NLP in particular are finding wider and wider applications across the society, in medicine and in mental health (Wang et al., 2022). A recent review of the use of AI and NPL in mental health has identified five major categories of their usage: to extract clinical symptoms, to classify severity of illnesses, to compare different therapies, to provide psychopathological clues and to challenge the current nosography. For example, an NLP system may aim at detecting suicidal tendencies from the electronic health records, at predicting their severity, at differentiating more vs less adequate deliveries of therapy, or at identifying new psychopathologies based on the use of languages (Le Glaz et al., 2021). Our current study to apply the NLP to rate the quality of the automatic thought records may belong to the third category, namely to distinguish different levels of appropriateness of the therapeutic process.

One of the earlier attempts to apply the NLP to cognitive therapy was a system to classify dysfunctional thoughts into their categories such as “all-or-nothing,” “negative predictions,” “discounting the positive” etc. The researchers collected examples of dysfunctional thoughts from cognitive therapy textbooks and used their classification in the supervised learning. The system seems to have been partially successful but not yet to a clinically applicable degree (Wiemer-Hastings et al., 2004). As the ability of the NLP progresses, more complex systems have been proposed. Burger et al. used the NLP software to classify automatic thoughts provided by crowd-sourced volunteers in terms of their matching schemas as identified by human experts. Among the several models trialed, those based on the recurrent neural networks (RNNs) appeared to perform the best but this study did not provide any clinical applications (Burger et al., 2021). Kawakami et al. developed an NLP-based system to rate the quality of thought records. They trained the RNN to rate the five thought record components (event, thought, mood, behavior, physical symptoms) in accordance with the experts’ judgment of their appropriateness. They achieved an accuracy between 0.79 and 0.84 for the five components in the internal cross-validation. The authors are now using this system in their iCBT app to provide feedback to the users when their responses were rated as unlikely to be appropriate by the RNN (Kawakami et al., 2021).

Distinctive from these previous studies, our system was able to demonstrate the discriminatory power of the NLP-based system through external validation. Not only were we able to demonstrate satisfactory discrimination in the internal threefold cross-validation, the mismatching thought-feeling pairs as judged by the NLP system had poorer agreement with the external expert ratings and also led to smaller effectiveness when submitted to cognitive restructuring than those for which the NLP system could correctly predict the thought-feeling matching. These findings suggest that prompting the participants to update/revise the thought that underlie their feelings when the NLP found mismatch between them might be able to lead to more accurate identification of automatic thoughts, to enable more effective cognitive restructuring, and eventually to lead to greater reduction in depression after the treatment.

Some weaknesses of the current study may include the following. First, while showing evidence of internal validity and some external validity, the current study remains essentially exploratory and the ultimate proof of the value of the system is yet to be seen by trialing it in a clinical trial. Second, the NLP is showing very rapid evolution. Although the T5 is currently one of the most advanced NLP systems, newer and better models are being developed. The data to be fed into the NLP are also accumulating as we offer the current CR module to the users. New analyses using a different LLM and an even larger dataset of thought records may improve the performance of the system. However, the model developed in this study already appears promising and ready to be tested out with the real-world users.

We are currently developing a smartphone CR module which incorporates the AI system as demonstrated by the current study. It will provide some feedback to the app users to revisit their automatic thought when there was discordance between the self-reported feeling and the feeling as predicted by the AI system based on the automatic thought. We are currently planning to test out this system in the platform trial of various CBT modules (Furukawa et al., 2023) and to see if such advice can ultimately augment the effect of the iCBT.