No Medication for My Child! A Naturalistic Study on the Treatment Preferences for and Effects of Cogmed Working Memory Training Versus Psychostimulant Medication in Clinically Referred Youth with ADHD

The intervention part of the study was conducted between November 2012 and June 2015 at Lucertis (previously Virenze-RIAGG) Maastricht, an outpatient center for children and adolescents with mental health problems. Youth who had been referred to the center were subjected to a standard intake procedure, which consisted of unstructured clinical interviews with the youngster and his/her parents, a semi-structured diagnostic (DSM-IV-based) interview, an observation in the play room (for children in the preschool and primary school age), a contact with the teacher at school, administration of a developmental anamnesis survey and a set of standardized questionnaires (i.e., the Achenbach Scales of Empirically-Based Assessment [32]), and eventually a (neuro)psychological test battery consisting of for example an intelligence test (in most cases the Wechsler Intelligence Scale for Children [33]) or a performance-based attention test (e.g., Test of Everyday Attention for Children (TEA-Ch) [34]). After the intake, a multidisciplinary team consisting of psychologists and other mental health workers, and at least one psychiatrist used all the gathered information to make a diagnosis and a DSM-IV-based classification for the child as well as to formulate a treatment plan.

Out of the 97 eligible participants, it appeared that five were already or had been taking psychostimulant medication. Because these participants were no longer naïve to one of the interventions, they were discarded from the study. Two further participants refused to participate because they did not want to participate in the extra assessment sessions. Thus, eventually 90 youngsters and their parents agreed to participate by signing the informed consent form, on which they also indicated the selected intervention, the reason(s) for this choice, and the level of confidence in the intervention as being helpful for their child (1 = not helpful at all, 10 = extremely helpful). The characteristics of the 90 participating youngsters are displayed in Table 1. The total sample consisted of 58 boys and 32 girls, who had a mean age of 129.59 months (10.80 years). The majority of participants were children aged 6–12 years (i.e., 71.1%) of whom most were in a regular primary school; others were adolescents aged 13 to 16 years (28.9%) who typically received education in secondary schools with varying levels. The mean IQ score of the participants in this sample was 103.13 (range: 80–144). With regard to the families: these frequently had a low to medium socio-economic background (84.0%) and in about one-third of the cases parents were divorced. Most youth had a father and a mother with an original Dutch origin; others had at least one parent coming from another country (e.g., Germany, Indonesia, Turkey, Morocco). All youth had a diagnosis of ADHD combined type, ADHD predominantly inattentive type, or ADHD-NOS and note that many of them also suffered from comorbid psychiatric conditions (e.g., parent–child relational problem, pervasive developmental disorder, relational problem due to a mental disorder), which was not an exclusion criterion.

As can be seen in the flowchart presented in Fig. 1, for 35 youngsters and their parents (38.9%) Cogmed WMT was the treatment of choice, 30 (33.3%) preferred the stimulant medication, and 25 (27.8%) chose the combination of WMT and medication as intervention. The three groups differed with regard to a number of demographic characteristics. Participants in the Cogmed WMT group were significantly younger than those in the stimulant medication and combined treatment groups [mean ages being 116.77 months, SD = 20.39 vs. 138.20, SD = 35.32 and 137.20, SD = 29.93 months; F(2,87) = 5.71, p < .01]. Further, level of interference associated with ADHD (as rated by parents on the ADIS) was higher in the psychostimulant medication and combined treatment groups as compared to the Cogmed WMT group [means being 6.35, SD = 0.98 and 5.98, SD = 1.02 vs. 5.21, SD = 1.12; F(2,87) = 10.02, p < .001]. Finally, youngsters in the stimulant medication group and the combined treatment group more frequently had a low socio-economic status than youth in the Cogmed WMT group, with percentages being 65.4 and 52.2 versus 25.0%; χ²(1) = 9.98, p < .01.

Three youngsters prematurely dropped out of the study (two of them took the pre-treatment assessment but then never started with the intervention and also did not show up for the post-treatment assessment; one child initially gave permission to participate but then refused to complete the pre-treatment assessment), leaving 87 participants for the intent-to-treat analysis (33 for Cogmed WMT, 29 for stimulant medication, and 25 for the combination intervention). Nine youngsters dropped out during the intervention part of the study: five (15.1%) in the WMT condition, three (10.3%) in the medication condition, and one (4.0%) in the combined intervention condition. Within the latter condition, four participants renounced the Cogmed WMT after having started with the medication treatment and naturally “switched” to the medication condition. Thus, ultimately there were 78 participants for the completer analysis (28 for Cogmed WMT, 30 for stimulant medication, and 20 for the combination intervention). In passing, it is relevant to note that participants in the WMT conditions (i.e., Cogmed WMT and combination intervention) were only considered as completers in case they fully adhered to the Cogmed protocol and conducted all 25 sessions.

Stimulant medication was provided by four psychiatrists who were experienced in treating ADHD and other psychiatric conditions in children and adolescents. In the current study, methylphenidate—Ritalin was most often prescribed, followed by extended release variants such as Medikinet or Concerta. In most cases, psychiatrists initially started with the prescription of Ritalin, but sometimes for practical reasons (e.g., difficulties with the regular administration of the pills) or due to side effects changed to other types of medication.

The frequency/duration of the contacts with the Cogmed coaches and the psychiatrists were similar for youth in the Cogmed WMT and stimulant medication groups. There was a starting session during which (a) the psychiatrist conducted a brief medical examination and provided information about the treatment with psychostimulant medication and its procedure, while (b) the Cogmed coaches explained the rational of the WMT training and provided instructions about the use of the program and the role of the parents (or the other person who was going to assist the child in conducting the training). One week later, there was a brief follow-up session (usually by telephone) to check whether youth had started with the intervention and to address eventual questions and problems. If indicated, there were additional telephone and email contacts and occasionally even face-to-face sessions during the further course of the intervention. At the end of the training period, there was a closing session in the Cogmed WMT group during which an evaluation of the intervention took place and youth received a certificate for successfully completing the program. Around this time, children and adolescents in the stimulant medication group visited the psychiatrist for a follow-up consult. Youth in the combined treatment group had sessions and follow-up contacts with both the psychiatrist and the Cogmed coaches, and thus received twice as much care as youth in the mono-intervention groups.

The primary outcome measure was the ADHD-Questionnaire (ADHD-Q [35]), which comprises 18 items covering DSM-defined symptoms of ADHD in three domains: inattention (6 items; e.g., “has difficulties with sustaining attention in tasks or play activities”), hyperactivity (6 items; e.g., “fidgets with hands and feet or squirms in seat”), and impulsivity (6 items; e.g., “blurts out the answer before the question has been completed”). Parents, teachers, or children/adolescents themselves indicate how frequently the pertinent behavior has been present during the past 2 weeks, using a 5-point rating scale with 0 = not, 1 = now and then, 2 = regularly, 3 = often, and 4 = very often. Previous research by Scholte and Van der Ploeg [35] has shown that this rating scale is reliable in terms of internal consistency (with Cronbach’s alpha generally being well above 0.80) and temporal stability (with a 2-weeks test–retest correlation of 0.95). Further, confirmatory factor analysis has demonstrated that symptoms of inattention, hyperactivity, and impulsivity were satisfactorily represented in the scale, providing support for the construct validity of the ADHD-Q [36, 37]. Finally, scores on this questionnaire are positively associated with educational and behavioral problems, and differentiate between youth with and without an ADHD diagnosis [35, 36].

The Strengths and Difficulties Questionnaire (SDQ [38, 39]) consists of 25 items describing negative and positive attributes of children and adolescents that can be allocated to 5 subscales of 5 items each: apart from the hyperactivity-inattention problems (e.g., “restless, overactive, cannot stay still for long”, “easily distracted, concentration wanders”), there are also subscales measuring emotional symptoms (e.g., “many fears, easily scared”), conduct problems (e.g., “often loses temper”), peer problems (e.g., “rather solitary, prefers to play alone”), and prosocial behaviour (e.g., “helpful if someone is hurt, upset or feeling ill”). Parents, teachers, and youth themselves are asked to score each item on a 3-point scale with 0 = ‘not true’, 1 = ‘somewhat true’, and 2 = ‘certainly true’. Subscale scores can be computed by summing scores on relevant items (after recoding reversed items). In the present study, we employed the hyperactivity-inattention problems subscale (range 0–10) and a combined ‘other problems’ score that was based on the emotional symptoms, conduct problems, and peer problems subscales (range 0–30). The impact supplement [40] was used to enquire further about chronicity, distress, social impairment, and burden to others associated with youth’s ADHD problems. At post-treatment, the SDQ included two additional follow-up questions that specifically asked about the effects produced by the intervention: (1) did the intervention reduce the problems? and (2) did the intervention help in other ways, e.g., make the problems more bearable? Research has shown that the SDQ has adequate psychometric properties. More specifically, the internal consistency and test–retest stability of the SDQ are satisfactory [39]. Furthermore, correlations among parent, teacher, and self-report SDQ scores are moderate but compare favourably to cross-informant correlations as obtained with other psychopathology measures [39]. Evidence has also been obtained for the validity of the SDQ. That is, SDQ scores were found to correlate in the expected way with other measures of psychopathology [41] and discriminate well between youth with and without psychopathological symptoms [40]. Finally, the SDQ has proven to be an effective index for tapping ADHD symptomatology and comorbidities in clinically referred children and adolescents [42‐44].

The Dutch version of the Behavior Rating Inventory of Executive Functioning (BRIEF [45]) is a 75-item parent-report questionnaire that intends to assess executive functioning in youth aged 5 to 18 years. Parents have to rate whether their child’s behavior is ‘never’, ‘sometimes’, or ‘often’ a problem. The BRIEF contains eight subscales reflecting impairments in various domains: inhibition (i.e., to control impulses and to stop engaging in a behavior), shifting (i.e., to move easily from one activity to another, to tolerate change, to switch attention), emotional control (i.e., to regulate emotional responses adequately), initiative (i.e., to start an activity and to independently generate ideas or problem solving strategies), working memory (i.e., to hold information when completing a task or performing a mental operation), planning and organization (i.e., to anticipate future events, to set goals and develop steps, to grasp the main idea), order (i.e., to put and keep order in play and living spaces), and behavior monitoring (i.e., to evaluate one’s own performance and behaviors towards other people). In the present study, we employed the BRIEF total score, a behavioral regulation index (which is composed of the subscales inhibition, shifting, and emotional control) and a meta-cognition index (which consists of the subscales initiative, working memory, planning and organization, order, and behavioral monitoring) [46]. We also considered the working memory subscale separately given its relevance as an outcome variable for the Cogmed WMT. The psychometric qualities of the BRIEF are good [45], and this appears also true in samples of children and adolescents with ADHD [47‐49].

Verbal and visuospatial working memory were measured with respectively the Digit Span test of the Wechsler Intelligence Scale for Children (WISC [33]) and the Corsi Block-tapping task [50]. During the Digit Span test, participants are verbally presented with a sequence of numerical digits, which they are asked to recall either in the normal (forward) or the reversed (backward) order. Increasingly longer sequences are tested until participants are no longer capable of correctly reproducing the sequence of the digits. In the present study, a total Digit Span score was obtained by summing the number of correctly reproduced sequences on the forward and backward trials (range 0–30). The Corsi Block-tapping task is the visuospatial equivalent of the Digit Span test. It requires participants to reproduce a sequence of movements by tapping blocks in the same (forward) or reversed (backward) order as the examiner did on a board with nine blocks placed at fixed, pseudorandom positions. The Corsi Block-tapping task also yielded a total score that ranged between 0 and 30. There is sufficient empirical support showing that performance on verbal and visuospatial span tasks indeed reflect working memory capacity [51], which appears to be significantly impaired in youth with ADHD.

The Stroop interference task [52] is a widely employed neuropsychological test that is thought to measure selective attention and response inhibition [53]. The test consists of three cards containing 100 stimuli each, which have to be processed as fast as possible. The first card requires the participants to read the names of colors that are printed in black ink. The second card asks the participants to name the color of squares that are printed in various colors. The third and final card consists of words describing colors that are printed with ink in a color that is incongruent with the word (e.g., the word ‘green’ printed in red). For each card, the response time is measured, but the interference effect is typically defined as the response time to card 3 minus the response time to card 2. A larger value is indicative for lower capacity of and thus greater problems with selective attention and response inhibition. A meta-analysis by Homack and Riccio [54] has demonstrated that children and adolescents with ADHD consistently exhibit poorer performance on the Stroop task as compared to individuals without a clinical diagnosis, although it should also be noted that youth with other psychiatric problems tend to display similar problems.

The Bourdon-Vos test [55] is a paper-and-pencil measure of sustained attention that in The Netherlands is frequently used in the neuropsychological assessment battery of children and adolescents suspect of ADHD. It is a cancellation test consisting of a single sheet on which 33 rows with 24 small patterns of three, four, or five dots. The four dots patterns are the targets that have to be cancelled in a normal reading order, as fast and as accurately as possible. A number of variables can be derived from this test. Because the experimenter measures the time needed to complete each of the 33 rows, a mean row completion time as well as a standard deviation can be obtained that are indicative for respectively the speed and the regularity of the participant’s performance. The smaller the mean row time and its accompanying standard deviation, the better a person’s ability to regulate and maintain attentional focus. In addition, omissions (i.e., missed targets), corrections (i.e., immediately noticed cancellations of non-targets), and errors (i.e., cancelled non-targets) are recorded, which together form an index of accuracy. A comprehensive psychometric evaluation is lacking, but based on the data published in the original manual [55], the Dutch evaluation committee of psychological tests (http://​www.​cotandocumentati​e.​nl) has concluded that the Bourdon-Vos test has reasonable reliability and validity.

The TEA-Ch [34] is a neuropsychological test battery for measuring youth’s attentional capacity. In this study, the following four subtests were employed: (1) the sky search subtest provides an index of selective attention (i.e., the ability to search for relevant information while ignoring irrelevant, distracting information) and consists of an A3-sheet depicting the sky above a city which is full with spaceships that fly in pairs. For 108 of the pairs the two spaceships are different, but for the 20 target pairs both spaceships are identical. Participants are instructed to search for the identical target pairs and to mark them as quickly as possible. The time needed to complete the test and the number of correctly identified target items are recorded, and eventually yield a time-per-target score. This score is corrected for motor speed by administrating a motor control version of the task during which children have to mark pairs of spaceships on a separate A3-sheet that only displays the target items. (2) the Score! Subtest measures sustained attention (i.e., the ability to preserve attention and alertness while doing a task that is not implicitly stimulating), and consists of ten trials during which children have to silently count (i.e., without the assistance of fingers) the number of tones that are produced by the computer. Each trial presents between 9 and 15 identical tones, which are separated by silent inter-stimulus intervals of variable duration (i.e., 500–5000 ms). At the end of each trial, children report the number of sounds that they have counted. This subtest yields a total score, which indicates the number of correctly counted trials (range 0–10). (3) the sky search dual task (DT) assesses divided attention (i.e., the ability to regulate, coordinate, and plan complex attention processes) and asks participants to complete a second version of the sky search subtest while simultaneously counting the number of tones produced by the computer (as was done during the Score! Subtest). The time needed to complete the test, the number of correctly identified targets and the number of correct counts are used to compute a weighted time-per-target score. (4) The Score! DT is also measuring divided attention and again consists of ten trials; this time children not only have to count the number of tones, but they also have to listen to a news bulletin (also presented by the computer) in which the name of an animal is mentioned. At the end of each trial, children have to report the number of tones they have counted as well as the animal that was mentioned in the bulletin. A total score can be derived by summing the number of correctly counted trials and the number of correctly reported animals (range 0–20). The psychometric qualities of the TEA-Ch appear satisfactory. Most of the subtests have been found to possess sufficient to good test–retest reliability [56, 57]. Furthermore, research has also provided support for the validity of the TEA-Ch, as evidenced by significant correlations with other indexes of attention (e.g., Stroop color naming task) and measures of intelligence and school performance [34, 57]. In addition, children with attention-deficit and hyperactivity disorder, who are known for their attention problems, perform significantly worse on this test as compared to other clinically referred youths [58, 59].

The ADHD module of the Anxiety Disorders Interview Schedule for youth [60] was administered to parents to confirm the main clinical diagnosis of the children and adolescents. This semi-structured interview assesses the frequency and intensity of (in this case: ADHD) symptoms as well as their interference in order to check various DSM-IV based criteria [1] and to ultimately establish a diagnosable condition. There is evidence indicating that the ADIS for youth has satisfactory test–retest [61] and inter-rater reliability [62], and the concurrent validity of the ADHD module has also demonstrated to be good [63].

One year after the post-intervention assessment had taken place, a research assistant contacted the parent(s) of the participating youth by telephone with a number of qualitative follow-up questions pertaining to (1) the current level of ADHD symptoms of the child/adolescent and the degree to which this problem interfered with daily functioning, (2) a retrospective evaluation of the intervention that the child had received for reducing attention-deficit and/or hyperactivity problems (1 = not helpful at all, 2 = somewhat helpful, 3 = helpful, 4 = extremely helpful), (3) whether the child/adolescent had continued to use the stimulant medication, and or—in case he/she had only received the Cogmed intervention—had started to use this type of medication in the past year, and (4) whether the child/adolescent had received other (additional) interventions targeting ADHD in the past year.

The Statistical Package for Social Sciences (SPSS Version 21) was used for calculating descriptive statistics and for performing the 3 (intervention groups: Cogmed WMT vs. stimulant medication vs. combined treatment) × 2 (assessment occasions: pre vs. post) analyses of variance (ANOVAs), with last factor being a repeated measure. The ANOVAs were conducted two times: the first time as an intent-to-treat analysis—which means that comparisons were made on the basis of the treatments that were initially chosen by parents and youth, thus regardless of eventual dropouts—and the second time as a completer analysis—which involved comparisons based on the treatments that were eventually completed. Because groups differed on a number of other variables (i.e., age, level of interference, SES, treatment confidence), ANOVAs were also conducted while correcting for these variables (ANCOVAs). In general, no significant effects of these covariates were found and therefore we decided to report the results of the uncorrected analyses. The one exception involved the analyses of the performance-based tests, which were all corrected for age. In keeping with what has been reported elsewhere in the literature [64], youth’s performance on this type of tests was found to improve significantly with increasing age. Finally, for the primary outcome measure—ADHD symptoms as measured with the ADHD-Q—effect sizes (Cohen’s d [65]) were computed for the pre-to-post changes observed within each of the intervention groups.