Spanish Validation of the Problem Behaviors Assessment–Short (PBA-s) for Huntington’s Disease

Huntington’s disease (HD) is a rare neurodegenerative disorder with an autosomal dominant inheritance that usually affects people in their forties. It progresses toward a severe functional decline and dementia, finally causing death, frequently due to comorbid complications such as respiratory infections and severe trauma after a fall.¹ HD manifestations include motor disturbances, cognitive impairment, and neuropsychiatric symptoms. Classically, motor disturbances defined the onset of the disease, even though several studies have shown the presence of cognitive and behavioral changes prior to the onset of a movement disorder.^2–4

Neuropsychiatric symptoms are frequent in HD, as referenced in many studies.^5–7 A recent study⁸ of the European Huntington’s Disease Network Behavioral Phenotype Working Group using the behavioral section of the Unified Huntington’s Disease Rating Scale (UHDRS)⁹ confirmed that depression, irritability/aggression, obsessive-compulsive behaviors (OCBs), and apathy are highly prevalent neuropsychiatric symptoms in HD. They concluded that some of these symptoms are also distressing and are often unresolved by treatment in all stages of HD, with apathy being the key neuropsychiatric symptom occurring most often in advanced HD stages. These symptoms lead to important impairments in the caregivers’ quality of life.¹⁰

Despite its high prevalence, some neuropsychiatric symptoms have often been underappreciated, underdiagnosed, and undertreated.⁸ Neuropsychiatric symptoms also result in impairment in quality of life and are a considerable burden on health care systems, so assessing them accurately is very important to provide the best care to HD patients.¹¹

However, there have been few scales that were specifically designed to address psychiatric disturbances in HD patients to date. The most relevant ones are the UHDRS Behavioral Assessment, the Problem Behaviors Assessment for HD (PBA-HD),⁵ and its short version, the Problem Behaviors Assessment-Short (PBA-s).¹²

The UHDRS was developed in 1996 to provide a uniform assessment of the clinical features and course of HD. The behavior component of the UHDRS is an 11-item scale that was used as the global psychiatric measure in most HD studies.⁹

The PBA-HD is a semistructured behavioral interview designed for rating the presence, frequency, and severity of behavioral abnormalities in patients with Huntington’s disease.⁵ The PBA-HD is a 40-item instrument intended for use by trained psychiatrists with experience in the assessment of patients with neuropsychiatric disorders. The scale was developed via the compilation of psychiatric symptoms that patients and relatives frequently reported or those that are often referred to in the literature due to clinical interest in the field of dementia syndromes affecting frontal lobes.⁵ The instrument was developed in two phases. First, a five-point (0–4) scale was included for rating the severity and frequency of each symptom recorded as present or absent for the period since the patient’s last visit to the clinic, as done in the UHDRS. Second, strict scoring criteria for rating the severity of each symptom were included, and frequency criteria for every symptom were also defined to improve reliability. In order to maintain compatibility with the UHDRS Behavioral Scale, the 4-week period immediately prior to the interview was chosen for the assessment. Relative to its psychometric properties, the PBA-HD showed good feasibility; in their original study, the authors referred to good reliability (inter-rater mean weighted kappa: 0.86; test-retest mean weighted kappa: 0.94).⁵ Kingma et al.¹³ created a Dutch translation of the PBA-HD in 2008. This version was able to replicate the previous results: an interrater reliability of 0.82 for severity scores and 0.73 for frequency scores in a larger sample than that used in Craufurd et al.’s original study.

The PBA-s,¹² an 11-item semistructured clinical interview, was developed by the EHDN Behavioral Phenotype Working Group for use in the REGISTRY study,¹² where a shorter scale was necessary. The PBA-s is an instrument that evaluates severity and frequency of these symptoms: depressed mood, suicidal ideation, anxiety, irritability, angry/aggressive behavior, apathy, perseveration, obsessive-compulsive behavior, paranoid thinking, hallucinations, and disoriented behavior. To improve reliability, suggested prompts are offered to the interviewers, and criteria for rating frequency and severity are provided. The frequency scores range from 0 (symptom absent) to 4 (present all the time). The severity scores are defined independently for each item, ranging from 0 (symptom absent) to 4 (symptom causing severe problems). Severity and frequency scores are multiplied to obtain a total score for each item. The final score is rated considering all available information: that given by the patient and relatives, as well as the interviewer’s own observations of the patient’s behavior. Patients and informants are asked to identify the presence or absence of the symptoms during the last month, but in order not to exclude important symptoms that could have occurred during the last year (the recommended period to apply the scale within the longitudinal studies that are usually performed in the HD population), a “worst” score is added to set the severity of the symptom during that period of time.

Recently, the psychometric proprieties of the PBA-s have been reported using the data from two large studies: TRACK-HD and REGISTRY.¹⁴ The reliability study included data from 410 PBA-s video-recorded interviews, conducted with a total of 308 participants. The authors concluded that the PBA-s demonstrated good reliability. The mean kappa was 0.74 for severity and 0.76 for frequency scores, whereas the weighted kappa (allowing scores to differ by 1 point) was 0.94 for severity and 0.92 for frequency scores.

McNally et al.¹⁵ conducted a study that aimed to determine the validity of the PBA-s for Huntington’s disease as a clinical outcome measure using Rasch analysis. This study highlighted important limitations of the PBA-s to achieve this goal. Some amendments could lead to a better construct validity: PBA-s severity items could be combined to form a valid total score measuring a unidimensional construct, with reduction to a four-point scale. Attempts to achieve fit to the Rasch model for the PBA-s frequency score were more problematic.

Due to the difficulty in applying the diagnostic criteria for major psychiatric conditions, the PBA-s may be a good alternative to approach the psychological and behavioral manifestations of HD. It is recommended that the PBA-s be used in all HD studies where a behavioral assessment is needed as a comprehensive screen for the most common neuropsychiatric symptoms in HD. REGISTRY, a Europe-wide study run by the EURO-HD Network, and Enroll,¹⁶ a worldwide HD study, incorporated the PBA-s as a core assessment. Spain and other Spanish-speaking countries participated in these studies, so validating the PBA-s was absolutely necessary.

Methods

Design and Participants

A prospective observational multicenter study was carried out between January 2013 and December 2014. One hundred seventeen participants were recruited with the same criteria from five Spanish centers within the European Huntington Disease Network: 80 (68.4%) from the Hospital Mare de Déu de la Mercè (Barcelona), 18 (15.38%) from the Hospital Ramón y Cajal (Madrid), 11 (9.4%) from the Hospital Universitario (Burgos), five (4.27%) from the Hospital Clinic (Barcelona), and three (2.56%) from the Hospital Virgen de Arriaxaca (Murcia). The Ethics Committee, CEIC Hermanas Hospitalarias (Barcelona, Spain), approved the study protocol in accordance with the Declaration of Helsinki of 1964, revised in Edinburgh in 2000. Written informed consent of the participants was obtained.

The group of participants was comprised of 117 subjects (104 outpatients and 13 inpatients). The distribution within the inpatient group was nine patients admitted to the neuropsychiatry hospitalization ward and four patients admitted to the neuropsychiatry day care hospital. The mean stay of these inpatients was 148 days. Patients were categorized into five groups: people at risk for HD, premanifest expansion mutation carriers, manifest HD patients, control expansion negative, and control without a family history of HD, following the structure of the REGISTRY study. In our sample, 77 participants were manifest HD patients, nine people were at risk, 12 were asymptomatic carriers, and 19 were controls. Informants were coded following the instructions of the author as follows: spouse or partner (N=34), parent (N=12), sibling (N=10), child (N=5), other relative (N=2), friend or neighbor (N=1), professional care worker (N=10), other (N=3), no informant-subject came alone (N=40).

Measures

A sociodemographic data sheet and three questionnaires were used: PBA-s, UHDRS Total Functional Capacity (TFC), and Neuropsychiatric Inventory (NPI).¹⁷ The NPI was administered the same day but after the PBA-s, with the informant alone; thus, this instrument is based on responses given by a well-informed caregiver. The sociodemographic data sheet provided information about many characteristics of the group, including age, gender, educational level, and other variables, such as CAG repetition number when suitable, disease stage, and rater category.

The TFC is an ordinal component of the UHDRS. Scores on the TFC represent five stages in the neurodegenerative disease process. Lower scores represent greater functional impairment: stage I represents scores of 13–11; stage II, scores of 10–7; stage III, scores of 6–3; stage IV, scores of 2–1; and stage V, a score of 0.

The NPI is an instrument designed to capture most of the more frequent psychiatric and behavioral symptoms in dementia, with a similar structure to the PBA-s. Twelve items are included: delusions, hallucinations, agitation, dysphoria, anxiety, apathy, irritability, euphoria, disinhibition, aberrant motor behavior, nighttime behavior disturbances, and appetite and eating abnormalities. The frequency and severity of every item are measured on the basis of scripted questions administered to the patient’s caregiver, and a total score of every single item is obtained by multiplying the scores of the different domains.

Procedure

The validation process was carried out following the recommended methodological approaches for translation, adaptation, and cross-cultural validation of research instruments.¹⁸ The process began by first making a direct translation, which was done by professionals with a suitable level of English and psychopathological expertise and who were experienced in Huntington’s disease. Each translator performed his or her translation independently, and, after several meetings, a consensus version was obtained. Psychiatrists and neuropsychologists from other units of the hospital, unrelated to HD, reviewed the first version in order to evaluate its comprehensibility and suitability. A pilot study was carried out using a limited number of subjects, and this was designed to evaluate the instrument’s feasibility (evaluate the instructions, items, and response format clarity as well as application time). Afterward, and once this preliminary version was reviewed, a bilingual translator created a back translation to compare the differences between the translated version and the original text.

To evaluate reliability, we videotaped the interview of the first 30 patients enrolled consecutively and who gave informed consent to perform it. One rater (JR-I) re-evaluated his own interview after one month to measure intra-rater reliability, and two different raters (MF and CM) evaluated their interviews independently to evaluate inter-rater reliability. Concurrent validity was also measured. We chose to compare the results of the PBA-s Spanish version with those obtained by simultaneously applying the NPI.

Statistical Analysis

Descriptive statistics of the reference group are reported. Internal consistency was tested using the Cronbach’s α coefficient.¹⁹ Intra- and inter-rater reliability were assessed using Cohen’s kappa (κ).²⁰ Cohen’s kappa takes into account disagreement between the two raters, but not the degree of disagreement. This is especially relevant when the ratings are ordered, and therefore, the weighted kappa (κ_w) was also calculated.²¹ Weighted kappa allows scores to differ by 1 point. A Spearman correlation coefficient was computed between the total scores of the PBA-s and NPI (N=46), to test the concurrent validity of the scale, and between the PBA-s and the TFC scores. This statistical analysis was performed using the IBM SPSS Statistics (version 23.0).

In order to evaluate the factor structure of the severity of symptoms, we carried out a factor analysis on the severity scores (Table 1). Specifically, we applied the factanal function from the R statistical package²² to perform a maximum-likelihood factor analysis with varimax rotation. To avoid confounding effects from both unrelated controls and former patients with negative test outcomes, these two subgroups were excluded from the analysis (leaving an N=98). From the pool of symptoms, “delusions” and “hallucinations” were discarded, as less than 10% of individuals scored nonnull values for the severity of both symptoms.

Results

Demographical and Clinical Data

All relevant demographical and clinical baseline data are highlighted in Table 2. Ninety-two subjects (78.6%) were receiving psychiatric medication, most of them in combination therapy regimen: 51 (43.6%) were taking antidepressants, 46 (39.3%) antipsychotics, 38 (32.5%) anxiolytics, and 11 (9.4%) antiepileptic drugs. Twenty-nine patients (24.8%) were on specific treatment with antichorea therapy (tetrabenazine and/or amantadine).

In order to analyze the clinical characteristics of the neuropsychiatric symptomatology in our sample, a symptom with a 0 score is considered as absent and a 1 score as slight or questionable, whereas scores ≥2 are considered clinically relevant. Severity of the symptoms was classified as mild (scores 1 and 2) or moderate–severe (scores 3 and 4). In the tables below, neuropsychiatric symptoms of every participant category are shown, either in the last month or the last year. Results for every single present symptom are shown. In order to focus on the clinical relevance of the symptoms, the absolute number and the percentage of those considered as moderate–severe within each present symptom (scores 3 and 4) are shown.

Focusing on the neuropsychiatric symptoms of the premanifest and manifest HD participants (N=98), the results by disease stage are shown in Table 3.

Psychometric Properties

The internal consistency of the PBA-s, as indicated by the calculation of the Cronbach’s α coefficient, was Cronbach’s α=0.79, and most of the items contributed to it (Table 4). The independent analysis of the scores of the three raters involved in the study obtained similar results: Cronbach’s α=0.771, 0.755, and 0.602, respectively.

To test the inter-rater reliability of the scale, three raters independently video recorded and evaluated 30 patients (Table 5). The test-retest reliability was also evaluated in 30 patients and was found to be very high for all items (both in severity and in frequency measurements).

Table 6 shows the correlations found among the items assessing similar psychopathological symptoms. All correlations found are congruent and strong enough, especially in those items measuring the same neuropsychiatric symptom, for example, irritability (r=0.918, p<0.01), hallucinations (r=0.835, p<0.01), anxiety (r=0.827, p<0.01), or apathy (r=0.828, p<0.01). Other items, such as angry/aggressive behavior, correlate strongly with agitation (r=0.829, p<0.01), as well as depressed mood and dysphoria (r=0.835, p<0.01), as expected.

Stage TFC was significantly correlated with five PBA-s items: apathy (r=0.564, p<0.001), perseveration (r=0.525, p<0.001), delusion/paranoid thinking (r=0.340, p<0.001), hallucinations (r=0.266, p<0.001), and disorientation (r=0.724, p<0.001), while not the rest: anxiety (r=−0.101, p=0.280), obsession (r=0.182, p=0.051), depression (r=−0.153, p=0.102), suicide (r=0.088, p=0.347), irritability (r=0.050, p=0.597), and aggressiveness (r=0.177, p=0.058).

Factor Analysis

After running the factanal function with an increasing number of factors, the factor analysis solution with four factors was the first solution to adequately explain the data (χ²=8.87, df=6, p=0.181), accounting for 56% of the total variance. As summarized in Table 7, the four latent factors found may be related to irritability, affective alterations, apathy, and perseverance.

Discussion

To our knowledge, this is the first time that the PBA-s scale has been validated in another language. The results suggest that the Spanish version of the PBA-s is a valid and reliable scale for measuring the severity and frequency of neuropsychiatric symptoms in HD patients.

Regarding inter-rater reliability, our results are similar to those Callaghan et al.¹⁴ obtained in their recent study measuring the psychometric properties of the PBA-s (κ_w were 0.94 for severity scores and 0.92 for frequency scores in a sample of 410 individuals). Although using the PBA-HD, we can also affirm that our results are in line with those of Craufurd et al.⁵ (κ_w were 0.86 for severity scores and 0.84 for frequency scores in a sample of 38 individuals included in the reliability analysis) and Kingma et al.¹³ (κ_w=0.86 for severity scores and 0.84 for frequency scores in a subset of 63 subjects). Test-retest reliability also achieved substantial agreement, in line with that observed in the original Craufurd et al. study⁵ (κ_w==0.94 for severity scores and 0.70 for frequency scores in a subset of 15 subjects). This psychometric property was not reflected in the studies by Callaghan et al. or Kingma et al.

The internal consistency was good (α=0.79) and was equivalent to the results in Kingma et al.’s study.¹³ The PBA-s correlates strongly with the NPI, a validated scale frequently used to assess psychopathology in Alzheimer’s disease and other brain disorders with a similar structure, showing its concurrent validity and therefore its accuracy in assessing neuropsychiatric symptoms in HD patients.

As found in other studies, apathy is strongly correlated with TFC stages.^5,13,23 Correlation with TFC was also found to be significant in these symptoms: perseveration, delusion/paranoid thinking, hallucinations, and disorientation, showing their increasing prevalence in more advanced stages of HD, although the low prevalence of the psychotic symptoms should be considered. The remaining symptoms do not correlate, perhaps due to the different prevalence and severity scores encountered within the disease’s stages. Although not significantly, anxiety and depression were found to correlate inversely with TFC stages, according to other studies.²⁴

The results of the factor analysis indicate four latent factors, accounting for 56% of the total variance—apathy, irritability, depression, and perseveration—which is consistent with similar analyses performed by other authors. Callaghan et al.¹⁴ extracted three factors after a principal-components analysis: apathy, irritability, and depression. The same factors were identified in studies conducted by Craufurd et al.⁵ and Kingma et al.¹³, using the PBA-HD. Rickards et al.²⁵ performed a factor analysis on 1,690 completed UHDRS-behavioral assessments, finding four factors: depression, drive/executive function, irritability/aggression, and psychosis. In their study, the factor “drive/executive function” included apathy, perseveration, and compulsion. In our study, “perseveration” includes perseverative thinking/behavior and obsessive-compulsive thinking/behavior. This fact might show a behavioral cluster involving the frontal lobe dysfunction.

Our results showed that neuropsychiatric symptoms are very frequent in HD. In our sample, only eight participants (6.8%) scored 0 on all of the items during last month, a much smaller percentage than that encountered in other studies. Van Dujin et al.,⁸ using the UHDRS-b, found that the 27% of the participants had a score of 0 on all five behavioral subscales in the previous month. This may be explained by taking into account the profile of our participants, most of whom were assessed in a neuropsychiatry unit, where they were provided care specifically due to the presence of psychiatric symptomatology. Focusing on manifest HD patients, apathy was the most frequent symptom during the previous month, as found in other studies.⁵ Anxiety was the second most frequent, followed by irritability and disoriented behavior. Apathy, perseverative thinking/behavior, and angry/aggressive behavior correlated with TFC in the evaluation of the previous year. Psychotic symptoms, hallucinations, and delusions were less frequent, showing that psychosis is a very infrequent finding in this population, as encountered in other studies.⁷ Regarding the presence of neuropsychiatric symptomatology, depending on functional stage, our results, despite being evaluated by different scales, are quite similar to those Van Dujin et al.⁸ obtained, although they made their analysis based on clusters that were previously reported in another study after a principal-components analysis.⁶

Although we cannot make an exact comparison with Callaghan et al.’s study results,¹⁴ our subset of 98 mutation carriers presents higher percentages of neuropsychiatric symptoms, especially moderate–severe symptoms (scores 3 and 4).

A very important symptom, suicidal ideation, is present in 11.2% of the mutation carrier group, similar to that encountered by Callaghan et al.¹⁴ (11.7%) and higher than that in Craufurd et al.’s⁵ study using the PBA-HD (8.0%) and in Hubers et al.’s²⁶ study (8.0%) using the UHDRS-b. If only cases with moderate-severe scores are considered, the prevalence decreases to 3.3%. Nevertheless, this figure is higher than the 1.5% figure Callaghan et al.¹⁴ reported.

It is important to note that in the control group without HD family history (caregivers), there is a high prevalence of anxiety, irritability, and depressed mood, revealing the impact of the HD care provision on the mental health of caregivers. This is an issue that is very important to note in the comprehensive treatment of the disease, which should include the whole family afflicted by HD.²⁷

This study has several noteworthy strengths. It followed a strict methodology for the cultural adaptation and validation of the scale, systematically analyzing the main psychometric characteristics. This is a multicenter study, performed by different evaluators with different types of clinical training (psychologists, psychiatrists, neurologists, nurses). The study was carried out using a large sample, with 98 patients with HD and 19 healthy controls and relatives, in different clinical settings (inpatients and outpatients). Unlike previous studies, our study aimed to highlight the severity of neuropsychiatric symptoms, showing their prevalence in a stratified manner that distinguishes mild severity (scores 1, 2) from moderate to severe (scores 3, 4) for each of the different categories of participants. Severity of symptoms has also been presented both in the last month and in the last year, according to the original structure of the PBA-s, something that had not previously been done.

There also are several limitations to this work that merit mention. Having the greatest number of assessments carried out in the neuropsychiatry unit of the Hospital Mare de Déu de la Mercè may have biased the results, as this unit focuses on the treatment of neuropsychiatric symptoms in patients with HD. The samples used for the analysis of reliability may not be great in terms of number, but ultimately, 90 assessments conducted on 30 subjects were available. A further limitation of our study is that we could only use NPI with 46 patients because this instrument is intended to be completed by a well-informed caregiver, and in some cases, no such figure was available.

In our experience, training is fundamental to increase the reliability of the scale. The authors recommend using suggested prompts in a nonrestrictive manner and changing the order of the items if necessary. We decided to commence the assessment focusing on the less distressing items to ameliorate anxiety due to examination and to improve patient collaboration. Moreover, we recommend using the suggested prompts more literally, thereby ensuring greater reliability.

Conclusions

This study provides evidence of the validity of the Spanish version of the PBA-s, showing good internal validity, intra-rater and inter-rater reliability, and strong correlations with other psychopathological rating scales. A final version is now available for use by Spanish language professionals.