Abstracts

ORIGINAL ARTICLE

Translation, adaptation, and preliminary validation of the Brazilian version of the Behavior Problems Inventory (BPI-01)

Tradução, adaptação e validação preliminar da versão brasileira do Behavior Problems Inventory (BPI-01)

Gisele da Silva Baraldi^I; Johannes Rojahn^II; Alessandra Gotuzo Seabra^III; Luiz Renato Rodrigues Carreiro^IV; Maria Cristina Triguero Veloz Teixeira^V

^IPsychologist. MSc in Developmental Disorders, Graduate Program in Developmental Disorders, Universidade Presbiteriana Mackenzie, São Paulo, SP, Brazil

^IIPsychologist. PhD in Clinical Psychology. Professor, Applied Developmental Psychology Program, Department of Psychology, George Mason University, Fairfax, VA, USA

^IIIPsychologist. MSc and PhD in Experimental Psychology, Universidade de São Paulo (USP), São Paulo SP, Brazil. Professor, Graduate Program in Developmental Disorders, Universidade Presbiteriana Mackenzie, São Paulo, SP, Brazil

^IVPsychologist. MSc and PhD in Human Physiology, Institute of Biomedical Sciences, USP, São Paulo SP, Brazil. Professor, Graduate Program in Developmental Disorders, Universidade Presbiteriana Mackenzie, São Paulo, SP, Brazil

^VPsychologist. MSc and PhD in Psychology and Philosophy of Healthcare, Center of Health Sciences, Universidade Federal de Santa Catarina (UFSC), Florianópolis, SC, Brazil. Professor, Graduate Program in Developmental Disorders, Universidade Presbiteriana Mackenzie, São Paulo, SP, Brazil

^{Correspondence} Correspondence Profa. Maria Cristina Triguero Veloz Teixeira, Centro de Ciências Biológicas e da Saúde, Universidade Presbiteriana Mackenzie, Rua da Consolação, 896, prédio 28, 1º andar, Consolação, 01302-000 - São Paulo, SP - Brazil, Tel./Fax: +55 (11) 2114-8707. E-mail: cris@teixeira.org

ABSTRACT

Introduction: Children with atypical development often present behavior problems that impair their psychosocial adaptation.

Objective: To describe the cultural adaptation to Brazilian Portuguese of the Behavior Problems Inventory (BPI-01), as well as preliminary indicators of instrument reliability and validity.

Methods: The process involved translation, back-translation, and cultural adaptation of the instrument. Psychometric properties (reliability and validity) were assessed comparing scores obtained with the BPI-01, the Wechsler Intelligence Scales for Children and Adults, the Child Behavior Checklist for Ages 6-18 (CBCL/6-18), and the Autism Screening Questionnaire (ASQ). The sample comprised 60 children (30 typically developing and 30 atypically developing).

Results: The cultural adaptation process was considered adequate. Internal consistency of the BPI-01 was satisfactory, with a Cronbach's alpha of 0.65 for the self-injurious behavior scale, 0.82 for stereotyped behaviors, and 0.91 for aggressive/destructive behaviors. Considering a mean frequency of 0.5, the receiver operating characteristic (ROC) curve revealed 80% sensitivity and 3% specificity in the stereotyped behavior scale, 50 and 10% in aggressive/destructive behaviors, and 76 and 6% in self-injurious behaviors, respectively. Low-to-moderate correlations were observed between BPI-01, ASQ, and CBCL/6-18 scores.

Conclusion: BPI-01 showed good psychometric properties, with satisfactory preliminary indicators of reliability, convergent validity, and sensitivity for the diagnosis of atypical development.

Keywords: Intellectual disability, inventory, reliability, validity, behavior problems, psychological assessment.

RESUMO

Introdução: Crianças com desenvolvimento atípico frequentemente apresentam problemas de comportamento que prejudicam sua adaptação psicossocial.

Objetivo: Descrever o processo de adaptação cultural para português do Brasil do Behavior Problems Inventory (BPI-01), bem como indicadores preliminares de sua fidedignidade e validade.

Método: O processo envolveu as etapas de tradução, retrotradução e adaptação cultural do instrumento. Para a verificação das propriedades psicométricas (fidedignidade e validade), foram comparados os escores obtidos com o BPI-01, as Escalas de Inteligência Wechsler para Crianças e Adultos, o Inventário dos Comportamentos de Crianças e Adolescentes de 6 a 18 anos (CBCL/6-18) e o Questionário de Rastreamento de Autismo (Autism Screening Questionnaire, ASQ). A amostra foi composta por 60 crianças (30 com desenvolvimento típico e 30 com atípico).

Resultados: O processo de adaptação cultural foi considerado adequado. A consistência interna do BPI-01 foi satisfatória, com alfa de Cronbach de 0,65 para a escala de comportamentos autoagressivos, 0,82 para comportamentos estereotipados e 0,91 para comportamentos agressivos/destrutivos. Considerando-se uma frequência média de 0.5, a curva receiver operating characteristic (ROC) verificou sensibilidade de 80% e especificidade 3% na escala de comportamentos estereotipados, 50 e 10% para comportamentos agressivos/destrutivos, e 76 e 6% para comportamentos autoagressivos, respectivamente. Foram verificadas correlações de baixas a moderadas entre os escores do BPI-01, do ASQ e do CBCL/6-18.

Conclusões: O BPI-01 apresentou boas características psicométricas, com indicadores preliminares satisfatórios de fidedignidade, validade convergente e sensibilidade para o diagnóstico de desenvolvimento atípico.

Descritores: Deficiência intelectual, inventário, fidedignidade, validade, problemas de comportamento, avaliação psicológica.

Introduction

People with intellectual disability often present a typical behavioral repertoire with varying degrees of severity, including aggression, self-injury, and stereotyped behaviors. Behavior problems can cause physical injury to the aggressors themselves and to other individuals, impairing adaptation to social environments, including the family and school^1-5 - a scenario that underscores the importance of assessing this phenomenon and plan interventions when necessary. Some examples of typical aggressive responses are kicking, pushing, biting, scratching, and destroying things. With regard to self-injury, the responses most commonly reported are self-biting, self-hitting (different body parts), self-scratching, self-pinching, inserting foreign bodies into body openings, skin picking, hair pulling, and teeth grinding. Among stereotyped behaviors, the body, gestural, and verbal behaviors most widely investigated include oddly moving parts of the body, whirling, rubbing hands, clapping, screaming, smelling objects or parts of the body, and staring firmly at parts of the body and/or objects.^2,5

Many of the instruments designed to assess behavior problems related with aggression, self-injury, stereotyped behaviors, irritability, and other indicators of social adaptation in people with intellectual disability are available in English language only. Examples include the Adaptive Behavior Scale,⁶ the Reiss Screen for Maladaptive Behaviors,⁷ Assessment of Dual Diagnosis,⁸ Repetitive Behavior Scale - Revised,⁹ Modified Overt Aggression Scale,¹⁰ and the Behavior Problems Inventory (BPI-01).¹

In Brazil, two studies have described the cross-cultural adaptation and preliminary validation of instruments that assess overall indicators of behavior problems, namely, the Aberrant Behavior Checklist (ABC)¹¹ and the Child Behavior Checklist for Ages 6-18 (CBCL/6-18).¹² The latter instrument has undergone preliminary validation of the version for ages 4-18 and can be used in children with both typical and atypical development,¹³ as it assesses different behavior problems characteristic of children with both types of development (withdrawal, anxiety, thought problems, aggression, self-injury, stereotyped behaviors, challenge, and attention problems, among others).¹²

For the ABC,¹¹ in turn, only partial data are available regarding its cross-cultural adaptation to Brazilian Portuguese. ABC scales assess irritability/agitation/crying, lethargy/social withdrawal, stereotypic behavior, hyperactivity, and inappropriate speech. According to previous studies, this instrument allows to measure these behavior problems and to evaluate treatment response.¹¹ Notwithstanding, it does specifically assess behavioral repertoires associated with aggression, self-injury, and stereotyped behaviors often observed in people with intellectual disability and requiring close monitoring.¹⁴ The availability of standardized instruments focusing on these behavior problems could help identify and treat these problems. In contrast, the absence of such instruments poses difficulties to many healthcare and education professionals.^6,11

Differently from the ABC,¹¹ the BPI-01 was specifically developed to assess aggression, self-injury, and stereotyped behaviors. The BPI-01 is aimed at people with intellectual disability and severe psychiatric disorders of different ages and adaptive functioning levels.¹ BPI-01 items were developed based on literature reviews of other reference instruments, which helped compose its scales (self-injurious behavior, stereotyped behavior, and aggressive behavior).¹⁵

The present study was motivated by the need still observed in Brazil for standardized instruments to assess specific behavior problems. The objectives of the study were divided into two stages, as follows: 1) to describe the translation and cultural semantic adaptation of the BPI-01 into Brazilian Portuguese; and 2) to describe preliminary indicators of instrument validity. The first page involved translation, back-translation, and conceptual review of the instrument. The second stage assessed preliminary indicators of internal consistency, convergent validity, sensitivity and specificity of the adapted instrument. Psychometric data are considered preliminary due to the small sample size analyzed.

Method

Data collection

The present study was approved by the Research Ethics Committee of Universidade Presbiteriana Mackenzie (protocol no. CEP/UPM no. 1219/04/2010 and CAAE no. 0027.0.272.000-10). The author of the original version of the BPI-01 authorized the translation and cultural adaptation of the instrument into Brazilian Portuguese.

Data collection was conducted in two stages. In the first page, the instrument was translated into Brazilian Portuguese, back-translated into English, and then each item was conceptually revised to generate the final writing of the Brazilian version of the inventory. In the second stage, a pilot study was conducted to assess preliminary indicators of validity of the adapted instrument.

Translation protocol

Data obtained during instrument translation, back-translation, and conceptual review were analyzed according to the recommendations of Pasquali²¹ for psychological scales. Translation and back-translation were performed independently by bilingual professionals with expertise in development and developmental disorders. Following translation, the inventory was back-translated into English. Finally, each item comprising the inventory was subjected to conceptual review to compare translated and back-translated items against the original version of the instrument. This step was carried out with the participation of the two translators involved and the group of authors. The clarity and objectivity of each item included in the resulting Portuguese version was also evaluated by a focal group lasting for approximately 1 hour and including eight mothers of children with atypical development. The final Brazilian Portuguese version of the BPI-01 is presented in ^{Appendix 1}Appendix 1.

Participants

A total of 60 children and adolescents aged 6 to 18 years and their caretakers (parents and/or guardians) were included. The only caretaker inclusion criterion was being with the child for at least 6 hours daily. Children were divided into two groups paired by sex and age. One group included 30 children and adolescents with atypical development and different developmental disorders diagnosed by pediatric neurologists and geneticists. Of these, 22 attended regular schools and eight attended special schools. The other group comprised 30 children and adolescents with typical development and no mental disability. Overall characteristics of both groups are presented in Table 1.

Instruments

All the scales used in the study are described below.

Wechsler Intelligence Scales for Children Aged 6-16 (WISC-III)¹⁶ and Wechsler Intelligence Scales for Adults over Age 16 (WAIS-III).¹⁷ These scales were used to assess mental disability and/or normal intelligence development in the groups. The participants' intelligence quotient (IQ) was calculated based on results obtained with the block design and vocabulary subtests.^18,19

Translated Brazilian version of the BPI-01.¹ This questionnaire was answered by the caretakers. The BPI-01 includes 52 items divided into three scales covering three types of behavior problems. The translated, adapted version of the instrument resulting from the first stage of the study as described below was used. Each behavior problem item was rated according to its frequency (never = 0, monthly = 1, weekly = 2, daily = 3, hourly = 4) and severity (mild = 1, moderate = 2, severe = 3). The self-injurious behavior scale includes 14 items; the stereotyped behavior scale, 24; and the aggressive/destructive behavior scale, 11 items. The caretaker is asked to rate only those behaviors that have occurred at least once over the past 2 months. In addition to the behaviors described, each scale includes one item where the informer can report any other behavioral problems not covered by the instrument.

Brazilian version of the CBCL/6-18.¹² This instrument was also answered by the caretakers, with a focus on the children's behavior in the past 6 months. The CBCL/6-18 assesses different competences in the areas of activities, social relationship, and academic performance, and also covers behavior problems through different scales. Raw scores are converted into standardized T scores according to the child's/adolescent's age and sex.¹² These T scores, in turn, allow to classify behavioral profiles into three categories, namely, normal, threshold, and clinical. The behavior problems assessed by the CBCL/6-18 include syndrome-based scales for anxiety/depression, withdrawal/depression, somatic complaints (this scale was excluded from the analysis), social problems, thought problems, attention problems, rule-breaking behavior, and aggressive behavior. In addition, the following scales are based on the Diagnostic and Statistical Manual of Mental Disorders (DSM): affective problems, anxiety problems, somatic problems, attention deficit/hyperactivity problems, oppositional defiant problems, and conduct problems. Other measures include internalizing problems (including anxiety/depression, somatic complaints, and withdrawal/depression), externalizing problems (rule-breaking behavior and aggressive behavior), and the total emotional/behavioral problem scale.¹² This inventory was included in our study to assess preliminary indicators of the validity of the Brazilian version of BPI-01, as the internalizing, externalizing, and total problem scales contain items that measure constructs similar to those of the BPI-01, e.g., self-injury, stereotyped repetitive movements, odd behaviors, cruelty, destructive behavior, and aggressive behavior towards others, among others.

Autism Screening Questionnaire (ASQ).²⁰ This questionnaire includes 40 questions organized into the following scales: reciprocal social interaction, social communication, and repetitive stereotyped behavior. Questions are closed and each can be scored with 0 (behavior absent) or 1 (behavior present). A score of 15 is considered the cutoff point for the presence of invasive development or autism spectrum disorders, and a score of 22 is considered enough to diagnose autism. The questionnaire should be answered by the parents or guardians of individuals aged 4 years or older with suspected autism spectrum disorders.²⁰ The ASQ includes items that are compatible with those present in the BPI-01 for the assessment of stereotyped behaviors, e.g., mannerisms, odd ways of moving hands and/or fingers, moving parts of the body, etc. This questionnaire was included so that we could assess preliminary indicators of validity of the Brazilian Portuguese version of the BPI-01.

Data analysis

Internal consistency was measured using Cronbach's alpha coefficients, with item-total correlation and alpha coefficient (in case an item had to be excluded). Coefficients were interpreted as follows: alpha values above 0.80 were considered desirable; values between 0.60 and 0.80 were considered recommended for clinical practice; and values below 0.60 were considered acceptable for use in research only (not recommended for clinical practice). As a result, any alpha value above 0.60 was interpreted to indicate satisfactory internal consistency within the scope of the present study.²²

The occurrence of behavior problems in the group with atypical development vs. the control group was compared using the nonparametric Mann-Whitney test, with significance set at p < 0.05. This nonparametric test was used because the requirements for the use of parametric tests were not met, i.e., the measures employed did not show a normal distribution and were not of interval, as they resulted from the conversion of Likert scales into scores. Because of our small sample size, in this analysis we chose to dichotomize the frequency of behavior problems assessed by the BPI-01 into 1 and 0, with score 1 accounting for the presence of behaviors monthly, weekly, daily, and hourly, and 0 accounting for the absence of behaviors (never).

Convergent validity was assessed using Spearman's correlation coefficient, comparing BPI-01 scores vs. ASQ and CBCL/6-18 scores. Results showing p < 0.05 were considered statistically significant. Sensitivity and specificity were assessed using the receiver operating characteristic (ROC) curve for different frequency cutoff points. This measure was used to distinguish between children in the atypical development group and those in the other group. For this analysis, dichotomized mean scores obtained in each of the three scales comprising the BPI-01 were calculated, followed by calculation of the arithmetic mean of each scale of the BPI-01.

Results

Translation, back-translation, and conceptual review

During conceptual review of the translated and back-translated versions in comparison with the original version of the instrument, three non-compatible items were identified in the back-translated version of the stereotyped behavior scale. In two of them, the translation choice was maintained, but in the third case and adaptation was deemed necessary, as follows: the original "Spinning own body" was translated as "Girar o próprio corpo" and back-translated as "Spinning," however the translated version was maintained as was; the original "Maintaining bizarre body postures" was translated as "Manter posturas corporais estranhas" and back-translated as "Maintain weird body postures," but again the final translated version was maintained as was; conversely, the original item "Gazing at hands or objects" was translated as "Esfregar-se" and back-translated as "Gaze steadily at one's hands or objects" - this item was adapted and rendered in the final translated version as "Esfregar-se com as mãos ou com objetos."

Mothers taking part in the focal group considered unclear item 5 of the self-injurious behavior scale. In the original English version, this item was rendered as "Vomiting and rumination (deliberate regurgitation of swallowed food with rumination)." Following the back-translation and conceptual review stages, this item was presented in Brazilian Portuguese as "Vomitar e ruminar, vômito deliberado da comida ingerida com ruminação." Based on the considerations made in the focal group, it was rewritten and rendered as "Vomitar e ruminar, regurgitar de propósito comida ingerida com ruminação."

Pilot study for the preliminary assessment of internal consistency, convergent validity, sensitivity, and specificity of the translated instrument

Preliminary indicators of internal consistency (instrument reliability) revealed a Cronbach's alpha of 0.65 in the self-injurious behavior scale. Item-total correlations were low to moderate overall, which may explain the relatively low coefficient obtained, probably a result of the great variety of behaviors included in this category. In the stereotyped behavior scale, Cronbach's alpha was 0.91, revealing adequate internal consistency. Finally, the Cronbach's alpha coefficient obtained for the aggressive/destructive behavior scale was 0.82, also satisfactory.

Assessment of mental disability and/or normal intelligence development using the WISC-III¹⁶ and WAIS-III¹⁷ allowed to identify two samples, namely, one without mental disability and the other with mental disability of varying degrees according to the DSM-IV, i.e., IQ below 80 (compatible with the classification of intellectually disabled).²³ The number of participants with intellectual disability was distributed as follows in the atypical development group: six children showed moderate mental disability (mean IQ = 47.8), 18 mild mental disability (mean IQ = 60.9), two children had borderline mental disability (mean IQ = 77), and four children showed a mean IQ value of 82.7, corresponding to absence of mental disability but intelligence below the average. The 30 children with typical development presented IQ values ranging from 80 to 129 (mean = 122.3), corresponding to absence of mental disability. Comparison of behavior problems between the two groups revealed that all frequency means obtained in the BPI-01 scales were higher in the group with mental disability than in the group without disability, pointing to a higher number of behavior problems in the former group, with statistically significant differences in terms of both frequency and severity of the problems assessed (Table 2).

Table 3 shows Spearman's correlation coefficients obtained for BPI-01 scores vs. ASQ and CBCL/6-18 scores. Preliminary indicators of convergent validity of the BPI-01 in relation to the ASQ revealed adequate coefficients for the self-injurious and stereotype behavior scales, whereas the aggressive/destructive behavior scale showed poorer correlations. With regard to the CBCL/6-18, most scales showed positive and statistically significant correlations with BPI-01.

Sensitivity and specificity, analyzed based on the area under the ROC curve, yielded values of 0.86 for the self-injurious behavior scale, 0.88 for the stereotyped behavior scale, and 0.70 for the aggressive/destructive behavior scale. Preliminary results of the pilot study conducted to establish possible cutoff points to differentiate between the groups are shown in Table 4. In all scales, a frequency of 0.5 was the one that best allowed to distinguish between the group with atypical development and the control group, with 80% sensitivity and 3% specificity in the stereotyped behavior scale, 50% sensitivity and 10% specificity in the aggressive/destructive behavior scale, and 76% sensitivity and 6% specificity in the self-injurious behavior scale.

Discussion

In the first stage of this study, which consisted of the translation and back-translation of the BPI-01, most of the back-translated items were equivalent to their original versions in English. Results of the focal group were positive with regard to the conceptual review undertaken, as only one item raised doubts, whereas all the remainder were adequately understood by participating mothers.

Assessment of instrument reliability based on Cronbach's alpha coefficients showed adequate indicators of internal consistency for two of the scales comprising the BPI-01, namely, stereotyped behaviors and aggressive/destructive behaviors, and acceptable ones for the third scale, self-injurious behavior scale. It is possible that our small sample size has caused aggressive/destructive behaviors to vary greatly and, as a result, show poor internal consistency. This hypothesis should be confirmed in future studies conducted in Brazil with larger samples (for instance, Lundqvist⁴ assessed 915 subjects with intellectual disability and observed better internal consistency indicators for the BPI-01, also using Cronbach's alpha coefficients).

Inter-group comparison revealed a higher number of behavior problems in the atypically developing group, which is compatible with previous findings.²⁴ In addition, as also described by Rojahn et al.,¹ stereotyped and self-injurious behaviors were more frequent in the group of atypically developing children.

Convergent validity indicators based on the analysis of correlations between BPI-01 and ASQ scores showed that only self-injurious and stereotyped behaviors as assessed by the BPI-01 showed moderate coefficients with the ASQ. Different results were observed for the aggressive/destructive behavior scale, where a low correlation was found. To some extent, we did not expect to find moderate correlations among all scales, as not all the constructs measured by the BPI-01 are also found in the ASQ (for example, the latter does not include items specifically directed at the assessment of aggression). Notwithstanding, the ASQ includes items that assess self-injurious and stereotyped behaviors, both gestural and verbal, which may have contributed to the higher correlations obtained for a mean frequency of 0.50 in the self-injurious behavior and especially in the stereotyped behavior scales of the BPI-01.

Convergent validity between CBCL/6-18 and BPI-01 scores revealed positive, statistically significant correlations for most scales. The CBCL/6-18 has been used to describe behavioral profiles of children with developmental disorders and mental or intellectual disability.^25-29 Based on the correlations found, we could argue that the presence of behavior problems identified by the BPI-01 may suggest the presence of other behavior problems assessed by the CBCL/6-18.

The ROC curve, more precisely the area under the curve, was used in an attempt to identify cutoff points to differentiate between children with typical and atypical development. A test is sensitive when it is able to detect individuals with a given characteristic. Specificity, in turn, for diagnostic purposes, refers to the test's ability to accurately detect the absence of a diagnosis. Taking into consideration that no data are currently available on the sensitivity and specificity of the English, original version of the BPI-01,¹ the present ROC analyses indicate possible cutoff points to distinguish between behavior problems in children with typical vs. atypical development (despite the preliminary nature of such analyses given our small sample size). We should also emphasize that the values here assessed reflect dichotomized mean scores obtained for each scale.

Of the three scales comprising the BPI-01, the self-injurious behavior scale was the one with the highest specificity. Indeed, these are behavioral problems more frequently observed in people with atypical development or chronic psychotic disorders.^30-32 Future studies are warranted to assess larger samples, broader age ranges, levels of education, as well as other types of neurobehavioral developmental disorders.

Future studies should include larger samples and calculate correlations for the frequency and severity of each behavior problem. On the one hand, aggressive behaviors may occur rarely but with extreme severity; on the other, it is possible to find reports of frequent episodes of self-biting, however of mild severity. Because the BPI-01 evaluates behavior considering these two criteria (frequency and severity), it can help in the decision-making process, favoring more objective treatment decisions based on individual needs.¹

Conclusion

Within the goal of producing a Brazilian Portuguese version for the BPI-01, the conceptual review of all items after their translation and back-translation allowed to check back-translated items against the regional scale and make adjustments where necessary. Moreover, the focal group contributed to further adapting the instrument and making it accessible to the caretakers of children with atypical development included in the pilot study.

Application of this new, adapted version to a small sample showed that the correlation between BPI-01, CBCL/6-18, and ASQ scores was adequate, i.e., that preliminary convergent validity indicators were acceptable. Notwithstanding, future studies should be conducted to identify other psychometric indicators. In addition, the ROC curve here obtained revealed that self-injurious behavior problems were the ones that best allowed to differentiate between children with typical and atypical development. In the other two scales, in turn, cutoffs were higher and did not show a satisfactory relationship between sensitivity and specificity.

Future studies with larger and more heterogeneous samples are warranted to validate and further standardize the instrument for use in the Brazilian population. These studies will allow to assess the influence of other factors (e.g., age, education level, sex, intelligence level, adaptive functioning, psychosocial stimulation, type of developmental disorder, and speech abilities, among others) on the presence of behavior problems, especially self-injurious and stereotyped behaviors, in children with both typical and atypical development.

Submitted May 08 2012, accepted for publication Nov 21 2012.

No conflicts of interest declared concerning the publication of this article

Financial support: Fundo Mackpesquisa, Universidade Presbiteriana Mackenzie, São Paulo, SP, Brazil.

Appendix 1

Publication Dates

History