Introduction

Epidemiological research in the past decades has pointed out the importance of assessing impairment associated with mental health problems when determining the prevalence rates of mental illness [2, 12, 19]. In general, the terms ‘impairment’ and ‘impact’ refer to consequences of present psychiatric symptomatology for the child regarding their distress or functioning [2, 19]. In contrast, ‘burden’ reflects the problems for significant others, i.e. the parents and other family members [3]. The need to assess the impact of present symptoms is also reflected in the current DSM-IV [1] and ICD-10 [30] diagnostic manuals.

One important reason to consider the impairment associated with mental health problems is to avoid the overinclusive identification of psychiatric cases. The prevalence rates reported in previous studies were typically halved [22] or at least considerably reduced [5, 6, 11, 25, 26] when adjusted for present impairment. However, impairment criteria are not only relevant in ascertaining the validity of caseness; they also have the potential to identify children and adolescents who are not detected when only symptom-related diagnostic criteria are applied [2, 10, 20]. Angold et al. [2] pointed out that even if all potential symptoms are adequately assessed, diagnostic computer algorithms are not able to reproduce poorly specified diagnoses such as the ‘not otherwise specified’ diagnoses in the DSM-IV. This failure causes a substantial problem when trying to estimate the overall rates of mental health problems. As shown by Ford et al. [15] in the British Mental Health Survey, a fifth of psychiatric cases had non-operationalised or ‘not otherwise specified’ disorders. Furthermore, such patients constitute a considerable proportion of mental health service users [2, 7]. Consequently, in order to identify these individuals with impairing, but (with regard to reported symptoms) sub-threshold conditions in epidemiological studies, the assessment of impairment, in addition to symptoms, is necessary.

This argument is even more relevant when detailed diagnostic interviews covering a broad range of symptoms cannot be conducted in large surveys. Instead, short screening instruments are administered which operationalise the diagnostic criteria with respect to symptoms. For example, the strengths and difficulties questionnaire (SDQ) has proven to be an useful tool for predicting the likelihood of a psychiatric disorder in population samples [18, 29]. However, it has also been shown that children with symptoms that are not well covered by the SDQ, such as anxiety or eating disorders, are likely to remain undetected [18]. In these instances, the assessment of impairment offers a good way to identify children suffering from disorders that are not sufficiently assessed, as well as those children with impairing symptoms below the cut-off score. Goodman [17] introduced an extension of the SDQ – the impact supplement – and demonstrated by discriminating between children from a community and a treatment sample that parent- as well as self-reported impact scores were significantly superior to symptom scores in predicting caseness [17].

The assessment of impairment and burden due to mental health problems may also be helpful in identifying potential barriers to treatment. The burden that parents experience as a result of their child’s mental health problem has been shown to be an important predictor of the use of mental health care, for all degrees of psychopathology [3]. Perceived parental burden, however, is also highly associated with the child’s level of impairment. Since children’s access to mental health care is highly dependent on their parents, not only the child’s perspective, but also the parents’ perception of impairment is of special interest. In particular, the association between reported symptomatology in different problem areas and perceived impairment might differ between parents and children.

The present paper examines how frequently difficulties due to mental health problems are reported in the general population, and it investigates the levels of associated distress and functional impairment in different areas of life. The influence of assessing the parents’ versus the child’s perspective will be addressed. The proportion of children identified by symptom criteria, impairment criteria or both will be reported, and the properties of those children who ‘only’ report impairment without noticeable symptom scores will be investigated. Furthermore, the connection between reported symptoms in different problem areas and impairment will be investigated from the perspective of both parent and child raters.

Methods

Recruitment and sampling

Details on the conceptualisation, design and procedure of the mental health module (BELLA study) are described in detail in Ravens-Sieberer & Kurth [21]. Briefly, the participants of the BELLA study were randomly recruited from the national representative sample of 17,641 families participating in the German Health Interview and Examination Survey for Children and adolescents (KiGGS) conducted by the Robert Koch-Institute. The KiGGS and the BELLA survey took place between May 2003 and May 2006 in 167 cities and communities, selected by means of a stratified random process. The overall response rate was 66.6% (KiGGS). A random selection of 4,199 families from the KiGGS sample with children aged 7–17 were asked on their visit to the examination centre to participate in the BELLA study. Of these eligible families, 70% agreed to participate and 68% (1,389 girls and 1,474 boys) could be surveyed. In each family, one parent was questioned with a standardised computer-assisted telephone interview. Children aged 11 and older were questioned as well. In addition, the participants were asked to fill in a mailed paper and pencil questionnaire. Sample data were weighted to correct for sample deviation caused by the age-, gender-, regional- and citizenship-structure of the German population (reference data 31 December 2004).

Instruments

The strengths and difficulties questionnaire

Children and adolescents with symptoms of mental health problems were identified by means of the SDQ [16]. Parents and children (if at least 11 years old) reported frequencies of 25 positive and negative attributes in almost identically worded questionnaire versions. The items were assigned to five subscales: behavioural problems, emotional problems, hyperactivity, peer-problems, and prosocial behaviour. Each item was rated to be ‘not true’, ‘somewhat true’ or ‘certainly true’ and was scored as 0, 1, or 2. The 20 items of the four problem areas were summed up to generate a total difficulties score. This score was categorised into three groups indicating a ‘normal’, ‘borderline’, or ‘abnormal’ amount of symptoms [19] – here referred to as symptom caseness.

The extended version of the SDQ:SDQ impact supplement

The SDQ impact supplement [17] was developed in order to receive additional information on psychosocial disability to comply with the requirements of the World Health Organization [31]. In its first ‘perceived difficulties’ item, the supplement enquires whether the respondent thinks that the young person has ‘no’, ‘minor’, ‘severe’, or ‘definite’ problems in at least one of the following areas: emotions, concentration, behaviour, or being able to get on with other people. Since most respondents from the general population do not report any problems, they can skip all the remaining questions. If at least ‘minor’ problems are reported, the respondent is asked further questions about associated distress as well as social impairment in the areas of home life, friendships, classroom learning, and leisure activities. The answer options of these five items were coded (‘not at all’ = 0, ‘only a little’ = 0, ‘quite a lot’ = 1, ‘a great deal’ = 2) and summed up to generate the ‘impact score’. This scoring system implies a threshold effect since only in the presence of considerable impairment does the impact score rise above zero [17]. An impact score of 1 (here also referred to as borderline score) is interpreted as possible but not definite caseness whereas a score of 2 or more (an abnormal score) indicates probable caseness. In accordance with Goodman [17], this variable will be referred to as impact caseness.

A further item – referred to as the ‘burden rating’ – enquires if the reported difficulties put a burden on the family. The answer options for this are ‘not at all’ (0), ‘only a little’ (1), ‘quite a lot’ (2) and ‘a great deal’ (3). A further question refers to the chronicity of the problems.

Further instruments

Detailed descriptions of the instruments are given in Ravens-Sieberer & Kurth [21]. Briefly, depressive symptoms were assessed using the Centre for Epidemiological Studies Depression Scale for Children (CES-DC) [14] and the Depression Inventory for Children and Adolescents (DIKJ) [27]. Symptoms of anxiety were assessed using the 5-item short version of the Screen for Child Anxiety Related Emotional Disorders (SCARED) instrument [8]. Externalising problems were assessed by means of the subscales ‘aggressive behaviour’ and ‘delinquent behaviour’ in the German version of the Child Behavior Checklist (CBCL) [4]. Attention deficit-/hyperactivity disorders (ADHD) were assessed by means of the 10-item Conners’ Scale [9] and the ‘hyperkinetic disorders’ questionnaire (FBB-HKS) [13]. The CBCL scale and the FBB-HKS were only administered to parents whereas all other instruments were applied to the child and the parent.

Statistical analysis

Gender-specific frequencies of perceived difficulties, chronicity, impact ratings, and the burden rating were calculated for the self-report of youth aged 11–17 years. For parent-reported data, frequencies were calculated separately for the 7 to 10 and 11 to 17-year-old age groups. Odds ratios (OR) and 95% confidence intervals (CI) were calculated to compare the reported impairment of males and females. Impact caseness, as determined by self-reported versus parent-reported data was cross-tabulated. Spearman’s rank correlation coefficients between parent-reported and self- reported impairment were calculated.

Frequencies of symptom caseness, impact caseness and overlapping caseness were calculated and are reported separately for the parent- and self-reports. Univariate generalised linear models were employed to determine the estimated marginal means of psychometric instruments in the different caseness groups, adjusting for the covariates of age and gender. The analysis was conducted twice, once including exclusively parent-reported data and once including exclusively self-reported data.

Multiple logistic regression analyses were conducted in order to investigate the explanatory value of symptom caseness as a predictor variable for impairment-related outcomes as dependent variables. The SDQ total difficulties score as well as its subscale scores were divided into normal and borderline versus abnormal ratings. Dichotomised impairment-related outcomes were perceived difficulties (‘definite’ and ‘severe’), impact caseness (impact score ≥ 2), and burden to others (‘quite a lot’ and ‘a great deal’). Odds ratios adjusted for age and gender were calculated. Regression analyses including a SDQ subscale as predictor were also adjusted for the other SDQ subscales in order to explore the unique contribution of each symptom scale in explaining the three outcomes.

The statistical analyses are based on the weighted sample data to represent the age-, gender-, regional- and citizenship-structure of the German population (reference data 31 December 2004). All analyses were performed with SPSS version 15.0.

Results

Sample

The SDQ was administered during family visits to the examination centre and was completed by 2,833 parents of the 2,863 families (99%) participating in the BELLA study. The self-reported SDQ was filled out by 1,889 of the 1,913 children older than 10 years (99%). The SDQ impact supplement was included in a postal questionnaire that all parents and all adolescents aged from 11 to 17 years received after their visit in the examination centre. A total of 425 families failed to return the parent’s or child’s questionnaire and were excluded from the analysis. Additionally, 101 families were excluded due to missing values in the SDQ total score, subscale scores, SDQ impact supplement filter question or in the impact ratings.

This paper refers to the 2,337 of the 2,863 families that provided full data with respect to the extended version of the SDQ (82%). In families with children aged 7–10 years old, complete data were available from 840 parents (88%). In the 1,913 families with children aged 11 years and older complete self- and parent-reported data were available for 1,497 cases (78%).

After exclusion of families who failed to provide full data the mean SDQ total difficulties score of the remaining sample decreased (p < 0.001). Thus, the proportions of children with borderline and abnormal SDQ symptom scores according to their parents’ report are smaller in the present sample (borderline score 5.7% instead of 6.2% and abnormal score 6.6% instead of 7.2%) whereas for the child self-report no such difference was observed.

Prevalence of perceived difficulties, impact and burden

Table 1 presents the frequencies of reported impairment separated by rater group (parent vs. self) and age group (7–10 years old vs. 11–17 years old).

Table 1 Prevalence of impairment: Percentages of self-reported and parent-reported impairment (SDQ impact supplement)
Full size table

With respect to age groups, parents perceive difficulties more frequently in the younger children (OR = 1.73; CI 1.26–2.38), particularly for boys. However, with respect to reported distress and impact ratings, similar amounts of impairment are reported.

Parent-reported data for both age groups display considerable gender-differences with approximately twice as many boys as girls experiencing definite or severe difficulties or school-related impairment. However, this gender-specific impairment is not reflected in the self-reported data. Some significant gender-differences even point in different directions in parent- and self-report. In the older age group, regarding difficulties in home life and the burden on others, the higher risk for boys according to the parent-report (home life: OR = 1.57; burden: OR = 1.82), is opposed by a lower risk for boys according to the self-report (OR = 0.52 and OR = 0.46, respectively). The aggregated impact score reflects the divergence between the rating perspectives and indicates higher impact caseness in boys according to parent-reported data, while girls are more affected according to self-reported data.

The rating of chronicity (data not shown) indicates that parents not only report more frequently impairment in boys, but also a longer duration with more than 30% of the boys having difficulties for more than 1 year compared to only 20% of the girls (in the older age group). In the self-reported data, higher chronicity is also observed for boys, however, on a lower level and with smaller gender-related differences: only 17% of the boys and 12% of the girls report durations longer than a year.

In general, the boys report consistently less distress and impairment than their parents. The largest differences are found regarding difficulties in home life and burden for others, where only ∼3% of male adolescents report problems compared to 9.1% (home life) and 11.4% (burden) of the parents. In girls, self-reported frequencies of impairment correspond better with parent estimates. However, while the percentage of ‘definite’ and ‘severe’ perceived difficulties is similar in both rating perspectives (4.1% versus 4.3%), in the girls’ self-report ‘minor’ difficulties are reported more often (data not shown).

In both rating perspectives and age groups, the children without any impairment represent approximately 80% of all cases. A total of 11–13% of the sample report only one area of impairment. Two areas of impairment are reported by 4–7%, and the percentage of cases with three or four impaired areas ranges between 1.5 and 3% (depending on age group and rater). Irrespective of age group and rater, most families exclusively report impairment in school (8–9%) or in home life (1–3%), followed by cases with impairment in school and home life (1–4%).

Agreement on impact caseness between parent- and child-reports

Table 2 displays the agreement of parents and children regarding impact caseness. In 70.3% of the sample neither the child nor the parent report relevant impairment. However, only a few of the self- and parent-reports match regarding borderline (1.9%) and abnormal (3.3%) levels of impairment. In 10.7% of the cases, even diametrically opposed impact ratings (‘normal’ vs. ‘abnormal’) are found. Gender-specific analyses give comparable results (data not shown) with the exception that the type of oppositional judgement differs. In girls, 5.1% report ‘normal’ impairment when the parent reports ‘abnormal’ impairment, and in 5.6% of the cases the reverse situation is observed. Conversely, in boys, 8.9% report ‘normal’ impairment when the parent-report indicates ‘abnormal’ impairment, while it is the other way around in only 1.8% of the ratings.

Table 2 Percentage of self-reported and parent-reported impact caseness (N = 1,497)
Full size table

Spearman’s rank correlation coefficient between parent-reported and self-reported impact caseness categories is ρ = 0.29 (n = 1497; p < 0.001) and does not differ between boys and girls or between different age groups. Correlation coefficients for the single items of the SDQ impact supplement range between ρ = 0.23 (impact on leisure time) and ρ = 0.33 (distress). Again, hardly any differences between the correlation coefficients of boys and girls can be seen.

Symptom caseness and impact caseness

In self- as well as in parent-reported data most children display normal SDQ total difficulties scores as well as normal SDQ impact ratings (77%). In a smaller group of children, borderline or abnormal ratings in both scores coincide (5 and 8% in the self- and parent-reports, respectively). Another subgroup is defined by borderline and abnormal total difficulties scores when no impairment is reported (self-report: 6%; parent-report: 3%). However, for a larger proportion of the sample (approximately 12% of children), impairment is reported even though a normal total difficulties score was recorded. While borderline impact scores are found for half of these children (7% in self-report and 6% in parent-report), severe impairments are observed in 5 and 6% of these children, respectively.

Tables 3 and 4 show that children with abnormal impairment but normal SDQ total difficulties scores obtain significantly higher scores regarding several emotional and behavioural problems compared to children with normal symptom and impact scores. In most cases, they also display higher symptomatology than children with borderline or abnormal total difficulties scores who are not impaired. In general, their scores are close to the children who meet symptom and impairment criteria, especially regarding self-reported depression (DIKJ). The CES-DC mean score of the children with ‘only’ abnormal impairment even falls in the range of the cut-off scores suggested for depression and dysthymia by Fendrich et al. [14]. Children with weaker (borderline) impairment and normal SDQ symptom scores also have constantly increased scores in all psychometric instruments under study. However, their symptomatology is lower than that of children with abnormal impairment.

Table 3 Self-reported data: mean (SE) of psychometric instruments in children aged 11–17 assigned to different SDQ symptom caseness and SDQ impact caseness categories
Full size table
Table 4 Parent-reported data: mean (SE) of psychometric instruments in children aged 11–17 assigned to different SDQ symptom caseness and SDQ impact caseness categories
Full size table

Associations between symptom caseness and perceived difficulties, impact caseness and burden

Table 5 shows that in both rating perspectives all SDQ problem subscales significantly contribute to explaining perceived difficulties, impact caseness and burden. Only the peer-problem subscale is not significantly associated with the perceived difficulties in the parent-report, nor is it a significant predictor for the burden rating in the self-report. In both rating perspectives, the prosocial behaviour subscale provided no significant contribution to the prediction of any of the three outcome variables.

Table 5 Proportions of definite/severe perceived difficulties, abnormal impact caseness (≥2) and children being a burden to others in children aged 11–17 years with normal and borderline versus abnormal SDQ total difficulties and subscale scores (within self-reported and parent-reported data) (N = 1,497)
Full size table

In general, within the parent-reported data, strong associations between SDQ symptom caseness and reported impairment are found, with ORs for the total difficulties score ranging between OR = 19.2 (impact caseness) and OR = 23.5 (perceived difficulties). No SDQ subscale shows a particularly strong association with the impairment outcomes. Conversely, in the self-reported data emotional problems display the strongest association with all three impairment-related outcomes. However, apart from this subscale and peer problems (regarding perceived difficulties), the odds ratios are considerably lower than in parent-reported data.

Discussion

Impairment due to mental health problems is reported for and by a noticeable percentage of children and adolescents. In this representative population sample, 12% of the children displayed ‘only’ impairment whereas at the same time the amount of reported symptoms did not indicate a mental health problem. Six percent even suffered from severe impairment by mental health problems which remained undetected by the SDQ total difficulties score. Even though the BELLA study employed questionnaire screening methods to assess symptom caseness and impact caseness, these results are comparable to those of the Great Smoky Mountains Study [2], which found 14.2% of children with impairment but without a diagnosis.

Regarding the 12% of children in our study who reported impairment but no high symptom score, the severity of the reported problems needs to be discussed. The results suggest that two groups – the 6% with ‘only’ borderline impairment and the 6% with ‘only’ abnormal impairment – need to be distinguished.

The group of children having borderline impairment and sub-cut-off symptoms predominantly results from the high percentage of participants who reported ‘minor’ perceived difficulties regarding ‘emotions, concentration, behaviour or being able to get on with other people’ (25–36%). Upon further enquiry, most of them reported only one area with considerable impairment. School plays a major role here, being in many cases the single source of social incapacity. This finding not only indicates how the child’s difficulties in concentrating, behaving appropriately or interacting well with other people may interfere with his/her educational opportunities. It also hints at the necessity to critically examine the conditions in school, given that approximately one in ten children suffers from impairment in this important area. Briefly, this first group displays noticeable but not comprehensive impairment. These children have slightly increased symptoms regarding emotional and behavioural problems, which, however, do not interfere with their functioning to any great extent. However, in line with the results of Goodman [17], high chronicity was observed indicating long-term disadvantageous conditions for a young person’s development.

Conversely, in the second group of ‘only’ impaired children (‘abnormal’ impact), comprehensive impairment can be observed that affects two areas of functioning on average (data not shown). The analysis also indicates more emotional and behavioural problems in this group. Self-reported CES-DC scores hint towards clinically significant symptoms of depression in a sizeable proportion of these children [14]. These results correspond to the findings of Pickles et al. [20] who reported that many children who were below the symptom diagnostic threshold for depression experienced marked impairment. Similarly, in parent-reported data, considerably increased scores were observed for these abnormally impaired children on emotional as well as behavioural scales. The high scores on the instruments focusing on depression, anxiety, and externalising disorders support Goodman’s conclusion [17] that an impact score of 2 or more is a good indicator of definite caseness and indicates that this group includes children with serious mental health problems. However, this group of children may not be considered in reported prevalence estimates, since they remain undetected when only symptoms reflected by the SDQ total difficulties score are taken into account.

Even though the comparison between groups with different combinations of symptom caseness and impact caseness supported the validity of the SDQ screening questionnaire, the results also show that present impairment and borderline to abnormal symptoms scores do not necessarily overlap. The 6% of children that can be identified as being severely impaired only by the impact score demonstrate the important contribution of information provided by measures of impairment. Consequently, psychosocial impairment should be considered not only in those children given a diagnosis or in those displaying high symptom scores, but also in children who report lower degrees of symptomatology. Even though within the BELLA study no diagnosis is available as a gold standard, the data support the assumption that the failure to identify children with poorly specified diagnoses or impairing sub-threshold conditions, as pointed out by Angold et al. [2], applies also to screening questionnaires. Furthermore, findings regarding the additionally administered psychometric instruments suggest that the assessment of impairment gives the ability to recognise children who remained undetected by the total difficulties score due to its imperfect sensitivity.

Assessment of impairment has to consider that the parents’ judgements differ considerably from self-reported impairment. The extent of this discrepancy depends not only on the area of impairment in question but, particularly, on the gender of the child. Regarding gender as risk factor for specific kinds of impairment, none of the significant gender-differences observed in the parent-report is reflected in the self-report and vice versa. The higher risk of boys regarding overall difficulties, school problems, home life and being a burden for the family can only be observed in the parent-report; findings are partly even opposed by a higher risk for girls according to the self-reported data. This result is likely to be a consequence of the gender-specific trends in mental health problems, with boys more frequently displaying externalising problems that are connected to more obvious limitations, while girls in this age group tend to show internalising problems [23] where interferences with functioning are less observable. Additionally, gender-specific attitudes in males, such as the denial of a problem that may be interpreted as a weakness, may account for the consistently lower frequencies of self-reported impairment in boys.

The different information contained in the parents’ and the adolescents’ report on impairment is also emphasised by the moderate agreement between parents and adolescents regarding impact caseness. Again, gender-differences are found, although they are with respect to the kind of disagreement, but not the degree, with boys rating themselves as less impaired than the girls do.

The differences between parents’ and children’s perceptions were also reflected in the results of multiple regression analyses, which found different contributions of single problem subscales in explaining the impairment outcomes of the rating perspectives. In general, parent-reported data proved to be more consistent regarding the co-occurrence of symptoms and impairment. With respect to the different problem areas, scales representing externalising problems such as hyperactivity and conduct problems tended to be of higher importance in parent-reported data compared to the self-report. Conversely, in line with the results of Van Roy et al. [28], emotional problems played a central role in predicting experienced impairment in the self-reported data. Despite having broad confidence intervals, these results seem plausible since children may be not aware of the consequences of their behavioural problems or may cover up a problem, whereas a parent is more likely to be ignorant of a child’s emotional status [24]. However, these results contain important implications for families seeking treatment. Since emotional problems are associated with a lower awareness of their resulting impairment in adults, it has to be assumed that in these cases parents are less likely to seek professional help. Since, however, these problems are the most important predictor of impairment in the perception of the child, the child’s perspective deserves increased attention.

Some limitations regarding the presented results need to be discussed. First, a few weeks passed between the administration of the SDQ symptom questionnaire and the administration of SDQ impact supplement. Although this time delay should not be of considerable importance since all items refer to a time frame of half a year prior to filling in the questionnaire, studies administering both parts at the same time might find closer associations. Second, regarding the reported amount of impairment, the slightly positive selection of participants has to be considered. However, the strict exclusion strategy described above allowed for full comparisons between parent- and self-reported data. Third, it should be kept in mind that more broadly defined impairment due to mental health problems was assessed and that impairment ratings were not tied to symptom areas.

In summary, analyses of reported distress and impairment in a large representative population of children and adolescents showed that the assessment of impairment provides important information beyond the knowledge of symptoms and helps to identify an otherwise undetected high risk group. However, in the assessment of impairment, gender-specific issues have to be taken into account. Not only do boys and girls report different levels of impairment, but their parents also perceive their children’s impairment differently depending on the child’s gender. Furthermore, systematic differences between childrens’ and parents’ reports regarding the association between symptoms in different problem areas and reported impairment lead to the conclusion that the child’s perspective should be given special attention in the assessment of impairment.