Introduction
Assessment of adolescent behavior problems often relies on the reports of family members who experience the adolescent in everyday life. The influence of the family context on these problem behavior judgments deserves special attention because the family provides a unique context for judging problem behavior. Acquaintanceship or familiarity among family members is high (i.e., family members know each other well and thus have access to a rich source of information about each other’s problem behavior) and family members share the same family context (i.e., the general context in which family members observe and experience each other’s problem behavior is more or less the same). The present study is concerned with judgments of problem behavior within families with adolescent children. It focuses on how family members judge each other’s problem behavior.
A considerable amount of research has been devoted to evaluating the reliability and validity of knowledgeable informant’s perspectives on family behavior (Achenbach et al. 1987; Cook and Goldstein 1993; Heinonen et al. 2006; Jacob and Windle 1999; Renk et al. 2008; Schwarz et al. 1985; Vierhaus and Lohaus 2008). However, very little of this research has taken advantage of models of interpersonal perception in hypothesizing the sources of informant disagreement on such ratings. In fact, there is a well developed literature on factors affecting interpersonal perceptions and the accuracy of such judgments (Haselton and Funder 2006; Kenny and Albright 1987; Kenny and DePaulo 1993; Letzring et al. 2006; Malloy and Albright 1990). In this paper, we evaluate Kenny’s (1994) model of interpersonal perception as a means of understanding knowledgeable informant reports of problem behavior.
According to Kenny’s (1994) model, one person’s perception of another will be due to three systematic sources. The first is called a perceiver effect. It is present if characteristics of the person making the judgment affect his or her perception of the target. This component could reflect a process of assimilation; the tendency to perceive others according to preconceptions of what others, in general, are like (i.e., a schema). The second source is called a target effect. This is the component that researchers hope they are measuring when they ask informants to judge a target; that is, it generally reflects characteristics of the person being judged (e.g., attractiveness, agreeableness, or externalizing behavior). The third source of interpersonal perceptions is called a relationship effect. A relationship effect represents a person’s perception of another person in particular, above and beyond their perceiver and target effects. For example, Susan’s relationship toward Michael on the variable of warmth measures the extent to which she perceives Michael as warm controlling for her general tendency toward perceiving others as warm and Michael’s general tendency to be perceived as warm by others.
Most of the theory-driven research on interpersonal perception is based on social-psychological studies of people of varying degrees of acquaintance (for a review see Kenny et al. 1994). Few of these studies involve perceptions within natural groups such as the family, so they tend to focus on individual and dyadic level factors such as perceiver, target, and relationship effects. However, when members of a group develop a group identity, as they do in families, group level factors may also influence interpersonal perceptions. This point of view is illustrated by measures of family functioning that assume that family members are similar in some way. A well-known example of such a measure is the family environment scale (FES: Moos and Moos 1981). The items in such measures are designed to assess similarity in family members. For example, “People in my family look out for each other” is the type of item used in such measures. If family members are similar with respect to a problem behavior, this group-level effect will also affect family members’ judgments of the behavior. In the family version of SRM this is called a family effect (Cook 1993, 1994; Kashy and Kenny 1990).
There is evidence from previous research indicating that these factors—perceiver, target, relationship, and family—affect judgments of child or adolescent’s problem behavior. In social-psychological studies, about one-third of the total variance in interpersonal perceptions is typically found to be due to the target, even when the perceivers know the target fairly well (Kenny 1994). Consistent with this, clinical researchers have found modest correlations between different family members’ ratings of a child or adolescent’s problem behavior (see e.g., Achenbach et al. 1987; Renk et al. 2008; Van Der Ende and Verhulst 2005; Vierhaus and Lohaus 2008). These findings imply that characteristics of targets (i.e., the children or adolescents) produce small amounts of agreement in family members’ ratings of the problem behavior.
The relatively small effect the target has on ratings of his or her behavior may be due, in part, to the relatively idiosyncratic theories about targets (Park et al. 1994). This would correspond to a relationship effect in Kenny’s model. These effects are often interpreted from the residual variances in latent variable models of rater agreement (e.g., Cook and Goldstein 1993; Jacob and Windle 1999). However, there is a substantial amount of research indicating the presence of more general perceiver effects that affect such judgments. In clinical research, for example, levels of psychopathology, particularly depression, may influence parents’ ratings of children’s behavioral problems, especially of children’s internalizing problems (see e.g., De Los Reyes and Kazdin 2005; Kroes et al. 2003). In a study by Heinonen et al. (2006) a parent’s own dispositional optimism and pessimism was found to have a role in shaping his or her perceptions of the child’s behaviour. In research on family relationships involving non-clinical families, these perceiver or actor effects tend to dominate in the explanation of individual ratings (e.g., Cook 2000).
The idea that characteristics of the family might affect family member ratings of problem behavior has not been addressed directly. Nonetheless, there is some evidence that characteristics of the family such as ethnicity or race, family stress, and SES are related to family members’ judgments of a child’s or adolescent’s problem behavior (see e.g., De Los Reyes and Kazdin 2005). In family relationships research, the family factor tends to be small and often does not have reliable variance at all. This depends largely on the variable being studied, however. If problem behavior tends to run in families, then similarity among family members could be a significant determinant of problem behavior ratings.
That characteristics of perceivers and the family may significantly affect family members’ ratings of problem behavior has important implications for the diagnosis and treatment of children’s behavioral problems. Informant ratings on instruments such as the child behavior checklist (CBCL; Achenbach 1991) are widely used to assess children’s problem behavior and, thus, to guide clinical interventions and assess clinical outcomes. If such ratings are due to the characteristics of a disturbed rater or a disturbed family environment, then the evaluation would be fatally flawed. Assessing the functioning of the rater and the family may be as important as assessing the behavior of the target person when measures obtained from knowledgeable informants are to be used. In the present study, the social relations model (SRM: Kenny and La Voie 1984; Cook 1994; Cook and Kenny 2004; Kashy and Kenny 1990) is used to test data from a family round-robin design in which each family member rates the problem behavior of all other family members.
Hypotheses
Based on existing research in which evidence was provided that perceiver, target, and family factors affect judgments of child or adolescent’s problem behavior (Achenbach et al. 1987; De Los Reyes and Kazdin 2005; Kroes et al. 2003; Van Der Ende and Verhulst 2005), our first hypothesis was that perceiver, target, and family effects would all be found in family members’ perceptions of each other’s problem behavior. We will also study the relative importance of the perceiver, target, and family effects for explaining within-family perceptions of problem behavior. No explicit hypotheses could be made about the relative importance of the SRM effects because SRM has not been applied to perceptions of problem behaviour. However, we expect percentages to be close to the percentages found in the studies by Kenny (1994) and Branje et al. (2003). Kenny (1994) summarized the relative contributions of these sources of variance in perceptions of personality and concluded that 20% of the variance could be attributed to the perceiver, 15% of the variance to the target, and 20% to the relationship. The relative contribution of the family effect was not assessed. In the study by Branje et al. (2003) somewhat lower target effects (more or less 10% except for conscientiousness) in personality judgments were found and average family variance was 4%.
Previous research showed greater correspondence evident in ratings of externalizing problems compared with internalizing problems (Achenbach et al. 1987, Duhig et al. 2000). Because it is easier to reach consensus on problem behavior that is more observable, our second hypothesis was that the absolute value of the amount of variance explained in family members’ perceptions of problem behavior by target effects would be greater for perceptions of externalizing problem behaviour than for perceptions of internalizing problem behavior. In the Method section (Statistical Analyses), we explain why we could not estimate relationship effects.
Method
Participants
Families were approached via 16 secondary schools in The Netherlands. Schools with different types of curriculum were selected in three different regions. At each school, the study was explained in several school classes. A letter was handed to the students addressed both to the parents and to the students themselves, informing them about the goals and procedures of the research project, the criteria with respect to participating, and the registration procedure. The criteria for participating in this study were being part of a household with (a) two parents and (b) at least two adolescents between the ages of 11 and 18. Families were given Euro 45.5 for their participation in this study . The students brought the letter home. If all family members (both parents and both adolescents) were willing to participate in the study, families could enlist in the study by sending the application form back to the research group. Seventy families participated in wave 1. The reason for not participating could be that families did not meet the criteria for participating (e.g., being part of a household with two parents and at least two adolescents between the ages of 11 and 18) or that not all family members (both parents and both adolescents) were willing to participate in the study. At wave 2 one family dropped out of the study because the parents were divorced. Therefore, results are reported with respect to 69 families in both waves. The adolescents were distinguished by their birth order as the older adolescent and the younger adolescent. At baseline (wave 1), the age of the older adolescents (35 boys, 34 girls) was on average 15.6 years (SD = 1.31) and the age of the younger adolescents (32 boys, 37 girls) on average 13.3 years (SD = 1.18). The average age of fathers and mothers was 47 and 45 years respectively, ranging from 40 to 58.6 years and from 36.9 to 55.4 years. The highest level of education was secondary education for 37% of the fathers and 51% of the mothers; whereas 30% of the fathers and 29% of the mothers finished higher education and 33% of the fathers and 20% of the mothers obtained a university degree. Of the fathers, 97% were employed, whereas 74% of the mothers were employed. Ninety-eight percent of the participants were Dutch.
Measures
Externalizing and Internalizing Problem Behavior
The Nijmegen Problem Behavior List (NPBL; De Bruyn et al. 2005) was used to assess family members’ problem behavior. The NPBL is a research instrument designed for studies that focus on family members’ perceptions of each other’s problem behavior (e.g. Delsing et al. 2005; Janssens et al. 2005). The items of the NPBL describe subclinical variants of the type of behavior that is assessed in the work of Achenbach (see Achenbach and Rescorla 2001). Confirmatory factors analyses have revealed that the items reflected two broad unidimensional factors; externalizing and internalizing problem behavior. The factorial validity of the NPBL across informants and targets (parents and adolescents) and time was proven (De Bruyn et al. 2005).
The scale for externalizing problem behavior consists of seven items (e.g., “This person fights a lot”) and the scale for internalizing problem behavior consists of nine items (e.g., “This person withdraws from other people”). The items are rated on 7-point Likert scales anchored at the extremes by 1 = very untrue to 7 = very true.
In this study, fathers, mothers, the older adolescents, and the younger adolescents rated each other on the 16 items of the NPBL. In each of these ratings the items appeared to assess two factors; externalizing and internalizing problem behavior. Cronbach’s alpha’s for family members perceptions of each other’s problem behavior were on average α = .75 with a range of α = .63 to α = .86 for externalizing problem behavior, and α = .72 with a range of α = .56 to α = .81 for internalizing problem behavior. In the second wave approximately 10 months later, Cronbach’s alpha’s were on average α = .77 with a range of α = .56 to α = .84 for externalizing problem behavior, and α = .74 with a range of α = .67 to α = .78 for internalizing problem behavior.
Statistical Analyses
SRM Perceiver, Target, and Family Effects in Perceived Problem Behavior
LISREL 8.50 (Jöreskog and Sörbom 2001) was used to estimate the components of the social relations model (Kenny and La Voie 1984). The application of SRM to family data requires data from a round-robin design (i.e. each family member was asked to rate the problem behavior of the other family members). In our two parent-two adolescent families, this design resulted in 12 judgments of problem behavior (four family members rating three family members each). According to the family version of the SRM (Cook 1993, 1994; Kashy and Kenny 1990), four sources of systematic variance can be identified in each of these 12 judgments of problem behavior; a perceiver effect, a target effect, a relationship effect, and a family effect. For instance, mother’s perception of adolescent’s problem behavior can be specified as a function of the mother perceiver effect, the adolescent target effect, the mother-adolescent relationship effect, and the family effect.
A SRM-analysis can be conceived as a confirmatory analysis, in which the SRM components are the latent variables and the round-robin data are the observed variables (Cook 1994). Thus, the components of the SRM were estimated by fitting the SRM to the variance-covariance matrix of the 12 measures of perceived problem behavior. Contrary to confirmatory analysis, the factor loadings are usually fixed at 1.0 and the variance in the latent variables is estimated. Twelve observed variables (four persons reporting on three other persons) were available to estimate twelve relationship effects (four persons times three relationships). Because latent variables as the relationship effects require a minimum of two indicators (i.e. observed variables) in order to be estimated, a latent variable for the relationship effects could not be estimated. Latent variables for the other SRM effects could be estimated because more than two indicators were available for each of them (for the SRM perceiver and target effects three indicators were available, and for the family effect 12 indicators were available). For example, for the perceiver effect of mother all three observed variables of mother reporting on her three family members can serve as indicators. Consequently, the systematic variance in the observed measure that is due to the relationship, became part of the residual variance, the variance that is not explained by the family, actor, or partner factors (Cook 1994).
The second step of fitting the SRM consists of specifying two kinds of reciprocity correlations: individual and dyadic reciprocity correlations. The individual reciprocity correlations are specified by correlating each family member’s perceiver and target effects. If mothers’ perceiver and target effects are significantly correlated, it means that mothers who perceive other family members to be high in problem behavior are seen as high in problem behavior themselves. Dyadic reciprocity correlations are specified by correlating the residual terms representing two person’s unique relationships with each other. For example, adolescents who uniquely perceive their mothers to be high in problem behavior may be uniquely seen by their mothers to be high in problem behavior too. Individual reciprocity correlations were estimated separately for fathers, mothers, older adolescents, and younger adolescents. Dyadic reciprocity correlations were estimated for the father–mother, father–older adolescent, father–younger adolescent, mother–older adolescent, mother–younger adolescent, and older adolescent–younger adolescent dyads.
Model Evaluation
The fit of the models was evaluated using the χ2 overall model test, the Bentler comparative fit index (CFI), and the root mean square error of approximation (RMSEA). We used the CFI because it has been recommended for relatively small samples (Bentler 1990), and the RMSEA (Steiger 1990) because it is considered one of the best indicators of fit regardless of sample size. CFI values of .90 or greater reflect an adequate fit. Browne and Cudeck (1993) suggest that values of the RMSEA of .05 or less indicate a close fit, values between .05 and .08 indicate an adequate fit, whereas values greater than .10 indicate the need for improvement in the model.
Results
Data Preparation and Screening
Prior to conducting the analyses, the assumptions of normality were checked. In the externalizing problem behavior data, values of skewness ranged from .736 to 2.982 (from 1.028 to 2.564 in the second wave) and values of kurtosis ranged from .013 to 12.502 (from .655 to 9.332 in the second wave). After the logarithmic transformation of the externalizing problem behavior data, the maximum values of skewness and kurtosis were 1.306 and 1.942 (1.301 and 1.914 in the second wave), respectively. For the sake of clarity, the logarithmic transformation was also performed on the internalizing problem behavior data. This transformation resulted in maximum absolute values of −.751 (−1.075 second wave) for skewness and 1.088 (1.431 second wave) for kurtosis. As a result of the logarithmic transformation of the data, the values for skewness and kurtosis were acceptable. Most researchers consider values greater than 3.0 as “extremely” skewed and values from 8.0 to over 20.0 have been described as indicating “extreme” kurtosis (Kline 1998).
The means and standard deviations of the 12 measures of problem behavior are shown in Table 1, together with their intercorrelations (means and standard deviations are taken from the raw scores before the logarithmic transformation). Family members reported relatively low levels of problem behavior (scores between 2 and 3 on a 7-point Likert-scale for internalizing problem behavior and scores between 1 and 2 on the same scale for externalizing problem behavior). Furthermore, each family member’s ratings of different targets (i.e., the externalizing and internalizing problem behavior of the other three family members) were positively associated with each other. All three ratings (i.e., the ratings of fathers, mothers, and the siblings) on the externalizing and internalizing problem behavior of the older and younger adolescents were positively associated, except for the younger adolescent’s internalizing problem behavior. The three ratings (i.e., the ratings of the partner and both of the adolescent children) on father’s and mother’s externalizing and internalizing problem behavior were not significantly associated.
Table 1
Means and standard deviations of and correlations among ratings of problem behavior in 69 four-person families at wave 1
Relationship | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1. | Father–mother | – | .57** | .64** | .36** | .28* | .27* | .14 | .27* | .19 | .15 | .25* | .16 |
2. | Father–Ol.Ad. | .55** | – | .67** | .45** | .53** | .30* | .40** | .45** | .46** | .14 | .23 | .33** |
3. | Father–Yo.Ad. | .59** | .47** | – | .28* | .22 | .34** | .19 | .16 | .37** | .28* | .30* | .30* |
4. | Mother–father | .01 | .23 | .21 | – | .67** | .70** | .36** | .34** | .26* | .19 | .25* | .22 |
5. | Mother–Ol.Ad. | .09 | .56** | .16 | .58** | – | .81** | .51** | .54** | .42** | .12 | .12 | .30* |
6. | Mother–Yo.Ad. | .14 | .07 | .41** | .55** | .46** | – | .41** | .35** | .41** | .21 | .17 | .33** |
7. | Ol.Ad.–father | −.07 | −.07 | −.10 | .43** | .15 | .18 | – | .82** | .69** | .21 | .07 | .40** |
8. | Ol.Ad.–mother | .07 | −.09 | −.02 | .24* | .07 | .12 | .79** | – | .64** | .15 | .21 | .39** |
9. | Ol.Ad.–Yo.Ad. | .02 | .03 | .21 | .29* | .21 | .42** | .66** | .69** | – | .32** | .27* | .51** |
10. | Yo.Ad.–father | .29* | .11 | .09 | .00 | −.05 | −.02 | .11 | .10 | .05 | – | .84** | .71** |
11. | Yo.Ad.–mother | .33** | .04 | .14 | −.01 | −.09 | .04 | .13 | .25* | .13 | .91** | – | .68** |
12. | Yo.Ad.–Ol.Ad. | .17 | .36** | .13 | .15 | .33** | .03 | .03 | .07 | .04 | .68** | .63** | – |
M
| Int. Prb. | 2.67 | 2.84 | 2.88 | 2.82 | 2.86 | 2.90 | 2.74 | 2.70 | 2.90 | 2.34 | 2.39 | 2.44 |
Ext. Prb. | 1.45 | 1.70 | 1.60 | 1.48 | 1.71 | 1.59 | 1.60 | 1.53 | 1.91 | 1.51 | 1.41 | 1.97 | |
SD | Int. Prb. | .72 | .81 | .67 | .76 | .81 | .79 | .85 | .84 | 1.00 | .67 | .74 | .69 |
Ext. Prb. | .46 | .67 | .45 | .55 | .79 | .54 | .71 | .74 | .86 | .53 | .47 | .94 |
SRM Perceiver, Target, and Family Effects in Perceived Problem Behavior
First, we assessed with SRM analysis whether significant perceiver and target variances and a significant family variance could be found in family members’ perceptions of each other’s problem behavior. We tested two models, one for externalizing problem behavior and one for internalizing problem behavior. The fit of the externalizing problem behavior model was acceptable. The chi-square test was significant, χ2 (N = 69, df = 47) = 72.84, p = .01, indicating a lack of fit. However, a RMSEA of .08 and a CFI of .96 indicated that the model could be accepted without modification. For perceptions of internalizing problem behavior, the SRM model also fitted the data adequately. The chi-square test was significant, χ2 (N = 69, df = 47) = 64.62, p = .04, but a RMSEA of .06 and the CFI of .97 indicated an adequate fit.
Both models were analyzed again approximately ten months later to determine if the findings would replicate. In the second wave, the initial fit of the model for perceived externalizing problem behavior was not adequate. The modification indices indicated that allowing the residuals of fathers’ ratings on the problem behavior of the older and younger adolescents to covary would improve the fit of the model. This adjustment did not alter the significance of effects and only slightly changed their variance. The adjusted model resulted in χ2 (N = 69, df = 46) = 64.73, p = .04. The RMSEA was .07 and the CFI .97, which indicated that there was an adequate fit for the model. The fit of the model for perceived internalizing problem behavior did not differ much from the first wave: χ2 (N = 69, df = 47) = 64.46, p = .05, RMSEA of .04 and CFI of .96. Overall, these values suggest that the SRM fits the data on perceptions of internalizing problem behavior reasonably well in the second wave.
The variances of the SRM effects for perceived externalizing and internalizing problem behavior in both the first and second wave are shown in Table 2. The significance of the between-family variance in the various SRM effects was tested with a one-tailed T-test. As shown in the columns labelled “First Wave, Externalizing Problems” and “Second Wave, Externalizing Problems”, we found that all four perceiver and target effects (for fathers, mothers, older adolescents, and younger adolescents) and the family effect explained significant amounts of variance in perceptions of externalizing problem behavior in the first wave as well as in the second wave. This indicates that within-family perceptions of externalizing problem behavior are due to the way fathers, mothers, older adolescents, and younger adolescents perceive externalizing problem behavior in their family members (i.e., perceiver effects for fathers, mothers, older adolescents, younger adolescents); the degree to which fathers, mothers, older adolescents, and younger adolescents elicit perceptions of externalizing problem behavior in their family members (i.e., target effects for fathers, mothers, older adolescents, younger adolescents); and the general level of externalizing problem behavior that family members perceive among each other (i.e., family effect). Our hypothesis that perceiver and target effects and the family effect could be found in family members’ perceptions of each other’s problem behavior is thus supported for perceptions of externalizing problem behavior.
Table 2
Variance estimates for the social relations model analyses of perceived externalizing and internalizing problem behavior: perceiver, target, and family effects
Source of variance | Role | First wave | Second wave | ||
|---|---|---|---|---|---|
Externalizing problems | Internalizing problems | Externalizing problems | Internalizing problems | ||
Perceiver | Father | 0.042*** | 0.031*** | 0.039*** | 0.037*** |
Mother | 0.065*** | 0.030*** | 0.056*** | 0.029*** | |
Adolescent 1 | 0.085*** | 0.078*** | 0.071*** | 0.038*** | |
Adolescent 2 | 0.068*** | 0.083*** | 0.071*** | 0.076*** | |
Target | Father | 0.005* | 0.003 | 0.014** | 0.012** |
Mother | 0.010** | 0.010** | 0.007* | 0.004 | |
Adolescent 1 | 0.026*** | 0.035*** | 0.022*** | 0.033*** | |
Adolescent 2 | 0.014*** | 0.020*** | 0.006* | 0.018*** | |
Family | 0.018** | 0.011* | 0.018** | 0.012** | |
As can be seen from column in Table 2 labelled “First Wave, Internalizing Problems”, one can see that we found four significant perceiver effects (for fathers, mothers, older adolescents, and younger adolescents), three significant target effects (for mothers, older adolescents, and younger adolescents), and a significant family effect in perceptions of internalizing problem behavior in the first wave. The non-significant target effect for fathers indicates that across families, there was no variability in how much fathers are perceived to have internalizing problems. As can be seen in the column labelled “Second Wave, Internalizing Problems”, we also found four reliable perceiver effects (for fathers, mothers, older adolescents, and younger adolescents) and a reliable family effect in perceptions of internalizing problem behavior in the second wave. The variances of the target effects of fathers, older adolescents, and younger adolescents were significant, but the target effect of mothers was not significant. Thus, we found that within-family perceptions of internalizing problem behavior could be attributed to (a) the degree to which fathers, mothers, older adolescents, and younger adolescents perceive internalizing problem behavior in their family members (i.e., the perceiver effects), (b) the degree to which older adolescents and younger adolescents elicit perceptions of internalizing problem behavior in their family members (i.e., the target effects), and (c) the general level of internalizing problem behavior that family members perceive among each other (family effect). Within-family perceptions of the degree to which mothers elicit perceptions of internalizing problem behavior (i.e., target effect mothers) in the first wave did not replicate in the second wave. In the second wave, within-family perceptions of the degree to which fathers elicit perceptions of internalizing problem behavior (target effect fathers) were found that had not been significant in the first wave. For perceptions of internalizing problem behavior, our hypothesis is supported with respect to two target effects (for both adolescents), four perceiver effects (for fathers, mothers, older adolescents, and younger adolescents), and the family effect. Our hypothesis was only partially supported with regard to two target effects (for both adolescents) in perceptions of internalizing problem behavior.
Other findings that were not directly related to our hypotheses emerged from the estimation of individual and dyadic reciprocity correlations. We found some significant individual reciprocity correlations. For externalizing problem behavior, a perceiver-target correlation was found for older adolescents (r = .67, t = 6.64, p ≤ .01 first wave; r = .58, t = 5.36, p ≤ .01 second wave), indicating that older adolescents who perceive more externalizing problem behavior in other family members tend to be perceived by their family members as higher in externalizing problem behavior. In the first wave, we also found a perceiver-target correlation for younger adolescents (r = .29, t = 2.42, p ≤ .05) with respect to externalizing problem behavior. This implies that younger adolescents who perceive more externalizing problem behavior in other family members tend to be perceived by their family members as higher in externalizing problem behavior.
In the second wave, the perceiver-target correlations for older adolescents (r = −0.31, t = −2.58, p ≤ .01) and younger adolescents (r = −.29, t = −2.48, p ≤ .01) were significant for internalizing behavior. Older and younger adolescents who perceive more internalizing problem behavior in other family members tend to be perceived by their family members as lower in internalizing problem behavior. We found no significant dyadic reciprocity correlations for perceptions of externalizing and internalizing problem behavior in either the first or the second wave.
The Relative Contributions of Perceiver, Target, and Family Effects in Perceptions of Externalizing and Internalizing Problem Behavior
The percentage of variance explained by the SRM effects was assessed for each of the 12 family members’ perceptions of each other’s problem behavior. For example, the total variance in fathers’ perceptions of mothers’ externalizing problem behavior is the sum of the perceiver variance for fathers (see Table 2, .042), the target variance for mothers (see Table 2, .010), the family variance (see Table 2, .018), and any remaining unexplained variance (.301, not in Table 2). In this case the sum is .371. The relative perceiver variance in fathers’ perceptions of mothers’ externalizing problem behavior is computed by dividing the fathers’ perceiver variance by the total variance (i.e., .042/.371 = 11.32%). The contributions of the different SRM effects to the variance in perceptions of externalizing problem behavior as well as in perceptions of internalizing problem behavior were next averaged for all 12 family members’ perceptions of each other’s problem behavior. In Table 3, the relative amounts of variance accounted for by the SRM perceiver and target effects and the family effect are presented.
Table 3
The relative amounts of variance in perceptions of externalizing and internalizing problem behavior accounted for by the srm perceiver, target, and family effects
Source of variance | First wave | Second wave | ||
|---|---|---|---|---|
Externalizing problems (%) | Internalizing problems (%) | Externalizing problems (%) | Internalizing problems (%) | |
SRM perceiver effects | 30.69 | 25.39 | 24.06 | 20.39 |
SRM target effects | 6.41 | 7.85 | 4.46 | 7.50 |
SRM family effects | 8.64 | 4.81 | 7.27 | 5.03 |
Unexplained/SRM relationship effects | 54.25 | 61.95 | 64.23 | 67.09 |
As can be seen in Table 3, SRM perceiver variance explained 24–31% of the variance in perceptions of externalizing problem behavior and 20–25% in perceptions of internalizing problem behavior. SRM target variance explained 4.5–6.5% of the variance in perceptions of externalizing problem behavior and 7.5–8% in perceptions of internalizing problem behavior. Finally, SRM family variance explained 7–9% of the variance in perceptions of externalizing problem behavior, and 5% of the variance in perceptions of internalizing problem behavior. Our hypothesis that the absolute value of the amount of variance explained in family members’ perceptions of problem behavior by target effects is greater for judgments of externalizing problem behavior than for judgments of internalizing problem behavior was not supported. We did not find evidence for greater agreement between family members in their ratings of externalizing problem behavior compared with their ratings of internalizing problems. Our findings suggest a tendency of greater correspondence in perceptions of internalizing problem behavior.
About 54–64% of the variance in externalizing problem behavior and 62–67% of the variance in internalizing problem behavior is unexplained. Considering that the systematic relationship variance became part of the residual variance (see section “Method”), part of this variance can be accounted for by characteristics of specific relationships between family members (i.e., SRM relationship effects). However, because these components also contain non-systematic (error) variance, we can not draw conclusions about their relative importance. For those components only containing systematic variance, the largest absolute value of the amount of variance explained in family members’ perceptions of problem behavior is accounted for by the SRM perceiver effects. For externalizing behavior, the absolute value of the amount of variance explained by the SRM family effect is slightly greater than the average for the SRM target effects, whereas for internalizing problem behavior the absolute value of the amount of variance explained by the SRM target effects is slightly greater than by the SRM family effect.
Discussion
Kenny’s (1994) model of interpersonal perception proved to be a valuable means of understanding knowledgeable informant reports of problem behavior. The model enabled us to identify the sources of informant disagreement on these reports. SRM analysis showed that within-family perceptions of externalizing and internalizing behaviors are consistently due to three sources of variance: perceiver, target, and family effects.
Perceiver effects were shown to affect family member’s reports on externalizing and internalizing problem behavior. The SRM perceiver variances were significant for each family member with respect to both perceptions of externalizing and internalizing problem behavior. These results were replicated in the second wave. This suggests that even though family members are well-acquainted with each other and therefore, in theory, should be able to differentiate between each other, the perceiver still rates the problem behavior of different family members rather in the same way. This could reflect a process of assimilation, the tendency to perceive others according to preconceptions of what others, in general, are like (i.e., a schema). These preconceptions may reflect individual differences in how the person generally sees others or in how the person sees the specific group being targeted, in this case the family (e.g. people in my family show high problem behavior). For the perception of family members, the self likely serves as a basis for creating assimilation (Kenny 1994). This is called assumed similarity (perceivers think others are similar to them). The perceiver’s own problem behavior may serve as a basis for rating the problem behavior of family members. The process of assimiliation and assumed similarity may explain previous findings that levels of psychopathology, particularly depression, may influence parents’ ratings of children’s behavioral problems, especially of children’s internalizing problems (see e.g., De Los Reyes and Kazdin 2005; Kroes et al. 2003). Mothers with psychopathological symptoms, for example, may over-report their children’s internalizing problems because they think their children are similar to them (assumed similarity). The projection hypothesis (i.e. Morretti et al. 1985) seems consistent with this. This hypothesis assumes that mothers may project symptoms of their own psychological states in their children. Another potential explanation could be that the symptoms of these mothers may lead to a preconception or schema in which their family member’s internalizing problems is overstated (process of assimilation). This is consistent with research which has shown cognitive biases in reaction to ambiguous stimuli by anxious and aggressive parents (Barrett et al. 1996; Strassberg 1997).
We found that within-family perceptions of externalizing and internalizing problem behavior can be explained by characteristics of individual family members on whose problem behaviors are reported (i.e. the targets). The SRM target variances were significant for each family member with respect to perceptions of externalizing problem behavior and for both of the adolescents in perceptions of internalizing problem behavior in both waves. Generally, these findings show that different family members agree on the problem behavior of individual family members. So, we measured what we were hoping to measure when we asked family members to judge each other, that is, characteristics of the family member being judged. This is in agreement with the correlations (although modest) that clinical researchers have found between different family members’ ratings of a child or adolescent’s problem behavior (see e.g., Achenbach et al. 1987; Van Der Ende and Verhulst 2005). Characteristics of the targeted family members (e.g. the problem behavior of children or adolescents) could have produced (small amounts of) agreement in family members’ ratings of the problem behavior. However, for perceived internalizing problem behavior, the target variance for fathers in the first wave and the target variance for mothers in the second wave were not significant. Consequently, in some instances differences in perceptions of internalizing problem behavior are not all explained by characteristics of the person being rated. It may be that the internalizing problem behavior of fathers and mothers is not always observable by all family members because parents may not show their emotional problems to their children.
Finally and importantly, we found significant SRM family variance in family members’ perceptions of each other’s externalizing and internalizing problem behavior, both in the first as well as in the second wave. Thus, family members perceptions of each other’s externalizing and internalizing problem behavior can be explained by characteristics of the family as a group. The application of the SRM enabled us to show that characteristics of the family affect an individual’s perceptions of externalizing and internalizing problem behavior when characteristics of individual family members (i.e. SRM perceiver and target effects) are taken into account. A family or contextual effect on informant reports of problem behavior has not been previously reported, but is in accordance with evidence that characteristics of the family such as ethnicity or race, family stress, and SES are related to family members’ judgments of a child or adolescent’s problem behavior (see e.g., De Los Reyes and Kazdin 2005). It is also consistent with measures of family functioning that assume that family members are similar in some way. Because family members may be similar with respect to externalizing and internalizing problem behavior, the group-level effect affects family members’ judgments of problem behavior. Problem behavior may run in families, and similarity among family members is a significant determinant of problem behavior ratings.
We did not find evidence of greater agreement between family members in their ratings of externalizing problem behavior compared with their ratings of internalizing problems. Our findings suggest a tendency of greater correspondence in perceptions of internalizing problem behavior. This finding is difficult to explain. It is in contrast with previous studies that found greater levels of correspondence for informants’ ratings of child externalizing problem behavior compared with informants’ ratings of internalizing problems (Achenbach et al. 1987; Duhig et al. 2000).). It is possible that previous studies might have come to different findings if effects due to characteristics of family members who report on family members (i.e. perceiver effects) and contextual effects of whole-family functioning (i.e. family effect) had been controlled as in our study. Another reason is that in this study relatively well-functioning families participated that show a wider range of internalizing than externalizing problem behavior.
Other findings that were not directly related to our hypotheses emerged from the estimation of individual and dyadic reciprocity correlations. The results for the individual reciprocity correlations with respect to externalizing problem behavior show that adolescents who perceive other family members as high in externalizing problem behavior are perceived by those family members to be high on problem behavior themselves. Baumrind (1980) described how such feedback loops may occur within families: “Within a reciprocal and interacting system such as the family, individuals produce by their own behavior the environmental conditions that affect their own as well as others’ behavior. One person’s behavior is simultaneously a response to environmental stimuli and a stimulus to others’ responses within the interactive system of social exchange” (p. 640). Internalizing problem behavior of adolescent family members does not appear to be reciprocally determined. In the second wave, adolescents who perceived less internalizing problem behavior in their family members were perceived as higher on internalizing problem behavior themselves and vice versa. A social comparison process seems to be operating with respect to internalizing problems of adolescents. Internalizing adolescents may use themselves as a baseline for rating the internalizing problem behavior of others. Thus, the more adolescents perceive themselves as having internalizing problems, the less they may rate others as having internalizing problem behavior. It could also be that being “internally oriented” makes it more difficult to “see” these problems in other people, at least for adolescents. This conclusion cannot be drawn firmly, however, because we did not find individual reciprocity correlations for adolescents with respect to internalizing problem behavior in the first wave.
Several limitations of this study should be noted. Kashy and Kenny (1990) have suggested that as few as 50 families may be adequate for SRM analysis. Nonetheless, our study would be stronger and more sensitive if it would have had a larger sample than 69 families. We also have to stress that the generalizability of this study is limited to relatively high functioning intact families with adolescent children who participated in this study. Results may be different for low-functioning families, families with a more varied composition, and families with younger children. Finally, the SRM provides information about the sources of variance in perceptions of problem behavior, but it does not identify the constructs that cause or explain the variance. For example, if mother perceives her adolescent child in particular as high in problem behavior, we do not know what specific characteristics of the family interpersonal relationship between mother and her adolescent child (e.g., mother or her adolescent child’s gender, parenting style) elicit this perception in mother. Another limitation of the study is the little variance in problem behavior. Despite the fact that there was little variance in problem behaviour, the between-family variance in the various SRM effects was significant. Almost all SRM effects explained significant amounts of variance in perceptions of externalizing and internalizing problem behaviour. Notwithstanding the limitations mentioned, this study advances the literature on problem behavior by identifying the sources of informant disagreement on knowledgeable informant reports of problem behavior by means of Kenny’s (1994) model of interpersonal perception. A family or contextual effect on informant reports of problem behavior has not been previously reported.
In general, the level of target variance is relatively low in this study and the perceiver variance is somewhat higher compared to the studies of Kenny (1994), and Branje et al. (2003). Most variance is explained by perceiver effects, followed by target or family effects dependent on the type of problem behavior. Our study shows that characteristics of perceivers (for the biggest part) and the family significantly affect family members’ ratings of problem behavior. As described in the introduction this may have important implications for the diagnosis and treatment of children’s behavioral problems because informant ratings on instruments such as the Child Behavior Checklist (CBCL; Achenbach 1991) are widely used to assess children’s problem behavior and, thus, to guide clinical interventions and assess clinical outcomes. If the rater or family environment is disturbed and these ratings are dependent on them, the ratings would be fatally flawed. Therefore, this study underscores the importance of not only assessing the behavior of the target person, but also assessing characteristics of the rater and the family when measures obtained from knowledgeable informants are to be used.
We would conclude that the clinical use of family members’ ratings to assess problem behavior of individuals is not valid, because these ratings are confounded by perceiver and family effects. Considering that invalid ratings of an individual’s problem behaviour may have important negative implications for the diagnosis and treatment of individuals, it is important to separate the true score variance representing characteristics of the targets from the potentially distorting effects of perceivers and the family. The SRM or latent variable approach can be an effective means to handle measurement problems by removing confounding sources of variance (perceiver and family effects) from the measure of an individual’s problem behaviour (the partner effect). SRM analysis can be used in applied contexts, for example, to assess the problem behaviour of a single family member (see e.g., Cook and Kenny 2004). To compare individual families and family members with each other, however, standard scores have to be developed. Further research will need to focus on developing standard scores for specific family configurations and age groups.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Open AccessThis is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.