Skip to main content
SpringerLink
Log in

Factors Influencing the Reliability and Validity of Observation Data

  • Chapter
Family Interaction and Psychopathology

Part of the book series: Applied Clinical Psychology ((NSSB))

Abstract

During the past 3 decades, researchers and clinicians alike have become increasingly interested in the nature of family interaction in their attempts to understand, modify, and prevent such diverse maladaptations as schizophrenia, depression, alcoholism, and childhood aggression. Overall, such investigations have sought to (a) construct and evaluate theoretical models of marital and parent—child behavior with the aim of understanding the development and functioning of family systems; (b) identify family interactions that reliably differentiate problem from nonproblem families as a first step toward developing more effective methods of treatment and prevention; and (c) assess the influence of various treatment approaches on the nature of intrafamilial interactions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Anthony, N. (1971). Comparisons of clients’ standard, exaggerated, and matching MMPI profiles. Journal of Consulting and Clinical Psychology, 27, 253–256.

    Google Scholar 

  • Aragona, J., and Eyberg, S. (1981). Neglected children: Mother’s report of child behavior problems and observed verbal behavior. Child Development, 52, 596–602.

    Article  PubMed  Google Scholar 

  • Arrington, E. (1943). Time sampling in studies of social behavior: A critical review of technique and results with research suggestions. Psychological Bulletin, 40, 81–124.

    Article  Google Scholar 

  • Bales, R. F. (1950). Interaction Process Analysis. New York: Addison-Wesley.

    Google Scholar 

  • Bell, R. Q. (1964). Structuring parent—child interaction for direct observation. Child Development, 35, 1009–1020.

    PubMed  Google Scholar 

  • Belsky, J. ( 1977, March). Mother—infant interaction at home and in the laboratory: The effect of context. Paper presented at the Society for Research in Child Development conference, New Orleans.

    Google Scholar 

  • Bernal, M. F., Gibson, D. M., Williams, D. E., and Pesses, D. I. (1971). A device for automatic audio tape recording. Journal of Applied Behavior Analysis, 4, 151–156.

    Article  PubMed  Google Scholar 

  • Blurton Jones, N. G., and Woodson, R. H. (1979). Describing behavior: The ethologists’ perspective. In M. E. Lamb, S. J. Suomi, and G. R. Stephenson (Eds.), Social Interaction Analysis. Madison: University of Wiscons in Press.

    Google Scholar 

  • Borduin, C. M., and Henggeler, S. W. (1981). Social class, experimental setting, and task characteristics as determinants of mother-child interaction. Developmental Psychology, 17, 209–214.

    Article  Google Scholar 

  • Bronfenbrenner, U. (1974). Developmental research, public policy, and the ecology of childhood. Child Development, 45, 1–5.

    Article  Google Scholar 

  • Brookhart, J., and Hock, E. (1976). The effects of experimental context and experimental background on infants’ behavior toward their mothers and a stranger. Child Development, 47, 333–340.

    Article  Google Scholar 

  • Campbell, D. T., and Stanley, J. C. (1963). Experimental and quasi-experimental designs for research and teaching. In N. L. Gage (Ed.), Handbook of research on teaching. Chicago: Rand McNally.

    Google Scholar 

  • Canter, F. (1963). Simulation of the California Personality Inventory and the adjustment of the simulator. Journal of Consulting Psychology, 27, 253–256.

    Article  PubMed  Google Scholar 

  • Carroll, J. B. (1961). The nature of the data, or how to choose a correlation coefficient. Psychometrika, 26, 347–372.

    Article  Google Scholar 

  • Christensen, A., and Hazzard, A. (1983). Reactive effects during naturalistic observation of families. Behavioral Assessment, 5, 349–362.

    Google Scholar 

  • Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213–220.

    Article  PubMed  Google Scholar 

  • Cohen, J. (1969). Statistical power analysis for the behavioral sciences. New York: Academic Press.

    Google Scholar 

  • Cohen, J. (1972). Weighted chi square: An extension of the kappa method. Educational and Psychological Measurement, 32, 61–74.

    Article  Google Scholar 

  • Cone, J. D. (1977). The relevance of reliability and validity for behavioral assessment. Behavior Therapy, 8, 411–426.

    Article  Google Scholar 

  • Conte, J. R. (1979). An experimental investigation of subject reactivity to observation of video camera and human observer (Doctoral dissertation, University of Washington). Dissertation Abstracts International, 40 (6-A), 3535A.

    Google Scholar 

  • Cronbach, L. J., Gleser, G. C., Nanda, H., and Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley.

    Google Scholar 

  • Crowne, D. D., and Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting and Clinical Psychology, 24, 349–354.

    Article  Google Scholar 

  • Eyberg, S., and Robinson, E. (1983). Dyadic Parent Child Interaction Coding System: A manual. Psychological Documents, 13. ( Ms. No. 2582 )

    Google Scholar 

  • Fenigstein, A., Scheier, M. F., and Buss, A. H. (1975). Public and private self-consciousness: Assessment and theory. Journal of Consulting and Clinical Psychology, 43, 522–527.

    Article  Google Scholar 

  • Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 378–382.

    Article  Google Scholar 

  • Fleiss, J. I.., Nee, J. C. M., and Landis, J. R. (1979). The large sample variance of kappa in the case of different sets of raters. Psychological Bulletin, 86, 974–977.

    Article  Google Scholar 

  • Floyd, F. J., and Markman, H. J. (1983). Observational biases in spouse observation: Toward a cognitive/behavioral model of marriage. Journal of Consulting and Clinical Psychology, 51, 450–457.

    Google Scholar 

  • Glass, G. V., and Stanley, J. C. (1970). Statistical methods in education and psychology ( 2nd ed. ). Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Goldfried, M., and Kent, R. (1972). Traditional versus behavioral personality assessment: A comparison of methodological and theoretical assumptions. Psychological Bulletin, 77, 409–420.

    Article  PubMed  Google Scholar 

  • Gottman, J. M. (1980). Analyzing for sequential connection and assessing interobserver reliability for the sequential analysis of observational data. Behavioral Assessment, 2, 361–368.

    Google Scholar 

  • Hadley, T., and Jacob, T. (1973). Relationship among four measures of family power. Journal of Personality and Social Psychology, 27, 6–12.

    Article  Google Scholar 

  • Hadley, T. R., and Jacob, T. (1976). The measurement of family power: A methodological study. Sociometry, 39, 384–395.

    Article  Google Scholar 

  • Haley, J. (1967). Cross-cultural experimentation: An initial attempt. Human Organization, 26, 110–117.

    Google Scholar 

  • Hannum, J. W., and Mayer, J. M. (1984). Validation of two family assessment approaches. journal of Marriage and Family, 46, 741–748.

    Article  Google Scholar 

  • Harris, A. (1969). Observer effect on family interaction. Unpublished doctoral dissertation, University of Oregon.

    Google Scholar 

  • Harris, F. C., and Lahey, B. B. (1982). Subject reactivity in direct observational assessment: A review and critical analysis. Clinical Psychology Review, 2, 523–538.

    Article  Google Scholar 

  • Hartmann, D. P. (1977). Considerations in the choice of interobserver reliability estimates. journal of Applied Behavior Analysis, 10, 103–116.

    Article  PubMed  Google Scholar 

  • Haynes, S. N., and Horn, W. F. (1982). Reactivity in behavioral observation: A review. Behavioral Assessment, 4, 369–385.

    Google Scholar 

  • Haynes, S. N., Chavez, R. E., and Samuel, V. (1984). Assessment of marital communication distress. Behavioral Assessment, 6, 315–321.

    Google Scholar 

  • Henggeler, S. W., Borduin, C. M., Rodick, J. D., and Tavormina, J. (1979). Importance of task content for family interaction research. Developmental Psychology, 15, 660–661.

    Article  Google Scholar 

  • Herbert, J., and Attridge, C. (1975). A guide for developers and users of observation systems and manuals. The American Education Research journal, 12, 1–20.

    Article  Google Scholar 

  • Hollenbeck, A. R. (1978). Problems of reliability on observational research. In G. Sackett (Ed.), Observing behavior: Vol. 2. Data collection and analysis methods. Baltimore: University Park Press.

    Google Scholar 

  • Hoover, L. K., and Rinehart, H. H. (1968). The effect of an outside observer on family interaction. Unpublished manuscript.

    Google Scholar 

  • Hughes, H. M., and Haynes, S. N. (1978). Structured laboratory observation in the behavioral assessment of parent-child interactions: A methodological critique. Behavior Therapy, 9, 428–447.

    Google Scholar 

  • Jacob, T. (1975). Family interaction in normal and disturbed families: A methodological and substantive review. Psychological Bulletin, 82, 33–65.

    Article  PubMed  Google Scholar 

  • Jacob, T. (1976). Behavioral assessment of marital dysfunction. In M. Hersen and A. Bellack (Eds.), Behavioral assessment: A practical handbook. New York: Pergamon Press.

    Google Scholar 

  • Jacob, T., and Davis, J. (1973). Family interaction as a function of experimental task. Family Process, 12, 415–427.

    Article  Google Scholar 

  • Jacob, T., Grounds, L., and Haley, R. (1982). Correspondence between parents’ reports on the Behavior Problem Checklist. journal of Abnormal Child Psychology, 4, 593–608.

    Article  Google Scholar 

  • Jacob, T., Rushe, R. H., and Tennenbaum, D. L. (1986). Alcoholism and family interaction: An experimental paradigm. Unpublished manuscript, University of Pittsburgh.

    Google Scholar 

  • Johnson, S. M., and Bolstad, O. D. (1973). Methodological issues in naturalistic observation: Some problems and solutions for field research. In L. A. Hamerlynck, L. C. Handy, and E. J. Mash (Eds.), Behavior change: Methodology, concepts and practice. Champaign, IL: Research Press.

    Google Scholar 

  • Johnson, S. M., and Bolstad, O. D. (1975). Reactivity to home observation: A comparison of audio recorded behavior with observers present or absent. Journal of Applied Behavioral Analysis, 8, 181–185.

    Article  Google Scholar 

  • Johnson, S. M., Christensen, A., and Bellamy, G. T. (1976). Evaluation of family intervention through unobtrusive audio recordings: Experiences in “bugging” children. journal of Applied Behavior Analysis, 9, 213–219.

    Google Scholar 

  • Jones, R. R., Reid, J. B., and Patterson, G. R. (1975). Naturalistic observation in clinical assessment. In P. McReynolds (Ed.), Advances in Psychological Assessment (Vol. 3). San Francisco: Jossey-Bass.

    Google Scholar 

  • Kazdin, A. E. (1977). Artifact, bias, and complexity of assessment: The ABC’s of reliability. Journal of Applied Behavior Analysis, 10, 141–150.

    Article  PubMed  Google Scholar 

  • Kazdin, A. E. (1982). Observer effects: Reactivity of direct observation. In D. P. Hartman (Ed.), Using observers to study behavior. San Francisco: Jossey-Bass.

    Google Scholar 

  • Kent, R. N., Kanowitz, J., O’Leary, K. D., and Cheiken, M. (1977). Observer reliability as a function of circumstances of assessment. journal of Applied Behavioral Analysis, 10, 317–324.

    Article  Google Scholar 

  • Krahn, G. L., and Gabriel, R. M. (1984). Quantifying categorical observations of social interactions through multidimensional scaling. Developmental Psychology, 20, 833–843.

    Google Scholar 

  • Lytton, H. (1974). Comparative yield of three data sources in the study of parent-child interaction. Merrill-Palmer Quarterly, 20, 53–64.

    Google Scholar 

  • Lytton, H. (1979). Disciplinary encounters between young boys and their mothers and fathers: Is there a contingency system? Developmental Psychology, 15, 256–268.

    Article  Google Scholar 

  • Margolin, G. (1978). Relationships among marital assessment procedures: A correlational study. Journal of Consulting and Clinical Psychology, 46, 1556–1558.

    Article  Google Scholar 

  • Margolin, G., and Weiss, R. L. (1978). Comparative evaluation of therapeutic components associated with behavioral marital treatments. Journal of Consulting and Clinical Psychology, 46, 1476–1486.

    Article  PubMed  Google Scholar 

  • Mash, E. J., and Johnston, C. (1982). A comparison of the mother-child interactions of younger and older hyperactive and normal children. Child Development, 53, 1371–1381.

    Article  PubMed  Google Scholar 

  • Mezzich, J. E., Kraemer, H. C., Worthington, D. R. L., and Coffman, G. A. (1981). Assessment of agreement among several raters formulating multiple diagnoses. Journal of Psychiatric Research, 16, 29–39.

    Article  PubMed  Google Scholar 

  • Mitchell, S. K. (1979). Interobserver agreement, reliability and generalizability of data collected in observational studies. Psychological Bulletin, 86(2), 376–390.

    Google Scholar 

  • Moos, R., and Moos, B. S. (1981). Family Environment Scale: Manual. Palo Alto: Consulting Psychologists Press.

    Google Scholar 

  • Murrell, S. A. (1971). Family interaction variables and adjustment of nonclinic boys. Child Development, 42, 1485–1494.

    Article  Google Scholar 

  • Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill.

    Google Scholar 

  • Oliven, M. E., and Reiss, D. (1984). Family concepts and their measurement: Things are seldom what they seem. Family Process, 23, 33–48.

    Article  Google Scholar 

  • Olson, D. H. (1985). Commentary: Struggling with congruence across theoretical models and methods. Family Process, 24, 203–207.

    Article  Google Scholar 

  • Olson, D. H., and Portner, J. (1983). Family adaptability and cohesion evaluation scales. In E. E. Filsinger (Ed.), Marriage and family assessment. Beverly Hills, CA: Sage Publications.

    Google Scholar 

  • Olson, D. H., and Ryder, R. G. (1970). Inventory of Marital Conflicts (IMC): An experimental interaction procedure. Journal of Marriage and the Family, 32, 443–448.

    Article  Google Scholar 

  • O’Neill, M. S., and Alexander, J. F. (1970, April). Family interaction as a function of task characteristics. Paper presented at Rocky Mountain Psychological Association, Salt Lake City, UT.

    Google Scholar 

  • O’Rourke, V. (1963). Field and laboratory: The decision-making behavior of family groups in two experimental conditions. Sociometry, 26, 422–435.

    Article  Google Scholar 

  • Patterson, G. R. (1982). A social learning approach: Vol. 3. Coercive family process. Eugene, OR: Castalia.

    Google Scholar 

  • Patterson, G. R., and Cobb, J. A. (1971). A dyadic analysis of “aggressive” behavior. In J. P. Hill (Ed.), Minnesota symposia on child psychology (Vol. 5 ). Minneapolis: University of Minnesota Press.

    Google Scholar 

  • Patterson, G. R., and Moore, D. (1979). Interactive patterns as units of behavior. In M. E. Lamb, S. J. Suomi, Sc G. R. Stephenson (Ed.), Social interaction analysis: Methodological issues. Madison: University of Wiscons in Press.

    Google Scholar 

  • Patterson, G. R., Sc Reid, J. B. (1970). Reciprocity and coercion: Two facets of social systems. In C. Neuringer and J. D. Michael (Eds.), Behavior modification in clinical psychology. New York: AppletonCentury-Crofts.

    Google Scholar 

  • Roberts, R., Jr., and Renzaglia, A. (1965). The influence of tape recording on counseling. Journal of Counseling Psychology, 12, 10–16.

    Google Scholar 

  • Robinson, E. A., and Price, M. G. (1980). Pleasurable behavior in marital interaction: An observation study. Journal of Consulting and Clinical Psychology, 48, 117–118.

    Google Scholar 

  • Rosenberg, M. J. (1969). The conditions and consequences of evaluation apprehension. In R. Rosenthal and R. L. Roshow (Eds.), Artifact in behavioral research. New York: Academic Press.

    Google Scholar 

  • Rosenthal, R. (1966). Experimenter effects in behavioral research. New York: Appleton-Century.

    Google Scholar 

  • Ross, G., Kagan, J., Zelazo, P., and Kotelchuck, M. (1975). Separation protest in infants in home and laboratory. Developmental Psychology, 11, 256–257.

    Article  Google Scholar 

  • Scott, W., and Wertheimer, M. (1962). Introduction to psychological research. New York: Wiley.

    Google Scholar 

  • Sigafoos, A., Reiss, D., Rich, J., and Douglas, E. (1985). Pragmatics in the measurement of family functioning: An interpretive framework for methodology. Family Process, 24, 189–203.

    Article  PubMed  Google Scholar 

  • Skinner, H., Steinhauer, P., and Santa-Barbara, J. (1983). Family Assessment Measure. Toronto, Ontario: Addiction Research Foundation.

    Google Scholar 

  • Skinner, H. A., Steinhauer, P. D., and Santa-Barbara, J. (1983). The Family Assessment Measure. Canadian Journal of Community Mental Health, 3 (2), 91–104.

    Google Scholar 

  • Stein, S.J., Girodo, M., and Dotzenroth, S. (1982). The interrelationships and reliability of a multilevel behavior-based assessment package for distressed couples. Journal of Behavioral Assessment, 4, 343–360.

    Google Scholar 

  • Steinglass, P. (1980). Assessing families in their own homes. American journal of Psychiatry, 137, 1523–1529.

    PubMed  Google Scholar 

  • Susman, E. J., Peters, D. J., and Steward, R. (1976). Naturalistic observational child study: A review. Paper presented at the 4th Biannual Southeastern Conferences on Human Development, Nashville.

    Google Scholar 

  • Tennenbaum, D. L. (1980). The effect of observer salience on family interaction in the home. Unpublished master’s thesis, University of Pittsburgh.

    Google Scholar 

  • Tennenbaum, D. L., and Jacob, T. ( 1985, November). An investigation of reactivity effects in spouse observation. Poster presented at Association for the Advancement of Behavior Therapy ( AABT) Convention, Houston.

    Google Scholar 

  • Tronick, E., Als, H., and Brazelton, T. B. (1977). Mutuality in mother-infant interaction. journal of Communication, 7, 74–79.

    Article  Google Scholar 

  • Vidich, A. J. (1956). Methodological problems in the observation of husband-wife interactions. Marriage and Family Living, 18, 234–239.

    Article  Google Scholar 

  • Vincent, J. P., Friedman, L. C., Nugent, J., and Messerly, L. (1979). Demand characteristics in observations of marital interaction. Journal of Consulting and Clinical Psychology, 47, 557–566.

    Article  PubMed  Google Scholar 

  • Volkin, J. I., and Jacob, T. (1981). The impact of spouse monitoring on target behavior and recorder satisfaction. Journal of Behavioral Assessment, 3, 99–109.

    Article  Google Scholar 

  • Watzlawick, P. (1966). A structured family interview. Family Process, 5, 256–271.

    Article  Google Scholar 

  • Weber, S. J., and Cook, T. D. (1972). Subject effects in laboratory research: An examination of subject roles, demand characteristics, and valid interference. Psychological Bulletin, 77, 273–295.

    Article  Google Scholar 

  • Weinrott, M. R., Garrett, B., and Todd, N. (1978). The influence of observer presence on classroom behavior. Behavior Therapy, 9, 900–911.

    Article  Google Scholar 

  • Weiss, R. L., and Perry, B. A. (1983). The Spouse Observation Checklist: Development and clinical application. In E. E. Filsinger (Ed.), Marriage and family assessment. Beverly Hills, CA: Sage Publications.

    Google Scholar 

  • Weiss, R. L., and Summers, K. J. (1983). Marital Interaction Coding System-Ill. In E. Filsinger (Ed.), Marriage and family assessment. Beverly Hills, CA: Sage Publications.

    Google Scholar 

  • Wells, K. C., McMahon, R. J., Forehand, R., and Griest, D. L. (1980). Effect of a reliability observer on the frequency of positive parent behavior recorded during naturalistic parent-child interactions. Journal of Behavioral Assessment, 2, 65–69.

    Article  Google Scholar 

  • White, G. D. (1973). Effects of observer presence on mother and child behavior. Unpublished doctoral dissertation, University of Oregon, Eugene.

    Google Scholar 

  • White, G. D. (1977). The effects of observer presence on the activity level of families. Journal of Applied Behavior Analysis, 10, 734.

    Article  Google Scholar 

  • Wieder, G. B., and Weiss, R. L. (1980). Generalizability theory and the coding of marital interactions. Journal of Consulting and Clinical Psychology, 48, 469–477.

    Article  PubMed  Google Scholar 

  • Wiggins, J. S. (1973). Observational techniques: 1. Generalizability and facets of observation. Personality and prediction principles of personality assessment. Reading, MA: Addison-Wesley.

    Google Scholar 

  • Yarrow, M. R., and Waxier, C. Z. (1979). Observing interaction: A confrontation with methodology. In R. B. Cairns, (Ed.), The analysis of social interaction: Methods, issues, and illustrations. New York: Erlbaum.

    Google Scholar 

  • Zajonc, R. B. (1965). Social facilitation. Science, 149, 269–274.

    Article  PubMed  Google Scholar 

  • Zegiob, L. E., and Forehand, R. (1975). Maternal interactive behavior as a function of socioeconomic status, race and sex of child. Child Development, 46, 564–568.

    Article  Google Scholar 

  • Zegiob, L. E., and Forehand, R. (1978). Parent-child interactions: Observer effects and social class differences. Behavior Therapy, 9, 118–123.

    Article  Google Scholar 

  • Zegiob, L. E., Arnold, S., and Forehand, R. (1975). An examination of observer effects in parent-child interactions. Child Development, 46, 507–512.

    Google Scholar 

  • Zuckerman, E., and Jacob, T. (1979). Task effects in family interaction. Family Process, 18, 47–53.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Family Studies, 210 Family and Consumer Resources Building, University of Arizona, Tuscon, AZ, 85721, USA

    Theodore Jacob

  2. Department of Psychiatry, University of Pittsburgh, Pittsburgh, PA, 15213, USA

    Daniel L. Tennenbaum

  3. Crippled Children’s Division, Oregon Health Sciences University, Portland, OR, 97202, USA

    Gloria Krahn

Authors
  1. Theodore Jacob
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel L. Tennenbaum
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gloria Krahn
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Arizona, Tucson, Arizona, USA

    Theodore Jacob

Rights and permissions

Reprints and permissions

Copyright information

© 1987 Springer Science+Business Media New York

About this chapter

Cite this chapter

Jacob, T., Tennenbaum, D.L., Krahn, G. (1987). Factors Influencing the Reliability and Validity of Observation Data. In: Jacob, T. (eds) Family Interaction and Psychopathology. Applied Clinical Psychology. Springer, Boston, MA. https://doi.org/10.1007/978-1-4899-0840-7_8

Download citation

  • DOI: https://doi.org/10.1007/978-1-4899-0840-7_8

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4899-0842-1

  • Online ISBN: 978-1-4899-0840-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation