Top

Gepubliceerd in:

01-11-2014 | Original Paper

Between Black and White: Examining Grey Literature in Meta-analyses of Psychological Research

Auteurs: Sarah P. Bellefontaine, Catherine M. Lee

Gepubliceerd in: Journal of Child and Family Studies | Uitgave 8/2014

Abstract

The inclusion of grey literature in meta-analyses and reviews is controversial. We examine both the advantages and challenges of including grey literature in meta-analyses. An exemplar meta-analysis of behavioral parenting interventions on parent behavior, child behavior, and parent adjustment outcomes is used to demonstrate these issues. It also explores the influence of the inclusion of grey literature on outcomes, including whether effect sizes are affected by the inclusion of grey literature and describes the challenges of searching for grey literature using traditional search engines such as Google and Yahoo. Homogeneity and publication bias are also examined. Based on these results, recommendations are presented for meta-analysts and researchers.

vorige artikel Change in Age-Specific, Psychosocial Correlates of Risky Sexual Behaviors Among Youth: Longitudinal Findings From a Deep South, High-Risk Sample

volgende artikel Assessing Specific Discipline Techniques: A Mixed-Methods Approach

Abby, M., Massey, M. D., Galandiuk, S., & Polk, H. C, Jr. (1994). Peer review is an effective screening process to evaluate medical manuscripts. Journal of the American Medical Association, 272, 105–107. doi:10.1001/jama199403520020031008.PubMedCrossRef

American Academy of Child and Adolescent Psychiatry (AACAP) (2006) Policy statement on evidence-based practice. Retrieved Sept 10, 2012 from http://aacap.org/cs/root/policy_statements/ evidence_based_practice.

American Psychological Association. (APA) (2005). Policy statement on evidence-based practice in psychology. Retrieved from http://www.apa.org/practice/resources/evidence/evidence-based-statement.pdf.

American Psychological Association. (APA). (2010). Publication manual of the American Psychological Association. Washington, DC, APA.

American Psychological Association, Task force on the future of psychology. (2009). 2009 Presidential Task Force on the Future of Psychology Practice Final Report. Retrieved from http://www.apa.org/ pubs/info/reports/future-practice.pdf.

Auger, C. P. (1998). Information sources in grey literature (guides to information sources). (4th ed.). London: Bowker Saur.

Banks, M. A. (2010). Blog posts and tweets: The next frontier for grey literature. In D. J. Farace & J. Schopfel (Eds.), Grey literature in library and information studies. Berlin: De Gruyter Saur.

Barwick, M., Boydell, K. M., Stasiulis, E., Ferguson, H. B., Blase, K., & Fixsen, D. (2008). Research utilization among children’s mental health providers. Implementation Science, 3, 19. doi:10.1186/1748-5908-3-19.PubMedCrossRefPubMedCentral

Bellefontaine, S. P., & Lee, C. M. (2013). A Meta-Analytic Examination of Behavioral Parenting Interventions in the Treatment of Children’s Behavior Problems. Manuscript submitted for publication.

Benzies, K. M., Premji, S., Hayden, K. A., & Serrett, K. (2006). State-of-the-evidence reviews: Advantages and challenges of including grey literature. World views on Evidence-based Nursing, 3, 55–61. doi:10.1111/j17416787200600051x.CrossRef

Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006). The empirical status of cognitive-behavioral therapy: A review of meta-analyses. Clinical Psychology Review, 26, 17–31. doi:10.1016/jcpr200507003.PubMedCrossRef

Coad, J., Hardicare, J., & Devitt, P. (2006). Searching for and using grey literature. Nursing Times, 102, 35–36.PubMed

Conn, V. S., Valentine, J. C., Cooper, H. M., & Rantz, M. J. (2003). Grey literature in meta-analyses. Nursing Research, 52, 256–261.PubMedCrossRef

Cook, D. J., Guyatt, G. H., Ryan, G., Clifton, J., Buckingham, L., Willan, A., et al. (1993). Should unpublished data be included in meta-analyses? Current convictions and controversies. The Journal of the American Medical Association, 269, 2749–2753. doi:10.1001/jama199303500210049030.CrossRef

Council of Editors. (2011). Summary report of journal operations, 2010. American Psychologist, 66, 405–406. doi:10.1037/a0023978.CrossRef

De Graaf, I., Speetjens, P., Smit, F., de Wolff, M., & Tavecchio, L. (2008). Effectiveness of the triple P positive parenting program on parenting: A meta-analysis. Family Relations, 57, 553–566. doi:10.1177/0145445508317134.CrossRef

Decullier, E., Lhéritier, V., & Chapuis, F. (2005). Fate of biomedical research protocols and publication bias in France: Retrospective cohort study. BMJ, 331, 19. DOI: http://dx.doi.org/10.1136/bmj.38488.385995.8F.

Dretzke, J., Davenport, C., Few, E., Barlow, J., Stewart-Brown, S., Bayliss, S., et al. (2009). The clinical effectiveness of different parenting programmes for children with conduct problems: A systematic review of randomized controlled trials. Child and Adolescent Psychiatry and Mental Health, 3, 1–10. doi:10.1186/1753-2000-3-7.CrossRef

Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A. W., Cronin, E., et al. (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. Public Library of Science ONE, 3, e3081. doi:10.1371/journal.pone.0003081.PubMedPubMedCentral

Emrick, C. D. (1975). A review of psychologically oriented treatment of alcoholism. Journal of Studies on Alcohol, 36, 88–108.PubMed

Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting and Clinical Psychology, 16, 319–324.CrossRef

Eysenck, H. J. (1978). An exercise in mega-silliness. American Psychologist, 33, 517. doi:10.1037/0003066X335517a.CrossRef

Fletcher, R., Freeman, E., & Matthey, S. (2011). The impact of behavioural parent training on fathers’ parenting: A meta-analysis of the triple p—positive parenting program. Fathering, 9, 291–312. doi:10.3149/fth0903291.CrossRef

Frater, J., Myohanen, L., Taylor, E., & Keith, L. (2007). What would you tell me if I said grey literature? The New York academy of medicine grey literature report. Journal of Electronic Resources in Medical Libraries, 4, 145–153.CrossRef

Furlong, M., McGilloway, S., Bywater, T., Hutchings, J., Smith, S. M., & Donnelly, M. (2012). Behavioural and cognitive-behavioural group-based parenting programmes for early-onset conduct problems in children aged 3 to 12 years. Cochrane Database for Systematic Reviews, 2012(2), 1–344. doi:10.1002/14651858CD008225pub2.

Gilgun, J. F. (2005). The four cornerstones of evidence-based practice in social work. Research on Social Work Practice, 15, 52–61. doi:10.1177/1049731504269581.CrossRef

Glass, G. V. (1977). Integrating findings: The meta-analysis of research. Review of Research Education, 5, 351–379. doi:10.3102/0091732X005001351.

Gokhale, P. (1997). Grey Literature Varieties–Definitional Problems. In Third International Conference on Grey Literature: Perspectives on the Design and Transfer of Scientific and Technical Information, 13–14 November 1997.

Hamilton, J. (2005). Clinician’s guide to evidence-based practice. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 494–498.PubMedCrossRef

Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6, 203–217.PubMedCrossRef

Hedges, L. V., & Pigott, T. D. (2004). The power of statistical tests for moderators in meta-analysis. Psychological Methods, 9, 426–445. doi:10.1037/1082-989X94426.PubMedCrossRef

Hopewell, S., & Clarke, M. (2001). Methodologists and their methods: Do methodologists write up their conference presentations or is it just 15 minutes of fame? International Journal of Technology Assessment in Health Care, 17, 601–603.PubMed

Hopewell, S., McDonald, S., Clarke, M. J., & Egger, M. (2008). Grey literature in meta-analyses of randomized trials of health care interventions. The Cochrane Library, 1–18.

Hunsley, J., & Mash, E. J. (2008). Developing criteria for evidence-based assessment: An introduction to assessments that work. In J. Hunsley & E. J. Mash (Eds.), A guide to assessments that work (pp. 3–14). New York: Oxford University Press.

Kaminski, J. W., Valle, L. A., Filene, J. H., & Boyle, C. L. (2008). A meta-analytic review of components associated with parent training program effectiveness. Journal of Abnormal Child Psychology, 36, 567–589. doi:10.1007/s1080200792019.PubMedCrossRef

Levant, R. F. (2006). Making psychology a household word. American Psychologist, 61, 383–395.PubMedCrossRef

Light, R. J., & Smith, P. V. (1971). Accumulating evidence: Procedures for resolving contradictions among different research studies. Harvard Educational Review, 41, 429–471.

Lundahl, B., Risser, H. J., & Lovejoy, M. C. (2006). A meta-analysis of parent training: Moderators and follow-up effects. Clinical Psychology Review, 26, 86–104. doi:10.1016/jcpr200507004.PubMedCrossRef

Lundahl, B. W., Tollefson, D., Risser, H., & Lovejoy, M. C. (2008). A meta-analysis of father involvement in parent training. Research on Social Work Practice, 18, 97–106. doi:10.1177/1049731507309828.CrossRef

Martin, J. L. R., Perez, V., Sacristan, M., & Alvarez, E. (2005). Is grey literature essential for a better control of publication bias in psychiatry? An example from three meta-analyses of schizophrenia. European Psychiatry, 20, 550–553. doi:10.1016/jeurpsy200503011.PubMedCrossRef

Maughan, D. R., Christiansen, E., Jenson, W. R., Olympia, D., & Clark, E. (2005). Behavioral parent training as a treatment for externalizing behaviors and disruptive behavior disorders: A meta-analysis. School Psychology Review, 34, 267–286.

Mays, G. P., & Hogg, R. A. (2012). Expanding delivery system research in public health settings: Lessons from practice-based research networks. Journal of Public Health Management Practice, 18, 485–498. doi:10.1097/PHH0b013e31825f75c9.PubMedCrossRefPubMedCentral

McAuley, L., Pham, B., Tugwell, P., & Moher, D. (2000). Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? The Lancet, 356, 1228–1231.CrossRef

McCart, M. R., Priester, P. E., Davies, W. H., & Azen, R. (2006). Differential effectiveness of behavioral parent- training and cognitive-behavioral therapy for antisocial youth: A meta-analysis. Journal of Abnormal Child Psychology, 34, 527–543. doi:10.1007/s1080200690311.PubMedCrossRef

McLeod, B. D., & Weisz, J. R. (2004). Using dissertations to examine potential bias in child and adolescent clinical trials. Journal of Consulting and Clinical Psychology, 72, 235–251. doi:10.1037/0022-006X722235.PubMedCrossRef

Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999). Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. The Lancet, 354, 1896–1900. doi:10.1016/S01406736(99)041495.CrossRef

Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., et al. (2010). CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trial. British Medical Journal, 340, c869–c897. doi:10.1136/bmjc869.PubMedCrossRefPubMedCentral

Nowak, C., & Heinrichs, N. (2008). A comprehensive meta-analysis of triple p—positive parenting program using hierarchical linear modeling: Effectiveness and moderating variables. Clinical Child and Family Psychology Review, 11, 114–144. doi:10.1007/s1056700800330.PubMedCrossRef

Pappas, C., & Williams, I. (2011). Grey literature: Its emerging importance. Journal of Hospital Librarianship, 11, 228–234. doi:10.1080/15323269.2011.587100.CrossRef

Piquero, A. R., Farrington, D. P., Welsh, B. C., Tremblay, R., & Jennings, W. G. (2009). Effects of early family/parent training programs on antisocial behavior and delinquency. Journal of Experimental Criminology, 5, 83–120. doi:10.1007/s112920099072x.CrossRef

Posner, B. P., & Bellefontaine, S. P. (2011).ReSearch [Computer Software].

Provos, N., McClain, J., Wang, K. (2006, November). Search worms. Paper presented at the 4^th Workshop on Recurring Malcode, Fairfax, VA. Retrieved from http://www.citi.umich.edu/u/provos/papers/ search_worms.pdf.

Reyno, S. M., & McGrath, P. J. (2006). Predictors of parent training efficacy for child externalizing behavior problems—a meta-analytic review. Journal of Child Psychology and Psychiatry, 47, 99–111. doi:10.1111/j.14697610200501544x.PubMedCrossRef

Roth, B. J. (2010). An academic perspective on grey literature. Archaeologies: Journal of the World Archaeological Congress, 6, 337–345. DOI: 10.1007/s11759-010-9141-9.

Royle, J., & Blythe, J. (1998). Promoting research utilisation in nursing: The role of the individual, organisation, and environment. Evidence-Based Nursing, 1, 71–72. doi:10.1136/ebn1371.CrossRef

Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996). Evidence-based medicine: What it is and what it isn’t. British Medical Journal, 312, 71–72. doi:10.1136/bmj312702371.PubMedCrossRefPubMedCentral

Sacks, H. S., Reitman, D., Pagano, D., & Kupelnick, B. (1996). Meta-analysis: An update. Mount Sinai Journal of Medicine, 63, 216–224.PubMed

Schulz, K. F., Altman, D. G., Mohe, D., & for the CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. British Medical Journal, 340, 698–702. doi:10.1136/bmjc332.CrossRef

Scullion, P. A. (2002). Effective dissemination strategies. Nurse Researcher, 10, 65–67.PubMedCrossRef

Serketich, W. J., & Dumas, J. E. (1996). The effectiveness of behavioral parent training to modify antisocial behavior in children: A meta-analysis. Behavior Therapy, 27, 171–186. doi:10.1016/S0005-7894(96)80013X.CrossRef

Seymour, D. J. (2010). Sanctioned inequity and accessibility issues in the grey literature in the United States. Archaeologies: Journal of the World Archaeological Congress, 6, 233–269. DOI: 10.1007/s11759-010-9144-6.

Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 36, 753–760.

Smith, M., Glass, G., & Miller, T. (1980). The benefits of psychotherapy. Baltimore: John Hopkins University Press.

Thomas, R., & Zimmer-Gembeck, M. J. (2007). Behavioral outcomes of parent-child interaction therapy and triple p-positive parenting program: A review and meta-analysis. Journal of Abnormal Child Psychology, 35, 475–495. doi:10.1007/s1080200791049.PubMedCrossRef

Weisz, J. R., Weiss, B., Alicke, M. D., & Klotz, M. L. (1987). Effectiveness of psychotherapy with children and adolescents: A meta-analysis for clinicians. Journal of Consulting and Clinical Psychology, 55, 542–549. doi:http://www.wjh.harvard.edu/~jweisz/pdfs/1987c.pdf.PubMedCrossRef

Wilson, D. B., & Lipsey, M. W. (2001). The role of method in treatment effectiveness research: Evidence from meta-analysis. Psychological Bulletin, 6, 413–429.

Wood, J. (2008). Methodology for dealing with duplicate study effects in meta-analysis. Organizational Research Methods, 11, 79–95. doi:10.1177/1094428106296638.CrossRef

Titel: Between Black and White: Examining Grey Literature in Meta-analyses of Psychological Research
Auteurs: Sarah P. Bellefontaine
Catherine M. Lee
Publicatiedatum: 01-11-2014
Uitgeverij: Springer US
Gepubliceerd in: Journal of Child and Family Studies / Uitgave 8/2014
Print ISSN: 1062-1024
Elektronisch ISSN: 1573-2843
DOI: https://doi.org/10.1007/s10826-013-9795-1

Andere artikelen Uitgave 8/2014

Evidence-Based Assessment in Children and Adolescents with Obsessive–Compulsive Disorder

Original Paper

Maternal Behavior Prior to Parenting as a Transgenerational Predictor of Offspring Behavior

Original Paper

Emerging Adults’ Behavior Problems and Coping Following Hurricanes

Original Paper

Keith Robinson and Angel L. Harris: Broken Compass: Parental Involvement with Children’s Education

Book Review

Maternal Relationship Insecurity and Depressive Symptoms as Moderators of Home Visiting Impacts on Child Outcomes

Original Paper

Adolescent Girls and Their Mothers Talk About Experiences of Binge and Loss of Control Eating

Original Paper