Skip to main content
main-content
Top

Tip

Swipe om te navigeren naar een ander artikel

Gepubliceerd in: Journal of Child and Family Studies 8/2014

01-11-2014 | Original Paper

Between Black and White: Examining Grey Literature in Meta-analyses of Psychological Research

Auteurs: Sarah P. Bellefontaine, Catherine M. Lee

Gepubliceerd in: Journal of Child and Family Studies | Uitgave 8/2014

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

The inclusion of grey literature in meta-analyses and reviews is controversial. We examine both the advantages and challenges of including grey literature in meta-analyses. An exemplar meta-analysis of behavioral parenting interventions on parent behavior, child behavior, and parent adjustment outcomes is used to demonstrate these issues. It also explores the influence of the inclusion of grey literature on outcomes, including whether effect sizes are affected by the inclusion of grey literature and describes the challenges of searching for grey literature using traditional search engines such as Google and Yahoo. Homogeneity and publication bias are also examined. Based on these results, recommendations are presented for meta-analysts and researchers.
Literatuur
go back to reference American Psychological Association. (APA). (2010). Publication manual of the American Psychological Association. Washington, DC, APA. American Psychological Association. (APA). (2010). Publication manual of the American Psychological Association. Washington, DC, APA.
go back to reference American Psychological Association, Task force on the future of psychology. (2009). 2009 Presidential Task Force on the Future of Psychology Practice Final Report. Retrieved from http://​www.​apa.​org/​ pubs/info/reports/future-practice.pdf. American Psychological Association, Task force on the future of psychology. (2009). 2009 Presidential Task Force on the Future of Psychology Practice Final Report. Retrieved from http://​www.​apa.​org/​ pubs/info/reports/future-practice.pdf.
go back to reference Auger, C. P. (1998). Information sources in grey literature (guides to information sources). (4th ed.). London: Bowker Saur. Auger, C. P. (1998). Information sources in grey literature (guides to information sources). (4th ed.). London: Bowker Saur.
go back to reference Banks, M. A. (2010). Blog posts and tweets: The next frontier for grey literature. In D. J. Farace & J. Schopfel (Eds.), Grey literature in library and information studies. Berlin: De Gruyter Saur. Banks, M. A. (2010). Blog posts and tweets: The next frontier for grey literature. In D. J. Farace & J. Schopfel (Eds.), Grey literature in library and information studies. Berlin: De Gruyter Saur.
go back to reference Bellefontaine, S. P., & Lee, C. M. (2013). A Meta- Analytic Examination of Behavioral Parenting Interventions in the Treatment of Children’s Behavior Problems. Manuscript submitted for publication. Bellefontaine, S. P., & Lee, C. M. (2013). A Meta- Analytic Examination of Behavioral Parenting Interventions in the Treatment of Children’s Behavior Problems. Manuscript submitted for publication.
go back to reference Coad, J., Hardicare, J., & Devitt, P. (2006). Searching for and using grey literature. Nursing Times, 102, 35–36. PubMed Coad, J., Hardicare, J., & Devitt, P. (2006). Searching for and using grey literature. Nursing Times, 102, 35–36. PubMed
go back to reference Conn, V. S., Valentine, J. C., Cooper, H. M., & Rantz, M. J. (2003). Grey literature in meta-analyses. Nursing Research, 52, 256–261. PubMedCrossRef Conn, V. S., Valentine, J. C., Cooper, H. M., & Rantz, M. J. (2003). Grey literature in meta-analyses. Nursing Research, 52, 256–261. PubMedCrossRef
go back to reference Cook, D. J., Guyatt, G. H., Ryan, G., Clifton, J., Buckingham, L., Willan, A., et al. (1993). Should unpublished data be included in meta-analyses? Current convictions and controversies. The Journal of the American Medical Association, 269, 2749–2753. doi: 10.​1001/​jama199303500210​049030. CrossRef Cook, D. J., Guyatt, G. H., Ryan, G., Clifton, J., Buckingham, L., Willan, A., et al. (1993). Should unpublished data be included in meta-analyses? Current convictions and controversies. The Journal of the American Medical Association, 269, 2749–2753. doi: 10.​1001/​jama199303500210​049030. CrossRef
go back to reference Dretzke, J., Davenport, C., Few, E., Barlow, J., Stewart-Brown, S., Bayliss, S., et al. (2009). The clinical effectiveness of different parenting programmes for children with conduct problems: A systematic review of randomized controlled trials. Child and Adolescent Psychiatry and Mental Health, 3, 1–10. doi: 10.​1186/​1753-2000-3-7. CrossRef Dretzke, J., Davenport, C., Few, E., Barlow, J., Stewart-Brown, S., Bayliss, S., et al. (2009). The clinical effectiveness of different parenting programmes for children with conduct problems: A systematic review of randomized controlled trials. Child and Adolescent Psychiatry and Mental Health, 3, 1–10. doi: 10.​1186/​1753-2000-3-7. CrossRef
go back to reference Emrick, C. D. (1975). A review of psychologically oriented treatment of alcoholism. Journal of Studies on Alcohol, 36, 88–108. PubMed Emrick, C. D. (1975). A review of psychologically oriented treatment of alcoholism. Journal of Studies on Alcohol, 36, 88–108. PubMed
go back to reference Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting and Clinical Psychology, 16, 319–324. CrossRef Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting and Clinical Psychology, 16, 319–324. CrossRef
go back to reference Fletcher, R., Freeman, E., & Matthey, S. (2011). The impact of behavioural parent training on fathers’ parenting: A meta-analysis of the triple p—positive parenting program. Fathering, 9, 291–312. doi: 10.​3149/​fth0903291. CrossRef Fletcher, R., Freeman, E., & Matthey, S. (2011). The impact of behavioural parent training on fathers’ parenting: A meta-analysis of the triple p—positive parenting program. Fathering, 9, 291–312. doi: 10.​3149/​fth0903291. CrossRef
go back to reference Frater, J., Myohanen, L., Taylor, E., & Keith, L. (2007). What would you tell me if I said grey literature? The New York academy of medicine grey literature report. Journal of Electronic Resources in Medical Libraries, 4, 145–153. CrossRef Frater, J., Myohanen, L., Taylor, E., & Keith, L. (2007). What would you tell me if I said grey literature? The New York academy of medicine grey literature report. Journal of Electronic Resources in Medical Libraries, 4, 145–153. CrossRef
go back to reference Furlong, M., McGilloway, S., Bywater, T., Hutchings, J., Smith, S. M., & Donnelly, M. (2012). Behavioural and cognitive-behavioural group-based parenting programmes for early-onset conduct problems in children aged 3 to 12 years. Cochrane Database for Systematic Reviews, 2012(2), 1–344. doi: 10.​1002/​14651858CD008225​pub2. Furlong, M., McGilloway, S., Bywater, T., Hutchings, J., Smith, S. M., & Donnelly, M. (2012). Behavioural and cognitive-behavioural group-based parenting programmes for early-onset conduct problems in children aged 3 to 12 years. Cochrane Database for Systematic Reviews, 2012(2), 1–344. doi: 10.​1002/​14651858CD008225​pub2.
go back to reference Gokhale, P. (1997). Grey Literature Varieties–Definitional Problems. In Third International Conference on Grey Literature: Perspectives on the Design and Transfer of Scientific and Technical Information, 13– 14 November 1997. Gokhale, P. (1997). Grey Literature Varieties–Definitional Problems. In Third International Conference on Grey Literature: Perspectives on the Design and Transfer of Scientific and Technical Information, 1314 November 1997.
go back to reference Hamilton, J. (2005). Clinician’s guide to evidence-based practice. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 494–498. PubMedCrossRef Hamilton, J. (2005). Clinician’s guide to evidence-based practice. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 494–498. PubMedCrossRef
go back to reference Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6, 203–217. PubMedCrossRef Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6, 203–217. PubMedCrossRef
go back to reference Hopewell, S., & Clarke, M. (2001). Methodologists and their methods: Do methodologists write up their conference presentations or is it just 15 minutes of fame? International Journal of Technology Assessment in Health Care, 17, 601–603. PubMed Hopewell, S., & Clarke, M. (2001). Methodologists and their methods: Do methodologists write up their conference presentations or is it just 15 minutes of fame? International Journal of Technology Assessment in Health Care, 17, 601–603. PubMed
go back to reference Hopewell, S., McDonald, S., Clarke, M. J., & Egger, M. (2008). Grey literature in meta-analyses of randomized trials of health care interventions. The Cochrane Library, 1–18. Hopewell, S., McDonald, S., Clarke, M. J., & Egger, M. (2008). Grey literature in meta-analyses of randomized trials of health care interventions. The Cochrane Library, 1–18.
go back to reference Hunsley, J., & Mash, E. J. (2008). Developing criteria for evidence-based assessment: An introduction to assessments that work. In J. Hunsley & E. J. Mash (Eds.), A guide to assessments that work (pp. 3–14). New York: Oxford University Press. Hunsley, J., & Mash, E. J. (2008). Developing criteria for evidence-based assessment: An introduction to assessments that work. In J. Hunsley & E. J. Mash (Eds.), A guide to assessments that work (pp. 3–14). New York: Oxford University Press.
go back to reference Kaminski, J. W., Valle, L. A., Filene, J. H., & Boyle, C. L. (2008). A meta-analytic review of components associated with parent training program effectiveness. Journal of Abnormal Child Psychology, 36, 567–589. doi: 10.​1007/​s1080200792019. PubMedCrossRef Kaminski, J. W., Valle, L. A., Filene, J. H., & Boyle, C. L. (2008). A meta-analytic review of components associated with parent training program effectiveness. Journal of Abnormal Child Psychology, 36, 567–589. doi: 10.​1007/​s1080200792019. PubMedCrossRef
go back to reference Light, R. J., & Smith, P. V. (1971). Accumulating evidence: Procedures for resolving contradictions among different research studies. Harvard Educational Review, 41, 429–471. Light, R. J., & Smith, P. V. (1971). Accumulating evidence: Procedures for resolving contradictions among different research studies. Harvard Educational Review, 41, 429–471.
go back to reference Martin, J. L. R., Perez, V., Sacristan, M., & Alvarez, E. (2005). Is grey literature essential for a better control of publication bias in psychiatry? An example from three meta-analyses of schizophrenia. European Psychiatry, 20, 550–553. doi: 10.​1016/​jeurpsy200503011​. PubMedCrossRef Martin, J. L. R., Perez, V., Sacristan, M., & Alvarez, E. (2005). Is grey literature essential for a better control of publication bias in psychiatry? An example from three meta-analyses of schizophrenia. European Psychiatry, 20, 550–553. doi: 10.​1016/​jeurpsy200503011​. PubMedCrossRef
go back to reference Maughan, D. R., Christiansen, E., Jenson, W. R., Olympia, D., & Clark, E. (2005). Behavioral parent training as a treatment for externalizing behaviors and disruptive behavior disorders: A meta-analysis. School Psychology Review, 34, 267–286. Maughan, D. R., Christiansen, E., Jenson, W. R., Olympia, D., & Clark, E. (2005). Behavioral parent training as a treatment for externalizing behaviors and disruptive behavior disorders: A meta-analysis. School Psychology Review, 34, 267–286.
go back to reference McAuley, L., Pham, B., Tugwell, P., & Moher, D. (2000). Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? The Lancet, 356, 1228–1231. CrossRef McAuley, L., Pham, B., Tugwell, P., & Moher, D. (2000). Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? The Lancet, 356, 1228–1231. CrossRef
go back to reference McCart, M. R., Priester, P. E., Davies, W. H., & Azen, R. (2006). Differential effectiveness of behavioral parent- training and cognitive-behavioral therapy for antisocial youth: A meta-analysis. Journal of Abnormal Child Psychology, 34, 527–543. doi: 10.​1007/​s1080200690311. PubMedCrossRef McCart, M. R., Priester, P. E., Davies, W. H., & Azen, R. (2006). Differential effectiveness of behavioral parent- training and cognitive-behavioral therapy for antisocial youth: A meta-analysis. Journal of Abnormal Child Psychology, 34, 527–543. doi: 10.​1007/​s1080200690311. PubMedCrossRef
go back to reference Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999). Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. The Lancet, 354, 1896–1900. doi: 10.​1016/​S01406736(99)041495. CrossRef Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., & Stroup, D. F. (1999). Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. The Lancet, 354, 1896–1900. doi: 10.​1016/​S01406736(99)041495. CrossRef
go back to reference Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., et al. (2010). CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trial. British Medical Journal, 340, c869–c897. doi: 10.​1136/​bmjc869. PubMedCrossRefPubMedCentral Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., et al. (2010). CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trial. British Medical Journal, 340, c869–c897. doi: 10.​1136/​bmjc869. PubMedCrossRefPubMedCentral
go back to reference Nowak, C., & Heinrichs, N. (2008). A comprehensive meta-analysis of triple p—positive parenting program using hierarchical linear modeling: Effectiveness and moderating variables. Clinical Child and Family Psychology Review, 11, 114–144. doi: 10.​1007/​s1056700800330. PubMedCrossRef Nowak, C., & Heinrichs, N. (2008). A comprehensive meta-analysis of triple p—positive parenting program using hierarchical linear modeling: Effectiveness and moderating variables. Clinical Child and Family Psychology Review, 11, 114–144. doi: 10.​1007/​s1056700800330. PubMedCrossRef
go back to reference Piquero, A. R., Farrington, D. P., Welsh, B. C., Tremblay, R., & Jennings, W. G. (2009). Effects of early family/parent training programs on antisocial behavior and delinquency. Journal of Experimental Criminology, 5, 83–120. doi: 10.​1007/​s112920099072x. CrossRef Piquero, A. R., Farrington, D. P., Welsh, B. C., Tremblay, R., & Jennings, W. G. (2009). Effects of early family/parent training programs on antisocial behavior and delinquency. Journal of Experimental Criminology, 5, 83–120. doi: 10.​1007/​s112920099072x. CrossRef
go back to reference Posner, B. P., & Bellefontaine, S. P. (2011). ReSearch [Computer Software]. Posner, B. P., & Bellefontaine, S. P. (2011). ReSearch [Computer Software].
go back to reference Royle, J., & Blythe, J. (1998). Promoting research utilisation in nursing: The role of the individual, organisation, and environment. Evidence-Based Nursing, 1, 71–72. doi: 10.​1136/​ebn1371. CrossRef Royle, J., & Blythe, J. (1998). Promoting research utilisation in nursing: The role of the individual, organisation, and environment. Evidence-Based Nursing, 1, 71–72. doi: 10.​1136/​ebn1371. CrossRef
go back to reference Sacks, H. S., Reitman, D., Pagano, D., & Kupelnick, B. (1996). Meta-analysis: An update. Mount Sinai Journal of Medicine, 63, 216–224. PubMed Sacks, H. S., Reitman, D., Pagano, D., & Kupelnick, B. (1996). Meta-analysis: An update. Mount Sinai Journal of Medicine, 63, 216–224. PubMed
go back to reference Schulz, K. F., Altman, D. G., Mohe, D., & for the CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. British Medical Journal, 340, 698–702. doi: 10.​1136/​bmjc332. CrossRef Schulz, K. F., Altman, D. G., Mohe, D., & for the CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. British Medical Journal, 340, 698–702. doi: 10.​1136/​bmjc332. CrossRef
go back to reference Seymour, D. J. (2010). Sanctioned inequity and accessibility issues in the grey literature in the United States. Archaeologies: Journal of the World Archaeological Congress, 6, 233–269. DOI: 10.​1007/​s11759-010-9144-6. Seymour, D. J. (2010). Sanctioned inequity and accessibility issues in the grey literature in the United States. Archaeologies: Journal of the World Archaeological Congress, 6, 233–269. DOI: 10.​1007/​s11759-010-9144-6.
go back to reference Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 36, 753–760. Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 36, 753–760.
go back to reference Smith, M., Glass, G., & Miller, T. (1980). The benefits of psychotherapy. Baltimore: John Hopkins University Press. Smith, M., Glass, G., & Miller, T. (1980). The benefits of psychotherapy. Baltimore: John Hopkins University Press.
go back to reference Thomas, R., & Zimmer-Gembeck, M. J. (2007). Behavioral outcomes of parent-child interaction therapy and triple p-positive parenting program: A review and meta-analysis. Journal of Abnormal Child Psychology, 35, 475–495. doi: 10.​1007/​s1080200791049. PubMedCrossRef Thomas, R., & Zimmer-Gembeck, M. J. (2007). Behavioral outcomes of parent-child interaction therapy and triple p-positive parenting program: A review and meta-analysis. Journal of Abnormal Child Psychology, 35, 475–495. doi: 10.​1007/​s1080200791049. PubMedCrossRef
go back to reference Wilson, D. B., & Lipsey, M. W. (2001). The role of method in treatment effectiveness research: Evidence from meta-analysis. Psychological Bulletin, 6, 413–429. Wilson, D. B., & Lipsey, M. W. (2001). The role of method in treatment effectiveness research: Evidence from meta-analysis. Psychological Bulletin, 6, 413–429.
Metagegevens
Titel
Between Black and White: Examining Grey Literature in Meta-analyses of Psychological Research
Auteurs
Sarah P. Bellefontaine
Catherine M. Lee
Publicatiedatum
01-11-2014
Uitgeverij
Springer US
Gepubliceerd in
Journal of Child and Family Studies / Uitgave 8/2014
Print ISSN: 1062-1024
Elektronisch ISSN: 1573-2843
DOI
https://doi.org/10.1007/s10826-013-9795-1

Andere artikelen Uitgave 8/2014

Journal of Child and Family Studies 8/2014 Naar de uitgave