ReviewRegistered trials report less beneficial treatment effects than unregistered ones: a meta-epidemiological study in orthodontics
Introduction
Randomized clinical trials are regarded as the gold standard in comparative efficacy research and form the basis for translating research evidence to clinical practice [1]. Among their advantages, methodological transparency is crucial and entails registration of the trial design protocol in a public domain prior to trial initiation to improve accountability in the conduct and reporting of research [2]. Trial protocols can be used post hoc to compare the original plan with subsequent procedures and analyses [3], thereby potentially reducing the risk of data dredging. A priori trial registration can additionally safeguard against bias-related phenomena such as delayed publication or nonpublication of trials, selective reporting of outcomes, manipulation of the analysis plan, and counting covert duplicate publications within systematic reviews as separate trials [3], [4], [5], [6].
Insights on trial characteristics systematically associated with treatment effects can be gleamed through meta-epidemiological studies. This is a subgenre of the big family of overviews of reviews, wherein data from a collection of meta-analyses is integrated and classified according to a specific study-level trait to empirically assess its influence on treatment effects [7].
A recent comprehensive meta-epidemiological study reported that nonregistered or retrospectively registered trials tend to show inflated treatment effects [8]. Although the effect was not statistically significant, this is confirmed by cross-sectional overviews of randomized trials in cardiology [9] or general medicine [10] that report weak associations between positive findings and trial registration, although the effects were not always consistent. It is also important to note that these analyses were based on qualitative evaluations of trial results by using either a P value cutoff or the trialists' interpretation of the results in text, both of which can be problematic.
The aim of this meta-epidemiological study is to provide empirical evidence of possible differences in the results of registered and nonregistered randomized clinical trials in orthodontics as a sign of bias, including its direction and magnitude.
Section snippets
Protocol, eligibility criteria, and registration
The protocol for this overview of reviews and trials was made a priori based loosely on the format of a systematic review, registered in PROSPERO (CRD42017072043), and all post hoc changes to the protocol were appropriately noted. According to the criteria set a priori, eligibility included parallel randomized clinical trials (or meta-analyses thereof) of human patients on any experimental intervention compared to a conventional or control group with any binary/continuous outcome in
Study selection
The literature search yielded a total of 1,241 hits electronically and 18 hits manually as of September 2, 2017; 216 of which, proceeded to full text assessment after eliminating duplicates and ineligible studies by title or abstract (Fig. 1 Supplementary Table 3a-3g). Finally, a total of 48 trials on similar comparisons were identified as eligible for inclusion in the present meta-epidemiological study. These were included in 16 separate meta-analyses on seven different topics, with some trial
Principal findings
The present overview summarizes empirical evidence up to September 2017 about the effects of trial registration on the results of orthodontic randomized clinical trials. Empirical evidence from 16 meta-analyses and 4,988 patients indicated that registered trials report considerably less beneficial treatment effects compared to nonregistered trials, which can be interpreted as signs of bias of moderate magnitude. The fact that bias from lack of registration was found to be of moderate magnitude
Acknowledgments
The authors thank Sophy K Barber (Leeds Dental Institute), Peter Miles (Caloundra, Queensland), Mueez Shedam (Rangoonwala Dental College and Research Centre), and Yanhua Xu (Kunming Medical University School) for providing details of their studies.
References (40)
- et al.
Clinical trial registration: a statement from the International committee of medical journal editors
Lancet
(2004) - et al.
Publication of statistically significant research findings in prosthodontics & implant dentistry in the context of other dental specialties
J Dent
(2015) - et al.
Empirical evaluation of which trial characteristics are associated with treatment effect estimates
J Clin Epidemiol
(2016) - et al.
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration
J Clin Epidemiol
(2009) - et al.
Bias and small-study effects influence treatment effect estimates: a meta-epidemiological study in oral medicine
J Clin Epidemiol
(2014) - et al.
Basic study design influences the results of orthodontic clinical investigations
J Clin Epidemiol
(2015) - et al.
Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry
J Clin Epidemiol
(2008) - et al.
Increasing value and reducing waste in research design, conduct, and analysis
Lancet
(2014) - et al.
Active implementation strategy of CONSORT adherence by a dental specialty journal improved randomized clinical trial reporting
J Clin Epidemiol
(2014) - et al.
An assessment of quality characteristics of randomised control trials published in dental journals
J Dent
(2010)
Comparison of protocols and registry entries to published reports for randomised controlled trials
Cochrane Database Syst Rev
Impact of covert duplicate publication on meta-analysis: a case study
BMJ
Publication and related biases
Health Technol Assess
Association between trial registration and treatment effect estimates: a meta-epidemiological study
BMC Med
Association of cardiovascular trial registration with positive study findings: epidemiological Study of Randomized Trials (ESORT)
JAMA Intern Med
Association between trial registration and positive study findings: cross sectional study (Epidemiological Study of Randomized Trials-ESORT)
BMJ
Reported methodological quality of split-mouth studies
J Clin Periodontol
Does published orthodontic research account for clustering effects during statistical data analysis?
Eur J Orthod
Comparison of treatment effect sizes associated with surrogate and final patient relevant outcomes in randomised controlled trials: meta-epidemiological study
BMJ
Cited by (20)
Enhancing the quality of reporting of orthodontic clinical research
2024, Seminars in OrthodonticsMaking ERP research more transparent: Guidelines for preregistration
2021, International Journal of PsychophysiologyCitation Excerpt :Results showed that 57% of the studies published before 2000 showed a significant benefit of the intervention, as opposed to only 8% of trials published after 2000.7 Similar results were reported in a (preregistered) meta-analysis of meta-analyses of orthodontics and dentofacial orthopedics studies: registered trials reported less favorable intervention effects compared to unregistered trials (Papageorgiou et al., 2018). Preregistration can also help identify whether funding sources are correlated with study outcome, potentially uncovering questionable practices due to (undisclosed) conflicts of interest.
Development of a core outcome set for use in routine orthodontic clinical trials
2020, American Journal of Orthodontics and Dentofacial OrthopedicsAssessing Publication Bias: a 7-Step User’s Guide with Best-Practice Recommendations
2023, Journal of Business and Psychology
Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Data availability: The datasets and commands of this study are openly available through Zenodo (http://doi.org/10.5281/zenodo.1186318).
Conflicts of interest: The authors declare no conflicts of interest to disclose.