Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: A systematic review
Introduction
Randomized controlled trials (RCTs) are important sources of evidence in health care [1]. Apart from the limitations that are inherent to the study design, readers must rely upon the report of the trial to establish its credibility. Unfortunately, many trials are not reported in sufficient detail to ensure that potential sources of bias have been avoided [2]. In order for readers to be able to appreciate the internal and external validities of RCTs, they must be able to fully understand exactly how the RCT was conducted. Discrepancies in the way RCTs were reported led to the development of the Consolidated Standards of Reporting Trials (CONSORT) statement in 1996 and a revision in 2001 [3], [4]. The 1996 publication of the CONSORT statement led to an improvement in the reporting of RCTs [5].
However, given the large volumes of scientific publications produced each year, many readers refer to the abstracts of RCTs rather than the entire report [6]. In addition, conference proceedings containing RCTs are limited to abstracts. This implies that abstracts of RCTs should contain sufficient information to be useful to readers. This can be achieved by recommending that information in abstracts be provided under specific headings and have a uniform “structure”. Structured abstracts were introduced to the medical literature in the 1980s [7]. They were found to be well accepted by readers, to contain more information and to facilitate the review of conference proceedings. However, they are often lengthier [7]. The acknowledgment of these issues in relation to RCTs led to the development of the CONSORT extension for abstracts [8].
The CONSORT statement for abstracts comprises 17 items in eight sections which are: the title, authors contact details, trial design, methods (participants, interventions, objective, outcomes, randomization, blinding), results (numbers randomized, recruitment, numbers analyzed, outcome, harms), conclusions, trial registration and funding. The statement describes what information authors of RCT abstracts are expected to report [8].
A number of studies have investigated the quality of reporting after the publication of this extension for abstracts. Poor adherence to the statement was noted in general medicine journals, [9] and also in specific fields of medicine such as anesthesia [10]. A recent scoping review noted that higher journal impact factor, later date of publication, industry funding, multisite, larger studies and studies of pharmacological interventions were more likely to adhere to guidelines. On the other hand studies with positive results were less likely to adhere to the CONSORT statement [11]. In addition, journal endorsement can lead to improved quality of reporting [8], [11].
These studies have either focused on the effect of journal endorsement [12], specific medical specialties [10], [13], [14], [15], [16], specific countries [17] or without adjustments for potential confounders [9].
In order to evaluate the role of the CONSORT extension for abstracts in improving the reporting of abstracts, it is necessary to appraise changes in reporting quality over time, taking into account the factors that influence reporting quality [11].
Our objectives were to compare abstracts of RCTs published in high-impact general medicine journals before and after the publication of the CONSORT extension for abstracts, while controlling for potential confounders.
Section snippets
Methods
We conducted a search of the United States National Institutes for Health (NIH) database PubMed (August 2013) for RCTs published in the years 2007 and 2012 in high-impact general medicine journals with wide readership: The Journal of the American Medical Association (JAMA), the New England Journal of Medicine (NEJM), the Lancet, the British Medical Journal (BMJ), Annals of Internal Medicine (AIM) and the Canadian Medical Association Journal (CMAJ). Our search strategy included terms for RCTs
Statistical analyses
Firstly, we determined the mean number of items reported (0–17) for each year and estimated the unadjusted and adjusted differences using a two-sample t-test and generalized estimation equations (GEEs) respectively [20]. The means are reported with standard deviations (SDs). The mean differences and adjusted means are reported with 95% CIs and p-values.
Secondly, we compared compliance with the 17 items of the CONSORT statement for abstracts for years 2007 versus 2012 using individual
Results
Our search retrieved 445 articles of which 199 and 230 were published in 2007 and 2012 respectively. We then randomly selected 100 for each year. Ten (8 from 2007 and 2 from 2012) articles did not meet our inclusion criteria and were replaced by another random selection. A flow diagram of our study selection process is shown in Fig. 1.
Agreement was considered high for inclusion of articles (Kappa = 0.60 [95% CI 0.29, 0.92]; p < 0.001; given the maximum attainable Kappa = 0.60). Overall we compared
Discussion
In this study, we report differences in the quality of reporting of abstracts in a random selection of articles published in major general medicine journals in 2012 and 2007 using the CONSORT extension for abstracts [8]. For some items there was an improvement while for others, reporting quality seemed to stagnate or regress. Overall, there was an improvement in the quality of reporting of abstracts since the publication of the CONSORT extension for abstracts in 2008, with an average of 3 more
Conclusion
In 2012 compared to 2007 there has been some improvement in reporting of RCT abstracts in top-tier medical journals according to the CONSORT for abstract checklist, with regard to the number of items reported and for specific items, but there is still room for improvement. Journal endorsement and implementation of the CONSORT statement for abstracts are warranted.
References (22)
- et al.
CONSORT for reporting randomised trials in journal and conference abstracts
Lancet
(2008) - et al.
Reporting quality of abstracts of randomized controlled trials published in leading orthodontic journals from 2006 to 2011
Am J Orthod Dentofacial Orthop
(2012) - et al.
Reporting quality of abstracts of randomized controlled trials published in dental specialty journals
J Evid Based Dent Pract
(2013) - et al.
Oncology trial abstracts showed suboptimal improvement in reporting: a comparative before-and-after evaluation using CONSORT for abstract guidelines
J Clin Epidemiol
(2014) - et al.
An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods
J Clin Epidemiol
(2004) - et al.
Randomized controlled trials
Am J Roentgenol
(2004) Randomised controlled trials: a user's guide
BMJ books
(1998)- et al.
Improving the quality of reporting of randomized controlled trials. The CONSORT statement
JAMA
(1996) - et al.
The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials
J Am Podiatr Med Assoc
(2001) - et al.
Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review
Med J Aust
(2006)
Comparing data accuracy between structured abstracts and full-text journal articles: implications in their use for informing clinical decisions
Evid Based Med
Cited by (60)
Methodology reporting improved over time in 176,469 randomized controlled trials
2023, Journal of Clinical EpidemiologyEvaluation of reporting quality of abstracts of randomized controlled trials regarding patients with COVID-19 using the CONSORT statement for abstracts
2022, International Journal of Infectious DiseasesDo not make clinical decisions based on abstracts of healthcare research: A systematic review
2021, Journal of Clinical EpidemiologyCitation Excerpt :The main finding is that even after the release of the CONSORT-A and PRISMA-A, the reporting quality of abstracts of RCTs and SRs still remains suboptimal with much room for improvement in all healthcare areas. Eleven overviews [3,5,29,31-33,35,38,42,43,47] that compared articles (RCTs or SRs) published before and after the release of such abstracts guidelines also presented only small improvements over time, sometimes showing a statistical significant change but the magnitude of change was not large enough to be considered sufficiently well reported. Abstracts reporting quality across all healthcare areas were poor, even if the reporting quality has varied across the same healthcare area (e.g. within dentistry one overview concluded that 62.5% of abstracts were fully reported items, [46] while another overview concluded that this percentage was 28.1% [38]).
A call for consensus guidelines on classification and reporting of methodological studies
2020, Journal of Clinical EpidemiologyAbstracts of Low Back Pain Trials Are Poorly Reported, Contain Spin of Information, and Are Inconsistent With the Full Text: An Overview Study
2019, Archives of Physical Medicine and Rehabilitation
- 1
CONSORT: Consolidated Standards of Reporting Trials.
- 2
RCT: Randomized controlled trial.
- 3
IRRs: Incidence rate ratios.
- 4
CI: Confidence interval.