Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: A systematic review

https://doi.org/10.1016/j.cct.2014.05.012Get rights and content

Abstract

Background

We sought to determine if the publication of the Consolidated Standards of Reporting Trials (CONSORT)1 extension for abstracts in 2008 had led to an improvement in reporting abstracts of randomized controlled trials (RCTs).2

Methods

We searched PubMed for RCTs published in 2007 and 2012 in top-tier general medicine journals. A random selection of 100 trial abstracts was obtained for each year. Data were extracted in duplicate on the adherence to the CONSORT extension for abstracts. The primary outcome was the mean number of items reported and the secondary outcome was the odds of reporting each item. We also estimated incidence rate ratios (IRRs).3

Results

Significantly more checklist items were reported in 2012 than in 2007: adjusted mean difference was 2.91 (95% confidence interval [CI]4 2.35, 3.41; p < 0.001). In 2012 there were significant improvements in reporting the study as randomized in the title, describing the trial design, the participants, and objectives and blinding. In the Results section, trial status and numbers analyzed were also reported better. The IRRs were significantly higher for 2012 (IRR 1.32; 95% CI 1.25, 1.39; p < 0.001) and in multisite studies compared to single site studies (IRR 1.08; 95% CI 1.03, 1.15; p = 0.006).

Conclusions

There was a significant improvement in the reporting of abstracts of RCTs in 2012 compared to 2007. However, there is still room for improvement as some items remain under-reported.

Introduction

Randomized controlled trials (RCTs) are important sources of evidence in health care [1]. Apart from the limitations that are inherent to the study design, readers must rely upon the report of the trial to establish its credibility. Unfortunately, many trials are not reported in sufficient detail to ensure that potential sources of bias have been avoided [2]. In order for readers to be able to appreciate the internal and external validities of RCTs, they must be able to fully understand exactly how the RCT was conducted. Discrepancies in the way RCTs were reported led to the development of the Consolidated Standards of Reporting Trials (CONSORT) statement in 1996 and a revision in 2001 [3], [4]. The 1996 publication of the CONSORT statement led to an improvement in the reporting of RCTs [5].

However, given the large volumes of scientific publications produced each year, many readers refer to the abstracts of RCTs rather than the entire report [6]. In addition, conference proceedings containing RCTs are limited to abstracts. This implies that abstracts of RCTs should contain sufficient information to be useful to readers. This can be achieved by recommending that information in abstracts be provided under specific headings and have a uniform “structure”. Structured abstracts were introduced to the medical literature in the 1980s [7]. They were found to be well accepted by readers, to contain more information and to facilitate the review of conference proceedings. However, they are often lengthier [7]. The acknowledgment of these issues in relation to RCTs led to the development of the CONSORT extension for abstracts [8].

The CONSORT statement for abstracts comprises 17 items in eight sections which are: the title, authors contact details, trial design, methods (participants, interventions, objective, outcomes, randomization, blinding), results (numbers randomized, recruitment, numbers analyzed, outcome, harms), conclusions, trial registration and funding. The statement describes what information authors of RCT abstracts are expected to report [8].

A number of studies have investigated the quality of reporting after the publication of this extension for abstracts. Poor adherence to the statement was noted in general medicine journals, [9] and also in specific fields of medicine such as anesthesia [10]. A recent scoping review noted that higher journal impact factor, later date of publication, industry funding, multisite, larger studies and studies of pharmacological interventions were more likely to adhere to guidelines. On the other hand studies with positive results were less likely to adhere to the CONSORT statement [11]. In addition, journal endorsement can lead to improved quality of reporting [8], [11].

These studies have either focused on the effect of journal endorsement [12], specific medical specialties [10], [13], [14], [15], [16], specific countries [17] or without adjustments for potential confounders [9].

In order to evaluate the role of the CONSORT extension for abstracts in improving the reporting of abstracts, it is necessary to appraise changes in reporting quality over time, taking into account the factors that influence reporting quality [11].

Our objectives were to compare abstracts of RCTs published in high-impact general medicine journals before and after the publication of the CONSORT extension for abstracts, while controlling for potential confounders.

Section snippets

Methods

We conducted a search of the United States National Institutes for Health (NIH) database PubMed (August 2013) for RCTs published in the years 2007 and 2012 in high-impact general medicine journals with wide readership: The Journal of the American Medical Association (JAMA), the New England Journal of Medicine (NEJM), the Lancet, the British Medical Journal (BMJ), Annals of Internal Medicine (AIM) and the Canadian Medical Association Journal (CMAJ). Our search strategy included terms for RCTs

Statistical analyses

Firstly, we determined the mean number of items reported (0–17) for each year and estimated the unadjusted and adjusted differences using a two-sample t-test and generalized estimation equations (GEEs) respectively [20]. The means are reported with standard deviations (SDs). The mean differences and adjusted means are reported with 95% CIs and p-values.

Secondly, we compared compliance with the 17 items of the CONSORT statement for abstracts for years 2007 versus 2012 using individual

Results

Our search retrieved 445 articles of which 199 and 230 were published in 2007 and 2012 respectively. We then randomly selected 100 for each year. Ten (8 from 2007 and 2 from 2012) articles did not meet our inclusion criteria and were replaced by another random selection. A flow diagram of our study selection process is shown in Fig. 1.

Agreement was considered high for inclusion of articles (Kappa = 0.60 [95% CI 0.29, 0.92]; p < 0.001; given the maximum attainable Kappa = 0.60). Overall we compared

Discussion

In this study, we report differences in the quality of reporting of abstracts in a random selection of articles published in major general medicine journals in 2012 and 2007 using the CONSORT extension for abstracts [8]. For some items there was an improvement while for others, reporting quality seemed to stagnate or regress. Overall, there was an improvement in the quality of reporting of abstracts since the publication of the CONSORT extension for abstracts in 2008, with an average of 3 more

Conclusion

In 2012 compared to 2007 there has been some improvement in reporting of RCT abstracts in top-tier medical journals according to the CONSORT for abstract checklist, with regard to the number of items reported and for specific items, but there is still room for improvement. Journal endorsement and implementation of the CONSORT statement for abstracts are warranted.

References (22)

  • P. Fontelo et al.

    Comparing data accuracy between structured abstracts and full-text journal articles: implications in their use for informing clinical decisions

    Evid Based Med

    (2013)
  • Cited by (60)

    • Do not make clinical decisions based on abstracts of healthcare research: A systematic review

      2021, Journal of Clinical Epidemiology
      Citation Excerpt :

      The main finding is that even after the release of the CONSORT-A and PRISMA-A, the reporting quality of abstracts of RCTs and SRs still remains suboptimal with much room for improvement in all healthcare areas. Eleven overviews [3,5,29,31-33,35,38,42,43,47] that compared articles (RCTs or SRs) published before and after the release of such abstracts guidelines also presented only small improvements over time, sometimes showing a statistical significant change but the magnitude of change was not large enough to be considered sufficiently well reported. Abstracts reporting quality across all healthcare areas were poor, even if the reporting quality has varied across the same healthcare area (e.g. within dentistry one overview concluded that 62.5% of abstracts were fully reported items, [46] while another overview concluded that this percentage was 28.1% [38]).

    View all citing articles on Scopus
    1

    CONSORT: Consolidated Standards of Reporting Trials.

    2

    RCT: Randomized controlled trial.

    3

    IRRs: Incidence rate ratios.

    4

    CI: Confidence interval.

    View full text