Selecting the language of the publications included in a meta-analysis: Is there a tower of babel bias?

doi:10.1016/0895-4356(94)00098-B

Journal of Clinical Epidemiology

Volume 48, Issue 1, January 1995, Pages 159-163

https://doi.org/10.1016/0895-4356(94)00098-B Get rights and content

Abstract

Although they usually claim a very thorough search to retrieve every pertinent work, most meta-analyses published in English restrict their search to papers which were also published in English. We reviewed all the meta-analyses printed from 1 January 1991 to 1 April 1993 in 8 medical journals published in English and selected those who stated linguistic restrictions for inclusion in the analysis. The computerized search methods used in these meta-analyses were then duplicated looking specifically for publications written in the excluded languages. Each meta-analysis was then redone with identical statistical tests to determine if its conclusions would have been different if the paper(s) absent only for linguistic reasons had been included. A total of 36 meta-analyses of which 28 had language restrictions were identified. The computer searches yielded 19 papers scientifically acceptable but excluded for linguistic reasons. Eleven of these articles were retained as having the potential to modify their corresponding 7 meta-analyses. One meta-analysis which concluded that selective decontamination of the digestive tract in intensive care units did not produce a significant change in mortality between treatment and control patients (OR 0.70; 95% CI 0.45–1.09) would have arrived at a different conclusion (OR 0.67; 95% CI 0.47–0.95) if a paper written in German in a Swiss journal had been included in the analysis. Our study demonstrates that, in at least one out of 36 consecutive meta-analyses the exclusion of papers for linguistic reasons produced results different from those which would have been obtained if this exclusion criteria had not been used.

References (52)

R Rosenthal
The “File drawer problem” and tolerance for null results
Psychol Bull
(1979)
K Dickersin et al.
Publication bias and clinical trials
Controlled Clin Trials
(1987)
CM Vandenbroucke-Grauls et al.
Effect of selective decontamination of the digestive tract on respiratory tract infections and mortality in the intensive care unit
Lancet
(1991)
KI Kennedy et al.
Contraceptive efficacy of lactational amenorrhoea
Lancet
(1992)
QE Whiting-O'Keefe et al.
Methotrexate and histologic hepatic abnormalities: a metaanalysis
Am J Med
(1991)
S Sazawal et al.
Meta-analysis of intervention trials on case-management of pneumonia in community settings
Lancet
(1992)
MT Nurmohamed et al.
Low-molecular-weight heparin versus standard heparin in general and orthopaedic surgery: a meta-analysis
Lancet
(1992)
R Collin et al.
Avoidance of large biases and large random errors in the assessment of moderate treatment effects: the need for systematic overviews
Stat Med
(1987)
KK Hoffmann et al.
Transparent polyurethane film as an intravenous catheter dressing. A meta-analysis of the infection risks (see comments)
JAMA
(1992)
E Abrutyn et al.
Intrathecal therapy in tetanus. A meta-analysis
JAMA
(1991)

TF Imperiale et al.

A meta-analysis of low-dose aspirin for the prevention of pregnancy-induced hypertensive disease

JAMA

(1991)

PS Denker et al.

Fasting serum insulin levels in essential hypertension. A meta-analysis

Arch Intern Med

(1992)

Chemotherapy in advanced ovarian cancer: an overview of randomised clinical trials. Advanced Ovarian Cancer Trialists Group

Br Med J

(1991)

KK Steinbert et al.

A meta-analysis of the effect of estrogen replacement therapy on the risk of breast cancer

JAMA

(1991)

AL Peters et al.

Insulin plus a sulfonylurea agent for treating type 2 diabetes

Ann Intern Med

(1991)

LR Sutherland et al.

Sulfasalazine revisited: a meta-analysis of 5-aminosalicylic acid in the treatment of ulcerative colitis

Ann Intern Med

(1993)

KK Teo et al.

Effects of intravenous magnesium in suspected acute myocardial infarction: overview of randomised trials

Br Med J

(1991)

T Poynard et al.

Beta-adrenergic-antagonist drugs in the prevention of gastrointestinal bleeding in patients with cirrhosis and esophageal varices

N Engl J Med

(1991)

PG Shekelle et al.

Spinal manipulation for low-back pain

Ann Intern Med

(1992)

JE Pope et al.

A meta-analysis of the effects of nonsteroidal anti-inflammatory drugs on blood pressure

Arch Intern Med

(1993)

RC Brownson et al.

Cigarette smoking and adult leukemia. A meta-analysis

Arch Intern Med

(1993)

WW Fawzi et al.

Vitamin A supplementation and child mortality. A meta-analysis

JAMA

(1993)

BL Kasiske et al.

Elective cyclo-sporine withdrawal after renal transplantation. A meta-analysis

JAMA

(1993)

CM Ripsin et al.

Oat products and lipid lowering. A meta-analysis

JAMA

(1992)

VI Lugo-Miro et al.

Comparison of different metronidazole therapeutic regimens for bacterial vaginosis. A meta-analysis

JAMA

(1992)

WM Macharia et al.

An overview of interventions to improve compliance with appointment keeping for medical services

JAMA

(1992)

Cited by (428)

Hematopoietic cell-based and non-hematopoietic cell-based strategies for immune tolerance induction in living-donor renal transplantation: A systematic review
2023, Transplantation Reviews
Despite its use to prevent acute rejection, lifelong immunosuppression can adversely impact long-term patient and graft outcomes. In theory, immunosuppression withdrawal is the ultimate goal of kidney transplantation, and is made possible by the induction of immunological tolerance. The purpose of this paper is to review the safety and efficacy of immune tolerance induction strategies in living-donor kidney transplantation, both chimerism-based and non-chimerism-based. The impact of these strategies on transplant outcomes, including acute rejection, allograft function and survival, cost, and immune monitoring, will also be discussed.
Databases such as PubMed, Scopus, and Web of Science, as well as additional online resources such as EBSCO, were exhaustively searched. Adult living-donor kidney transplant recipients who developed chimerism-based tolerance after concurrent bone marrow or hematopoietic stem cell transplantation or those who received non-chimerism-based, non-hematopoietic cell therapy using mesenchymal stromal cells, dendritic cells, or regulatory T cells were studied between 2000 and 2021. Individual sources of evidence were evaluated critically, and the strength of evidence and risk of bias for each outcome of the transplant tolerance study were assessed.
From 28,173 citations, 245 studies were retrieved after suitable exclusion and duplicate removal. Of these, 22 studies (2 RCTs, 11 cohort studies, 6 case-control studies, and 3 case reports) explicitly related to both interventions (chimerism- and non-chimerism-based immune tolerance) were used in the final review process and were critically appraised. According to the findings, chimerism-based strategies fostered immunotolerance, allowing for the safe withdrawal of immunosuppressive medications. Cell-based therapy, on the other hand, frequently did not induce tolerance except for minimising immunosuppression. As a result, the rejection rates, renal allograft function, and survival rates could not be directly compared between these two groups. While chimerism-based tolerance protocols posed safety concerns due to myelosuppression, including infections and graft-versus-host disease, cell-based strategies lacked these adverse effects and were largely safe.
There was a lack of direct comparisons between HLA-identical and HLA-disparate recipients, and the cost implications were not examined in several of the retrieved studies. Most studies reported successful immunosuppressive weaning lasting at least 3 years (ranging up to 11.4 years in some studies), particularly with chimerism-based therapy, while only a few investigators used immune surveillance techniques. The studies reviewed were often limited by selection, classification, ascertainment, performance, and attrition bias.
This review demonstrates that chimerism-based hematopoietic strategies induce immune tolerance, and a substantial number of patients are successfully weaned off immunosuppression. Despite the risk of complications associated with myelosuppression. Non-chimerism-based, non-hematopoietic cell protocols, on the other hand, have been proven to facilitate immunosuppression minimization but seldom elicit immunological tolerance. However, the results of this review must be interpreted with caution because of the non-randomised study design, potential confounding, and small sample size of the included studies. Further validation and refinement of tolerogenic protocols in accordance with local practice preferences is also warranted, with an emphasis on patient selection, cost ramifications, and immunological surveillance based on reliable tolerance assays.
Developmental language disorder in Chinese children: A systematic review of research from 1997 to 2022
2023, Brain and Language
Developmental language disorder (DLD) is one of the most common neurodevelopmental disorders. The influences of DLD on language development have been delineated in detail in English. The same is not true for Chinese, a group of Sinitic languages with distinct typological features that may modify the profile of DLD crosslinguistically. We conducted a systematic search of English and Chinese journal databases and reviewed 59 studies on the manifestations of DLD in Chinese. Methodological quality appraisal of the literature revealed several areas of improvement to enhance transparency and replicability. A bibliometric analysis indicated a steep growth trajectory of this literature. Examination of the participant selection and diagnostic criteria revealed limitations and calls for the development of assessment tools and increased knowledge of evidenced-based diagnostic practice. Areas of deficits demonstrated by Chinese children with DLD were synthesized qualitatively and discussed in light of the literature on clinical markers of DLD in English.
Perioperative complications of a transvaginal cervical cerclage in singleton pregnancies: a systematic review and meta-analysis
2023, American Journal of Obstetrics and Gynecology
Given that many studies report on a limited spectrum of adverse events of transvaginal cervical cerclage for preventing preterm birth, but are not powered to draw conclusions about its safety, the objective of this study was to conduct a systematic review with pooled risk analyses of perioperative complications and compare characteristics on the basis of indication for cerclage in singleton pregnancies.
Ovid MEDLINE, Ovid Embase, Web of Science, the Cochrane Central Register of Controlled Trials (CENTRAL), and the prospective trial registers ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform were searched from inception to April 2020.
All randomized controlled trials and both retrospective and prospective observational cohort studies reporting about complications in history-indicated cerclage, ultrasound-indicated cerclage, or physical examination-indicated cerclage were eligible. Studies were included if they contained original data on the occurrence of adverse events during surgery or within 24 hours after surgery.
The Cochrane risk of bias tool for randomized controlled trials and the Newcastle–Ottawa scale for cohort and case–control studies were used for the critical appraisal. The pooled risk assessment was conducted using meta and metafor packages in R (studio), version 4.0.3.
The search yielded 2328 potential studies; 3 randomized controlled trials, 3 prospective, and 38 retrospective cohort studies were included in the final analysis. Of the 4511 women with singleton gestations, 1561 (34.6%) underwent history-indicated cerclage, 1348 (29.9%) underwent ultrasound-indicated cerclage, and 1549 (33.3%) underwent physical examination-indicated cerclage. Most perioperative complications occurred in physical examination-indicated cerclage, especially hemorrhage (2.3%; 95% confidence interval, 0.0–7.6) and preterm premature rupture of membranes (2.5%; 95% confidence interval, 0.91–4.5). The fewest complications occurred in history-indicated cerclage, varying from 0.0% of preterm premature rupture of membranes (95% confidence interval, 0.0–1.7) to 0.9% of hemorrhage (95% confidence interval, 0.0–7.9). In ultrasound-indicated cerclage, the most common complication was hemorrhage (1.4%; 95% confidence interval, 0.0–4.1), followed by lacerations (0.6%; 95% confidence interval, 0.0–3.1) and preterm premature rupture of membranes (0.3%; 95% confidence interval, 0.0–0.8).
The highest risk of perioperative complications was observed in physical examination-indicated cerclage in comparison with ultrasound- and history-indicated cerclage. However, the occurrence of complications is poorly documented in the published literature, as is the timing of the complications (ie, perioperative or later in pregnancy). There is an urgent need for uniform complication reporting policy in both cohort studies and randomized controlled trials on cerclage.
Effectiveness, reach, uptake, and feasibility of digital health interventions for adults with hypertension: a systematic review and meta-analysis of randomised controlled trials
2023, The Lancet Digital Health
Digital health interventions are effective for hypertension self-management, but a comparison of the effectiveness and implementation of the different modes of interventions is not currently available. This study aimed to compare the effectiveness of SMS, smartphone application, and website interventions on improving blood pressure in adults with hypertension, and to report on their reach, uptake, and feasibility.
In this systematic review and meta-analysis we searched CINAHL Complete, Cochrane Central Register of Controlled Trials, Ovid Embase, Ovid MEDLINE, and APA PsycInfo on May 25, 2022, for randomised controlled trials (RCTs) published in English from Jan 1, 2009, that examined the effectiveness of digital health interventions on reducing blood pressure in adults with hypertension. Screening was carried out using Covidence, and data were extracted following Cochrane's guidelines. The primary endpoint was change in the mean of systolic blood pressure. Risk of bias was assessed with Cochrane Risk of Bias 2. Data on systolic and diastolic blood pressure reduction were synthesised in a meta-analysis, and data on reach, uptake and feasibility were summarised narratively. Grading of Recommendations, Assessment, Development, and Evaluation criteria were used to evaluate the level of evidence. The study was registered with PROSPERO CRD42021247845.
Of the 3235 records identified, 29 RCTs from 13 regions (n=7592 participants) were included in the systematic review, and 28 of these RCTs (n=7092 participants) were included in the meta-analysis. 11 studies used SMS as the primary mode of delivery of the digital health intervention, 13 used smartphone applications, and five used websites. Overall, digital health intervention group participants had a –3·62 mm Hg (95% CI –5·22 to –2·02) greater reduction in systolic blood pressure, and a –2·45 mm Hg (–3·83 to –1·07) greater reduction in diastolic blood pressure, compared with control group participants. No statistically significant differences between the three different modes of delivery were observed for both the systolic (p=0·73) and the diastolic blood pressure (p=0·80) outcomes. Smartphone application interventions had a statistically significant reduction in diastolic blood pressure (–2·45 mm Hg [–4·15 to –0·74]); however, there were no statistically significant reductions for SMS interventions (–1·80 mm Hg [–4·60 to 1·00]) or website interventions (–3·43 mm Hg [–7·24 to 0·38]). Due to the considerable heterogeneity between included studies and the high risk of bias in some, the level of evidence was assigned a low overall score. Interventions were more effective among people with greater severity of hypertension at baseline. SMS interventions reported higher reach and smartphone application studies reported higher uptake, but differences were not statistically significant.
SMS, smartphone application, and website interventions were associated with statistically and clinically significant systolic and diastolic blood pressure reductions, compared with usual care, regardless of the mode of delivery of the intervention. This conclusion is tempered by the considerable heterogeneity of included studies and the high risk of bias in most. Future studies need to describe in detail the mediators and moderators of the effectiveness and implementation of these interventions, to both further improve their effectiveness as well as increase their reach, uptake, and feasibility.
European Union's Horizon 2020 Research and Innovation Programme.
Effectiveness, reach, uptake, and feasibility of digital health interventions for adults with type 2 diabetes: a systematic review and meta-analysis of randomised controlled trials
2023, The Lancet Digital Health
Digital health interventions have shown promising results for the management of type 2 diabetes, but a comparison of the effectiveness and implementation of the different modes is not currently available. Therefore, this study aimed to compare the effectiveness of SMS, smartphone application, and website-based interventions on improving glycaemia in adults with type 2 diabetes and report on their reach, uptake, and feasibility.
In this systematic review and meta-analysis, we searched CINAHL, Cochrane Central, Embase, MEDLINE, and PsycInfo on May 25, 2022, for randomised controlled trials (RCTs) that examined the effectiveness of digital health interventions in reducing glycated haemoglobin A_1c (HbA_1c) in adults with type 2 diabetes, published in English from Jan 1, 2009. Screening was carried out using Covidence, and data were extracted following Cochrane's guidelines. The primary endpoint assessed was the change in the mean (and 95% CI) plasma concentration of HbA_1c at 3 months or more. Cochrane risk of bias 2 was used to assess risk of bias. Data on reach, uptake, and feasibility were summarised narratively and data on HbA_1c reduction were synthesised in a meta-analysis. Grading of Recommendations, Assessment, Development, and Evaluation criteria was used to evaluate the level of evidence. The study was registered with PROSPERO, CRD42021247845.
Of the 3236 records identified, 56 RCTs from 24 regions (n=11 486 participants), were included in the narrative synthesis, and 26 studies (n=4546 participants) in the meta-analysis. 20 studies used SMS as the primary mode of delivery of the digital health intervention, 25 used smartphone applications, and 11 implemented interventions via websites. Smartphone application interventions reported higher reach compared with SMS and website-based interventions, but website-based interventions reported higher uptake compared with SMS and smartphone application interventions. Effective interventions, in general, included people with greater severity of their condition at baseline (ie, higher HbA_1c) and administration of a higher dose intensity of the intervention, such as more frequent use of smartphone applications. Overall, digital health intervention group participants had a –0·30 (95% CI –0·42 to –0·19) percentage point greater reduction in HbA_1c, compared with control group participants. The difference in HbA_1c reduction between groups was statistically significant when interventions were delivered through smartphone applications (–0·42% [–0·63 to –0·20]) and via SMS (–0·37% [–0·57 to –0·17]), but not when delivered via websites (–0·09% [–0·64 to 0·46]). Due to the considerable heterogeneity between included studies, the level of evidence was moderate overall.
Smartphone application and SMS interventions, but not website-based interventions, were associated with better glycaemic control. However, the studies' heterogeneity should be recognised. Considering that both smartphone application and SMS interventions are effective for diabetes management, clinicians should consider factors such as reach, uptake, patient preference, and context of the intervention when deciding on the mode of delivery of the intervention. Nine in ten people worldwide own a feature phone and can receive SMS and four in five people have access to a smartphone, with numerous smartphone applications being available for diabetes management. Clinicians should familiarise themselves with this modality of programme delivery and encourage people with type 2 diabetes to use evidence-based applications for improving their self-management of diabetes. Future research needs to describe in detail the mediators and moderators of the effectiveness and implementation of SMS and smartphone application interventions, such as the optimal dose, frequency, timing, user interface, and communication mode to both further improve their effectiveness and to increase their reach, uptake, and feasibility.
EU's Horizon 2020 Research and Innovation Programme.
A quality assessment of a sample of herbal medicine systematic reviews using R-AMSTAR
2022, Phytomedicine Plus
The numbers of herbal medicine systematic reviews (SR) are increasing, and reviews need to be good quality to inform practice and research. This study assesses the quality of herbal medicine SR and makes recommendations to those reading and writing them. Two open datasets from published meta-reviews were used as a sample, in which R-AMSTAR (Revised Assessment of Multiple Systematic Reviews) was used to assess the method quality of 94 SR. This scrutinises 11 domains including search strategy, risk of bias and other components, and compiles a score. The datasets were examined in Microsoft Excel.
For the herbs studied, the numbers of SRs published had increased over time but quality standards had not altered. Reviews published with specialist groups and those with meta-analyses were higher quality. Overall, some methodological steps were performed well such as article searching. Other steps like pre-registering a protocol and providing a clear basis for selecting articles to assess could be enhanced.
To conclude, the quality of SR was variable in this sample, on par with other medical areas. The use of quality assurance tools could help improve authors’ skills, although some aspects of quality assessed were influenced by the complexity of botanical material and herbal medicine.

View all citing articles on Scopus

View full text

Selecting the language of the publications included in a meta-analysis: Is there a tower of babel bias?

Abstract

Psychol Bull

Controlled Clin Trials

Lancet

Lancet

Am J Med

Lancet

Lancet

Avoidance of large biases and large random errors in the assessment of moderate treatment effects: the need for systematic overviews

Stat Med

Transparent polyurethane film as an intravenous catheter dressing. A meta-analysis of the infection risks (see comments)

JAMA

Intrathecal therapy in tetanus. A meta-analysis

JAMA

A meta-analysis of low-dose aspirin for the prevention of pregnancy-induced hypertensive disease

JAMA

Fasting serum insulin levels in essential hypertension. A meta-analysis

Arch Intern Med

Chemotherapy in advanced ovarian cancer: an overview of randomised clinical trials. Advanced Ovarian Cancer Trialists Group

Br Med J

A meta-analysis of the effect of estrogen replacement therapy on the risk of breast cancer

JAMA

Insulin plus a sulfonylurea agent for treating type 2 diabetes

Ann Intern Med

Sulfasalazine revisited: a meta-analysis of 5-aminosalicylic acid in the treatment of ulcerative colitis

Ann Intern Med

Effects of intravenous magnesium in suspected acute myocardial infarction: overview of randomised trials

Br Med J

Beta-adrenergic-antagonist drugs in the prevention of gastrointestinal bleeding in patients with cirrhosis and esophageal varices

N Engl J Med

Spinal manipulation for low-back pain

Ann Intern Med

A meta-analysis of the effects of nonsteroidal anti-inflammatory drugs on blood pressure

Arch Intern Med

Cigarette smoking and adult leukemia. A meta-analysis

Arch Intern Med

Vitamin A supplementation and child mortality. A meta-analysis

JAMA

Elective cyclo-sporine withdrawal after renal transplantation. A meta-analysis

JAMA

Oat products and lipid lowering. A meta-analysis

JAMA

Comparison of different metronidazole therapeutic regimens for bacterial vaginosis. A meta-analysis

JAMA

An overview of interventions to improve compliance with appointment keeping for medical services

JAMA