Relation between sampling device and detection of abnormality in cervical smears: a meta-analysis of randomised and quasi-randomised studies
BMJ 1996; 313 doi: https://doi.org/10.1136/bmj.313.7068.1285 (Published 23 November 1996) Cite this as: BMJ 1996;313:1285
- a Departments of General Practice, Universities of Louvain (Belgium) and Maastricht (Netherlands)
- b Department of General Practice, University of Maastricht (Netherlands)
- Correspondence to: Professor Frank Buntinx, Department of General Practice, Universiteit Maastricht, PO Box 616–6200 MD Maastricht, Netherlands.
- Accepted 19 September 1996
Abstract
Objective: To assess the diagnostic yield of different sampling devices used in cervical screening.
Design: Meta-analysis of randomised and quasi-randomised studies.
Setting: All randomised and quasi-randomised studies comparing the yield of cytological or histological abnormalities when two or more different sampling devices were used.
Subjects: 85 000 patients included in 29 studies reported in 28 papers.
Main outcome measures: Pooled relative risk and 95% confidence interval of the yield of mild dysplasia or worse in smears recovered by each sampling method versus each other method with which it was compared; sensitivity or positive predictive value, or both, of cytological versus histological results in six studies from which sufficient data were available.
Results: There were no substantial differences in the yield of cytological abnormalities between the Ayre spatula, the Cytobrush, and the cotton swab used alone. There were also no substantial differences in the yield of cytological abnormalities between the extended tip spatula, the Ayre spatula combined with the Cytobrush or cotton swab, or the Cervex brush. The Ayre spatula, Cytobrush, or cotton swab used alone generally performed significantly worse than the combinations, the extended tip spatula, or the Cervex brush. There were no substantial differences in sensitivity or positive predictive value between the sampling methods.
Conclusions: These results support the use of either the extended tip spatula, a combination of any spatula plus the Cytobrush or cotton swab, or the Cervex brush for cervical screening.
Key messages
The Ayre spatula, Cytobrush, or cotton swab alone generally performed significantly worse than the combination of a spatula and Cytobrush or cotton swab, the extended tip spatula, or the Cervex brush
No substantial differences in sensitivity or positive predictive value were detected between the sampling methods
Results support the use of either the extended tip spatula, the combination of any spatula plus Cytobrush or cotton swab, or the Cervex brush for cervical screening
Introduction
During 50 years after the first publications on cervical screening1 2 and 30 years after the initial method of vaginal aspiration was abandoned3 several studies assessed the relative value of newer sampling methods. However, because of lack of statistical significance resulting from variations in the designs of the studies and small numbers in many we still do not know which sampling device is best.
In a previous review Buntinx et al found that using a spatula and the Cytobrush in combination yielded the highest proportion of smears with endocervical or metaplastic cells,4 an important indicator of sampling quality. The question remains, however, whether improved cervical sampling will lead to higher diagnostic yield. In order to determine which method achieves the best diagnostic yield we reviewed published randomised and quasi-randomised studies comparing the effects of different sampling techniques on the proportion of pathological smears recovered.
In most studies cytological criteria (proportions of smears showing dysplasia) are used as a measure of outcome. Some studies also present histological data. These results permit the additional calculation of sensitivity or positive predictive value, or both, for each sampling method.
Methods
RETRIEVAL OF PAPERS
An on line Medline search was augmented by a similar check of FAMLI (Family Literature Index), by carefully screening the references in all articles, and by contacting experts in the subject. “Cervix—neoplasms/Prevention and control” and “Vaginal—smears/all subheadings” were used as keywords for the initial computer search. Searches were updated to the end of 1994. No language restrictions were used. Publication bias was examined by using funnel plots for those comparisons for which at least four studies were available.
SELECTION OF PAPERS
Initially all controlled studies examining the relation between the presence of abnormality in a smear and the type of sampling device were selected. Generally in this review when cytological results are used as the outcome measure they refer to mild dysplasia or worse. When histological results are used they refer to a diagnosis of at least grade I cervical intraepithelial neoplasia or at least grade III cervical intraepithelial neoplasia in tissues obtained by biopsy or cone biopsy or during surgery. Papers were excluded if they lacked the necessary data; if no randomised, quasi-randomised, or paired design was used; or if methods or outcome were insufficienly defined.
Extended tip spatulas—that is, spatulas whose tips have been extended to permit endocervical sampling—exist in multiple shapes and materials. Though this may influence the recovery of pathological smears, all spatulas were classified as either classic Ayre spatulas or extended tip spatulas. When details were missing spatulas were assigned to the extended tip group.
Some studies lasted several years. As the numbers of smears examined increased so the credibility and precision of the studies increased. Some accounts were of studies to be followed up years later by a second report that covered a longer period.5 6 7 8 9 In these cases the most recent report was used. One report describing two completely different studies was treated as two separate reports.10
METHODOLOGICAL STRENGTH
Study methodology was reviewed by systematically identifying the number of smears examined, the setting of the study (population based, unselected, referred, or otherwise selected patients), the type of assignment of each patient to a sampling method (paired with or without randomisation of the sequence, randomisation, or quasi-randomisation), and the outcome measures (cytological or histological, or both). These indicators were judged independently by two reviewers, rare discordances being decided by consensus.
DESCRIPTIVE ANALYSIS
The relative risk and 95% confidence interval of the proportion of cytological abnormalities in smears obtained with one device versus another were used to express the results of the various studies.11 12 When histological data were available we used these as the standard when calculating sensitivity and positive predictive value (and 95% confidence intervals) of the cytological result. These were estimated separately for mild dysplasia or worse and severe dysplasia or worse (cytological) versus at least grade I cervical intraepithelial neoplasia and at least grade III cervical intraepithelial neoplasia (histological).
For all comparisons we estimated the statistical power of all available studies to detect a relative risk of 1.25. Therefore, standard errors were calculated from the confidence intervals after logistic transformation. Results below 0.99 were rounded to the nearest 5%.
STATISTICAL POOLING
Pooled relative risks and their 95% confidence intervals were calculated by a Mantel-Haenszel estimation. However, when the 2 test showed significant heterogeneity between studies a random effects method was applied by means of Fast-Pro 1.7 software.13
Results
Of 41 papers retrieved, 13 were excluded because of lacking essential crude data,14 15 the use of a cohort16 17 or pre-post5 6 15 18 19 20 21 design, extreme selection of patients,22 inadequate definition of method23 or outcome,24 or reporting partial results from a study on which a later report was available.5 One report described two different studies, which we considered separately.10 The 28 papers selected for the review therefore covered 29 studies.10 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 Seven studies concerned screening a non-selected population. In all other studies patients were selected or referred, generally for further evaluation after an abnormal smear. Ten studies used a paired design, two or more methods being used for the same patients during the same examination. In 18 studies patients were assigned to a method by randomisation (n = 8) or quasirandomisation (n = 10) (table 1). All studies presented cytological data for analysis, and in eight studies histological data were available also (table 1).
Methodological details of the 29 selected studies
CYTOLOGICAL RESULTS
2 Tests indicated significant between study heterogeneity in cytological findings in one comparison only. When comparing an extended tip spatula and the Cytobrush (with mild dysplasia used as cut off) heterogeneity seemed to be associated with one study10 showing a very high relative risk (7.32) based on 17 pathological smears only. Removing this study resulted in homogeneity with a significant relative risk of 0.93 (table 2). Only for the comparison of spatula plus Cytobrush versus spatula plus cotton swab did the funnel plot suggest publication bias in favour of the spatula plus Cytobrush. (A copy of all funnel plots is available from FB.)
Diagnostic yield in each study with power, 2 test for heterogeneity, pooled relative risk, and 95% confidence interval for each comparison
Mild dysplasia or worse as outcome measure
Ayre spatula—A combination of the Ayre spatula plus an endocervical device (cotton swab or Cytobrush) performed better than the Ayre spatula used alone in detecting dysplasia. Pooling of paired studies only resulted in a pooled relative risk of 1.13 (95% confidence interval 1.03 to 1.24) for the combination including the Cytobrush. Both studies comparing the Ayre spatula with the Ayre spatula plus cotton swab used a paired design. The extended tip spatula and the Cervex brush also performed better than the Ayre spatula, though not significantly so. Studies were in referred patients as well as in unselected populations. Results of all comparisons were homogeneous.
Extended tip spatulas performed better than the Ayre spatula or the endocervical devices used alone. Results with extended tip spatulas were comparable to those with the Ayre spatula plus Cytobrush combination (relative risk 1.02), the Cervex brush (1.00), and the Cytopick (0.92).
Cytobrush or cotton swab alone—Using the Cytobrush alone resulted in a higher yield of cytological abnormalities than with the cotton swab used alone (relative risk 1.41). Both devices, however, performed significantly worse than most devices with which they were compared. Comparisons with the Ayre spatula and extended tip spatulas gave non-significant results.
Ayre spatula plus cotton swab or Cytobrush—The combination of Ayre spatula plus cotton swab or Cytobrush performed significantly better than the Ayre spatula, cotton swab, or (for spatula plus cotton swab only) Cytobrush used alone. Comparisons of these combinations showed no significant differences. However, power for these comparisons was below 0.50.
Extended tip spatula plus cotton swab or Cytobrush—Comparison of the combination of extended tip spatula plus cotton swab or Cytobrush with extended tip spatula, Cervex brush, or Cytopick used alone generally showed no significant differences. Power was low for most studies of spatula plus the Cytobrush. The one (low power) study comparing the combinations showed a better, though non-significant outcome for the extended tip spatula plus the Cytobrush. When using either the Ayre spatula plus cotton swab or the Ayre spatula alone (depending of an everted Cervex (n = 252)), Garite and Feldman detected mild dysplasia or worse in 5% of smears, which was comparable to the 4.5% when using the spatula alone and much less than when using the combination (11%).30
Cervex brush—In none of the six comparisons including the Cervex brush was a significantly better or worse outcome recorded. Four of the six comparisons had high statistical power.
Other devices—No significant difference was found in any comparison including the Cytopick, a combination of Ayre spatula plus aspirator, the Profile brush, or the Bayne brush. In one paired study significantly more smears containing mild dysplasia or worse were obtained with the Multispatula than with the Ayre spatula.37 No comparison was examined in more than one study. Most studies had small sample sizes.
Severe dysplasia or worse as outcome measure
The results of most studies using severe dysplasia or worse as the outcome measure followed the trends of the those using mild dysplasia or worse as the outcome measure. Generally sample sizes were small and the results non-significant. The Ayre spatula plus cotton swab combination and the extended tip spatula plus cotton swab or Cytobrush combination, however, performed significantly better than the Ayre spatula and extended tip spatula, respectively.
HISTOLOGICAL COMPARISONS
Sensitivities and positive predictive values of smears obtained with different devices were similar. Almost all values were well within the 95% confidence intervals of the other values. Sample sizes for all these analyses were generally small, with denominators between 31 and 348.
For mild dysplasia or worse (cytological) versus at least grade III cervical intraepithelial neoplasia (histological) sensitivities were generally 85% or higher with positive predictive values of around 50%. Sensitivities were lower and positive predictive values higher in analyses using higher cut off points for cytological abnormality (table 3). To avoid drawing conclusions based on data that may not come from the same set of studies statistical pooling was not performed.
Sensitivity and positive predictive value of studies from which sufficient data were available
Discussion
Certain limitations should be borne in mind when considering this review. Important interobserver variation exists in cytotechnologists and pathologists.52 53 54 55 Hence in studies with only small numbers of smears cytological assessments are often done by one person. In large studies it may be difficult to adjust for this problem. In paired studies the sequence of assignment may influence results.56 57 Some researchers resolve this by changing the sequence continuously. Others place the technique of which they expect the best results in the position considered most disadvantageous. We fear that in some studies extended tip spatulas were reported as Ayre-type spatulas. Therefore, all relevant pooled relative risks were recalculated after combining data for Ayre-type spatulas and extended tip spatulas. This produced no substantial difference in the point estimate or significance of the original results.
Apart from sampling method, factors such as the training and experience of the sampler,58 the day of the menstrual cycle, and the population studied may influence outcome. We, however, have no reason to think that such bias selectively favoured one device over another. Obtaining a second cervical sample immediately after the first—even with the same device—increases the yield of cytological abnormalities by almost one third.59 60 61 This is important when comparing the yield of two combined samples (for example, obtained with any spatula plus cotton swab or Cytobrush) with the yield from one sample only. Only one funnel plot provided evidence for publication bias. A substantial role of publication bias in this review therefore seems unlikely. The review comprised randomised and quasi-randomised studies only. However, repeating the analysis with the inclusion of cohort16 17 and pre-post studies15 18 19 20 yielded closely similar results.
Taking into account the limitations several conclusions can be drawn. It is plainly not appropriate to sample with a Cytobrush, a cotton swab, or an Ayre spatula alone. In all comparisons they turned out to be worse, in many cases significantly so. Comparisons between a combination of spatula plus cotton swab or Cytobrush, an extended tip spatula, and the Cervex brush mostly showed no significant differences. This could result from a true similarity in diagnostic yield or from sample sizes that were too small to show statistically significant differences. Statistical power estimated for each comparison was high in some cases and very low in others. This suggests that for part of the comparisons additional research may be appropriate.
False positive results are unlikely to be selectively more frequent with any specific device. This is even less likely given the similar predictive values of the cytological results in relation to the histological results. We emphasise, however, that very small numbers of patients were included in these studies. In six studies sufficient histological data were available. All but one of these studies, however, comprised highly selected patients.
In summary, though some comparisons were based on small numbers, our findings support the advice to use either any spatula plus the Cytobrush or a cotton swab, a Cervex brush, or an extended tip spatula to obtain cervical smears. There is no sound evidence to support the use of other devices.
A copy of all funnel plots may be obtained by writing direct to FB. We acknowledge the invaluable help of Berna Schouten for logistical work, Andre Knottnerus for commenting on a previous version of this paper, and Emmanuel Lesaffre for advice on power calculations.
Footnotes
-
Funding None
-
Conflict of interest None.