Download PDF
Methods Inf Med 2004; 43(02): 163-170
DOI: 10.1055/s-0038-1633855
Original Article
Schattauer GmbH

Quality Assessments of HMO Diagnosis Databases Used to Monitor Childhood Vaccine Safety

J. Mullooly
1   Center for Health Research, Kaiser Permanente Northwest, Portland, Oregon, USA
,
L. Drew
1   Center for Health Research, Kaiser Permanente Northwest, Portland, Oregon, USA
,
F. DeStefano
2   Centers for Disease Control and Prevention, Atlanta, Georgia, USA
,
J. Maher
3   Oregon Department of Human Services, Portland, Oregon, USA
,
K. Bohlke
4   Group Health Cooperative, Seattle, Washington, USA
,
V. Immanuel
4   Group Health Cooperative, Seattle, Washington, USA
,
S. Black
5   Kaiser Permanente Southern California, Downey, California, USA
,
E. Lewis
5   Kaiser Permanente Southern California, Downey, California, USA
,
P. Ray
5   Kaiser Permanente Southern California, Downey, California, USA
,
C. Vadheim
6   Kaiser Permanente Northern California, Fresno, California, USA
,
M. Lugg
6   Kaiser Permanente Northern California, Fresno, California, USA
,
R. Chen
2   Centers for Disease Control and Prevention, Atlanta, Georgia, USA
› Author Affiliations
Further Information

Publication History

Publication Date:
05 February 2018 (online)

    Permissions and Reprints

Summary

Objective: To assess the quality of automated diagnoses extracted from medical care databases by the Vaccine Safety Datalink (VSD) study.

Methods: Two methods are used to assess quality of VSD diagnosis data. The first method compares common automated and abstracted diagnostic categories (“outcomes”) in 1-2% simple random samples of study populations. The second method estimates positive predictive values of automated diagnosis codes used to identify potential cases of rare conditions (e.g., acute ataxia) for inclusion in nested case-control medical record abstraction studies.

Results: There was good agreement (64-68%) between automated and abstracted outcomes in the 1-2% simple random samples at 3 of the 4 VSD sites and poor agreement (44%) at 1 site. Overall at 3 sites, 56% of children with automated cerebella ataxia codes (ICD-9 = 334) and 22% with “lack of coordination” codes (ICD-9 = 781.3) met objective clinical criteria for acute ataxia.

Conclusions: The misclassification error rates for automated screening outcomes substantially reduce the power of screening analyses and limit usefulness of screening analyses to moderate to strong vaccine-outcome associations. Medical record verification of outcomes is needed for definitive assessments.

Keywords

Databases - health maintenance organizations - International Classification of Diseases (ICD-9) - quality control - vaccines
 

  • References

  • 1 Chen RT, Glasser RP, Rhodes PH. et al. The Vaccine Safety Datalink Project: a new tool for improving vaccine safety monitoring in the United Sates. Pediatrics 1997; 99: 765-73.
    CrossrefPubMedGoogle Scholar
  • 2 Chen RT, DeStefano F, Davis RL. et al. The Vaccine Safety Datalink Project: immunization research in health maintenance organizations in the USA. WHO Bulletin 2000; 78: 186-94.
    PubMedGoogle Scholar
  • 3 Fisher ES, Whaley FS, Krushat WH. et al. The accuracy of Medicare’s hospital claims data: progress has been made, but problems remain. Am J Public Health 1992; 82: 243-8.
    CrossrefPubMedGoogle Scholar
  • 4 Office of Statewide Health Planning and Development. Report of results from the OSHPD re-abstracting project. Sacramento, CA: Office of Statewide Health Planning and Development; 1990.
    PubMedGoogle Scholar
  • 5 Baron JA, Weiderpass E. An introduction to epidemiological research with medical databases. Ann Epidemiol 2000; 10: 200-4.
    CrossrefPubMedGoogle Scholar
  • 6 Romano PS, Roos LL, Luft HS. et al. and the Ischemic Heart Disease Patient Outcomes Research Team A comparison of administrative versus clinical data: coronary artery bypass surgery as an example. J Clin Epidemiol 1994; 47 (03) 249-60.
    CrossrefPubMedGoogle Scholar
  • 7 Smith GS, Langlois JA, Buechner JS. Methodological issues in using hospital discharge data to determine the incidence of hospitalized injuries. Amer J of Epidemiol 1991; 134 (10) 1146-58.
    PubMedGoogle Scholar
  • 8 Feinstein AR. Unsolved scientific problems in the nosology of clinical medicine. Arch Intern Med 1988; 148: 2269-74.
    CrossrefPubMedGoogle Scholar
  • 9 McMahon LF, Smits HL. Can Medicare prospective payment survive the ICD-9-CM Disease Classification System?. Ann of Int Med 1986; 104: 562-6.
    PubMedGoogle Scholar
  • 10 Mullooly JP, Drew L, DeStefano F. et al. Quality of HMO vaccination databases used to monitor childhood vaccine safety. Am J Epidemiol 1999; 149: 186-94.
    CrossrefPubMedGoogle Scholar
  • 11 Mullooly JP. Misclassification model for persontime analysis of automated medical care databases. Am J Epidemiol 1996; 144: 782-92.
    CrossrefPubMedGoogle Scholar
  • 12 Farrington CP. Relative incidence estimated from case series for vaccine safety evaluation. Biometrics 1995; 51: 228-35.
    CrossrefPubMedGoogle Scholar
  • 13 Farrington CP, Nash J, Miller E. Case series analysis of adverse reactions to vaccines: A comparative evaluation. Am J Epidemiol 1996; 143: 1165-73.
    CrossrefPubMedGoogle Scholar
  • 14 Chin HL, Krall M. Implementation of a comprehensive computer-based patient record system in Kaiser Permanente’s northwest region. MD Comput 1997; 14: 41-5.
    PubMedGoogle Scholar
  • 15 Selby JB. Linking automated databases for research in managed care settings. Ann Intern Med 1997; 127 (Pt 2) 719-24.
    CrossrefPubMedGoogle Scholar