Download PDF
Methods Inf Med 2008; 47(04): 328-335
DOI: 10.3414/ME0500
Original Article
Schattauer GmbH

Predictive Value of ICD-9-CM Codes Used in Vaccine Safety Research

J. P. Mullooly
1   Center for Health Research, Kaiser Permanente Northwest, Portland, OR, USA
,
J. G. Donahue
2   Marshfield Clinic, Marshfield, WI, USA
,
F. DeStefano
3   Centers for Disease Control and Prevention, Atlanta, GA, USA
,
J. Baggs
1   Center for Health Research, Kaiser Permanente Northwest, Portland, OR, USA
,
E. Eriksen
1   Center for Health Research, Kaiser Permanente Northwest, Portland, OR, USA
,
for the VSD Data Quality Working Group › Author Affiliations
Further Information

Publication History

Received: 20 July 2007

accepted: 25 February 2008

Publication Date:
18 January 2018 (online)

    Permissions and Reprints

Summary

Objectives: To assess how well selected ICD-9-CM diagnosis codes predict adverse events; to model bias and power loss when vaccine safety analyses rely on unverified codes.

Methods: We extracted chart verification data for ICD-9-CM diagnosis codes from six Vaccine Safety Datalink (VSD) publications and modeled biases and power losses using positive predictive value (PPV) estimates and ranges of code sensitivity.

Results: Positive predictive values were high for type 1 diabetes (80%) in children, relative to WHO criteria, and intussusception (81%) in young children, relative to a standard published case definition. PPVs were moderate (65%) for inpatient and emergency department childhood seizures and low (21%) for outpatient childhood seizures, both relative to physician investigator judgment. Codes for incident central nervous system demyelinating disease in adults had high PPV for inpatient codes (80%) and low PPV for outpatient codes (42%) relative to physicians’ diagnoses. Modeled biases were modest, but large increases in frequencies of adverse events are required to achieve adequate power if unverified ICD-9-CM codes are used, especially when vaccine associations are weak.

Conclusions: ICD-9-CM codes for type 1 diabetes in children, intussusception in young children, childhood seizures in inpatient and emergency care settings, and inpatient demyelinating disease in adults were sufficiently predictive for vaccine safety analyses to rely on unverified diagnosis codes. Adverse event misclassification should be accounted for in statistical power calculations.

Keywords

ICD-9-CM codes - positive predictive values - vaccinesafety - bias - power reductions
 

  • References

  • 1 DeStefano F. The Vaccine Safety Datalink project. Pharmacoepidemiol Drug Saf 2001; 10 (Suppl. 05) 403-406.
    CrossrefPubMedGoogle Scholar
  • 2 World Health Organization.. International Classification of Diseases, 1975 Revision, Section 1.2. History and Development of Uses of the ICD. Geneva, Switzerland: World Health Organization; 1977
    Google Scholar
  • 3 Mullooly JP. Misclassification model for persontime analysis of automated medical care databases. Am J Epidemiol 1996; 144 (Suppl. 08) 782-792.
    CrossrefPubMedGoogle Scholar
  • 4 Mullooly J, Drew L, DeStefano F, Chen R, Okoro K, Swint E. et al. Quality of HMO vaccination databases used to monitor childhood vaccine safety. Vaccine Safety DataLink Team. Am J Epidemiol 1999; 149 (Suppl. 02) 186-194.
    CrossrefPubMedGoogle Scholar
  • 5 Mullooly J, Drew L, DeStefano F, Maher J, Bohlke K, Immanuel V. et al. Quality assessments of HMO diagnosis databases used to monitor childhood vaccine safety. Methods Inf Med 2004; 43 (Suppl. 02) 163-170.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 6 Mullooly JP, Pearson J, Drew L, Schuler R, Maher J, Gargiullo P. et al. Wheezing lower respiratory disease and vaccination of full-term infants. Pharmacoepidemiol Drug Saf 2002; 11 (Suppl. 01) 21-30.
    PubMedGoogle Scholar
  • 7 DeStefano F, Mullooly JP, Okoro CA, Chen RT, Marcy SM, Ward JI. et al. Childhood vaccinations, vaccination timing, and risk of type 1 diabetes mellitus. Pediatrics 2001; 108 (Suppl. 06) E112.
    CrossrefPubMedGoogle Scholar
  • 8 WHO Multinational Project for Childhood Diabetes.. WHO Diamond Project Group. Diabetes Care 1990; 13 (10) 1062-1068.
    CrossrefPubMedGoogle Scholar
  • 9 DeStefano F, Verstraeten T, Jackson LA, Okoro CA, Benson P, Black SB. et al. Vaccinations and risk of central nervous system demyelinating diseases in adults. Arch Neurol 2003; 60 (Suppl. 04) 504-509.
    CrossrefPubMedGoogle Scholar
  • 10 Chang EJ, Zangwill KM, Lee H, Ward JI. Lack of association between rotavirus infection and intussusception: implications for use of attenuated rotavirus vaccines. Pediatr Infect Dis J 2002; 21 (Suppl. 02) 97-102.
    CrossrefPubMedGoogle Scholar
  • 11 Tapiainen T, Bar G, Bonhoeffer J, Heininger U. Evaluation of the Brighton Collaboration case definition of acute intussusception during active surveillance. Vaccine 2006; 24 (Suppl. 09) 1483-1487.
    CrossrefPubMedGoogle Scholar
  • 12 Barlow WE, Davis RL, Glasser JW, Rhodes PH, Thompson RS, Mullooly JP. et al. The risk of seizures after receipt of whole-cell pertussis or measles, mumps, and rubella vaccine. N Engl J Med 2001; 345 (Suppl. 09) 656-661.
    CrossrefPubMedGoogle Scholar
  • 13 Davis RL, Marcuse E, Black S, Shinefield H, Givens B, Schwalbe J. et al. MMR2 immunization at 4 to 5 years and 10 to 12 years of age: a comparison of adverse clinical events after immunization in the Vaccine Safety Datalink project. The Vaccine Safety Datalink Team. Pediatrics 1997; 100 (Suppl. 05) 767-771.
    CrossrefPubMedGoogle Scholar
  • 14 O’Malley KJ, Cook KF, Price MD, Wildes KR, Hurdle JF, Ashton CM. Measuring diagnoses: ICD code accuracy. Health Serv Res 2005; 40 5 Pt 2 1620-1639.
    CrossrefPubMedGoogle Scholar
  • 15 Nilsson G, Petersson H, Ahlfeldt H, Strender LE. Evaluation of three Swedish ICD-10 primary care versions: reliability and ease of use in diagnostic coding. Methods Inf Med 2000; 39 4-5 325-331.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 16 Bergmann J. Health and clinical management – comparing the clinical outcome of computerized strategies. Findings from the section on health and clinical management. Methods Inf Med 2006; 45 (Suppl. 01) 26-28.
    PubMedGoogle Scholar
  • 17 Meystre S. Electronic patient records: some answers to the data representation and reuse challenges. Findings from the section on patient records. Methods Inf Med 2007; 46 (Suppl. 01) 47-49.
    PubMedGoogle Scholar
  • 18 Hviid A, Stellfeld M, Wohlfahrt J, Melbye M. Childhood vaccination and type 1 diabetes. N Engl J Med 2004; 350 (14) 1398-1404.
    CrossrefPubMedGoogle Scholar
  • 19 Nielsen GL, Sorensen HT, Pedersen AB, Sabroe S. Analyses of data quality in registries concerning diabetes mellitus -– a comparison between a population based hospital discharge and an insulin prescription registry. J Med Syst 1996; 20 (Suppl. 01) 1-10.
    CrossrefPubMedGoogle Scholar
  • 20 Hebert PL, Geiss LS, Tierney EF, Engelgau MM, Yawn BP, McBean AM. Identifying persons with diabetes using Medicare claims data. Am J Med Qual 1999; 14 (Suppl. 06) 270-277.
    CrossrefPubMedGoogle Scholar
  • 21 Miller DR, Safford MM, Pogach LM. Who has diabetes? Best estimates of diabetes prevalence in the Department of Veterans Affairs based on computerized patient data. Diabetes Care 2004; 27 (Suppl. 02) B10-B21.
    CrossrefPubMedGoogle Scholar
  • 22 Szeto HC, Coleman RK, Gholami P, Hoffman BB, Goldstein MK. Accuracy of computerized outpatient diagnoses in a Veterans Affairs general medicine clinic. Am J Manag Care 2002; 8 (Suppl. 01) 37-43.
    PubMedGoogle Scholar
  • 23 Wilson C, Susan L, Lynch A, Saria R, Peterson D. Patients with diagnosed diabetes mellitus can be accurately identified in an Indian Health Service patient registration database. Public Health Rep 2001; 116 (Suppl. 01) 45-50.
    PubMedGoogle Scholar
  • 24 Diabetes among young American Indians – Montana and Wyoming, 200500-2002. MMWR Morb Mortal Wkly Rep 2003; 52 (46) 1127-1129.
    PubMedGoogle Scholar
  • 25 Kramarz P, France EK, DeStefano F, Black SB, Shinefield H, Ward JI. et al. Population-based study of rotavirus vaccination and intussusception. Pediatr Infect Dis J 2001; 20 (Suppl. 04) 410-416.
    CrossrefPubMedGoogle Scholar
  • 26 Murphy TV, Gargiullo PM, Massoudi MS, Nelson DB, Jumaan AO, Okoro CA. et al. Intussusception among infants given an oral rotavirus vaccine. N Engl J Med 2001; 344 (Suppl. 08) 564-572.
    CrossrefPubMedGoogle Scholar
  • 27 Beitel AJ, Olson KL, Reis BY, Mandl KD. Use of emergency department chief complaint and diagnostic codes for identifying respiratory illness in a pediatric population. Pediatr Emerg Care 2004; 20 (Suppl. 06) 355-360.
    CrossrefPubMedGoogle Scholar
  • 28 Daley MF, Barrow J, Pearson K, Crane LA, Gao D, Stevenson JM. et al. Identification and recall of children with chronic medical conditions for influenza vaccination. Pediatrics 2004; 113 1 Pt 1 e26-e33.
    CrossrefPubMedGoogle Scholar
  • 29 Bonhoeffer J, Menkes J, Gold MS, de Souza-Brito G, Fisher MC, Halsey N. et al. Generalized convulsive seizure as an adverse event following immunization: case definition and guidelines for data collection, analysis, and presentation. Vaccine 2004; 22 5-6 557-562.
    CrossrefPubMedGoogle Scholar
  • 30 Aminoff MJ, Greenberg DA, Simon RP. Seizures and Syncope. NewYork: McGraw-Hill Co.; 2005
    Google Scholar
  • 31 Jackson LA, Carste BA, Malais D, Froeschle J. Retrospective population-based assessment of medically attended injection site reactions, seizures, allergic responses and febrile episodes after acellular pertussis vaccine combined with diphtheria and tetanus toxoids. Pediatr Infect Dis J 2002; 21 (Suppl. 08) 781-786.
    CrossrefPubMedGoogle Scholar
  • 32 Vestergaard M, Hviid A, Madsen KM, Wohlfahrt J, Thorsen P, Schendel D. et al. MMR vaccination and febrile seizures: evaluation of susceptible subgroups and long-term prognosis. JAMA 2004; 292 (Suppl. 03) 351-357.
    CrossrefPubMedGoogle Scholar
  • 33 Walker AM, Jick H, Perera DR, Knauss TA, Thompson RS. Neurologic events following diphtheria-tetanus-pertussis immunization. Pediatrics 1988; 81 (Suppl. 03) 345-349.
    PubMedGoogle Scholar
  • 34 Farrington P, Pugh S, Colville A, Flower A, Nash J, Morgan-Capner P. et al. A new method for active surveillance of adverse events from diphtheria/ tetanus/pertussis and measles/mumps/rubella vaccines. Lancet 1995; 345 8949 567-569.
    CrossrefPubMedGoogle Scholar
  • 35 Griffin MR, Ray WA, Mortimer EA, Fenichel GM, Schaffner W. Risk of seizures after measlesmumps-rubella immunization. Pediatrics 1991; 88 (Suppl. 05) 881-885.
    PubMedGoogle Scholar
  • 36 Gale JL, Thapa PB, Wassilak SG, Bobo JK, Mendelman PM, Foy HM. Risk of serious acute neurological illness after immunization with diphtheriatetanus-pertussis vaccine. A population-based case-control study. JAMA 1994; 271 (Suppl. 01) 37-41.
    CrossrefPubMedGoogle Scholar
  • 37 Kuno-Sakai H, Kimura M. Safety and efficacy of acellular pertussis vaccine in Japan, evaluated by 23 years of its use for routine immunization. Pediatr Int 2004; 46 (Suppl. 06) 650-655.
    CrossrefPubMedGoogle Scholar
  • 38 Hernan MA, Jick SS. Hepatitis B vaccination and multiple sclerosis: the jury is still out. Pharmacoepidemiol Drug Saf 2006; 15 (Suppl. 09) 653-655.
    CrossrefPubMedGoogle Scholar