Skip to main content
Log in

Patient-Reported Outcomes in Rheumatoid Arthritis

Assessing the Equivalence of Electronic and Paper Data Collection

  • Original Research Article
  • Published:
The Patient: Patient-Centered Outcomes Research Aims and scope Submit manuscript

Abstract

Background: The questionnaires used in clinical research have often been developed and validated using paper, and in such cases it is necessary to show that the electronic versions are equivalent to the originals.

Objective: To determine if electronic versions of questionnaires assessing severity and impact of rheumatoid arthritis (RA) are equivalent to the original paper versions.

Methods: Patients (n = 43; 31 female) aged 32–83 years (25 aged <60 years) took part in a single session during which they completed paper and electronic assessments in randomized order, with an interval of 45 minutes between the two modes. Electronic assessments were set up on a Palm® TX handheld device. Assessments included measures of pain, fatigue, disability, and health status.

Results: Scores were similar between the two modes. All effect sizes for electronic-paper differences were <0.2, and there was no overall tendency for one mode to show higher scores than the other. Intraclass correlation coefficients (ICCs) ranged from 0.72 to 0.96, and were generally similar to reported retest reliabilities of the scales in their paper versions. The Disability Index of the Health Assessment Questionnaire (HAQ-DI) showed an ICC of 0.96. ICCs for the EQ-5D health status scale were utility: 0.79; profile: 0.91; visual analog scale: 0.75. Most patients reported that both modes were easy to use. In general, patients preferred the electronic version over the paper, and this was true for the older as well as the younger patients.

Conclusions: Electronic versions of questionnaires assessing severity and impact of RA provide data that correspond closely to those of paper originals, are easy to use, and are acceptable to patients.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Table I
Table II
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Drummond HE, Ghosh S, Ferguson A, et al. Electronic quality of life questionnaires: a comparison of pen-based electronic questionnaires with conventional paper in a gastrointestinal study. Qual Life Res 1995; 4: 21–6

    Article  PubMed  CAS  Google Scholar 

  2. Crawley JA, Kleinman L, Dominitz J. User preferences for computer administration of quality of life instruments. Drug Inf J 2000; 34: 137–44

    Article  Google Scholar 

  3. Ring AE, Cheong KA, Watkins CL, et al. A randomized study of electronic diary versus paper and pencil collection of patient-reported outcomes in patients with non-small cell lung cancer. Patient 2008; 1(2): 105–13

    Article  PubMed  Google Scholar 

  4. Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 2008; 11(2): 322–33

    Article  PubMed  Google Scholar 

  5. Shields A, Gwaltney C, Tiplady B, et al. Grasping the FDA’s PRO guidance. Appl Clin Trials 2006; 15(8): 69–72

    Google Scholar 

  6. Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value Health 2009; 12(4): 419–29

    Article  PubMed  Google Scholar 

  7. Cohen SB, Strand V, Aguilar D, et al. Patientversus physician-reported outcomes in rheumatoid arthritis patients treated with recombinant interleukin-1 receptor antagonist (anakinra) therapy. Rheumatology 2004; 43(6): 704–11

    Article  PubMed  CAS  Google Scholar 

  8. EuroQol Group. EuroQol: a new facility for the measurement of health-related quality of life. Health Policy 1990; 16(3): 199–208

    Article  Google Scholar 

  9. Hurst NP, Jobanputra P, Hunter M, et al. Validity of Euroqol — a generic health status instrument — in patients with rheumatoid arthritis. Economic and Health Outcomes Research Group. Br J Rheumatol 1994; 33(7): 655–62

    Article  PubMed  CAS  Google Scholar 

  10. Fries JF, Spitz P, Kraines RG, et al. Measurement of patient outcome in arthritis. Arthritis Rheum 1980; 23(2): 137–45

    Article  PubMed  CAS  Google Scholar 

  11. Ramey DA, Fries JF, Singh G. The Health Assessment Questionnaire 1995: status and review. In: Spilker B, editor. Quality of life and pharmacoeconomics in clinical trials. Philadelphia (PA): Lippincott Williams & Wilkins, 1996: 227–37

    Google Scholar 

  12. Bruce B, Fries JF. The Health Assessment Questionnaire (HAQ). Clin Exp Rheumatol 2005; 23(5 Suppl. 39): S14–8

    PubMed  CAS  Google Scholar 

  13. Ramachandran S, Lundy J, Coons S. Testing the measurement equivalence of paper and touch-screen versions of the EQ-5D visual analog scale (EQ VAS). Qual Life Res 2008; 17(8): 1117–20

    Article  PubMed  Google Scholar 

  14. Cleeland CS, Ryan KM. Pain assessment: global use of the Brief Pain Inventory. Ann Acad Med Singapore 1994; 23(2): 129–38

    PubMed  CAS  Google Scholar 

  15. Melzack R. The short-form McGill Pain Questionnaire. Pain 1987; 30(2): 191–7

    Article  PubMed  CAS  Google Scholar 

  16. Melzack R. The McGill pain questionnaire: from description to measurement. Anesthesiology 2005; 103(1): 199–202

    Article  PubMed  Google Scholar 

  17. Yellen SB, Cella DF, Webster K, et al. Measuring fatigue and other anemia-related symptoms with the Functional Assessment of Cancer Therapy (FACT) measurement system. J Pain Symptom Manage 1997; 13(2): 63–74

    Article  PubMed  CAS  Google Scholar 

  18. Mallinson T, Cella D, Cashy J, et al. Giving meaning to measure: linking self-reported fatigue and function to performance of everyday activities. J Pain Symptom Manage 2006; 31(3): 229–41

    Article  PubMed  Google Scholar 

  19. Ware Jr JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Med Care 1992; 30(6): 473–83

    Article  PubMed  Google Scholar 

  20. Mystakidou K, Mendoza T, Tsilika E, et al. Greek brief pain inventory: validation and utility in cancer pain. Oncology 2001; 60(1): 35–42

    Article  PubMed  CAS  Google Scholar 

  21. Dolan P. Modeling valuations for EuroQol health states. Med Care 1997; 35(11): 1095–108

    Article  PubMed  CAS  Google Scholar 

  22. Spritzer K. SAS code for scoring 36-item health survey version 2.0 standard form (not acute!) [online]. Available from URL: http://gim.med.ucla.edu/FacultyPages/Hays/UTILS/ sf36v2-4-public.sas [Accessed 2007 Sep 4]

  23. Conover WJ, Iman R. Rank transformations as a bridge between parametric and nonparametric statistics. Am Stat 1981; 35: 124–9

    Google Scholar 

  24. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods 1996; 1(1): 30–46

    Article  Google Scholar 

  25. Vacha-Haase T, Thompson B. How to estimate and interpret various effect sizes. J Couns Psychol 2004; 51(4): 473–81

    Article  Google Scholar 

  26. Haywood KL, Garratt AM, Dziedzic K, et al. Generic measures of health-related quality of life in ankylosing spondylitis: reliability, validity and responsiveness. Rheumatology 2002; 41(12): 1380–7

    Article  PubMed  CAS  Google Scholar 

  27. Hurst NP, Kind P, Ruta D, et al. Measuring health-related quality of life in rheumatoid arthritis: validity, responsiveness and reliability of EuroQol (EQ-5D). Br J Rheumatol 1997; 36(5): 551–9

    Article  PubMed  CAS  Google Scholar 

  28. Kirwan JR, Reeback JS. Stanford Health Assessment Questionnaire modified to assess disability in British patients with rheumatoid arthritis. Br J Rheumatol 1986; 25(2): 206–9

    Article  PubMed  CAS  Google Scholar 

  29. Sullivan FM, Eagers RC, Lynch K, et al. Assessment of disability caused by rheumatic diseases in general practice. Ann Rheum Dis 1987; 46(8): 598–600

    Article  PubMed  CAS  Google Scholar 

  30. Ekdahl C, Eberhardt K, Andersson SI, et al. Assessing disability in patients with rheumatoid arthritis: use of a Swedish version of the Stanford Health Assessment Questionnaire. Scand J Rheumatol 1988; 17(4): 263–71

    Article  PubMed  CAS  Google Scholar 

  31. Guillemin F, Briancon S, Pourel J. Validity and discriminant ability of the HAQ Functional Index in early rheumatoid arthritis. Disabil Rehabil 1992; 14(2): 71–7

    Article  PubMed  CAS  Google Scholar 

  32. Harper R, Brazier JE, Waterhouse JC, et al. Comparison of outcome measures for patients with chronic obstructive pulmonary disease (COPD) in an outpatient setting. Thorax 1997; 52(10): 879–87

    Article  PubMed  CAS  Google Scholar 

  33. Fransen M, Edmonds J. Reliability and validity of the EuroQol in patients with osteoarthritis of the knee. Rheumatology (Oxford) 1999; 38(9): 807–13

    Article  CAS  Google Scholar 

  34. König HH, Ulshöfer A, Gregor M, et al. Validation of the EuroQol questionnaire in patients with inflammatory bowel disease. Eur J Gastroenterol Hepatol 2002; 14(11): 1205–15

    Article  PubMed  Google Scholar 

  35. Bushnell DM, Reilly MC, Galani C, et al. Validation of electronic data capture of the Irritable Bowel Syndrome — Quality of Life Measure, the Work Productivity and Activity Impairment Questionnaire for Irritable Bowel Syndrome and the EuroQol. Value Health 2006; 9(2): 98–105

    Article  PubMed  Google Scholar 

  36. Daut RL, Cleeland CS, Flanery RC. Development of the Wisconsin Brief Pain Questionnaire to assess pain in cancer and other diseases. Pain 1983; 17(2): 197–210

    Article  PubMed  CAS  Google Scholar 

  37. Ware Jr JE. SF-36 health survey update. Spine 2000; 25(24): 3130–9

    Article  PubMed  Google Scholar 

  38. Hagell P, Höglund A, Reimer J, et al. Measuring fatigue in Parkinson’s disease: a psychometric study of two brief generic fatigue questionnaires. J Pain Symptom Manage 2006; 32(5): 420–32

    Article  PubMed  Google Scholar 

  39. Strand LI, Ljunggren AE, Bogen B, et al. The Short-Form McGill Pain Questionnaire as an outcome measure: test-retest reliability and responsiveness to change. Eur J Pain 2008; 12(7): 917–25

    Article  PubMed  Google Scholar 

  40. Hanscom B, Lurie JD, Homa K, et al. Computerized questionnaires and the quality of survey data. Spine 2002; 27(16): 1797–801

    Article  PubMed  Google Scholar 

  41. Velikova G, Wright EP, Smith AB, et al. Automated collection of quality-of-life data: a comparison of paper and computer touch-screen questionnaires. J Clin Oncol 1999; 17(3): 998–1007

    PubMed  CAS  Google Scholar 

  42. Bushnell DM, Martin ML, Parasuraman B. Electronic versus paper questionnaires: a further comparison in persons with asthma. J Asthma 2003; 40(7): 751–62

    Article  PubMed  CAS  Google Scholar 

  43. Cook MR, Gerkovich MM, Graham C, et al. Effects of the nicotine patch on performance during the first week of smoking cessation. Nicotine Tob Res 2003; 5(2): 169–80

    Article  PubMed  CAS  Google Scholar 

  44. Gaertner J, Elsner F, Pollmann-Dahmen K, et al. Electronic pain diary: a randomized crossover study. J Pain Symptom Manage 2004; 28(3): 259–67

    Article  PubMed  Google Scholar 

  45. Kvien TKK, Mowinckel P, Heiberg T, et al. Performance of health status measures with a pen-based personal digital assistant. Ann Rheum Dis 2005; 64(10): 1480–4

    Article  PubMed  CAS  Google Scholar 

  46. Richter JG, Becker A, Koch T, et al. Self-assessments of patients via tablet PC in routine patient care: comparison with standardised paper questionnaires. Ann Rheum Dis 2008; 67: 1739–41

    Article  PubMed  CAS  Google Scholar 

  47. Yarnold PR, Stewart MJ, Stille FC, et al. Assessing functional status of elderly adults via microcomputer. Percept Mot Skills 1996; 82(2): 689–90

    Article  PubMed  CAS  Google Scholar 

  48. Begg A, Drummond G, Tiplady B. Assessment of post-surgical recovery after discharge using a pen computer diary. Anaesthesia 2003; 58(11): 1101–5

    Article  PubMed  CAS  Google Scholar 

  49. Kurt R, Bogner HR, Straton JB, et al. Computer-assisted assessment of depression and function in older primary care patients. Comput Methods Programs Biomed 2004; 73(2): 165–71

    Article  PubMed  Google Scholar 

  50. Millsopp L, Frackleton S, Lowe D, et al. A feasibility study of computer-assisted health-related quality of life data collection in patients with oral and oropharyngeal cancer. Int J Oral Maxillofac Surg 2006; 35(8): 761–4

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgments

This study was financially supported by AstraZeneca. Robert Carrington and Clare Battersby are employed by, and own shares in, AstraZeneca, who commissioned the clinical study on which this article is based. Brian Tiplady is employed by P RO Consulting, a division of invivodata inc., the study sponsor; owns shares in AstraZeneca; has acted as consultant for various companies; and will receive royalties from a forthcoming book on ePRO. Professor Stuart Ralston acts as a consultant for Novartis and Merck Pharmaceuticals.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Brian Tiplady.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tiplady, B., Goodman, K., Cummings, G. et al. Patient-Reported Outcomes in Rheumatoid Arthritis. Patient-Patient-Centered-Outcome-Res 3, 133–143 (2010). https://doi.org/10.2165/11535590-000000000-00000

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.2165/11535590-000000000-00000

Keywords

Navigation