Indexed in MEDLINE, SCI & KoreaMed
pISSN 1011-8934   eISSN 1598-6357
Open Access, Peer-reviewed, Weekly
    J Korean Med Sci. 2021 Aug 23;36(33):e212. English.
    Published online Jul 21, 2021.
    https://doi.org/10.3346/jkms.2021.36.e212
    © 2021 The Korean Academy of Medical Sciences.
    Original Article

    Patient-Reported Outcomes Measurement Information System: Translation and Linguistic Validation of Six Profile Domains for Korean Adults

    Youngha Kim,1,* Junghee Yoon,2,* Nayeon Kim,2,3 Mangyeong Lee,1,4 Danbee Kang,1,2 Hye Yun Park,2,5 Dongryul Oh,6 Ki-Sun Sung,7 Gee Young Suh,8 Jin Seok Ahn,9 and Juhee Cho1,2,3,4
      • 1Center for Clinical Epidemiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.
      • 2Department of Clinical Research Design and Evaluation, SAIHST, Sungkyunkwan University, Seoul, Korea.
      • 3Cancer Education Center, Samsung Medical Center, Seoul, Korea.
      • 4Department of Digital Health, SAIHST, Sungkyunkwan University, Seoul, Korea.
      • 5Division of Pulmonary and Critical Care Medicine, Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.
      • 6Department of Radiation Oncology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.
      • 7Department of Orthopedic Surgery, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.
      • 8Department of Critical Care Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.
      • 9Division of Hematology/Oncology, Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.
    • Address for Correspondence: Juhee Cho, PhD. Department of Clinical Research Design and Evaluation, SAIHST, Sungkyunkwan University, 81 Irwon-ro, Gangnam-gu, Seoul 06315, Korea. Email: jcho@skku.edu

      *Youngha Kim and Junghee Yoon equally contributed equally to this work.
    Received April 28, 2021; Accepted July 18, 2021.

    This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

    Abstract

    Background

    The purpose of the study was to translate and linguistically validate a Korean language version of the PROMIS (K-PROMIS) for the six profile adult domains: Fatigue, Pain Intensity, Pain Interference, Physical Function, Sleep Disturbance, and Ability to Participate in Social Roles and Activities.

    Methods

    A total of 268 items were translated into Korean according to the Functional Assessment of Chronic Illness Therapy multilingual translation methodology. Participants first completed approximately 27 to 35 items and were then interviewed to evaluate the conceptual equivalence of the translation to the original English language source. The K-PROMIS items that met the a priori threshold of ≥ 20% of respondents with comprehension difficulties in the cognitive interview.

    Results

    54 of the 268 items were identified as difficult items to comprehend for at least 20% of respondents in Round 1. The most frequently identified K-PROMIS domain on difficult items to comprehend was the Physical function (24.5%). Most items with linguistic difficulties were Fatigue and Physical function. Cultural difficulties were only included the Physical function and Ability to Participate in Social Roles and Activities domains. 25 of 54 items were slightly revised, and then these revised items were tested with additional six participants in Round 2, and most participants had no problems to understand modified items.

    Conclusion

    The six profile adult domains of K-PROMIS have been linguistically validated. Further psychometric validation of the K-PROMIS items will provide additional information of meaningful outcomes for chronic disease and clinical setting.

    Graphical Abstract

    Keywords
    Patient-Reported Outcomes; Patient-Reported Outcomes Measurement Information System; PROMIS; Validation; Korean Translation

    INTRODUCTION

    Over the years, patient-reported outcomes (PROs) have become important endpoints in clinical trials, health care, and the social sciences.1, 2 PROs are health outcomes that are directly reported by the patient without interpretation by a clinician or anyone else. Many studies have confirmed that PROs should be considered as the standard for measuring perceived health condition, symptoms, functional status, and health-related quality of life to adequately guide treatments and improve health outcomes.3, 4, 5, 6 In addition, PROs may provide information relevant to treatment choices, disease management, reimbursement decisions, and health policies in various clinical settings.7, 8

    However, most PRO measurement instruments have encompassed problems such as a narrow scope of health status and difficulties in standardized interpretation owing to irrelevant, incomprehensible, or poorly formulated questions.9 Thus, in 2004, the Patient-Reported Outcomes Measurement Information System (PROMIS®) was established as a multi-center, collaborative project funded under the US National Institutes of Health Initiative to improve the measurement of clinically important symptoms and outcomes.10 The PROMIS is based on the conceptual framework advocated by the World Health Organization, and has three key domains: physical, mental, and social health.11 These domains have a set of item banks to facilitate standardized assessment of PROs in adults and children. Initial development of the constructs and domains of PROMIS is detailed elsewhere.12 The item banks for the measurement of a large number of clinically important outcomes can be administered either as short forms (fixed subsets of the best items from an item bank) or through computerized adaptive testing (where a computer algorithm selects items from an item bank one by one based on answers to previous items, leading to a variable subset of items per person). The item banks were developed using item response theory (IRT) methods, which model the relationship between the level of the domain and the probability of giving a certain response to an item. Based on the IRT model, items are ordered in an item bank based on the item difficulty and this ordering allows for the selection of subsets of items for short forms or computer adaptive tests (CATs) and ensures that all short forms and CATs are measured on the same scale and that scores can be compared directly.

    The PROMIS has been translated and validated in various languages, and the domains of the adult item banks have been internationally adopted in clinical research and practice.13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23 The PROMIS has the potential to be global health scale considering as different item parameters could be used across countries and because its psychometric properties enable efficient assessments, which is valuable in clinical research and the evaluation of medical care.24, 25, 26 Thus, we aim to translate and linguistically validate a Korean version of the PROMIS (K-PROMIS) for six profile adult domains.

    METHODS

    Translation procedure

    We translated and validated six of the eight profile domains of the PROMIS (“fatigue,” “pain intensity,” “pain interference,” “physical function,” “sleep disturbance,” and “ability to participate in social roles and activities”; Supplementary Table 1) because the items in the depression and anxiety domains have already been linguistically validated. In addition, items available in Korean from Short Forms before this study (e.g., Short Form 4a items) were also excluded from the study.

    Fig. 1 provides an overview of the translation and linguistic validation procedure. A total of 268 items from six profile domains were translated into Korean according to the Functional Assessment of Chronic Illness Therapy (FACIT) multilingual translation methodology including forward and back translations, independent reviews of translation quality, and pilot testing including cognitive interviews.27, 28 This methodology is consistent with the guidelines for translation of PRO instruments and previously published translation and validation procedures of the PROMIS for non-English speaking populations. During the translation, we focused on semantic and conceptual equivalence of meaning with the original English PROMIS.

    Fig. 1
    Translation and linguistic validation process for the Korean version of the PROMIS profile domains.

    First, each English source item was forward translated into Korean by three independent bilingual professional translators who have licenses for Korean-English interpretation and translation. This step focused on capturing the essence of items rather than performing literal translations. Then, a fourth independent bilingual translator who was not involved with the forward translation process selected one of the three translations or created a hybrid version of the translated items. The reconciled version was then back translated by another set of three bilingual professional translators fluent in both English and Korean. Translators did not have medical background. While they were provided brief background information about PROMIS measurement system before the translations, no other information was provided to the translators. For quality assurance of the translation procedure, we organized an expert group consisting of 10 health professionals (six clinicians, two nurses, and two behavioral scientists) who were native speakers of Korean and fluent in English. The expert group reviewed the forward translations, reconciliations, and back translations based on the concept and definition provided by the PROMIS Statistical Center and identified inadequate expressions and concepts in the translation as well as any discrepancies between the original source items in English and the translated Korean items. If the previous translation steps were unacceptable, the experts selected the most appropriate translation for each item or provided alternate translations. In addition, the Korean Language and Culture Institute reviewed the translation for spelling errors and conceptual equivalence. The harmonization and quality assurance of the pre-final translation were performed by the PROMIS Statistical Center to address consistency and conceptual equivalence with existing translations in other languages if applicable, as well as between items.

    Cognitive interviewing procedure

    We conducted cognitive debriefing interviews to determine whether participants found any of the K-PROMIS items ambiguous or difficult to understand. This is an interview method designed to evaluate the comprehensibility, ease of response, and acceptability of the terminology, phrasing, response options, and format of a newly developed PRO measure.29

    Questionnaire scripts

    Debriefing scripts were created to guide the cognitive interviews. Each interview was conducted using a standardized script consisting of open-ended questions developed by the researchers. Each script contained 27 to 35 items to avoid burdening the participants in a single interview. The interviewer asked a series of open-ended questions, following a script, seeking comments with regard to the item stem, the response options, and the time frame (e.g., in the past seven days, now, etc.). Prior PROMIS cognitive debriefing studies have successfully used a similar approach.18, 23, 30, 31, 32 Data concerning participants' sociodemographic characteristics, including marital status, educational level, and monthly household income, were also gathered by self-report. Clinical information was obtained from electronic medical records. We calculated the total time (in minutes) taken to complete the K-PROMIS survey, excluding any breaks taken during the interview.

    Participants

    Sixty-seven adults aged 18 years or older who had seen a physician for chronic diseases or pain within the past five years, who had no concurrent psychiatric condition or cognitive or other impairments (e.g., visual) that would interfere with completing an interview, and who could speak, read, and write Korean as their primary language were recruited. Prespecified enrollment goals included the following: at least 30% of participants with an educational level of high school or older than 65 years, approximately equal gender representation, and at least five participants per item.28 Accrual of the sample was monitored prospectively to achieve these enrollment goals.

    Interviews

    Two rounds of cognitive interviews were conducted. Sixty-one participants were included in Round 1. After making the necessary revisions based on feedback from Round 1, six participants were included in Round 2. The cognitive interviews were performed in a private area of the outpatient clinic and consisted of the following two parts: administration of the K-PROMIS survey composed of a subset of K-PROMIS items, and a semi-scripted debriefing interview with cognitive probing for comprehension, clarity, and ease of judgment. Interviews were conducted by two experienced interviewers in conducting cognitive interviews.

    After completing the paper-and-pencil K-PROMIS survey, participants were asked to indicate items they found difficult to comprehend and those for which they had difficulty selecting a response option. The interviewer did not provide any assistance or advice and encouraged patients to complete the survey to the best of their ability on the basis of the instructions provided. After completing the survey, the cognitive debriefing interview was conducted. The script was similar to those used in previous PROMIS studies.30, 31, 33 Participants identified confusing text from the K-PROMIS items and rephrased them in their own words. The interviewer asked the participants to suggest alternative wording or phrasing to improve the clarity of items that were difficult to comprehend. The interviewer took detailed notes, and the interviews were recorded and transcribed for analysis.

    Iterative cycles of analysis and retesting

    For analysis of the individual K-PROMIS items, interview field notes and transcripts were compiled, abstracted, and summarized item by item. The interview data were examined for semantic and conceptual equivalence to the original English items. The expert group reviewed the results of each round of interview data analysis. The proportion of respondents exhibiting any level of difficulty or hesitation for an item or response option was calculated. K-PROMIS items that met the a priori threshold of ≥ 20% of respondents with comprehension difficulties in Round 1 were considered for rephrasing and retesting in Round 2.

    Items that at least 20% of the respondents found difficult to comprehend were classified under either linguistic or cultural difficulties. Linguistic difficulties were categorized into four issues: 1) “Do not know the meaning of the word” referred to a situation where the participants did not know or understand the meaning of a few words or sentences; 2) “Ambiguous meaning” referred to a situation where the participants knew the words but had a different interpretation than the intended meaning; 3) “Confused context” referred to a situation where the participants had an idea of the words but were confused about the exact meanings or were uncertain about the meanings in the correct context; and 4) “Others” referred to difficulties not included in the above three categories. Cultural difficulties were categorized into three issues: 1) “Differences in concepts or unfamiliar activities” referred to a situation where the participants did not understand the phrasing owing to conceptual differences or unfamiliar experiences; 2) “Differences in lifestyle and living environment” referred to a situation where the participants had difficulties responding because of differences in lifestyles and living conditions; and 3) “Different metric units” referred to a situation where the participants had difficulties understanding metric units.

    Item revision was considered on the basis of a detailed review of participants' responses and in the context of an effort to produce a final version that would be well comprehended by diverse respondents, including those who were older and had lower educational levels. Following this, the PROMIS Statistical Center performed the final quality review.

    Ethics statement

    This study was approved by the Institutional Review Board (IRB) of the Samsung Medical Center (IRB approval No. SMC-2017-03-103-012), and written informed consent was obtained from all participants.

    RESULTS

    A total of 67 outpatients participated in the cognitive interviews (61 participants in Round 1 and six participants in Round 2). The mean age was 57.1 years (age range: 18 to 86 years, standard deviation: 15.7), 55.2% were men, 71.6% were married, and 50.7% had an educational level of below high school (Table 1). Of the study participants, 38.8% either had an educational level of below middle school or were aged above 65 years.

    Table 1
    Characteristics of cognitive interview participants (n = 67)

    Although participants understood most of the K-PROMIS items during the cognitive interview in Round 1 (n = 61), there were several items that were difficult to comprehend. Fifty-four (20.2%) of the 268 K-PROMIS items were identified as difficult to comprehend for at least 20% of the respondents (Table 2). Among these 54 items, four items had both linguistic and cultural difficulties. Most K-PROMIS items involved linguistic difficulties, but items under “physical function” and “ability to participate in social roles and activities” involved difficulties with regard to both. Among the items that were difficult to comprehend, the most frequently identified K-PROMIS domain was “physical function” (24.5%). In particular, items which more than 60% of participants had difficulty in comprehending fell under “physical function” (10 items), “fatigue” (five items), and “ability to participate in social roles and activities” (one item).

    Table 2
    Number of translated items presenting comprehension difficulties during the cognitive interview in Round 1

    Linguistic and cultural difficulties were specifically categorized into four (Table 3) and three (Table 4) issues, respectively. Most items with linguistic difficulties were related to “fatigue” (11 items) and “physical function” (11 items). In terms of the issue “Do not know the meaning of the word,” participants were unfamiliar with the foreign words or Chinese characters, such as “mental energy,” “restrict,” or “feel limited.” Regarding “Ambiguous meaning,” four items (“household chores,” “feel less alert,” “flights of stairs,” and “labor”) in the “fatigue” and “physical function” domains presented difficulties for 60% or more of the participants. For instance, for the item “How often were you too tired to do your household chores?” five out of six participants (83.3%) said that they understood the meaning of the term but had no experience with household chores. Most participants (85.7%) understood “flights of stairs” to mean “steps.” We conducted a retest by modifying the phrase to “climb five floors with several stairs,” but most participants still did not understand. Through further review, we modified the phrase “climb five floors with several stairs” to “climb up five floors.”

    Table 3
    Key cognitive interview findings regarding items with linguistic difficulties in Round 1a

    Table 4
    Key cognitive interview findings regarding items with cultural difficulties in Round 1a

    For the issues grouped under “Confused context,” the K-PROMIS domain that involved the most difficulties in comprehension was “physical function” (seven items). Most participants (85.7%) understood the items “walk up and down two steps” and “climb up five steps” to mean going up and down “two steps” and “five steps” at a time. We added the word “one step at a time” to make the intended meaning clearer, and then conducted the cognitive interview in Round 2. At this point, more than 80% of the participants understood the intended meaning.

    Issues classified as “Others” were identified only under “fatigue.” We tried to use the same phrases or words that were previously translated in other domains in order to consistently match the meaning in whole domain in Korean. For instance, “How fatigued were you on average?” and “I feel fatigued” were translated into the Korean “How exhausted were you on average?” and “I was exhausted” in the FACIT-Fatigue scale. Of the participants, 66.7% said that there was no connection between “How fatigued were you” and “at its worst.”

    Cultural difficulties were only identified in the “physical function” (seven items) and “ability to participate in social roles and activities” (one item) domains. In terms of the issue of “Differences in concept or unfamiliar activities,” regarding the item “I have to limit social activities at home,” five out of seven participants (71.4%) said that they do not socialize at home. This was because in Korea, “social activities” refer to activities involving interaction with people outside the home.

    For the issue of “Differences in lifestyle and living environment,” all problematic items fell under “physical function.” As an item relevant to a different lifestyle, 85.7% of the participants did not understand what “cut food using eating utensils” meant. Americans cut food using eating utensils such as knives and forks, whereas Koreans use spoons and chopsticks. After discussing this issue with the PROMIS Statistical Center, we rephrased this item to “cut food using spoons or chopsticks.”

    Regarding the issue of “Different metric units,” many participants (85.7%) indicated that they found it difficult to judge exactly what distance “two miles (3 km)” or “10 miles (16 km)” represented and that they needed suitable examples. Koreans are familiar with the concept of judging distances in terms of the number of bus stops or blocks on the way to one's destination, but the distances between bus stops or blocks can vary widely depending on the area of residence. For a similar reason, about 71.4% of the participants needed practical examples for measurement units like a “2 kg object” or “heavy object (5 kg).”

    Of the 54 items, 25 were slightly revised after consultation with the PROMIS Statistical Center while 29 remained unchanged. These revised items were tested with an additional six participants in Round 2. During Round 2, most participants had no problems understanding the modified items. Three raters determined the types of issues associated with comprehension difficulties. Differences between raters were resolved by discussion until a consensus was reached.

    DISCUSSION

    In this study, we translated and linguistically validated the six adult profile domains of the PROMIS into Korean based on the standard FACIT multilingual translation methodology.27, 28 The majority of items in the K-PROMIS were well comprehended by Korean speakers, including older adults and those with lower educational levels. After two rounds of cognitive debriefing, we concluded that the K-PROMIS possesses satisfactory psychometric properties for use in clinical research and the care of Korean-speaking patients. However, despite adhering to rigorous procedures to maintain semantic and conceptual equivalence with the original PROMIS items, some linguistic and cultural issues persisted.

    Overall, the prevalence of linguistic issues in all domains, with the exception of “pain intensity,” ranged from 10% to 20%. However, only 15 items (14.1%) under “physical function” and one item (3.2%) under “ability to participate in social roles and activities” had cultural issues. This might be because, by conferring with the PROMIS Statistical Center, we were able to eliminate common cultural issues identified in PROMIS linguistic validation studies from other countries during the translation process. For example, measurement units (e.g., miles, blocks), definitions of “leisure activities” and “recreational activities,” and extended definitions of “yard work” for people living in apartments were discussed during translation. Yet, cultural adaptations were particularly required for “physical function” items in different living environments (e.g., yard work, carrying a laundry basket up) and tools used in daily life (e.g., utensils, can openers). In terms of metrics, while we employed those used in Korea (e.g., km, kg), patients still experienced difficulties because they do no use metrics in daily life. In Korea, people are more likely use examples instead of exact metrics or units to express an amount or distance. For example, people prefer to use the number of steps or time taken to get to a destination (e.g., 30 minutes) than exact units (e.g., 1.5 km). This is similar to previous studies. In an Arabic version, “walking more than a mile” was replaced by “walking a distance more than a bus stop” because the mile is not a well-known measure in Arabic culture.34

    The PROMIS was developed quantitatively with scientific accuracy for the purpose of making an evidence-based objective outcome measure available in clinical practice.35, 36 Yet, in this study, we found that individuals' experiences or situations affected their interpretations of the items. For example, regarding one item in the “fatigue” domain, “How often were you too tired to do your household chores?” some patients, especially older males, did not respond because they did not have experience of doing household chores. This is similar to a previous study, where patients could not answer some questions as they could not visualize the tasks/demands proposed.18 Thus, our study team considered including examples of household chores. However, the cognitive debriefing in the Dutch PROMIS validation study found that giving examples might lead to confusion or misunderstanding of items. For example, in the context of a question about physical activities, “Does your health now limit you in participating in active sports such as swimming, tennis, or basketball?,” patients were not able to think beyond these specific sports.37 Thus, after conferring with the PROMIS Statistical Center, we decided not to include examples or reference points. The issue of how patients' interpretations can vary based on such uncertainties should be further explored.

    The PROMIS item bank covers a broad range of levels because the items need to be evaluated based on IRT methods.38 However, participants had difficulties distinguishing between similar idiomatic expressions and sentence constructions in the “fatigue” domain. There were 10 words (tired, fatigued, sluggish, run-down, physically drained, exhausted, bushed, totally drained, wiped out, extremely exhausted) but patients could not differentiate between them. This was even more problematic with people with limited vocabulary.23, 30 Translation of these 10 different words to describe increasing levels of fatigue may have resulted in a different ordering (item difficulties) of the items in the translated language, potentially introducing differential item functioning. Further language-specific item calibration would be necessary if important differential item functioning is found.

    Similarly, there were some words/expressions that, though slightly different, were considered the same by participants. The study team acknowledged this issue during forward translation and tried to choose the translation with the greatest conceptual equivalence. However, participants still had difficulties in distinguishing between words. For example, participants could not differentiate “flight of stairs” from “steps,” “labor” from “work,” and “bending” from “stooping.” While not being able to differentiate between these words would not pose a problem in daily life, this ambiguity is detrimental to the evaluation based on IRT methods. It might be helpful for patients if they are provided with the exact definitions of words, or given pictures as guidance for words with ambiguous meanings.

    Furthermore, participants, especially those with low educational levels, had difficulties with loanwords such as “energy” or “recreational activities,” but we had to use them because they had no Korean equivalents. These items could not be substituted because the suggestions offered were mostly colloquial. This issue has been mentioned in previous studies involving testing PRO measures such as the Common Terminology Criteria for Adverse Events that have been translated into Korean.39

    This study has some limitations. First, as the setting was a single hospital, the participants were not representative of all Koreans. However, the sample possessed the demographic and clinical diversity to provide rigorous evidence of content validity in clinical settings. Therefore, additional testing to confirm its comprehensibility and cultural acceptability with regard to the general population with no experience of symptoms could be considered. Second, some items could not be modified because of the absence of Korean substitutes. Especially, four items included in the standard short forms were identified as difficult to comprehend for at least 20% of the respondents but items could not be modified as there were no suitable alternatives. Therefore, it is necessary to encourage clinicians and researchers who use the translated item banks in the target countries to share any culturally important content that might be missing. Furthermore, it might be reasonable to select different items in the standard short forms for use in Korea as these items could be problematic for Korean population. It is, therefore, necessary to encourage clinicians and researchers who use the translated item banks in the target countries to share any culturally important content that might be missing. Third, we conducted cognitive interviews with only six to seven participants per item; it might be better to increase this number to assess the level of comprehensibility and cognitive equivalence of the translation. However, this is higher than the number of participants recommended in the FACIT translation methodology for the debriefing of new items in the target country (five participants). In addition, we had an expert group who participated in the entire process, from linguistic validation to quality assurance to validation.

    Regardless of these limitations, this study has several strengths. First, we performed purposive sampling to include people with low health literacy and cognitive abilities. Of the sample, 38.8% either had an educational level of high school or below or were older than 65 years, and we sought an approximately equal gender representation. Second, we included patients who were experiencing symptoms and difficulties included in the PROMIS items. This increased the validity of the study as individuals without symptoms might have had more difficulties in comprehending PRO items. For example, orthopedic patients would report problems with physical function more accurately than people who do not experience those limitations or symptoms. Next, we matched the words and phrases to increase the consistency of meaning within and between domains, except for the mental health profile, for which linguistic validation had already been performed by another Korean team.14 In addition, we designed common questionnaires and principles for the cognitive interviews and coding methods to reduce bias among interviewers.39 Lastly, we followed standard methodology and guidelines, including the item definitions provided by the PROMIS team, and tried to achieve consensus with the PROMIS Statistical Center throughout the process to ensure measurement invariance across languages.

    Our study can contribute to the formation of a basis for the international measurement of PROs and formal process of PROMIS adoption across countries. Further psychometric validation of the K-PROMIS items will provide additional information regarding meaningful outcomes for chronic diseases and clinical settings.

    SUPPLEMENTARY MATERIAL

    Supplementary Table 1

    Translated Korean PROMIS item banks for adults

    Click here to view.(32K, doc)

    Notes

    Funding:This research was supported by a grant (18182MFDS407) from the Ministry of Food and Drug Safety in 2018, and Samsung Medical Center Research and Development Grant (SMX1210381).

    Disclosure:The authors have no potential conflicts of interest to disclose.

    Authors Contributions:

    • Conceptualization: Kim Y, Yoon J, Kim N, Lee M, Kang D, Park HY, Oh D, Sung KS, Suh GY, Ahn JS, Cho J.

    • Data curation: Kim Y, Yoon J.

    • Formal analysis: Kim Y, Yoon J.

    • Funding acquisition: Cho J.

    • Investigation: Kim Y, Cho J.

    • Methodology: Kim Y, Yoon J, Cho J.

    • Project administration: Kim Y, Yoon J, Cho J.

    • Resources: Cho J.

    • Supervision: Cho J.

    • Validation: Kim Y, Yoon J.

    • Writing - original draft: Kim Y, Yoon J.

    • Writing - review & editing: Kim Y, Yoon J, Kim N, Lee M, Kang D, Park HY, Oh D, Sung KS, Suh GY, Ahn JS, Cho J.

    ACKNOWLEDGEMENTS

    We thank the participants who participated in this study.

    References

      1. U.S. Food and Drug Administration. Guidance for industry: patient-reported outcomes measures: use in medical product development to support labeling claims. [Updated 2009]. [Accessed December 23, 2019].
        https://www.fda.gov/media/77832/download.
      1. Trotti A, Colevas AD, Setser A, Basch E. Patient-reported outcomes and the evolution of adverse event reporting in oncology. J Clin Oncol 2007;25(32):5121–5127.
      1. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res 2013;13(1):211.
      1. Howell D, Molloy S, Wilkinson K, Green E, Orchard K, Wang K, et al. Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol 2015;26(9):1846–1858.
      1. Denis F, Basch E, Septans AL, Bennouna J, Urban T, Dueck AC, et al. Two-year survival comparing web-based symptom monitoring vs routine surveillance following treatment for lung cancer. JAMA 2019;321(3):306–307.
      1. Ishaque S, Karnon J, Chen G, Nair R, Salter AB. A systematic review of randomised controlled trials evaluating the use of patient-reported outcome measures (PROMs). Qual Life Res 2019;28(3):567–592.
      1. Snyder CF, Jensen RE, Segal JB, Wu AW. Patient-reported outcomes (PROs): putting the patient perspective in patient-centered outcomes research. Med Care 2013;51(8) Suppl 3:S73–S79.
      1. Ahmed S, Berzon RA, Revicki DA, Lenderking WR, Moinpour CM, Basch E, et al. The use of patient-reported outcomes (PRO) within comparative effectiveness research: implications for clinical practice and health care policy. Med Care 2012;50(12):1060–1070.
      1. Snyder CF, Aaronson NK, Choucair AK, Elliott TE, Greenhalgh J, Halyard MY, et al. Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res 2012;21(8):1305–1314.
      1. Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, et al. The Patient-reported outcomes measurement information system (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care 2007;45(5) Suppl 1:S3–11.
      1. Riley WT, Rothrock N, Bruce B, Christodolou C, Cook K, Hahn EA, et al. Patient-reported outcomes measurement information system (PROMIS) domain names and definitions revisions: further evaluation of content validity in IRT-derived item banks. Qual Life Res 2010;19(9):1311–1321.
      1. Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, et al. The Patient-reported outcomes measurement information system (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol 2010;63(11):1179–1194.
      1. Haan EA, Terwee CB, Van Wier MF, Willigenburg NW, Van Deurzen DF, Pisters MF, et al. Translation, cross-cultural and construct validity of the Dutch-Flemish PROMIS® upper extremity item bank v2.0. Qual Life Res 2020;29(4):1123–1135.
      1. Choi H, Kim C, Ko H, Park CG. Translation and validation of the Korean version of PROMIS® pediatric and parent proxy measures for emotional distress. J Patient Rep Outcomes 2019;3(1):36.
      1. Devine J, Schröder LA, Metzner F, Klasen F, Moon J, Herdman M, et al. Correction to: Translation and cross-cultural adaptation of eight pediatric PROMIS® item banks into Spanish and German. Qual Life Res 2018;27(11):3057.
      1. Westmoreland K, Reeve BB, Amuquandoh A, van der Gronde T, Manthalu O, Correia H, et al. Translation, psychometric validation, and baseline results of the Patient-Reported Outcomes Measurement Information System (PROMIS) pediatric measures to assess health-related quality of life of patients with pediatric lymphoma in Malawi. Pediatr Blood Cancer 2018;65(11):e27353
      1. Devine J, Schröder LA, Metzner F, Klasen F, Moon J, Herdman M, et al. Translation and cross-cultural adaptation of eight pediatric PROMIS® item banks into Spanish and German. Qual Life Res 2018;27(9):2415–2430.
      1. Schnohr CW, Rasmussen CL, Langberg H, Bjørner JB. Danish translation of a physical function item bank from the Patient-Reported Outcome Measurement Information System (PROMIS). Pilot Feasibility Stud 2017;3(1):29.
      1. Jakob T, Nagl M, Gramm L, Heyduck K, Farin E, Glattacker M. Psychometric properties of a German translation of the PROMIS® depression item bank. Eval Health Prof 2017;40(1):106–120.
      1. Haverman L, Grootenhuis MA, Raat H, van Rossum MA, van Dulmen-den Broeder E, Hoppenbrouwers K, et al. Dutch-Flemish translation of nine pediatric item banks from the Patient-Reported Outcomes Measurement Information System (PROMIS)®. Qual Life Res 2016;25(3):761–765.
      1. Silva EC, Pinto Rde M, Mendonca TM, Silva CH. Brazilian-Portuguese translation and cultural adaptation of the sleep and wake disturbances domains of the Patient-reported-outcomes measurement information system (PROMIS). Cad Saude Publica 2014;30(7):1391–1401.
      1. Alves FS, Pinto RM, Mendonça TM, Silva CH. Portuguese-language translation and cross-cultural adaptation of the fatigue domain of patient-reported-outcomes measurement information system (PROMIS). Cad Saude Publica 2014;30(5):1103–1110.
      1. Terwee CB, Roorda LD, de Vet HC, Dekker J, Westhovens R, van Leeuwen J, et al. Dutch-Flemish translation of 17 item banks from the patient-reported outcomes measurement information system (PROMIS). Qual Life Res 2014;23(6):1733–1741.
      1. Bevans M, Ross A, Cella D. Patient-reported outcomes measurement information system (PROMIS): efficient, standardized tools to measure self-reported health and quality of life. Nurs Outlook 2014;62(5):339–345.
      1. Fischer HF, Wahl I, Nolte S, Liegl G, Brähler E, Löwe B, et al. Language-related differential item functioning between English and German PROMIS depression items is negligible. Int J Methods Psychiatr Res 2017;26(4):e1530
      1. Choi JS, Park YS, Kim JA, Park CS. International trends on patient-reported outcome measures for improving care quality and its implication for South Korea: focus on OECD PaRIS. Qual Improv Health Care 2019;25(1):11–28.
      1. Bonomi AE, Cella DF, Hahn EA, Bjordal K, Sperner-Unterweger B, Gangeri L, et al. Multilingual translation of the functional assessment of cancer therapy (FACT) quality of life measurement system. Qual Life Res 1996;5(3):309–320.
      1. Eremenco SL, Cella D, Arnold BJ. A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Eval Health Prof 2005;28(2):212–232.
      1. Willis GB. In: Cognitive Interviewing: a Tool for Improving Questionnaire Design. New York, NY, USA: SAGE Publications; 2004.
      1. Solorio R, Ayala NC, Paez E, Skalicky AM, Morales LS. Use of cognitive interviews to adapt PROMIS measurement items for Spanish speakers living with HIV. Aids Res Treat 2016;2016:8340863
      1. Christodoulou C, Junghaenel DU, DeWalt DA, Rothrock N, Stone AA. Cognitive interviewing in the evaluation of fatigue items: results from the patient-reported outcomes measurement information system (PROMIS). Qual Life Res 2008;17(10):1239–1246.
      1. Liegl G, Rose M, Correia H, Fischer HF, Kanlidere S, Mierke A, et al. An initial psychometric evaluation of the German PROMIS v1.2 physical function item bank in patients with a wide range of health conditions. Clin Rehabil 2018;32(1):84–93.
      1. Liu Y, Hinds PS, Wang J, Correia H, Du S, Ding J, et al. Translation and linguistic validation of the pediatric patient-reported outcomes measurement information system measures into simplified Chinese using cognitive interviewing methodology. Cancer Nurs 2013;36(5):368–376.
      1. Mahmoud GA, Rady HM, Mostafa AM. Cross cultural adaptation and validation of an Arabic version of selected PROMIS measures for use in rheumatoid arthritis patients. Egypt Rheumatol 2019;41(3):177–182.
      1. Rhee Y, Jun S, Choi SE. Concepts and applications of patient-reported outcomes & quality of life measure: practical recommendations for Korea. J Health Technol Assess 2015;3(1):48–58.
      1. Cho J, Yoon J, Kim Y, Park S. In: Development and Clinical Validation of Patient-Reported Outcome Measurements for Chronic Disease Patients for Precision Medicine. Cheongju, Korea: Korea Ministry of Food and Drug Safety; 2018.
      1. Oude Voshaar MA, Ten Klooster PM, Taal E, Krishnan E, van de Laar MA. Dutch translation and cross-cultural adaptation of the PROMIS® physical function item bank and cognitive pre-test in Dutch arthritis patients. Arthritis Res Ther 2012;14(2):R47.
      1. Bae JM. Indices for the responsiveness and interpretability in patient-reported outcomes. Korean J Fam Pract 2015;5(3):161–166.
      1. Cho J, Yoon J, Kim Y, Oh D, Kim SJ, Ahn J, et al. Linguistic validation of the US national cancer institute's patient-reported outcomes version of the common terminology criteria for adverse events in Korean. J Glob Oncol 2019;5(5):1–10.
    Cited by
    PubMed Central 1
    Scopus 2
    Web of Science 2
    MeSH Terms
    Adult*
    Chronic Disease
    Comprehension
    Fatigue
    Humans
    Information Systems*
    Linguistics*
    Psychometrics
    Surveys and Questionnaires
    Metrics
    Share
    Links to
    Figures
    slide

    1 / 1

    Tables
    slide
    slide
    slide
    slide

    1 / 4

    ORCID IDs
    Funding Information
    PERMALINK