Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

The Brazilian Version of the Edmonton Symptom Assessment System (ESAS) Is a Feasible, Valid and Reliable Instrument for the Measurement of Symptoms in Advanced Cancer Patients

  • Carlos Eduardo Paiva ,

    caredupai@gmail.com

    Affiliations Department of Clinical Oncology, Barretos Cancer Hospital, Barretos, São Paulo, Brazil, Learning and Research Institute, Barretos Cancer Hospital, Barretos, São Paulo, Brazil

  • Luciana Lopes Manfredini,

    Affiliation Cancer Hospital Children and Youth President Luiz Inacio Lula da Silva, Barretos Cancer Hospital, Barretos, São Paulo, Brazil

  • Bianca Sakamoto Ribeiro Paiva,

    Affiliation Learning and Research Institute, Barretos Cancer Hospital, Barretos, São Paulo, Brazil

  • David Hui,

    Affiliation Departments of Palliative Care and Rehabilitation Medicine, The University of Texas M. D. Anderson Cancer Center, Houston, Texas, United States of America

  • Eduardo Bruera

    Affiliation Departments of Palliative Care and Rehabilitation Medicine, The University of Texas M. D. Anderson Cancer Center, Houston, Texas, United States of America

Abstract

Purposes

To develop and validate a Portuguese version of the Edmonton Symptom Assessment System (ESAS) in Brazilian patients with advanced cancer.

Methods

The ESAS was translated and then back translated into Portuguese in accordance with international guidelines. The final version was approved by an Expert Committee after pilot testing on 24 advanced cancer patients. Subsequently, we evaluated the time to complete the assessment, the number of unanswered items, internal consistency, convergent validity, and known-group validity in a sample of 249 advanced cancer patients who completed the ESAS along with the European Organization for Research and Treatment of Cancer Core Quality of Life Questionnaire (EORTC QLQ-C30), Hospital Anxiety and Depression Scale (HADS), and Epworth Sleepiness Scale (ESS). A total of 90 clinically stable patients were retested after 4 to 96 hours (test-retest reliability), and 80 patients answered the ESAS after 21 (±7) days to measure scale responsiveness using an anchor-based method.

Results

The ESAS was completed in a mean time of only 2.2 minutes. The internal consistency was good (Cronbach's alpha = 0.861), and the removal of single items did not change the overall alpha value. For convergent validity, Spearman’s correlation coefficients between the ESAS symptom scores and the corresponding EORTC QLQ-C30 and ESS symptom scores ranged between 0.520 (95% CI = 0.424–0.605) and 0.814 (95% CI = 0.760–0.856), indicating moderate to strong correlations. Test-retest reliability values were considered adequate, with intraclass correlation coefficients ranging from 0.758 (95% CI = 0.627–0.843) to 0.939 (95% CI = 0.905–0.960).

Conclusions

ESAS is a feasible, valid and reliable multi-symptom assessment instrument for use in Brazil.

Introduction

Patients with advanced cancer frequently report multiple concomitant symptoms [1,2] that negatively affect their quality of life [3]. Therefore, the adequate control of symptoms is an essential part of supportive care in oncology. The number of symptoms found using systematic assessments has been shown to be tenfold higher than the number that are voluntarily reported [2]. Thus, routine and systematic symptom assessments with a validated symptom assessment tool should occur during patient visits to oncology clinics [4,5].

A number of symptom screening instruments have been developed, including those that target single and multiple symptoms [6,7]. The Edmonton Symptom Assessment System (ESAS) is a widely used, multiple-symptom assessment instrument that was first developed in 1991 by Bruera et al. to audit the symptoms of patients receiving palliative care [8]. Originally, the ESAS investigated eight symptoms using visual analog scales (0–100 mm). However, these scales have undergone several changes since they were first implemented. The most recent version of the ESAS evaluates 10 common symptoms using categorical numbered scales (0–10) [9,10]. The ESAS has been translated and validated in several languages and cultures [816]. Although it is frequently adapted in medical practice and research in Brazil, its psychometric properties have not been formally investigated to date.

Thus, we developed a Brazilian version of the ESAS by performing a translation and cultural adaptation and subsequently tested it on a sample of advanced cancer patients to measure its psychometric properties.

Methods

Ethics Statement

This study complied with the ethical standards of the Declaration of Helsinki and Brazilian National Health Council resolution no. 466/2012. It was approved by the Research Ethics Committee of the Barretos Cancer Hospital (BCH; Barretos, SP, Brazil) under number 100.425. Each of the included patients voluntarily signed a consent-to-participate form.

Study design and selection of participants

The present cross-sectional study employed methods for the translation and validation of assessment instruments. To be eligible, patients had to have been diagnosed with advanced cancer (locally advanced, relapsed or refractory or metastatic disease), at least 18 years old, and a native Portuguese-speaking Brazilian. Patients with any confusion, uncontrolled psychiatric disease, cognitive dysfunction, or any other disabling disease that could hinder their ability to answer the study questionnaires were not eligible to participate.

Phase I—Translation process

The ESAS is a well known symptom-intensity tool for assessing ten common symptoms (pain, fatigue, nausea, anxiety, depression, drowsiness, anorexia, well-being, dyspnea, and sleep disturbance) in cancer care [8]. The severity of each symptom is rated from 0 to 10, with 0 indicating that the symptom is absent and 10 describing the worst possible severity [17]. The total symptom distress score (TSDS) represents the sum of all ESAS symptoms (from 0 to 100) [8,18].

Five different versions of the ESAS (V1A, V1B, V1C, V1D, and V1E) were obtained by translating them from their original English versions into the target Portuguese/Brazil language. Translations were performed by five different Brazilian native speakers who were also fluent in English. These included a radiation oncologist, a medical oncologist, two engineers, and an English teacher; both oncologists were familiar with the scale, but had never had used it. The translation panel for the reconciliation of these five versions consisted of four authors (BSRP, LLM, EMB, and BSRP), who analyzed the detailed translations and synthesized them into a grouped version (V1A-E). Two native North American professional translators, who spoke Portuguese/Brazil fluently but had no prior knowledge of the ESAS, subsequently performed a back translation from the V1A-E version into English. Again, the same translation panel for reconciliation synthesized the two translations into a grouped version (V2A-B). A bilingual Expert Committee that was composed of five members (a medical oncologist, registered nurse, dentist, occupational therapist, and psychologist), all of whom had expertise in instrument validation studies, was assembled. Out of this group, the physician, nurse and occupational therapist had extensive experience in palliative care practice and research. This committee analyzed all of the translation documents using a structured protocol that has been described elsewhere [19] and produced the final adapted version of the ESAS to be pre-tested. Cognitive debriefing interviews on the Brazilian version of ESAS were conducted with 24 advanced cancer patients of different educational backgrounds who were representative of the target population. These patients were questioned about confusion, embarrassment, and comprehension difficulties, and when necessary, they were given suggestions. Patient understanding of the ESAS was also graded according to the interviewer’s opinion.

Phase II–Psychometric evaluation

Informed consent process and data collection.

A research coordinator from the Research Support Center (Barretos, SP, Brazil) identified potential participants via convenience sampling at the Clinical Oncology Department (outpatient clinics and inpatient ward) of the BCH by checking eligibility criteria from the medical charts. In sequence, interviewers contacted potential participants in person and informed them about the objectives and procedures of the study. Then, after asking simple initial questions (“what is your name?”, “where are you from?”, “what kind of disease are you treating?” and “which treatment are you receiving?”), the interviewers confirmed each patient’s ability to communicate in Brazilian Portuguese and identified cases of evident cognitive dysfunction. Those who voluntarily agreed to participate in the study completed the evaluation instruments.

All of the patients who were included in the validation phase of this study completed the ESAS, Hospital Anxiety and Depression Scale (HADS), Epworth Sleepiness Scale (ESS) and European Organization for Research and Treatment of Cancer Core Quality of Life Questionnaire (EORTC QLQ-C30). These instruments could be completed in either a self-administered or interviewer-assisted manner, depending on the choice of the participant. To assess test-retest reliability, 84 clinically stable participants were subjected to a second interview 4 to 96 hours later. To be considered clinically stable, a patient’s Karnofsky performance status at both of the two visits needed to be within 10% of each other. Following this, to assess the responsiveness (the ability of the instrument to detect changes in symptoms) of the ESAS, 80 patients retook the test after 21 (±7) days. For each symptom evaluated by the ESAS, participants answered as to whether it was better than, the same as, or worse than it was during the first evaluation. Data were collected from August 2012 to March 2014.

Validation instruments.

EORTC QLQ-C30. The 30-item EORTC QLQ-C30 (version 3.0) includes a scale that measures global health, five functioning scales (physical, role, emotional, cognitive, and social), three symptom scales (fatigue, nausea/vomiting, and pain), six single items addressing common symptoms (dyspnea, sleep disturbance, appetite loss, constipation, and diarrhea), and an additional item measuring financial difficulties. Items are rated on a Likert scale from 1 (not at all) to 4 (very much), with the exception of two global health items, which are rated from 1 (very poor) to 7 (excellent). All of the items are linearly transformed to a 0–100 scale. For the functioning and global quality of life scales, a higher score represents a better quality of life. In contrast, a higher score on the symptom scale represents greater symptom severity [20]. The EORTC QLQ-C30 has been previously validated in Brazil [21]. In the present study, Cronbach’s alpha ranged from 0.622 (cognitive functioning) to 0.852 (global health).

Hospital Anxiety and Depression Scale. The HADS questionnaire consists of 14 items with a 4-point Likert-type scale ranging from 0 (minimally present) to 3 (maximally present). It is commonly used to assess anxiety and depression among individuals with cancer and has been validated in Brazil [22,23]. HADS-A and HADS-D scores range from 0 to 21, with higher scores indicating greater distress. In the present study, Cronbach’s alphas were 0.78 (HADS-A) and 0.82 (HADS-D).

Epworth Sleepiness Scale. The ESS was developed to identify the occurrence of excessive daytime sleepiness. The ESS consists of 8 items that are rated using a 4-point scale (0–3) [24]. The higher the score, the greater the daytime sleepiness. The ESS has been previously validated in Brazil [25]. In the present study, Cronbach’s alpha was 0.808.

Karnofsky Performance Status. The KPS is an 11-point rating scale that ranges from 0 (dead) to 100 (normal function) and was initially developed to assess functional status in patients with cancer [26]. It has been used in previous Brazilian studies [27,28].

Statistical analysis

Clinical utility was estimated by measuring the mean (standard deviation [SD]) time to complete the ESAS and the number of missing items per ESAS item. Internal consistency was assessed using Cronbach’s alpha coefficient, and a value between 0.70 and 0.95 was considered adequate [29]. Test-retest reliability was measured using the intraclass correlation coefficient (ICC) at 4 to 96 hours after the first evaluation; a value ≥0.70 was considered adequate [29]. Regarding convergent validity, we hypothesized that the results of the HADS, EORTC QLQ-C30, and ESS, all of which measure similar symptoms as the ESAS, would be at least moderately correlated with the results of the ESAS. Pearson correlation coefficients of >0.6, 0.4–0.6, and <0.4 were considered strong, moderate, and poor correlations, respectively [30]. The known-group validity analysis was performed to evaluate the extent to which the instrument was able to discriminate between clinical subgroups of patients. The ESAS symptom scores were compared between groups of patients with different KPS values (≤70% versus >70%) and treatment locations (inpatient versus outpatient). The non-parametric Mann-Whitney test was used for these comparisons. Responsiveness analyses were carried out using an anchor-based strategy. After 21 (±7) days, patients were asked to classify their symptoms as worse than, the same as, or better than those experienced during their first study visit. Following this, median values were calculated for each category (worse, the same and better) and for each specific ESAS symptom. Differences were evaluated using a non-parametric, two-sided Wilcoxon signed rank test.

All statistical analyses were performed using SPSS software version 20.0 (Chicago, IL, USA). P-values of less than 0.05 were considered statistically significant.

Results

Phase I–Translation process

S1 Table describes the original version, the translated version, the back-translated version, the final adapted version, and core commentaries from the Expert Committee. During the translation process, the committee questioned the best terms in Portuguese to describe the concepts of fatigue and depression. Thus, we conducted another study to search for the best terms [31]. The findings from this study were then sent to the members of the Expert Committee; each one had the autonomy to decide whether or not to use the results of the aforementioned study. The ESAS was well comprehended by all of the patients. According to the interviewers’ perceptions, 1 (1/24, 4.1%), 2 (2/24, 8.3%), and 3 (3/24, 12.5%) patients only partially understood the fatigue, nausea and drowsiness items, respectively. However, no modification was necessary after the pre-testing, and the panel decided to use the final version of the ESAS (S2 Table) in the psychometric validation study.

Phase II–Psychometric evaluation

Originally, 293 patients were invited to participate in the study. Of them, 1 was considered a screening failure (patient did not presented advanced disease), and 43 refused to participate. Therefore, the final sample population comprised 249 advanced cancer patients.

The mean (SD) age was 55.1 (12.6) years. The mean (SD) KPS value was 78.1 (13.2). The majority of the patients were women (158/249, 63.5%), most of who had less than 8 years of formal education and actively worked (174/249, 70.0%). The mean (SD) wage was 2.94 (3.28) Brazilian minimum wages. The most common primary cancer types were breast (n = 85, 34.1%) and colorectal (44/249, 17.7%). There were 218 (218/249, 87.6%) and 50 (50/249, 20.1%) patients with distant metastasis and locally unresectable recurrence, respectively. The majority of the patients were receiving palliative chemotherapy (208/249, 83.5%). The demographic and clinical characteristics of the patients are detailed in Table 1.

thumbnail
Table 1. Demographic and clinical characteristics of the patients (n = 249).

https://doi.org/10.1371/journal.pone.0132073.t001

The lowest prevalence and mean symptom scores were nausea (23.3% and 0.99, respectively), dyspnea (29.3% and 1.39, respectively) and depression (26.5% and 1.26, respectively). The highest prevalence and mean symptom scores were sleep disturbance (61% and 3.2, respectively), anxiety (65.5% and 3.15, respectively) and drowsiness (60.5% and 3, respectively) (Table 2).

thumbnail
Table 2. Mean and median scores, missing values, and percentages of maximum and minimum scores.

https://doi.org/10.1371/journal.pone.0132073.t002

Instrument’s feasibility.

The ESAS was completed in a mean (SD) time of 2.24 (1.36) minutes. Out of 249 questionnaires administered, only 1 non-response item was observed, for which the patient did not understand the meaning of the word drowsiness (“sonolência”).

Internal consistency.

Cronbach’s alpha coefficient was 0.86 (95% confidence interval [CI] = 0.83–0.88). No single item significantly modified the internal consistency of the scale when deleted (Table 2).

Test-retest reliability.

The general test-retest ICC value was 0.94 (0.90–0.96). When considering isolated items, the lowest values were observed for drowsiness (0.76, 95% CI = 0.63–0.84) and fatigue (0.78, 95% CI = 0.66–0.86) (Table 2).

Convergent validity.

Convergent validity was assessed by determining the correlation between each ESAS symptom and a specific related measure from another validated instrument. All previously hypothesized correlations were further confirmed. In general, we observed strong correlations (r>0.6), with the exception of the following moderate correlations: ESAS-depression and emotional functioning (r = -0.57; 95% CI = -0.49−0.64), ESAS-depression and HADS-D (r = 0.52; 95% CI = 0.42–60), and ESAS-drowsiness and ESS global score (r = 0.55; 95% CI = 0.44–0.64) (Table 3).

Known-group validity.

As expected, the inpatients reported higher median symptom scores than the outpatients for all individual ESAS symptoms. Regarding the TSDS, the median values of the outpatients and inpatients were 16 and 37, respectively (Table 4). The same pattern was observed when comparing the median scores of the ESAS symptoms and the ESAS-TSDSs of the patients based on their KPS; all analyses revealed that the patients with low performance statuses (KPS≤70%) had higher scores than those with better performance statuses (KPS>70%) (Table 4).

Responsiveness.

It was expected that the patients with no perception of change from their first to their second study visit would have no statistically significant changes in their scores. This was confirmed for all ESAS symptoms, with the exception of dyspnea, for which the score for the first visit was slightly higher than that for the second (p = 0.029). Patients who reported feeling better at the second visit had significantly lower ESAS scores for pain (p = 0.038), fatigue (p = 0.008), depression (p = 0.027), loss of appetite (p = 0.001), feeling of well-being (p = 0.037), and sleep disturbance (p = 0.024) than at the first visit. The only ESAS symptom score that was significantly altered in the patients who perceived symptom deterioration was that for pain (p = 0.042). However, in general, a low number of patients were classified as having symptom deterioration, which could have impacted the statistical analyses (Table 5).

Discussion

The majority of patients with advanced cancer are exclusively treated by medical oncologists during most of the time that they receive palliative chemotherapy. Although the early integration of palliative care in the treatment of advanced cancer patients is recommended, it is typically only offered late during the course of the disease, even at comprehensive cancer centers [32]. A key approach to improving the health-related quality of life of cancer patients undergoing chemotherapy is the timely and effective control of uncomfortable symptoms [18,33]. Patients with advanced cancer frequently report many symptoms concomitantly; thus, the proper screening of these symptoms is essential in oncology care [5]. In this study, the ESAS was completed by advanced cancer patients, most of whom were undergoing palliative chemotherapy. In Brazil, the ESAS has been widely used in clinical practice and research, even without proper psychometric validation. However, it is rarely used in routine medical oncology. In our opinion, appropriate strategies for the screening of symptoms and the implementation of institutional flowcharts for the management of these symptoms should be a priority for excellence in clinical oncology services. The ESAS may play a valuable role in this endeavor. In our study, the ESAS was easily completed in a mean time of only 2.2 minutes.

Based on a previous "think-aloud" study of 20 advanced cancer patients, the ESAS was revised (ESAS-r) by Canadian researchers [34]. The revised version is easier to understand than the original as determined by Watanabe et al. [35]. Our results contradict the need to modify the ESAS, showing that it was easy to understand, and possessed only one unanswered item (due to lack of comprehension). Interestingly, our Brazilian version of ESAS contains two added words, which were inserted in brackets, for the definitions of fatigue and nausea. We believe that the addition of these words in brackets facilitated the understanding of these items. The ESAS-r was translated simultaneously and independently by another Brazilian research group [36]. Interestingly, the final versions of both instruments are quite similar, which validates the translation and adaptation processes performed by both studies. However, the psychometric properties of the Brazilian version of the ESAS-r have not yet been evaluated.

The translation process was conducted according to guidelines proposed by Guillemin et al. [37]. However, we conducted another study concurrent with the translation process to define the best terms in Portuguese (Brazil) to explain the concepts of "depression" and "fatigue" [31], as was performed for the Spanish validation of the ESAS [10]. It was difficult to create a conciliatory version (T1AB) with regard to these terms because several words were initially considered equally appropriate. Generally, the members of the Expert Committee decided on the best terms or phrases to be used in the translated questionnaire based on their personal opinions. However, the results of our simultaneous study were made available to the members of the Expert Committee, who had the autonomy to decide whether to use the results during the process of translating and adapting the ESAS. This method of empirically defining terms could be more frequently used in validation studies, depending on the unresolved issues encountered during translation processes. This method is feasible and appears to facilitate expert decisions.

The reliability of the ESAS was evaluated by calculating the internal consistency and the test-retest values. The internal consistency values that were calculated were considered adequate and were in accordance with those reported by previous ESAS validation studies [9,14,15,38]. Regarding test-retest reliability, our results were also considered quite adequate. We expected ICC values to be above 0.7, which were observed for all ESAS items and TSDSs. Although the ICC 95% CIs for fatigue, drowsiness and sleep disturbance included the lower limit of significance that we adopted, their values were all very close to 0.7.

The validity of the ESAS was assessed using convergent validity and also by comparing symptom scores between known distinct groups of patients. With regard to convergent validity, the ESAS symptom scores were correlated with items or subscales from other questionnaires that measured the same constructs and were previously validated in Brazil. In general, we observed significant correlations, and most were higher than expected (r>0.6). Appropriate correlations of ESAS scores with other instruments have been demonstrated in previous validation studies [9,11,16]. Known-group validation is another important psychometric property that is commonly evaluated in validation studies. Instruments with adequate sensitivities are required to detect clinical differences between groups that are known to be different [30]. Our results confirmed the previous hypothesis that median ESAS values are significantly higher in inpatients and those with worse functional statuses compared to outpatients and those with better functional statuses. Consistent with our findings, previous studies [10,16,38] have shown that the ESAS is a tool with adequate sensitivity for identifying patients with different conditions.

Responsiveness is the ability to detect clinically important differences that occur over time [30]. In the present study, we re-evaluated the ESAS scores after 14–28 days. The scores of patients in three different groups were analyzed as a function of their perceived changes in clinical conditions, which were classified as better, worse, or the same. In general, the median scores of the patients who considered themselves better were improved, and those of the patients who reported a worsened condition were decreased. However, we observed statistically significant differences in only some of the analyses, which can be explained by the small sample size. The patients who considered themselves better (of whom there was a greater number and therefore a larger sample size) had statistically significantly lower scores for pain, fatigue, depression, lack of appetite, feeling of well-being and sleep disturbance. The difference in pain was so profound in those who considered themselves worse that even with a sample size of only seven patients it was possible to obtain a statistically significant difference. From a statistical point of view, the inclusion of a higher number of patients with worse clinical conditions (reporting greater symptom burdens) would be beneficial. In these patients, symptomatic worsening is more frequent, and improvement, when it occurs, is supposedly more evident. Previous studies assessing the ESAS have had difficulties in demonstrating the responsiveness of the scale statistically [9,10,16]. A strength of our study is the evaluation of responsiveness using an anchor-based method. To the best of our knowledge, ours is the first validation study of the ESAS using this methodology in the analysis of responsiveness.

The present study has several limitations. One limitation is that we included patients with better functional conditions who were mostly in outpatient chemotherapy. Thus, we might have overestimated the instrument’s feasibility because patients in worse functional condition might experience greater difficulties in understanding or completing the ESAS items. However, even in considering these aspects, our results regarding its feasibility justify its routine use for the screening of symptoms in the oncology setting. Another limitation of this study is the small sample size that was used in the analysis of responsiveness. Because most patients showed clinical improvement or stability, our analysis was limited by the small proportion of patients who reported a worsening of symptoms. Our center is currently involved in a separate large, multicenter study to evaluate the responsiveness of the ESAS and to determine the minimal clinically important difference values.

Conclusions

The ESAS should be considered a reliable and valid instrument for use in Brazil to assess symptoms in advanced cancer patients. Our results revealed that this tool is easy to understand and can be quickly completed, suggesting that it could be used in routine practice in palliative care as well as in medical oncology clinics. A larger, ongoing study is being performed to confirm our findings regarding the responsiveness of the ESAS.

Supporting Information

S1 Table. Summary of the original, forward-translated, back-translated, and final adapted versions and core comments from the Expert Committee.

https://doi.org/10.1371/journal.pone.0132073.s001

(DOC)

S2 Table. Final version of the Edmonton Symptom Assessment System (ESAS) in Portuguese from Brazil.

https://doi.org/10.1371/journal.pone.0132073.s002

(DOC)

Acknowledgments

We would like to thank Marielle Borges and other members of the Research Support Center (NAP, Barretos Cancer Hospital) for their assistance with data collection.

Author Contributions

Conceived and designed the experiments: CEP LLM BSRP DH EB. Performed the experiments: CEP LLM. Analyzed the data: CEP BSRP. Contributed reagents/materials/analysis tools: CEP EB. Wrote the paper: CEP LLM BSRP DH EB.

References

  1. 1. Homsi J, Walsh D, Nelson KA, LeGrand SB, Davis M, Khawam E, et al. The impact of a palliative medicine consultation service in medical oncology. Support Care Cancer. 2002;10(4):337–342. pmid:12029434
  2. 2. Homsi J, Walsh D, Rivera N, Rybicki LA, Nelson KA, Legrand SB, et al. Symptom evaluation in palliative medicine: patient report vs systematic assessment. Support Care Cancer. 2006;14(5):444–453. pmid:16402231
  3. 3. Portenoy RK, Thaler HT, Kornblith AB, Lepore JM, Friedlander-Klar H, Coyle N, et al. Symptom prevalence, characteristics and distress in a cancer population. Qual Life Res. 1994;3(3):183–189. pmid:7920492
  4. 4. Rangachari D, Smith TJ. Integrating palliative care in oncology: the oncologist as a primary palliative care provider. Cancer J. 2013;19(5):373–378. pmid:24051609
  5. 5. Partridge AH, Seah DSE, King T, Leighl NB, Hauke R, Wollins DS, et al. Developing a service model that integrates palliative care throughout cancer care: the time is now. J Clin Oncol. 2014;32(29):3330–3336. pmid:25199756
  6. 6. Hearn J, Higginson IJ. Outcome measures in palliative care for advanced cancer patients: a review. J Public Health Med. 1997;19(2):193–199. pmid:9243435
  7. 7. Paiva CE, Barroso EM, Carneseca EC, de Pádua Souza C, Dos Santos FT, Mendoza López RV, et al. A critical analysis of test-retest reliability in instrument validation studies of cancer patients under palliative care: a systematic review. BMC Med Res Methodol. 2014;14:8. pmid:24447633
  8. 8. Bruera E, Kuehn N, Miller MJ, Selmser P, Macmillan K. The Edmonton Symptom Assessment System (ESAS): a simple method for the assessment of palliative care patients. J Palliat Care. 1991;7(2):6–9. pmid:1714502
  9. 9. Kwon JH, Nam S-H, Koh S, Hong YS, Lee KH, Shin S-W, et al. Validation of the Edmonton Symptom Assessment System in Korean patients with cancer. J Pain Symptom Manage. 2013;46(6):947–956. pmid:23628516
  10. 10. Carvajal A, Centeno C, Watson R, Bruera E. A comprehensive study of psychometric properties of the Edmonton Symptom Assessment System (ESAS) in Spanish advanced cancer patients. Eur J Cancer. 2011;47(12):1863–1872. pmid:21514818
  11. 11. Philip J, Smith WB, Craft P, Lickiss N. Concurrent validity of the modified Edmonton Symptom Assessment System with the Rotterdam Symptom Checklist and the Brief Pain Inventory. Support Care Cancer. 1998;6(6):539–541. pmid:9833305
  12. 12. Pautex S, Berger A, Chatelain C, Herrmann F, Zulian GB. Symptom assessment in elderly cancer patients receiving palliative care. Crit Rev Oncol Hematol. 2003;47(3):281–286. pmid:12962902
  13. 13. Strömgren AS, Goldschmidt D, Groenvold M, Petersen MA, Jensen PT, Pedersen L, et al. Self-assessment in cancer patients referred to palliative care: a study of feasibility and symptom epidemiology. Cancer. 2002;94(2):512–520. pmid:11900236
  14. 14. Chinda M, Jaturapatporn D, Kirshen AJ, Udomsubpayakul U. Reliability and validity of a Thai version of the edmonton symptom assessment scale (ESAS-Thai). J Pain Symptom Manage. 2011;42(6):954–960. pmid:21621964
  15. 15. Claessens P, Menten J, Schotsmans P, Broeckaert B. Development and validation of a modified version of the Edmonton Symptom Assessment Scale in a Flemish palliative care population. Am J Hosp Palliat Care. 2011;28(7):475–482. pmid:21398268
  16. 16. Moro C, Brunelli C, Miccinesi G, Fallai M, Morino P, Piazza M, et al. Edmonton symptom assessment scale: Italian validation in two palliative care settings. Support Care Cancer. 2006;14(1):30–37. pmid:15937688
  17. 17. Kang JH, Kwon JH, Hui D, Yennurajalingam S, Bruera E. Changes in symptom intensity among cancer patients receiving outpatient palliative care. J Pain Symptom Manage. 2013;46(5):652–660. pmid:23566756
  18. 18. Zimmermann C, Swami N, Krzyzanowska M, Hannon B, Leighl N, Oza A, et al. Early palliative care for patients with advanced cancer: a cluster-randomised controlled trial. Lancet. 2014;383(9930):1721–1730. pmid:24559581
  19. 19. Barroso EM, Carvalho AL, Paiva CE, Nunes JS, Paiva BSR. Tradução e adaptação cultural para o português (Brasil) do instrumento Vanderbilt Head and Neck Symptom Survey version 2.0 (VHNSS2.0) para avaliação de sintomas orais em pacientes com câncer de cabeça e pescoço. Braz J Otorhinolaryngol. 2015; in press.
  20. 20. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85(5):365–376. pmid:8433390
  21. 21. Paiva CE, Carneseca EC, Barroso EM, de Camargos MG, Alfano ACC, Rugno FC, et al. Further evaluation of the EORTC QLQ-C30 psychometric properties in a large Brazilian cancer patient cohort as a function of their educational status. Support Care Cancer. 2014; 22(8):2151–2160. pmid:24652051
  22. 22. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361–370. pmid:6880820
  23. 23. Pais-Ribeiro J, Silva I, Ferreira T, Martins A, Meneses R, Baltar M. Validation study of a Portuguese version of the Hospital Anxiety and Depression Scale. Psychol Health Med. 2007;12(2):225–235 pmid:17365902
  24. 24. Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep. 1991;14(6):540–545. pmid:1798888
  25. 25. Bertolazi AN, Fagondes SC, Hoff LS, Pedro VD, Menna Barreto SS, Johns MW. Portuguese-language version of the Epworth sleepiness scale: validation for use in Brazil. J Bras Pneumol. 2009;35(9):877–883. pmid:19820814
  26. 26. Karnofsky D, Burchenal J. The clinical evaluation of chemotherapeutic agents in cancer. In: MacLeod C, editor. Evaluation of chemotherapeutic agents. New York: Columbia University Press; 1949.
  27. 27. Paiva CE, Paiva BSR. Does the Palliative Performance Scale have added value over the Karnofsky Performance Status in ambulatory cancer patients receiving palliative care? J Palliat Med. 2014;17(3):264–265. pmid:24521252
  28. 28. Rugno FC, Paiva BSR, Paiva CE. Early integration of palliative care facilitates the discontinuation of anticancer treatment in women with advanced breast or gynecologic cancers. Gynecol Oncol. 2014;135(2):249–254. pmid:25173586
  29. 29. Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42. pmid:17161752
  30. 30. Fayers P, Machin D. Quality of Life: The Assessment, Analysis and Interpretation of Patient-reported Outcomes. 2nd ed. Chinchester, UK: John Wiley & Sons; 2007.
  31. 31. Paiva CE, Manfredini LL, Paiva BSR, Carneseca EC, Yennurajalingam S, Bruera E, et al. What are the best terms in Portuguese to explain the concepts of “fatigue” and “depression” in cancer patients? Palliat Support Care. 2014;1–6.
  32. 32. Hui D, Kim S- H, Kwon JH, Tanco KC, Zhang T, Kang JH, et al. Access to palliative care among patients treated at a comprehensive cancer center. Oncologist. 2012;17(12):1574–80. pmid:23220843
  33. 33. Temel JS, Greer JA, Muzikansky A, Gallagher ER, Admane S, Jackson VA, et al. Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med. 2010;363(8):733–42. pmid:20818875
  34. 34. Watanabe S, Nekolaichuk C, Beaumont C, Mawani A. The Edmonton symptom assessment system—what do patients think? Support Care Cancer. 2009;17(6):675–83. pmid:18953577
  35. 35. Watanabe SM, Nekolaichuk C, Beaumont C, Johnson L, Myers J, Strasser F. A multicenter study comparing two numerical versions of the Edmonton Symptom Assessment System in palliative care patients. J Pain Symptom Manage. 2011;41(2):456–68. pmid:20832987
  36. 36. Monteiro D da R, Almeida M de A, Kruse MHL. [Translation and cross-cultural adaptation of the Edmonton Symptom Assessment System for use in Palliative Care]. Rev Gaucha Enferm. 2013;34(2):163–171. pmid:24015476
  37. 37. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417–1432. pmid:8263569
  38. 38. Chang VT, Hwang SS, Feuerman M. Validation of the Edmonton Symptom Assessment Scale. Cancer. 2000;88(9):2164–2171. pmid:10813730