Main

Oral mucositis is a common consequence of chemotherapy and stem cell transplantation (SCT) that reduces quality of life and results in morbidity in adults and children with cancer (Sonis, 2011). Prevention of mucositis is important to patients, parents and health-care providers (Ethier et al, 2012). In order to determine the most effective preventive and treatment strategies, outcome measures are needed to reliably quantify the degree of oral mucositis and its resultant morbidity. Considerable effort has resulted in many different mucositis scales being developed for adults, and currently, there are several options for the valid measurement of mucositis among adults receiving chemotherapy and radiotherapy, including those undergoing SCT (Sonis et al, 2004). However, there remains a lack of validated instruments for assessing mucositis in children (Tomlinson et al, 2007).

Children have unique measurement issues compared with adults. Young children are more difficult to assess due to lack of co-operation with oral cavity examination. In addition, attribution of functional symptoms may not be linkable to an aetiology. For example, if a young child refuses to eat, it may be difficult to know whether this behaviour is related to mucositis, nausea or anorexia. The lack of a feasible, reliable and valid scoring system for mucositis in children has created an obstacle to interventional and epidemiological mucositis research in paediatric cancer (Qutob et al, 2013).

In order to address this gap in the literature, we formed a multi-disciplinary group with expertise in paediatric mucositis. Our previous work established the need for a new paediatric mucositis scale, generated items, drafted the scale and evaluated early psychometrics with a focus on understandability, content validity and acceptability from parents and children with cancer. Three iterations were required to arrive at the final version of the instrument that was then termed the Children’s International Mucositis Evaluation Scale (ChIMES; Tomlinson et al, 2007, 2008a, 2008b, 2009a, 2009b, 2010).

This manuscript describes the evaluation of reliability and validity of ChIMES in children with cancer or undergoing SCT. We hypothesised that ChIMES would be reliable (test–retest, inter-rater and internal consistency) and valid (convergent validity and responsiveness).

Materials and Methods

We conducted a multi-centre prospective study of children at increased risk of developing oral mucositis in order to evaluate the psychometric properties of ChIMES. We measured oral mucositis at baseline (conducted between day −2 to day 5 when mucositis was not expected) and then daily between Days 7 and 17 (when mucositis was expected) following start of chemotherapy or conditioning. Three groups of respondents were included: parent/guardian proxy respondents for children aged <12 years; child self-respondents aged 12–18 years; and child self-respondents aged 8 to <12 years.

The Research Ethics Board at The Hospital for Sick Children and all participating institutions approved the study. All participants provided written informed consent or assent as appropriate.

Subjects

Children aged 0 to18 years were eligible if they were to receive any of the following treatments: myeloablative SCT, 60 mg m−2 course−1 doxorubicin or 12 g m−2 methotrexate. We excluded respondents unable to read English and those who did not have the cognitive ability to complete instruments. Participants were recruited from: The Hospital for Sick Children, Toronto, ON, Canada; Children’s National Medical Center, Washington, DC, USA; and Lucile Salter Packard Children's Hospital at Stanford, Palo Alto, CA, USA.

Procedures

Respondents were approached in the inpatient or clinic setting before the start of the chemotherapy cycle or SCT procedure. Measures of mucositis were ChIMES, World Health Organisation (WHO) mucositis scale, mucositis visual analogue scale (VAS), National Cancer Institute Common Terminology Criteria (NCI-CTC) v3.0 functional/symptomatic mucositis scale and the Oral Mucositis Daily Questionnaire (OMDQ). The recall period for all instruments was the same day except for the OMDQ, which was the past 24 h.

A study training manual, which detailed standard operating procedures, was circulated to all participating sites to maximise consistency in evaluations. Participants were provided descriptions and pictures of erythema and ulcers. Participants were encouraged to consult with their doctor or nurse if they were unsure about their presence. A standard light source was not used across sites. Timing and conduct of assessments were further standardised to occur before eating or drinking and at approximately the same time every day. The order of evaluations was also standardised. The baseline assessment occurred between 2 days before initiation of chemotherapy or SCT conditioning until 5 days following chemotherapy initiation as mucositis is unlikely to occur within this time frame. Follow-up assessments were then conducted once daily between Days 7 to 17 after starting chemotherapy or conditioning. Responses were recorded in a paper diary. The evaluation of test–retest reliability occurred on Days 13 and 14, when maximum mucositis was expected.

For children aged <8 years, the parent completed all measures and the child did not participate. On Day 14, the parent answered the following three additional questions: (a) reported whether their child was experiencing any mouth pain or sores that day (yes or no); (b) reported whether oral mucositis had changed since the previous day (Day 13) on a five-point scale consisting of much worse, somewhat worse, no change, somewhat better and much better; and (c) completed the Faces Pain Scale—Revised (Hicks et al, 2001) to explore how this measure is associated with the pain question of ChIMES.

For children aged 12 years, the child completed all instruments although he/she could request assistance from the parent/guardian if necessary. These older children also completed the three additional Day-14 questions as outlined above. The parent did not participate in daily assessments although on Day 14, the parent or guardian also completed ChIMES so that inter-rater reliability could be examined. For children aged 8 to <12 years, only the parent, only the child or both could participate. In the case of child self-report, the child completed the three additional Day-14 questions and the parent completed Day-14 ChIMES.

Outcome measures

ChIMES

ChIMES (parent version illustrated in Appendix 1) consists of seven elements: (1) Amount of mouth or throat pain (ChIMES1), (2) Effect of mouth or throat pain on swallowing (ChIMES2), (3) Effect of mouth or throat pain on eating (ChIMES3), (4) Effect of mouth or throat pain on drinking (ChIMES4), (5) Receipt of pain medication (ChIMES5), (6) Receipt of pain medication for mouth or throat pain (ChIMES6), and (7) Presence of ulcers (ChIMES7). Because in young children failure to eat or drink may not be attributable to mucositis vs other aetiologies such as nausea or anorexia, the instrument allows the respondent to choose ‘I can’t tell’ if the respondent is uncertain if the cause of functional impairment is due to oral mucositis. ChIMES1–4 each received a score of 0–5 where 5 is the worst degree of symptoms. ChIMES5 received a score of 1 if the child had received pain medications and ChIMES6 received a score of 1 if the child received pain medications because of mucositis. Otherwise, ChIMES5 and 6 received a score of 0. Finally, ChIMES7 received a score of 1 if oral ulcers were present and 0 if absent. Any question that was scored as missing or ‘I can’t tell’ was excluded from the total possible score. If all the questions were answered, the maximum score was 23. The ChIMES Total Score was the sum of all scores; ‘I can’t tell’ responses and missing responses both received a score of 0. The ChIMES Percentage Score was the ChIMES Total Score over the total maximum score taking into account ‘I can’t tell’ responses (by subtracting these items from the maximum score) multiplied by 100. In other words, the ChIMES Total Score does not take into account ‘I can’t tell’ or missing responses as they are given a score of 0 and the ChIMES Total Score keeps the weighting of all components constant. In contrast, ChIMES Percentage Score does take into account ‘I can’t tell’ responses by changing the maximum score possible. Higher scores correspond to worse mucositis.

WHO

The WHO scale is based upon the ability to eat and drink combined with objective signs of mucositis, namely erythema and ulceration (World Health Organization, 1979). Visualisation of the oral cavity is critical for scoring, as the presence of oral ulcers delineates a WHO mucositis grade of 2 vs<2. It is one of the most commonly used outcome measures in clinical research (Sonis et al, 2004). WHO grade ranges from 0 to 4 where higher scores correspond to worse mucositis.

VAS

We used a horizontal 10-cm VAS anchored at 0 (no mouth or throat pain) and 10 (most severe mouth or throat pain) and asked the respondent to indicate that day’s level of pain. We have previously used this outcome measure to validate the Oral Mucositis Assessment Scale (OMAS) and as an outcome measure in a randomised trial (Sung et al, 2007a, 2007b).

NCI-CTC

The NCI-CTC is the standard adverse event reporting system used by the National Cancer Institute and it is widely used for grading oral mucositis (National Cancer Institute, 2003). The NCI-CTC v3.0 mucositis functional/symptomatic scale was used. NCI-CTC grade ranges from 0 to 5 where higher scores correspond to worse mucositis.

OMDQ

The OMDQ was developed through multiple focus groups and one-on-one interviews with cancer patients (Bellm et al, 2002; Stiff et al, 2006). It consists of seven questions that relate to: (1) Amount of mouth and throat pain (OMDQ1), (2) Effect of pain on sleeping (OMDQ2), (3) Effect on swallowing (OMDQ3), (4) Effect on drinking (OMDQ4), (5) Effect on eating (OMDQ5), (6) Effect on talking (OMDQ6) and (7) Amount of diarrhoea (OMDQ7). The OMDQ has been validated in paediatric cancer by parent proxy report and child self-report for all items other than for the diarrhoea item. Thus, OMDQ7 was not included in this study. Each component of the OMDQ was scored separately as an aggregate score has not been validated. For each component, the score ranges from 0 to 4 where higher scores correspond to worse mucositis.

Statistical considerations

All analyses were stratified by respondent type. ChIMES evaluations focused on ChIMES Total Score and ChIMES Percentage Score, but we also illustrated the properties of individual items. To evaluate the test–retest reliability of ChIMES, we calculated the Spearman’s correlation coefficient between Days 13 and 14 for those who reported no change in mucositis between these days and for all respondents. We hypothesised an r 0.7. To evaluate the inter-rater reliability of ChIMES, we calculated the Spearman’s correlation coefficient between parents and children aged 8 to <12 and 12–18 years on Day 14 and anticipated an r 0.5. We evaluated internal consistency by Cronbach’s alpha and anticipated an alpha 0.7 (Streiner and Norman, 1995).

To evaluate convergent validity of ChIMES, we hypothesised that ChIMES scores would be positively correlated with WHO, VAS, NCI-CTC and OMDQ. For this analysis, we described the Spearman’s correlation coefficients using all evaluations but in order to account for the same child providing multiple measures at baseline and on Days 7–17, we obtained the P values using a repeated-measures linear model with Proc Mixed in SAS (Cary, NC, USA). We anticipated a Spearman’s correlation of 0.35 based on our previous studies. To evaluate the responsiveness of ChIMES, we compared the ChIMES scores obtained at baseline with Day-14 evaluations in children who had oral mucositis on Day 14 and in all children. These two scores were compared using the Wilcoxon signed-rank test. An exploratory aim was to compare the Faces Pain Scale-Revised and the ChIMES pain question (ChIMES1) on Day 14 to address the question of whether the smiley faces scale used in ChIMES may be confounding pain affect and intensity. These scores were evaluated using the Spearman’s correlation coefficient.

The sample size was based on evaluating the test–retest reliability of ChIMES. Assuming that the r under the null hypothesis was 0.4 and under the alternate hypothesis was 0.7, an α of 0.05 and a β of 0.20, we planned to recruit at least 90 parent respondents to ensure that we had 45 who reported no change in oral mucositis between Days 13 and 14.

Results

Between 6 July 2010 and 29 April 2013, 222 potentially eligible respondents were evaluated. Figure 1 outlines the flow of participants; 30 refused and 7 were not evaluable, thus leaving 185 respondents in the final analysis. Of these, 98 were parent/guardian proxy respondents for children aged <12 years, 66 were child self-respondents aged 12–18 years and 21 were child self-respondents aged 8 to <12 years. Of the 34 children aged 8 to <12 years, 14 children agreed to self-report mucositis scores alongside their parents, 13 children refused and only their parents participated and 7 children participated alone without their parents. Table 1 illustrates the demographics of the study cohort stratified by respondent type. Approximately 40–45% of respondents had previous experience with mucositis.

Figure 1
figure 1

Flow diagram of participants stratified by respondent type.

Table 1 Demographics of the study cohort stratified by respondent type

Table 2 and Appendix 2 illustrate the evaluation of test-–retest reliability among respondents who reported no change in mucositis between Days 13 and 14 and among all respondents. Reliability was excellent for ChIMES Total Score and ChIMES Percentage Score with r>0.8 for all respondent types. In particular, among parent respondents reporting no change in mucositis, Spearman’s correlation coefficients were r=0.967 and 0.968 for the ChIMES Total and Percentage Scores, respectively. In the evaluation of individual items, the r was>0.7 for all respondents and the majority of items had r>0.9. In the evaluation of internal consistency, Cronbach’s α was 0.95 for parent, 0.93 for child respondents aged 12–18 years and 0.95 for child respondents aged 8 to <12 years.

Table 2 Test–retest reliability of ChIMES total and percentage scores measured on Days 13 and 14a

Table 3 demonstrates the evaluation of convergent construct validity and correlation between ChIMES and other measures of mucositis. All r values were>0.5 across all respondent types. Table 4 highlights the evaluation of responsiveness. Among those who reported mucositis on Day 14, the mean difference in ChIMES Total Scores was approximately 10 and the mean difference in ChIMES Percentage Scores was approximately 50. These differences were significantly different across respondent types. Among all respondents and all evaluations, the median (interquartile ranges (IQRs)) for ChIMES Total Scores for WHO=1 was 4.5 (3, 8); WHO=2 was 10 (6, 15); and WHO=3 or 4 was 20 (16, 22). The corresponding median (IQRs) ChIMES Percentages Scores for WHO=1 was 21.7 (13.0, 37.7); WHO=2 was 43.5 (26.1, 65.2); and WHO=3 or 4 was 87.0 (72.7, 95.7).

Table 3 Construct validation of ChIMES total and percentage scoresa
Table 4 Responsiveness of ChIMES total and percentage scores between baseline and day 14a

For the exploratory evaluation of the correlation between Faces Pain Scale-Revised and ChIMES1, the Spearman’s correlation coefficients were 0.906, 0.972 and 1.000 for parent, child respondents aged 12–18 years and child respondents aged 8 to <12 years, respectively.

Discussion

We have demonstrated that ChIMES, a new paediatric-specific measure of oral mucositis, is reliable, valid and responsive to change in children and adolescents with cancer and undergoing SCT. In order to decide whether to incorporate an instrument as an outcome measure in clinical trials, assessment of all of these properties is important. Our data also suggest that ChIMES may be used confidently for child self-report in those aged 12 years and likely for those as young as 8 years of age.

There are several components of ChIMES that are particularly relevant to children. First, ChIMES focuses on functional elements, as these were considered more clinically important rather than simply the presence of ulcers (Tomlinson et al, 2009b). Second, the assessment of ulcers is limited to a yes/no question rather than a detailed assessment as is conducted for the OMAS. With the OMAS (Sonis et al, 2001), nine sites of the oral cavity are evaluated for erythema and ulceration. Although we found the OMAS to be valid in paediatric cancer patients receiving chemotherapy (Sung et al, 2007b), the correlation coefficients between OMAS with WHO and VAS were much lower (0.56 and 0.37, respectively) compared with ChIMES. We also found OMAS difficult for some children given the time required for oral cavity evaluation. These issues suggest that ChIMES is preferable to OMAS for children, although OMAS is likely to be an excellent outcome measure for adult cancer patients. Third, ChIMES allows ‘I can’t tell’ responses. With WHO, the inability to attribute symptoms to aetiology and the inability to visualise the oral cavity result in a missing score. Given the importance of avoiding missing scores in clinical trials, these issues suggest that ChIMES may be a better measure for paediatric clinical trials compared with WHO (Little et al, 2012).

We calculated and evaluated two ChIMES outcomes, the ChIMES Total Score and the ChIMES Percentage Score. Our results suggest that either may be used in clinical trials. However, the incorporation of ‘I can’t tell’ responses may make the ChIMES Percentage Score preferable.

We had a limited number of children 8 to <12 years of age who self-reported mucositis scores, and there are several aspects of our trial that merit specific consideration. First, some children reported mucositis alongside their parents; we do not know whether the simultaneous completion of diaries by parents and children may have biased the child responses favourably. Second, we invited 34 children aged 8 to <12 years to our study and only 21 agreed to self-report mucositis scores. Thus, our results may not be generalisable to all 8- to <12-year olds. Third, the evaluation of responsiveness focused on those with mucositis on Day 14, and only six respondents met this criterion within the 8- to <12-year self-report respondent group. Consequently, the failure to show a statistically significant difference for the analysis is more likely related to inadequate power rather than lack of responsiveness in this age range. Further, the magnitude of the differences between baseline and Day 14 is similar to that seen in parent and child respondents aged 12–18 years.

The major strengths of our study include the multi-centre design, incorporation of multiple measures of mucositis and evaluation of responsiveness in addition to reliability and validity. However, our results must be interpreted in light of its limitations. As previously noted, the 8- to <12-year-old group is likely to be self-selected as a more compliant and motivated group. Second, we only included English speakers in this study. Another limitation is that we have little insight into whether parents were able to accurately assess subjective symptoms in the youngest children using the six-point scale or whether a dichotomous or simpler scale would have been preferable. Finally, another limitation is that we do not have clinically defined thresholds for ChIMES scores that delineate the presence or absence of mucositis or degree of severity (such as mild, moderate and severe).

Future work should focus on determining whether children aged <8 years can self-report mucositis and parent/guardian evaluation of mucositis in the youngest children. Future work should also focus on identifying clinically meaningful thresholds for ChIMES scores to categorise the presence of mucositis and degree of severity. Additionally, ChIMES should be translated into other languages and evaluated in other cultural contexts. In conclusion, ChIMES is a paediatric-specific measure of mucositis with favourable psychometric properties. It exhibits test–retest reliability, inter-rater reliability, internal consistency, construct validity and responsiveness. ChIMES should be incorporated into mucositis prevention and treatment clinical trials in paediatric cancer and SCT.