FormalPara Key Points

A pediatric patient-reported outcome has been developed to assess the symptoms of the common cold in children aged 6–11 years.

Combined concept elicitation and cognitive debriefing interviews employing creative methods were conducted with 39 children and ten of their parents.

The findings support the content validity of the final patient-reported outcome.

1 Introduction

The common cold, a viral infection of the upper respiratory tract, is the most frequently occurring human illness for which approximately 25 million individuals seek medical attention per year in the USA [1,2,3]. Children experience three to ten colds annually, accounting for 22 million missed days of school [4, 5]. The most frequently experienced symptoms are nasal, including rhinorrhea (runny nose), nasal congestion (stuffy nose), and sneezing. Sore throat can be present from the first day of symptoms, and a cough may develop on or around the fourth or fifth day. Low-grade fever, headache, and malaise can also be present [6]. Many children and their parents/caregivers lose sleep because the cough associated with the cold keeps them awake at night [3].

To conduct trials evaluating the efficacy of cold medications in children, assessments of common cold symptoms that can form trial endpoints are required and they should be appropriately developed and validated. Assessing cold symptoms in young children has the challenge that not all children will experience the same symptoms, which are generally mild and transient. Moreover, several symptoms (e.g., headache and chest congestion) are not directly observable, while others are highly variable (e.g., cough and sneezing). Consequently, assessment of symptom frequency or severity is best based on direct report from children through use of pediatric patient-reported outcome (PRO) instruments. Pediatric PRO instruments are available to assess conditions with symptoms that overlap with the common cold. These include the Strep PRO, which assesses sore throat [7] and the Pediatric Rhinoconjunctivitis Quality of Life Questionnaire [8], which assesses runny nose and sneezing. However, none providing a comprehensive assessment of cold symptoms are found in the literature.

Well-established methods for the development and validation of adult PROs are detailed in the PRO guidance for industry provided by the US Food and Drug Administration [9] and the reflection paper on health-reported quality of life by the European Medicines Agency [10]. Developing pediatric-focused assessments adds complications beyond those encountered in PRO development for adults [11, 12]. Foremost is the need to ensure, through rigorous qualitative research, that the concepts being assessed and the instrument instructions, items, response options, and recall periods are relevant and appropriate for the disease population of interest (context of use). Given the developmental changes in children’s physical, cognitive, and functional abilities, it is also important to conduct qualitative research within narrow age ranges to ensure that the PRO is appropriate for all intended ages. In the present research, the inherent difficulties in pediatric PRO development work have been addressed using concept elicitation and cognitive debriefing activities tailored to overcome some of those challenges.

The primary objective of this qualitative research was to conduct content validity testing of draft items that may comprise a pediatric PRO instrument assessing cold symptoms in children aged 6–11 years. The ultimate intention is that selected items from the pool of items could be included as endpoints in clinical trials to evaluate the efficacy of cough and cold medicines in children aged 6–11 years.

2 Methods

2.1 Sample and Recruitment

This was a qualitative interview study, conducted with 39 children aged 6–11 years, who were either currently experiencing a cold (n = 31) or had experienced a cold in the past 14 days (n = 8) but were otherwise healthy. The latter group was included because it is expected that by the end of the study period for a cold treatment trial, which could potentially use the PRO items, at least some of the participants will no longer be experiencing symptoms; therefore, it was considered important to test that children not experiencing symptoms were still able to understand and respond to the questions. Ten parents of participants aged 6–8 years were also interviewed (separately from their child) regarding how they thought their children would understand the items. Participants were recruited by a specialist patient recruitment agency from two locations in the USA: Los Angeles, California and Chicago, Illinois. The recruitment agency contacted parents in their database and informed them that if their child was experiencing a common cold (or had done so in the last few days) and would be willing and able to participate in an interview, they should respond. The recruitment agency then completed a screener with each parent/guardian to confirm eligibility. Written informed consent was then obtained from the parent/guardian and assent from the child prior to any other study activities.

The children had to have experienced at least one of the six symptoms of interest (headache, sinus pain/pressure, body aches/muscle aches, sore throat, cough, or chest congestion), in the past 14 days, as reported by their parent, to be considered for inclusion. The children were also required to have experienced at least one other symptom of the common cold (headache, sinus pain/pressure, body aches/muscle aches, sore throat, cough, chest congestion, nasal congestion [stuffy nose], runny nose, sneezing, or low fever). Diagnosis of cold symptoms was not confirmed by a clinician because parents do not usually take a child with the common cold to the doctor.

Children were excluded from the study if they had experienced cold symptoms continuously for more than 14 days; were experiencing severe cold symptoms such as high fever; were taking antibiotics; or had been diagnosed with sinusitis, otitis media, tonsillitis, strep throat, laryngitis, bronchitis, pertussis, or pneumonia. These children could be experiencing symptoms due to other conditions, thus their symptom experience could be different from that of the common cold. They were also excluded if they had mental, psychiatric, or cognitive conditions that might affect their ability to participate in the interview. Recruitment quotas ensured that the sample included children with a range of demographic characteristics and different combinations of the symptoms of interest, thus providing the ability to evaluate whether there were any age-related differences in comprehension or relevance of symptom concepts.

2.2 Interview Procedure

The interviews were split over two rounds to allow findings from the first round to be used to revise the original items for testing in the second round. Both children and parents were interviewed for approximately 1 h, but with breaks, particularly for the children. Parents were interviewed separately from their children. The interviews were conducted in either a research facility or in a room in a public building such as a library that was located conveniently for the participants. Each interview included concept elicitation activities (open-ended, exploratory questioning) and cognitive debriefing questions (testing of comprehension and relevance of the questions) [2, 13,14,15]. Experienced pediatric interviewers trained in qualitative interviewing techniques used a semi-structured interview guide to first elicit how children spontaneously talk about their cold symptoms to inform the development of the PRO items. Open-ended questions were designed to encourage spontaneous quotes regarding the children’s experience of cold symptoms using their own terminology [2, 14]. More direct questions were then asked, to ensure that feedback on all symptoms of interest was obtained.

Four creative tasks were employed during concept elicitation to encourage children to talk spontaneously about cold symptoms: free drawing, circling parts of the body on a diagram, card sorting, and a child–parent completion task. The free-drawing task asked children to draw what it feels like to have a cold and describe their drawing aloud. This method often helps nervous children to relax and draws out comments that might not be elicited from formal questioning [11, 16]. The ‘circle parts of your body’ task involved children circling areas of an outline of a human child’s upper body that felt different to them while experiencing the cold and describing how each felt.

The children were asked to sort a set of cards into “the order you think they should go in” that had the following response options associated with symptom severity printed on them: “not bad at all”, “a tiny bit bad”, “a little bad”, “bad”, “pretty bad”, “very bad”, and “really bad”. The interviewer presented the cards in a random order to establish whether children understood the severity continuum. The goal was to select five child-appropriate response options that align with the validated response scale in an adult PRO instrument for upper respiratory cold symptoms: none, very mild, mild, moderate, and severe [16]. Because the common cold is a generally mild condition, efforts were made to include more gradation toward the milder end of the response continuum (i.e., include an equivalent to very mild).

Following the concept elicitation questioning, children were asked to complete the draft PRO items using a think-aloud process during which they speak aloud their thoughts as they answer each question [17, 18]. This was followed by detailed cognitive debriefing questions to assess the content validity and clarity of each question, instruction, response scale, and recall period [2, 14, 19]. Ten parents of children aged 6–8 years were also asked whether they felt their children would be able to understand the instructions, questions, response options, and recall periods.

2.3 Generation and Refinement of Instrument

Draft instructions, items, response scales, and recall periods were based on the pediatric rhinitis literature, previously used measures of cold symptoms in adults, other pediatric PRO measures, and consideration of best practices in developing PRO items suitable for completion by children aged 6–11 years [9,10,11, 14, 17]. Efforts were made to keep items as short and simple as possible and avoid the use of double enquiry or overly clinical terminology in keeping with pediatric PRO best practice [11, 12, 15].

The children’s responses to the items during debriefing were recorded and summarized descriptively to provide insight on which items used the full response scale and may be more discriminating. Items with a strong floor (i.e., a high proportion of children choosing the lowest response) or ceiling (i.e., a high proportion of children choosing the highest response) effect and those with any one response option being overly favored were considered later with all qualitative evidence and their clinical relevance when considering whether any items might be candidates for deletion.

2.4 Qualitative Analysis

Interviews were audio recorded and transcribed verbatim. Qualitative analysis was performed by authors CM and KB with the oversight and guidance of RA. Qualitative analysis of verbatim transcripts involved sorting quotes by conceptual domain using thematic analysis and content analysis using Atlas.ti [20]. Thematic analysis involved reviewing transcripts and highlighting and coding quotes related to specific cold symptoms and concepts. Quotes related to a specific symptom/concept were then grouped from across transcripts during analysis and the findings summarized. This was the primary method of analysis. Content analysis was used where counts summarized findings. For example, the count of how many children depicted certain symptoms in the drawing task or the counts of how many children circled different parts of the body are examples of content analysis. Although assessing concept frequency is not a primary purpose of qualitative research, the number of children who mentioned a given concept was recorded to provide an indication of the relative importance of each concept. To ensure that the concepts elicited were fully explored, conceptual saturation was assessed: defined as the point at which no new relevant or important information emerges with the collection of more data [21]. Saturation was evaluated by splitting the child transcripts into four groups, each comprising ten or nine transcripts. The groups were based on the chronological order the interviews were conducted in, thus effectively random. The concepts reported in the first group of transcripts were then compared to those reported in the second group of transcripts. The concepts that emerged from the first two were then compared with the third and the concepts reported in the first three were compared with the final group. The point at which no new symptom concepts emerged was the point at which saturation was deemed to have been achieved.

3 Results

3.1 Demographics

3.1.1 Child Sociodemographic Characteristics

Thirty-nine children aged 6–11 years were included in the final study sample, with 19 participating in the first round of interviews and 20 in the second round. The interviews were conducted between February and June 2011 at a research facility or in quiet area of a public building (e.g., library). An additional 6-year-old child was unable to concentrate enough to participate. Sociodemographic characteristics for the total sample and sub-samples are presented in Table 1. Mean age was 8.2 years (range from 6 to 11 years), 19 (49%) were female. Just over half the sample were Caucasian (53.9%; n = 21), but other races and ethnicities were well represented.

Table 1 Child sociodemographic characteristics and cold symptoms assessed by parent at screening
Full size table

3.1.2 Child Cold Characteristics

The cold symptoms reported by each child’s parent during screening are shown in Table 1. Nearly all children had a cough as part of their cold (89.7%; n = 35). Four symptoms of interest (chest congestion, headache, sore throat, and body/muscle aches) were reported for approximately 50% of the sample. Only sinus pain/pressure was reported by a lower proportion of the sample (28.2%, n = 11). The most commonly reported symptoms after coughing were nasal congestion (61.5%; n = 24) and sneezing (61.5%; n = 24).

3.1.3 Sociodemographic Characteristics of Parents who were Interviewed

Ten parents of children with current colds were interviewed; mean age was 36.7 years (range from 31 to 49 years), and nine (90%) were female. Five parents (50%) were White/Caucasian, four (40%) were Hispanic/Latino, and one was Asian. Two parents (20%) had completed a college or university degree, while seven (70%) had completed some college.

3.1.4 Concept Elicitation Findings: Descriptions of Cold Symptoms by Children

The children were asked to describe their cold symptoms through open-ended questions such as “Tell me about your cold” and “What types of things happen when you have a cold?”. Table 2 summarizes the findings and descriptions used for each symptom across the sample, which were generally consistent. Conceptual saturation was achieved—all symptoms were reported in the first saturation checking group. Cough was the most commonly reported cold symptom, reported spontaneously by the majority (31/39) and by a further 4/39 only when probed directly. Chest symptoms, reported by 32/39 children (12 spontaneously and 20 only when probed directly), were most commonly described in terms of the chest feeling “full of mucus” (7/39), “hurting” (6/39), or a “squeezing” (3/39)/”tight” feeling (3/39). Twenty-eight of the 39 children spontaneously reported experiencing headaches, most often referring to their head “hurting”, despite only 18/39 parents reporting this symptom during screening. Thirty-six of the 39 children reported experiencing a sore throat (29 spontaneously, seven only when probed), most describing their throat “hurting”, while 25/39 children talked about feeling aches or weakness in their body (13 spontaneously, 12 only when probed). Four parents said their children (aged 6–8 years) would simply say that their body hurts.

Table 2 Symptom concepts reported, number of reports, and example quotes
Full size table

Although 11/39 parents reported that their children had sinus pain/pressure during screening, only 8/39 children talked about sinus pain/pressure spontaneously, with a further five reporting the symptom when asked directly but did not elaborate, making it unclear whether they truly understood the concept. Children described their sinus pain with a wider variety of descriptors than for other symptoms, referring to their face feeling “pushed”, “stretched”, or “pluggish” or describing a “squeezing” sensation around their nose and eyes or a pain that was “burning” or “hot”. Four of the ten parents of children aged 6–8 years did not believe their child would be capable of explaining sinus pain.

3.1.5 Drawing Task

Of the 32 children that completed the drawing task, 13 spontaneously drew at least one symptom of interest. Twelve children (aged 6–11 years) drew themselves coughing, an example of which is shown in Fig. 1. Two children drew what happens during their cold when they get headaches: a 9-year-old girl showed herself with a “cold rag” on her head and an 11-year-old girl drew herself saying “leave me alone” because she did not want to play with other children when she had a headache. One 11-year-old boy drew himself with chest congestion, clearly showing mucus built up in his chest. One 9-year-old boy highlighted his throat in the picture, explaining: “Here’s the throat where the mucus is.”

Fig. 1
figure 1

Child’s drawing of coughing (11-year-old boy)

Full size image

Q: “Do you want to just explain to me what you did?” “Um, (inaudible) like, it goes through your throat and then up the mouth.” “OK and that’s the mucus right there?” “Yeah. And then the tissue on the nose, like blowing. That’s all I really think of, just like blowing your nose and coughing up stuff.”

3.1.6 Circling Parts of the Body Task

Eighteen children (aged 6–11 years) circled the throat on the diagram, many saying it hurt during their cold. Thirteen children marked the head, explaining that they had experienced headaches. Nine children marked their chest on the diagram, and seven of nine children talked about coughing when asked why they identified the chest area. One 9-year-old child said he had marked his chest because he felt “mucus” in there. Six children marked the arms, reporting “weakness” and “achy” feelings. Two children marked the areas around their nose and eyes (two girls aged 6 and 11 years), with the 11-year-old child saying it felt “like a headache but wasn’t”, which suggests she may have been experiencing sinus pain but had difficulty articulating this.

3.2 Development of Draft Patient-Reported Outcome Items, Response Scales, Recall Periods, and Illustrations of Symptoms.

3.2.1 Item Content

Nineteen PRO items were tested in the first round of interviews with alternative wording used on showcards. Subsequent changes were tested in the second round. A tracking matrix was developed to document the changes made to the item content between interview rounds and present the rationale for the final items (Table 3). The items assessed the key cold symptom constellations reported and targeted for study including cough, chest congestion (chest congestion, chest tightness, chest pain, difficulty breathing), headache, sore throat, body/muscle aches, and sinus pain/pressure.

Table 3 Item tracking
Full size table

3.2.2 Response Scales

Two pictorial scales were tested along with the verbal descriptors for each response option: circles of increasing size to indicate severity, and boxes that became gradually filled to indicate frequency or “how much” (Fig. 2). Similar diagrams have been found to aid understanding of the gradation in a response scale in pediatric PRO assessments [17].

Fig. 2
figure 2

Example of patient-reported outcome items tested during cognitive debriefing activities

Full size image

3.2.3 Recall Period

Shorter recall periods (e.g., “since you woke up this morning”) are preferable for pediatric PROs as children can have difficulty recalling accurately over longer timeframes [11, 12]. Tying recall to a concrete event can also help younger children remember [12]. Clinical trials involving the assessment of the common cold may require multiple PRO assessments throughout a given day, thus requiring short well-defined recall periods. Several recall periods were tested for understanding and appropriateness for different symptoms, all for periods of less than 24 h. For immediate assessments of symptom severity, “right now” was tested with some symptoms.

3.2.4 Illustrations of Symptoms

An image of a child depicting the symptom accompanied each item to help children correctly think of the appropriate symptom. The child in the image was intended to be as neutral as possible in terms of sex, race, ethnicity, and emotion being portrayed, to reduce the risk that any child would not identify with the image or the response would be influenced by emotion [17].

3.3 Cognitive Debriefing Results and Refinement of the Draft Items

3.3.1 Cognitive Debriefing Results for the Item Content

Across the two rounds of cognitive debriefing, the items were largely well understood by children (Table 4 shows chest symptoms as an example). Items asking about cough, sore throat, and headache were the best understood and were most consistently interpreted without difficulty; however, some items, especially sinus pain, were less well understood or inconsistently interpreted and thus were deleted or reworded between the rounds of interviews. For example, based on evidence from the first round, one item on body aches was reworded from asking “how much did your body ache” to “how much does your cold make your arms and legs ache”. Reference to the child’s cold was added to help children separate this experience from other types of body aches (e.g., due to physical activity), while “body ache” was changed to “arms and legs ache” because most children identified aching in the arms and legs as due to their cold. The recall period of some items was adjusted to only retain the recall periods that were best understood. At least one item was retained for every symptom concept identified as relevant from the concept elicitation results.

Table 4 Cognitive debriefing findings and subsequent changes to chest symptom items
Full size table

3.3.2 Cognitive Debriefing of the Illustrations Depicting Symptoms

Across both rounds of interviews, 8/12 children said the illustrations depicting a child experiencing each individual symptom helped them to answer the questions. Five parents also commented on the usefulness of the pictures. However, three children misinterpreted the illustrations. One child thought that lines coming from the child’s mouth to depict coughing showed the child speaking, one child thought that areas highlighted in red depicted blood (the intent was to highlight the areas of the body where symptoms occur), and one child thought that the images looked like an alien rather than a child. Based on these findings, the images were revised by a professional artist to improve interpretation.

3.3.3 Card Sorting Task

Thirty-six of the 39 children sorted the cards containing response phrases into an order by which they were satisfied. Three 6- and 7-year-old children did not complete the task; two because of reading ability and one because of time constraints. Six children (aged 6–8 years) placed the cards in an order that represented how their cold had progressed chronologically, rather than in terms of severity. 33 of the remaining 30 children sorted the cards into an order of increasing severity. While children understood the continuum, the severity level of “pretty bad” and “really bad” was unclear, thus these response terms were not selected for use going forward.

3.3.4 Parent–Child Completion Task

When asked to complete the questions with their parent, 4/39 children (aged 8–11 years) answered the questions entirely by themselves without any input from their parents. The remaining 35/39 children were read the questions by their parents. Only one child, aged 11 years, asked his parent for clarification on a question during the completion task. During the task, 7/39 parents questioned their child’s answer on at least one item; however, only one 6-year-old girl changed her response. The other six children kept their original answer despite being challenged. Example quotes from this exercise are provided in Table 5.

In total, 25/34 parents believed their children would be able to answer the questions independently, but nine parents of children aged 6–8 years said it would be better if they either explained the items or reminded the children of the recall period. Six parents pointed out an inconsistency with what their children were reporting and what they had observed during the day, while 15 parents reminded their children of a specific time they may have experienced a symptom during the day. In addition, ten parents rephrased items to ensure their children understood, but none of this paraphrasing changed the intended meaning of the question. Most children (24/39) stated that they preferred their parents being there when answering the questionnaire (Table 5).

Table 5 Example quotes from the parent-child completion task relevant to the children’s ability to answer the questionnaire
Full size table

3.3.5 Item Response Distributions

Distributions of responses were reviewed for each item to provide additional insight into whether the items would discriminate among children differing in symptom severity. Cough and headache showed a good spread of responses with less spread for chest and sinus symptoms (see Table 6 for example response distributions for a few items).

Table 6 Item response distributions for example questions
Full size table

4 Discussion

Developing pediatric PROs and conducting concept elicitation and cognitive debriefing interviews with children, particularly those as young as 6 years of age, is widely recognized to bring additional challenges not encountered in adult PRO development [11, 12, 22]. To minimize such difficulties and overcome the challenges, the interviewers took time to build rapport and used creative tasks to engage the children and help them describe their symptoms. Because interviews were conducted in research facilities and public buildings, these steps helped the children feel comfortable in the less familiar and slightly more formal research setting. If the interviews had been conducted in the children’s homes, the children may have felt more at ease, but they also may have been more likely to become distracted and lose focus. The free drawing and circling of upper body parts with symptoms worked well in identifying symptoms and confirming comprehension, and provided support for the concepts elicited during open-ended questioning. The card-sorting task provided strong evidence that the wording chosen for the response scales was well understood by children of all ages and that they understood the severity continuum. The benefit of using these creative tasks to help make the children comfortable and to aid in eliciting meaningful information is consistent with the success other researchers have had in using creative techniques in concept elicitation [16, 23]. Similarly, the illustrations in the items helped children to focus on the correct symptom. As others have reported previously, the response scales of graduated size or fill encouraged distinction between different response levels across the sample [17].

Children aged 6–8 years had some difficulty with the more complex concepts of sinus pain/pressure and chest congestion. Very few children aged 9 years and older had any problems with comprehension. A few younger children also had difficulty reading some words—this was addressed through revisions following the first round of cognitive debriefing. Other researchers have noted that limitations in reading ability and attention can result in some children in the 6- to -8-year-old age range having difficulty completing PROs, which can impact the validity and reliability of measurement [11, 12, 24,25,26]. However, others have provided evidence that valid and reliable assessment in this age range is feasible [11, 12, 22]. The International Society for Pharmacoeconomics and Outcomes Research Pediatric Task Force and others have highlighted that children’s ability to provide valid and reliable responses depends first on the concept being assessed, but also the language and reading ability of the individual child [11]. Variation by concept was observed in our study, with the complex concept of sinus pain less well understood than simpler and more commonly experienced symptoms such as sore throat and stuffy nose. In the validation of the PRO questions developed here (to be published in a subsequent article), we took account of individual differences in the abilities of children of the same age by requiring a certain reading level as part of the inclusion criteria of the study. Future research might also consider showing the questionnaire items to kindergarten and first-grade teachers for their professional opinion on whether typical children in their classes would be able to independently complete the questions. Parent administration of a child-completed PRO is one solution to overcome barriers to collecting self-report data from younger children with reading/attention limitations. This study provides evidence that such an approach should not threaten the validity of the endpoints as most children kept their answers even when challenged by parents. However, providing parents with clear instructions and training that they should not paraphrase or influence their child’s responses is a further step that can be taken to ensure the report remains a self-report from the child.

One limitation of the study was the reliance on parent/caregiver reports to confirm participants met the inclusion criteria, which could have introduced bias. Nevertheless, the interview findings did consistently provide corroborating evidence that the children reported cold symptoms that their parents also reported. Another limitation of the study is that diagnosis of the common cold was not verified by a trained physician. As parents typically do not take a child to the doctor’s office for a common cold, this approach is acceptable and reflects the real-world management of the common cold. Another limitation is that the study was only conducted in the USA. Further research in other countries would be valuable to confirm that the findings reflect the way other cultures describe cold symptoms.

Children were able to describe all symptoms of the common cold, using consistent language to do so, irrespective of age. Feedback from children and their parents suggests that the majority found most PRO items were easy to understand, interpreted consistently, and as intended. The items have strong face and content validity for severity level, symptom description, and various recall periods. Responses from children with current colds were spread across the scale demonstrating that children used the scale appropriately; however, most selected options in the middle and lower end of the 5-point scales. This is expected considering that the common cold is a relatively mild, self-limiting condition. Evidence of items having strong floor or ceiling effects was considered along with all qualitative findings when making decisions around deletion of items. It is acknowledged as a limitation that the presence of floor or ceiling effects could, to some degree, have been influenced by the inclusion criteria of the study. However, this evidence was primarily considered when trying to choose between items measuring the same symptom rather than in consideration of deleting symptoms, i.e., to identify which wording resonated best with children for characterizing a given symptom. Moreover, as this was a relatively small qualitative sample, caution was employed in giving too much weight to evidence of floor or ceiling effects. Item response distributions were examined further in later psychometric evaluation in a larger sample.

5 Conclusions

This in-depth qualitative study has supported the identification of relevant symptom concepts and the development and refinement of PRO items to assess those concepts. The findings support the content validity of the items and suggest that they can be used with confidence in children aged 9 years and older. For children aged 6–8 years, it is recommended the items are administered with initial adult supervision to explain the more difficult concepts or through parent/interviewer administration. Subsequent to the conduct of this research, a psychometric validation study was conducted to evaluate the reliability and validity of the items in a larger sample of children with colds. Those findings will be reported elsewhere.