Using Mobile-Technology-Based Ecological Momentary Assessment (EMA) Methods With Youth: A Systematic Review and Recommendations

Abstract

Ecological momentary assessment (EMA) methods are increasingly used in social and health sciences, but the feasibility and best practices for using EMA with youth are not yet clear. We conducted a systematic review of studies that used self-report EMA methods with youth; the goal was to identify common approaches and challenges to implementation and develop recommendations for future research. We examined 54 peer-reviewed papers that reported on 24 unique studies. Papers were evaluated using a standardized, three-dimensional coding scheme focused on the following: (1) sample characteristics; (2) EMA data collection methods (sampling duration, frequency, hardware/software); (3) study implementation methods (technical/logistical challenges, training participants, compliance). Overall, the research suggests EMA can be successfully implemented with youth (age ∼ ≥7) from diverse backgrounds, but protocol adaptations may be necessary for younger children. Study design and implementation challenges and recommendations for research on youth are provided.

Naturalistic, ambulatory, real-time assessment methods have become increasingly popular research tools in the social and health sciences, including with child and adolescent populations. In recent years, technological advances (e.g., mobile smartphones) have also dramatically improved access to, and sophistication of, these approaches. These methods are identified by various names, including ecological momentary assessment (EMA), the experience sampling method (ESM), daily diary, and ambulatory assessment. Although there are distinctions between some of these methods, they share the common characteristics of involving the repeated, and often intensive, collection of information from individuals as they go about their daily lives. In this review we will use the term EMA to encompass these methods, as it is widely used in the social and health sciences. There are many methodological advantages to using EMA, which have been discussed at length elsewhere (Smyth & Heron, 2014; Smyth & Stone, 2003; Stone, Broderick, Shiffman, & Schwartz, 2004). Briefly, EMA provides reduced retrospective recall and associated biases when collecting self-report information (e.g., memory biases, self-concept biases), increased ecological validity, and the opportunity to examine within-person processes and temporal dynamics that occur over relatively short time periods (minutes, hours, days).

Advances in mobile technologies (e.g., mobile phones, smartphones, wearable devices) have provided new ways of capturing information from people as they go about their everyday lives. EMA approaches do not require the use of technology; in fact, in early EMA studies, participants completed paper-and-pencil assessments multiple times per day in provided paper diaries (Csikszentmihalyi & Larson, 1987), and this approach is still sometimes used, including in pediatric research (Valrie, Gil, Redding-Lallinger, & Daeschner, 2008). Mobile technology can, however, facilitate study implementation in several ways. First, assessments can be time- and date-stamped to better track compliance. This is important because studies of both adults and children suggest that participants are significantly more compliant—that is, they complete assessments at the designated times—when using electronic assessment devices versus paper assessments (Palermo, Valenzuela, & Stork, 2004; Stone etal., 2003). Second, technology allows for new assessment approaches, including presenting information in alternate formats (e.g., picture, video), multiple response option styles, and complex question branching; these advances can greatly enhance the amount and quality of EMA data. Third, as mobile technologies become more widely available, they will likely be used more often in future EMA research. Given these benefits and the increasing use of mobile technology in EMA research, in this review, we focus on studies that use newer mobile devices, such as mobile phones and smartphones.

EMA techniques were initially used to study behaviors and experiences in adult research participants. However, EMA methods also offer opportunities for studying youth and can provide unique advantages over more traditional research designs and data collection approaches. For example, children may face particular struggles in accurately reporting on experiences and behaviors when using retrospective self-report measures that require them to estimate the frequency or extent of their experiences over extended, and often unspecified, periods (Kamphaus & Frick, 2005). Youth may be able to provide more valid and reliable reports over the much shorter EMA recall periods (minutes, hours, days). In particular, and most relevant to pediatric psychology, research participants may be better able to report on experiences of pain, treatment adherence (e.g., medication, dietary recommendations), sleep, and disease symptoms across shorter recall periods. EMA approaches also provide opportunities to study children and adolescents during the course of their everyday lives. Collecting data from youth in the settings and situations they encounter each day—at school, with their families, in hospitals or clinics, and interacting with peers—allows researchers to study processes and experiences that vary across place and time, thereby enhancing ecological validity. Assessing youth in their everyday contexts addresses the problem that many of the contexts and experiences of interest to clinical and pediatric researchers studying children and adolescents are not easy or even possible to create using other research methods, such as laboratory-based studies (Smyth & Heron, 2014). Given these advantages, EMA tools are increasingly being used in studies of children and adolescents. However, difficulties can arise when developing protocols that are acceptable and feasible for youth participants and their caregivers. As a result, questions remain regarding the optimal ways to implement EMA with youth.

The goal of this review was to synthesize methodological information across studies that used self-report EMA with youth to identify common design considerations, implementation approaches, and challenges encountered, as well as to provide recommendations for future research using EMA approaches with children and adolescents. The studies we identified varied in their substantive foci and as such, we organized our review around three methodological topics: (1) sample characteristics, including youth age and other demographic characteristics; (2) EMA data collection methods, including procedures related to sampling duration and frequency and types of mobile devices used; (3) study implementation issues, including technical and logistical challenges, training of participants in EMA methods, and participant compliance. We conclude by identifying opportunities for future use of EMA, especially in pediatric psychology, and providing recommendations regarding best practices for researchers when designing, implementing, and reporting on EMA studies with children and adolescents.

Method

Studies published in English-language, peer-reviewed journals through May 2016 that used EMA methods were identified via searches in Web of Science, PsycINFO, and PubMed. Combinations of the following search terms (and variations) were used: EMA, ESM, ambulatory assessment, child, youth, and adolescent. Reference lists were reviewed to identify additional articles for inclusion. Inclusion criteria were: (1) published in an English-language peer-reviewed journal, (2) participants ≤18 years, (3) participants completed self-report EMA, which was operationalized as involving at least two daily assessments in natural settings, (4) EMA data were collected using mobile phone or other potentially Internet-connected mobile devices (smartphone, iPod Touch, etc.), (5) results of EMA were reported (i.e., not only methodological paper).

As most EMA studies are observational in nature, we developed a coding scheme for reviewing studies based on the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement (http://www.strobe-statement.org/), which provides a checklist of items that should be included in reports of observational studies. The first and last authors added EMA-specific items (e.g., frequency and type of EMA, technology used) based on their EMA expertise. The coding scheme was reviewed by all authors and included information about: sample characteristics (sample size, age, gender, race, socioeconomic status [SES], health status), data collection methods (sampling duration, frequency, hardware, software), and study implementation issues (technical issues, logistical challenges, adaptations for youth, training, compliance). Research assistants took part in a 2-hr training session and coded three practice articles. Articles were reviewed by at least two coders and the first and second authors reviewed all articles. The average agreement between coders across all categories was 96% (range across coding categories: 75–100%) and discrepancies were resolved by consensus with the first and second authors.

Results

Literature Search

Figure 1 shows the flow diagram of articles selected using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Guidelines. We identified 54 articles meeting inclusion criteria. There were 14 instances where more than one article was published from the same data set. In these cases, all articles were coded and information extracted was presented as a single study; the 54 eligible articles reflect 24 studies considered in this review. Table I provides sample and methodological characteristics of all studies reviewed.

Sample Characteristics

Study sample sizes varied from 6 to 303 participants, with a mean of 77 participants; most studies (k = 16, 67%) had ≤100 participants. Participant age ranged from 7 to 18 years, with a mean age across all studies of 13.5 years (range of average ages across all studies: 9.9–16.2 years). Most studies (k = 20, 83%) included a mix of genders, two studies included only females, one study included only males, and one study did not report gender. The race and ethnicity of participants varied widely; 38% of studies (k = 9) included mostly Caucasian youth, 29% (k = 7) included most or all minorities, and 33% (k = 8) did not report race/ethnicity. Child and family SES varied, but was inconsistently indexed across studies (e.g., by income, parent education), and sometimes not reported; available SES information is presented in Table I. Nearly three-quarters of studies (k = 17, 71%) selected participants based on physical or mental health status: nine studies (38%) enrolled participants with mental health concerns (e.g., depression, anxiety, attention deficit hyperactivity disorder, autism), four studies (17%) had participants with a physical health condition (e.g., diabetes, asthma), and four studies (17%) enrolled based on health behavior engagement (e.g., smoking, drinking, binge eating). Most studies were conducted in the United States (k = 17, 71%), four in Australia (17%), two in Germany (13%), and one in the Netherlands (4%).

EMA Data Collection Methods

EMA Sampling Duration

The total number of days of EMA data collection for 96% (k = 23) of the studies ranged from 4 to 31 days; one study had as many as 120 days of EMA. This study, however, involved data collection with adolescents in an outpatient substance use treatment program and required completing EMA for 1 month followed by a 1-month break with no EMA, four times across the total data collection period (for 120 days of total EMA data collection). Some variant of this type of study design—often referred to as a measurement burst design in which participants complete several days or weeks of EMA, followed by a break of weeks or months, and then complete additional days/weeks of EMA (Sliwinski, 2008)—was used in 10 studies (42%). In any single “burst” or “wave” of EMA data collection, two of the most common assessment durations were 1 week of EMA (k = 7, 29%) and 4 or 5 days of EMA (k = 7, 29%), typically including 2–3 weekdays (Thursday, Friday, Monday) and 2 weekend days (Saturday, Sunday). Most of the seven studies (n = 6, 86%) that assessed participants for 4 or 5 days used a measurement burst design and repeated the EMA weeks or months later for 8–25 days of EMA over the duration of the study.

EMA Sampling Frequency

Nearly all studies (k = 23, 96%) signaled participants to complete assessments throughout the course of the day at either random times unknown to the participant (k = 20, 83%) or fixed times (k = 4, 17%) that participants knew of in advance. Across all studies, participants were prompted to complete surveys between two and nine times per day (M = 4.4 times/day). When considering both the frequency and duration of EMA, in studies that required participants to complete EMA for ≤1 week (in a given wave), the participants were prompted to complete an average of 5.00 EMA/day, whereas studies that lasted >1 week averaged 3.25 EMA/day. This illustrates that, when collecting data over longer time periods, fewer EMA per day are generally used (presumably to reduce participant burden). In addition to signaling participants to complete assessments, in 25% of the studies (k = 6), participants were asked to self-initiate assessments in response to specific behaviors or experiences, such as after eating or following an interpersonal interaction (i.e., event-contingent sampling). In all but one study using event-based sampling, the researchers also signaled participants to complete assessments during other (nonevent) times.

EMA Hardware and Software

The two most commonly used mobile devices were smartphones (k = 11, 46%) and mobile phones (k = 11, 46%), with two studies using an iPod Touch to administer the EMA. In all but two studies (92%), participants were provided with the mobile device to use for the duration of the study; of those studies that did not provide devices, one study required participants to have a mobile phone and one study provided a phone only if the participant did not own one (17% of the sample). EMA surveys were delivered to mobile devices using a survey application (or “app”) in 12 of the studies (50%). The apps were developed in a range of formats, including custom software and commercially available options (see Technology column of Table I for hardware and software information). Of studies not using an app, seven (29%) used voice phone calls made by either the participant or research staff, two studies (8%) used an interactive voice response (IVR) system on mobile phones, two (8%) used text message EMA questions, and one study (4%) sent a text message with a Web site link that directed participants to a Web site survey optimized for mobile phones. In addition to collecting self-report EMA data, four studies (17%) also asked participants to wear an accelerometer to monitor activity levels and/or sleep, one study (4%) used a sensor on participants’ inhalers to track their asthma medication use, and one study (4%) had youth with diabetes use a glucose monitor.

Study Implementation Considerations

Technical and Logistical Challenges With EMA Implementation

Most studies (k = 14, 58%) did not report whether participants encountered any technical or logistical problems when implementing EMA. Two studies (8%) explicitly reported that they had no device loss or damage during the study. Of the remaining studies, problems included one or more instances of hardware or software malfunction resulting in data loss (k = 7) and poor cellular reception leading to inability to collect data (k = 2). Given that most studies failed to report on technical problems, we are limited in the conclusions we can draw from these data. No studies reported technical problems that are unique to youth samples, as all of these issues have also been reported in EMA studies conducted with adults (Kubiak & Krog, 2012).

Nearly two-thirds of the studies (k = 15, 63%) reported on adaptations made to the EMA hardware, EMA content, or EMA protocol to make them more appropriate and/or useable for youth. The most common modifications made were in studies that provided mobile devices; the devices were “locked down” or otherwise modified to limit the device capabilities (e.g., able to only access the survey app, answer-only cellular phones, disable phone during school hours) or with limited features designed for children (i.e., child-specific phones; k = 10, 42% of all studies). Some studies also described modifying the survey language for youth (e.g., using youth slang; k = 2, 8%) or adjusting the study protocol to make it more appropriate for youth (e.g., scheduling around youth planned activities). In the other one-third of studies, it was unclear whether modifications were used, as no modifications were reported.

One unique logistic challenge EMA studies with youth face is how to handle signaling or requiring self-reports during school hours. The most common ways studies in this review addressed this issue were as follows: not signaling participants during school (k = 14; 58%), allowing youth to create custom alarm schedules that avoided school hours or other activities (e.g., sporting events, after school activities; k = 3, 13%), or obtaining teachers’ approval for completing EMA during school hours (k = 2, 8%). One study addressed this issue by scheduling EMA data collection during the summer when school was not in session. This option is only appropriate, however, if researchers are interested in nonschool processes.

Training Participants in EMA Methods

We found that 71% of studies (k = 17) described how researchers explained the EMA protocol procedures and how they trained youth to use devices or complete surveys, although the level of detail provided varied. Of the studies that described the training procedures, training sessions were always directed at the youth participants, but in three studies (13%) a caregiver was also included in training. The content of training typically included instructions on how to use the mobile device (k = 14, 58%), and provided details about the questionnaires that would be administered via EMA (k = 14, 58%). A potentially important aspect of training is providing practice opportunities; 13 studies (54%) explicitly reported including practice time in training sessions to allow youth to familiarize themselves with the device, surveys, and study procedures.

Compliance With the EMA Protocol

As in all longitudinal studies following people over time, promoting participant compliance is critical. The majority of studies reported at least some data on completion rates (k = 23, 96%), which ideally would always be reported. Notably, this reporting rate is substantially higher than what has been seen previously; Hufford, Shields, Shiffman, Paty, & Balabanis (2002) found that only 44% of studies using electronic EMA reported completion rates, leading to a call for rates to be required in manuscripts using EMA methods (Stone & Shiffman, 2002). The range of study methods and variations in the types of compliance data reported in our reviewed studies make cross-study comparisons of completion rates challenging. However, of the studies that reported an overall completion rate, the average survey completion rate was 76%, with individual study rates ranging from 51% to 92%; these rates are similar to rates observed in studies with adult samples (Hufford etal., 2002). Survey completion is one aspect of compliance, but for studies that prompt participants to complete surveys, timeliness of responses to prompts is also important. Few studies (k = 4, 17%) reported timeliness data; those that did provided information regarding the average time lapse in responding to a prompt (e.g., average lapse time = 3 min) or the percent of EMA surveys responded to within a set period after a prompt (e.g., percent of EMA completed within 5 min of the prompt; see Table I for specific study results).

Studies reported a range of strategies for improving compliance. One-quarter of the studies (k = 6) used ongoing compliance monitoring and check-ins with youth regarding study procedures; these reminders were in addition to any alarms used to prompt surveys. For example, in four studies, research assistants had remote access to data as they were being entered and contacted youth via text or voice messaging during the study to provide compliance feedback and query for difficulties with the study protocol when compliance rates were low. Another approach used to enhance youth compliance was to link compliance with incentives; 38% (k = 9) reported using incentives to enhance compliance, including monetary incentives (i.e., cash) and technology-based incentives, such as providing a pre-paid SIM card youth could use to make phone calls or text message during the study.

Discussion

This systematic review examined studies that used mobile or smart phones or other potentially Internet-connected mobile devices to collect EMA data from children and/or adolescents in their everyday lives. Studies included in the review enrolled participants ages 7–18 years across a range of demographic and mental health characteristics, as well as pediatric conditions. A variety of EMA data collection methods were used to meet specific study needs, with study duration typically ranging from 4 to 31 days and participants completing surveys two to nine times per day. Most studies (63%) reported making adaptations to the mobile devices, survey software, or study protocol to accommodate youth participants. In general, EMA methods appeared to be acceptable to youth participants (and their caregivers), as there were no systematic patterns of problems or complaints reported across studies. Further, the average EMA survey completion rates (76%) were adequate and on par with EMA studies conducted with adults. Taken together, these data provide evidence that it is feasible to design EMA studies that are appropriate to implement with child (≥7 years of age) and adolescent samples.

The goal of this review was to identify commonly used EMA study designs and implementation challenges that arise with youth participants, and to apply this information in developing recommendations for researchers conducting EMA studies with children, adolescents, and pediatric patients. It is evident there is a growing interest in using mobile technology-based EMA approaches to study youth: our review revealed 54 eligible articles, representing 24 unique studies. Studies in this review were not limited by their topical content, which allowed us to include children and adolescents with pediatric conditions, those with primary mental health diagnoses, and healthy youth (see Table I). We hope that promoting best practice guidelines when designing, implementing, and reporting on EMA studies that use newer mobile technologies with youth may improve the interpretability and generalizability of EMA findings across a range of content areas and health conditions.

Two challenges emerged in our effort to develop methodological recommendations for conducting EMA studies with youth. First, methodological information was missing from some papers, limiting our ability to draw firm conclusions from existing literature. We are not the first to note this problem (and it is not unique to EMA studies); researchers noted this issue as early as 2002 (Stone & Shiffman, 2002), and more recently Liao and colleagues (2016) again identified missing methodological information as a concern in EMA research. In response, both groups of authors proposed general reporting guidelines for mobile technology-based EMA data collection studies. A second challenge with developing methodological recommendations for EMA studies with youth is that study design and implementation decisions are interconnected and dependent on one another. Although it is tempting to look for concrete design rules—for example, the minimum age at which a child can participate in an EMA study—there are rarely simple answers and instead design decisions nearly always require taking into account multiple aspects of the study protocol to make appropriate methodological decisions.

In developing study design recommendations based on this systematic review, we have attempted to address both of these challenges. Existing resources provide general guidance regarding EMA study design and implementation issues (see Mehl & Conner, 2012; Stone, Shiffman, Atienza, & Nebeling, 2007) and EMA reporting guidelines for manuscripts (Liao etal., 2016; Stone & Shiffman, 2002); thus, we highlight design and implementation decisions and reporting guidelines that are influenced by studying child or adolescent participants. Table II provides a summary of the key conclusions from this review and recommendations for researchers when designing and reporting on EMA studies with youth. In these recommendations, we have identified many of the key interdependencies between various study design features that were revealed in this systematic review, with a particular focus on how youth age or developmental level can influence EMA design decisions and reporting recommendations.

EMA Data Collection Methods Summary and Recommendations

As expected, data collection methods used across studies varied widely, as they were likely selected by researchers to meet specific research questions and sample needs. The papers included in this review addressed a range of research questions, and it is possible that when these studies were designed, the researchers had additional questions of interest that were not explicitly identified in individual papers. There are many different “types” of research questions that are particularly well-suited to be answered with EMA designs. In brief, EMA research questions can capitalize on the reduced retrospective recall of EMA (e.g., by comparing/contrasting EMA and retrospective reports), the increased real-world generalizability (e.g., using EMA to provide a more ecologically valid picture of real-world processes), or the multiple assessments collected over relatively short periods. The latter category provides opportunities for researchers to address sometimes novel research questions, including those regarding the moment-to-moment associations between variables, the temporal sequencing of events or experiences (i.e., within-person dynamics), the consideration of individual difference moderators of EMA relationships, and the examination of context effects (both those generally stable at the between-person level, such as neighborhood or school characteristics, and dynamic, time-varying contexts that are assessed via EMA, such as whom a child is with or their location at a given moment). These various types of research questions and others, including specific examples in child and family EMA research, have been discussed at length elsewhere (see Smyth & Heron, 2014 and Heron, Miadich, Everhart, & Smyth, in press).

Although decisions regarding the frequency and duration of EMA surveys and the complexity of the study design (i.e., using self-initiated surveys, prompted surveys, wearable sensors, etc.) must be made based on the research question the study is designed to answer, youth age and capabilities should also be considered. It may be difficult for younger children to remain engaged in and compliant with long or complex studies, but the recommendations in Table II describe strategies that can make EMA studies more accessible for children and adolescents. For example, measurement burst designs (Sliwinski, 2008)—which were used in >40% of the reviewed studies—can extend the EMA assessment period while reducing (sustained) participant burden through “bursts” or “waves” of EMA data collection followed by breaks where no data collection occurs. A burst design may be particularly suited for research on children and adolescents because it allows researchers to use EMA to study the nature of, and variations in, processes that occur over short timeframes (e.g., minutes, hours, days) in the context of a traditional, longer-term longitudinal design that examines developmental processes that emerge over months or years. Other novel approaches for assessing youth in daily life have been suggested. For instance, with both youth and adult samples, some researchers have used planned missingness with EMA questions (not every question is asked at every assessment time), and context-sensitive self-report assessments (an EMA signal can be automatically triggered by an outside event or experience). In one study included in this review, Dunton and colleagues (2016) used these approaches to study adolescents with asthma. Participants completed a subset of questions at each signal, and they also were provided with a Bluetooth-enabled sensor for their inhalers that communicated with the EMA survey program and prompted a survey 5 min after the participant used the inhaler. Both of these strategies attempted to reduce participant burden by limiting the number of questions asked and eliminating the need for adolescents to remember to initiate a survey after using their inhaler. Although these strategies have been used with adults and adolescents, they could also be implemented with children. It is important to note, however, that young children may find it challenging to take part in more complex EMA protocols (i.e., those that combine prompts and event-based sampling, self-report and non-self-report methods, measurement burst designs, etc.). It remains to be seen whether, with careful training and support from study staff or caregivers, younger children are able to provide reliable data in the context of more complex protocols. Further, the youngest participants in the reviewed studies were 7 years old, and it is unclear whether self-report EMA approaches can be adapted and used with children <7 years.

Although the focus of this review was on self-report EMA approaches with youth, 25% of these studies also used mobile sensors, such as wearable accelerometers or medication use tracking devices, to track real-world activities that do not require self-report. Many, although not all, of these non-self-report EMA methods are referred to as passive ambulatory data collection techniques because they require little or no effort on the part of the participant (Smyth & Heron, 2012). Although there is a smaller literature on the use of these devices for collecting ambulatory data from children and adolescents as compared with data from adults, researchers are increasingly using them in combination with self-report EMA. For example, in the pediatric diabetes literature, ambulatory blood glucose monitoring has been used in conjunction with adherence and self-care behaviors measured via EMA in a sample of adolescents with type 1 diabetes (Mulvaney etal., 2012). Moreover, passive non-self-report EMA methods may be particularly useful for collecting real-time data from children who cannot provide self-reports owing to their age or developmental level. For example, the Electronically Activated Recorder (EAR) device is a digital voice recorder that participants wear and that can be programed to capture ambient sound, data that provide information about contextual exposures (e.g., social interactions, arguments). The EAR device has been used in studies with children as young as 3–5 years old to study their family experiences, specifically level of family conflict (Slatcher & Robles, 2012). Although the lower age limit for providing active self-report EMA in the reviewed studies was 7 years old, using passive non-self-report data collection approaches is another way researchers can collect information from younger children in their everyday lives.

Study Implementation Considerations and Recommendations

Although many of the general recommendations and guidelines provided for EMA studies (Mehl & Conner, 2012; Stone, Shiffman, Atienza, & Nebeling, 2007) are relevant when implementing studies with youth participants, additional considerations for adaptations to the mobile devices (hardware and/or software), training procedures, and strategies for enhancing compliance should be considered (see Table II). Self-report EMA measures should incorporate many of the same approaches as non-EMA self-report questionnaires. For example, using pictorial response options or thermometers instead of traditional Likert scales for children’s reports of their moods or experiences have long been used in research with children (Jay etal., 1983; Kamphaus & Frick, 2005). As mobile technologies continue to advance, many devices and software can display images or video and/or audio files as part of self-report surveys, thus providing great flexibility in how EMA surveys can be adapted for child participants.

Technological advances also bring unique challenges to studies of children. For example, researchers as well as caregivers may have concerns about providing children and adolescents with mobile phones, smartphones, or other “connected” devices that could be used for nonstudy purposes. Of the studies included in this review, nearly two-thirds reported making adaptations to the mobile devices, surveys, or study protocol to reduce the potential for harm for youth participants. One common adaptation was to block or limit the ways in which youth could use study-provided mobile devices. For instance, Internet or text messaging capabilities may be turned off if the device is using resident app software that does not require these features, or for studies using voice calls to collect EMA data, answer-only phones can be used. Another emerging question is whether it is possible for youth (particularly adolescents) to use their own smartphones or mobile devices to participate in EMA studies. In the current review, two studies allowed participants to use their own mobile devices and only provided study-owned devices to participants who did not have access to an appropriate device (see Table I). In the United States in 2015, it was estimated that 68% of 13–14-year-olds and 76% of 15–17-year-olds had access to a smartphone and another 14–16% had access to at least a basic mobile phone (Pew Research Center, 2015). As mobile device ownership among adolescents continues to grow, using youth-owned devices for research studies may be increasingly possible and desirable, although there are downsides as well. Most notably, there can be difficulties with screen size/display consistency across devices (making standardization difficult or impossible). There also may be additional privacy concerns related to researchers having access to information on participants’ phones or participants using their own phones, particularly when reporting on private or nondesirable behaviors (e.g., substance use and sexual behavior).

Researchers also have many options when selecting software for EMA studies, which vary in cost and in complexity for development and implementation. Ranging from least to most expensive/complex, software solutions for implementing self-report EMA surveys can be categorized as using one of the following approaches: (1) voice or text message interactive interfaces; (2) cloud- or web-resident survey software that is optimized for delivery on mobile devices; (3) device resident software applications (“apps”; including off the shelf survey app development software, free open-source platforms, and custom survey app development). Each of these options has advantages and disadvantages. When selecting hardware and survey software for studies with youth, we suggest that researchers begin with available recommendations (Kubiak & Krog, 2012), explore multiple options both within and between these categories of software solutions, and consider identifying collaborators with experience with these tools. Throughout the process of selecting survey software, ensuring the software and survey are age and developmentally appropriate for youth participants is critical. Table II provides suggestions for adapting existing hardware and software to be implemented in EMA studies with youth, as well as recommendations for reporting adaptations.

Training participants in EMA procedures is important in all studies, but for studies with youth, training is especially important. As noted, children and even adolescents may have difficulty understanding self-report EMA measures and, especially, complex study procedures. Most studies that included information about training procedures provided participants with a description of the mobile device and how to use it, and about half provided time during the training for participants to practice the EMA survey and procedures. Allowing time for practice serves the dual purpose of reducing the likelihood that problems and questions will arise during the course of data collection as well as increasing participants’ confidence and competence in completing the EMA procedures. A caregiver or another adult who is included in the training session can also be asked to reinforce study procedures and troubleshoot with the youth participant if problems emerge during data collection, although only a minority of reviewed studies included a caregiver or another adult in training (13%). This strategy may be particularly useful with young children.

Another study implementation issue considered in this review was EMA compliance and strategies to enhance compliance. Across studies, the average EMA survey completion rates were similar to studies with adults (Hufford etal., 2002). Studies with youth used a range of strategies for enhancing compliance, including compliance monitoring during the data collection period and providing compliance-based incentives. Although these strategies are somewhat similar to those used with adult samples, their implementation with youth may be somewhat different (as described in Table II). For youth samples, using reminder strategies for enhancing compliance may be particularly important because developmental factors can limit children’s abilities to track the scheduling of EMA procedures. Even in adolescence, involvement in ongoing and emotionally demanding daily activities may increase the likelihood of forgetting or ignoring data collection schedules. Although not reported in any of the studies reviewed, researchers could also use the assistance of adults in the youth’s lives for providing reminders or troubleshooting with EMA procedures. For example, caregivers or other adults could be included in the initial study description and EMA training and be asked to assist youth with remaining compliant with study procedures. When caregivers are involved in reminders to younger children, it may be important to treat them as “citizen collaborators,” such that phone calls or other communications are not experienced as demands on family life. In adolescence, an increasing desire for autonomy may make it important that reminders do not come across as controlling (or nagging); automatic reminders to complete the EMA, including those with a humorous touch to foster positive emotion around study demands, may be a desirable approach to promote compliance. Depending on the technology and systems used to set up EMA surveys, it may also be possible to use automated systems that monitor compliance remotely and provide intervention (e.g., sending a text message to the youth or caregiver if one or more reports are missed), thus reducing the monitoring burden on the research team while enhancing compliance.

Developmental considerations also come into play with respect to incentives. Monetary incentives are often used in EMA studies with adults, and may also be appropriate for adolescents. Access to device features (e.g., Internet use, phone calls, text messaging) can also be used for incentives, particularly with adolescents, but some caution may be needed regarding the appropriate use of devices, privacy, and confidentiality, as discussed previously. Although monetary incentives may be appropriate for adolescents or adults participating along with children in EMA studies, other incentive approaches could be considered with younger children. As is the case in non-EMA studies with children, depending on the age of the child, developmentally appropriate prizes such as a sticker, pencil, notepad, or small toy may be used as an incentive after a child completes the study (regardless of how many assessments he or she completes). A calendar that displays the schedule for data collection with stickers to mark completed assessments may also be particularly effective for enhancing compliance in young children, as it visually displays children’s progress through the protocol. For all youth, timing of incentives is important in intensive longitudinal studies, and delaying gratification to receive an incentive after weeks or months of EMA may make those incentives less effective. Providing a small incentive at the beginning of data collection may pull youth in, and graduated incentives along the way, with a bonus for completing a targeted high percentage of assessments, may motivate continued involvement. Although not explicitly discussed in the reviewed studies, an important part of incentivizing the research process for some youth is to encourage their sense of involvement in efforts that may make a positive difference in the lives of others. Particularly across the transition to adolescence, youth strive for a sense of belonging and to develop a personal identity. Providing information about the translational goals of the project and project “branding” on t-shirts, hats, or backpacks, may encourage youth’s involvement as part of a larger effort.

Although not specific to studies with youth participants, researchers interested in conducting EMA studies should be aware of data management and analysis considerations during study planning and implementation. Large amounts of data are collected with EMA designs—potentially thousands or tens of thousands of data points—and the data are typically on different time scales and intervals across participants, which can be challenging to manage. For example, as part of an EMA study, adolescents may complete four randomly prompted assessments each day with one set of survey questions, another survey at the end of each day (summarizing daily experiences), and wear an ambulatory heart rate monitor that collects information at 15-s intervals. If this hypothetical protocol were followed for 1 week, it would result in up to 28 randomly prompted surveys, seven surveys at the end of each day, and >40,000 heart rate data points for each participant. Analyzing such data sets is complex, beginning from calculating statistical power during study planning, through identifying the appropriate statistical models to test theories and research questions that are of particular interest to researchers using EMA (e.g., regarding within-person processes, temporal ordering of events). As has been discussed in the EMA literature more broadly, existing psychological and health behavior theories are not always well-suited for EMA approaches (Riley etal., 2011). Pediatric researchers planning and executing EMA studies should take particular care with considering how EMA designs fit with existing and new theories (e.g., related to how variables might vary within-person or experiences might unfold over minutes, hours, or days), and adjust both study design and analyses accordingly. These data management and analysis considerations are not specific to studies with youth and, therefore, general handbooks and guides regarding best practices on these topics can serve as useful resources for youth and pediatric researchers interested in using EMA (for general analytic discussion and also power analysis, see Bolger & Laurenceau, 2013; data-analytic methods section of Mehl & Conner, 2012; Schwartz & Stone, 2007).

Future Directions in EMA Research With Youth

Based on our review of the methodological features of EMA studies, we have identified several areas where EMA research with youth participants can continue to advance. Studies with children and adolescents often are directed at characteristics and experiences of youth themselves, including their health and medical conditions, but there is also interest in other individuals in youth’s lives (e.g., caregivers, siblings, peers, medical providers, teachers), contextual factors (e.g., family functioning), and how these might impact, or be impacted by, youth. EMA studies are well-suited to examine these issues, particularly if EMA reports are collected not only from youth, but also youth’s relationship partners. Although not used in any of the studies included in this review, dyadic assessments have been used previously in studies with child–parent dyads (Whalen etal., 2006) and child–sibling participants (Rende, Slomkowski, Floro, & Jamner, 2009); such an approach can provide researchers with a more comprehensive understanding of youth experiences. One issue that arises in all multiple informant studies, regardless of whether EMA is used, is how to make sense of discordant information from different reporters given that different reporters may notice different features of their everyday natural settings and interpret the same experiences differently. Reporters may also be differentially reliable. Despite these challenges, future studies that use EMA approaches with children and adolescents should consider how collecting EMA data from multiple people in a single relationship unit might answer unique research questions relevant to child and adolescent development.

We also suggest that researchers in pediatric psychology consider incorporating EMA methods into their studies to obtain a more complete picture of youth’s daily experiences with chronic or acute medical conditions. For instance, EMA could be used to assess daily disease-related symptoms, mood, health behaviors (e.g., sleep, physical activity), and self-care behaviors. The study in this review by Dunton and colleagues (2016), for example, highlighted the utility of using EMA to measure daily asthma symptoms and assessed variability in stressors and contextual conditions. Results from this study suggested that when youth were outside and experiencing social stressors, they were more likely to experience asthma symptoms. Such findings have implications for psychoeducation and treatment recommendations for youth with asthma. EMA can also be used in pediatric samples to better determine when and how youth may be affected by their health conditions, as well as how such conditions may influence other domains of their daily lives (e.g., academic functioning, school attendance, sleep, family relationships).

In addition to an increased interest in using mobile technology to collect assessments in natural settings using EMA, researchers are also designing interventions or treatment programs that can be delivered as people go about their daily lives. This intervention approach—referred to as ecological momentary intervention (EMI; Heron & Smyth, 2010) or just-in-time intervention (Smyth & Heron, 2016; Spruijt-Metz & Nilsen, 2014)—has been used most often with adult participants. Some researchers, however, are beginning to use EMI with adolescents (Pramana, Parmanto, Kendall, & Silk, 2014), although it has yet to be implemented with pediatric samples. One of the unique opportunities and potential benefits of EMI is that EMA can be used to inform the tailoring of intervention content, as well as the timing of when specific intervention components are delivered to particular people. Thus, this approach allows for interventions to be tailored to real-world contexts and experiences, and using mobile technology, treatments can be provided to people at times when and places where they are most in need of intervention. Although the use of EMI with child and adolescent participants is relatively new, documenting the feasibility and acceptability of using EMA with youth is an important first step as this field continues to grow.

Another important area requiring attention is comprehensive reporting of EMA design decisions and features in EMA studies. Although, as others have noted, detailed reporting is necessary to appropriately interpret results (Liao etal., 2016; Stone & Shiffman, 2002), such information is also necessary when developing data-driven, best-practice recommendations for EMA studies with children and adolescents. For example, if more EMA studies with youth report details regarding training procedures, incentives, and compliance rates, future reviews and meta-analyses could use this information to identify whether and how specific training practices (e.g., including caregiver in training sessions, allowing time for youth to practice, providing compliance-based incentives) are linked to EMA protocol compliance (timeliness to responding, total completion rates, etc.). At the outset, we had hoped to use our review to answer these and other design-related questions regarding best practice. However, given the information reported in existing studies, it is not possible at this time. Instead, we have identified design recommendations and unique reporting recommendations for EMA studies with youth based on the current state of the literature (see Table II). It is hoped that, as more researchers become interested in conducting EMA studies with children and adolescents, these design and reporting recommendations will allow the field to continue to advance as best practice guidelines are refined.

Conclusions

This review synthesized methodological information across 24 studies that used mobile phones, smartphones, or other potentially connected devices to collect EMA self-report information from child or adolescent participants in their daily lives. Participants included children and adolescents with pediatric conditions, those with primary mental health diagnoses, and healthy youth. One of the goals of this review was to use existing research to develop a set of guidelines for researchers to apply when designing and implementing EMA studies with youth with a range of characteristics, across a variety of contexts, and when addressing a variety of research questions. As detailed in Table II, using EMA with children and adolescents can present unique challenges with respect to the most appropriate data collection methods (i.e., sampling duration and frequency, mobile technology selected, survey software used) and study implementation (i.e., adaptations required for youth, training, compliance, incentives). Nonetheless, it is evident that there is growing interest in EMA approaches for studying children and adolescents, and the studies reviewed provide evidence that it is feasible to design EMA studies that are appropriate for, and acceptable to, youth samples. In developing design recommendations, this review highlights that there is inconsistent reporting of key methodological information relevant to EMA study design, and we echo calls for standardized and consistent reporting of EMA methods. Adopting best practice guidelines—such as those outlined in Table II—when designing, implementing, and reporting on EMA studies with youth, may also help clarify what conclusions can be drawn from individual studies and inform future reviews and meta-analyses focused on EMA with youth samples.

Acknowledgments

We would like to thank the following people for assisting with coding articles included in this review: Jennifer Abrams, Sulamunn Coleman, Emma Gibbons, and Jackson Harper.

Funding

Conflicts of interest: None declared

References