User-centered Design of an Adjunct Smartphone App to Reduce Cannabis Use among Youth Diverted from the Juvenile Legal System

The study was conducted in partnership with a statewide family court located in the Northeastern United States that annually serves an average of 2,400 youth facing a range of juvenile offenses. Adolescent participants and their caregivers were recruited exclusively from our partner’s court-supervised diversion program, which seeks to minimize youths’ contact with the JLS and avoid detention. The program is intended for youth with a first-time or less serious second status offense (offenses that can only be committed by a minor, e.g., truancy, underage drinking) or a first-time delinquent offense (criminal offenses committed by a minor, e.g., arson, breaking and entering, felony assault and/or battery). Following their intake assessment, family court staff assign restorative justice (Development Services Group INC, 2021) conditions that the diverted youth must meet in order for their charges to be dismissed. Family court staff also use evidenced-based screening practices for mental health and SU concerns, and can recommend or mandate (i.e., require as a part of the diversion process) a range of local behavioral health services options, including SU treatment, as well as opportunities to participate in clinical research. The decision of whether a youth was mandated to complete the court’s TAU for SU was at the sole discretion of the intake worker. Recent internal demographic data (reported in Helseth et al., 2024) indicate that the court’s juvenile population is predominantly 12–17 years old (98%) and identifies as male (71%) and Non-Hispanic White (62%), with over-representation of Black/African American (22%) youth relative to youth state-wide.

To participate in Phase 1 or 2, diverted youth needed to: (1) be 14–18 years old, reflecting the stronger predictive value of peer influence on SU in late adolescence (Van Ryzin et al., 2012); (2) be able to speak and understand English; (3) have access to a smartphone; (4) report any past-year cannabis use during formal screening by the family court (i.e., the Massachusetts Youth Screening Instrument-Version 2 (Grisso & Barnum, 2006) or the “Car; Relax; Alone; Forget; Friend; Trouble” screener commonly referred to as the CRAFFT v 2.0 (Knight, 2016); and (5) be able to provide consent and/or assent. The research team confirmed eligibility during the initial recruitment call with youth. Caregivers were eligible for Phase 1 if they were: (1) the legal guardian of an eligible youth; and (2) able to speak and understand English or Spanish; all caregivers interviews were done in English, though one caregiver was a bilingual Spanish-speaker. Phase 1 adolescents and caregivers were enrolled independently (i.e., we did not enroll youth and caregivers as dyads).

A member of the research team approached families during the youth’s virtual or in-person court intake appointment, to ask if they were interested in being contacted about a research study for which they may be eligible to participate. Youth over the age of 18 and caregivers of interested minors were invited to sign a consent-to-contact form or provide verbal consent (as permitted by IRB approved procedures due to COVID-19) allowing the research team to review the youth’s court records for eligibility criteria and approach them for secondary screening and enrollment. When contacting families, the research team member would describe the active phase of the research study and invite interested diverted youth and/or caregivers to complete informed consent; parental consent and child assent were obtained for participants under the age of 18. Phase 1 recruitment began in December 2020 and ended in November 2021. Phase 2 recruitment was conducted from June 2022 to September 2022.

The primary goal of Phase 1 interviews was to identify the digital intervention needs and preferences of diverted youth who use cannabis, in order to guide initial development of the TECH app. Both caregivers and youth were asked about diverted youth’s technology access and the perceived feasibility and acceptability of digital interventions for diverted youth in general, in the family court context, and for the proposed TECH app. Interview questions covered key dimensions of the Behavior Intervention Technology (BIT) model (Mohr et al., 2014), an integrated conceptual framework that helps digital intervention developers to distill the “what, why, how, and when” of their intervention. Specifically, the guide confirmed “what” behaviors youth wanted help changing and “why” youth needed the proposed app (i.e., clinical and usage aims), “how” the app could help youth change their behavior (i.e., behavior change strategies), and which features would best achieve that objective (i.e., digital elements, design characteristics and workflows). On average, caregiver interviews lasted 24 min (SD = 6; range 14–30 min) while interviews with diverted youth lasted 51 min (SD = 11; range 40–78 min), reflecting solicitation of end-user feedback about anticipated TECH app features.

Following the BIT model, we specifically asked youth for guidance around five anticipated components of the TECH app we selected a priori to help achieve our theory-driven clinical and app usage aims. We anticipated targeting intrapersonal mechanisms of SU behavior change via features that would enhance user motivation to change SU or directly facilitate a user’s SU behavior change efforts. We sought to target interpersonal mechanisms of SU behavior change via interactive features that allowed supportive, prosocial engagement. Notably, all our anticipated behavior change components were identified as widely used components of digital interventions for young adult cannabis use in a recent meta-analysis (Côté et al., 2024). Finally, to achieve the BIT model’s theory-driven app usage aims—which seek to engage youth with the app enough to expose them to our clinically-oriented components, we anticipated including gamification(see Cugelman, 2013) and personalization components. Following established strategies for mobile app development (e.g., Santesteban-Echarri et al., 2021), diverted youth and caregivers were shown screenshots of several then-publicly available apps for cannabis use reduction or abstinence (e.g., Quit Cannabis (Memsta Apps, 2024), Grounded: Quit Weed Smoking [Grounded, 2024], Stop-Cannabis [Benarous et al., 2016]) to demonstrate how several anticipated TECH app components (i.e., personalization, behavior change, social interaction, gamification, motivation enhancement) might look or function. Diverted youth were also asked to rate the five anticipated components on a scale of perceived importance (1 = not important to 10 = very important) and select their top 3 must-have components.

All participants completed online surveys before being interviewed; youth completed their online survey, received the court’s TAU, then completed their individual interview. The family court’s TAU for diverted youth who screened positive for cannabis use at intake included a brief computer program called eCHECKUP-To-Go Cannabis (ECTG; San Diego State University Research Foundation, 2009). The program anonymously collects data about the participant’s SU and provides accurate information about peer use (e.g., peer normative feedback) and personalized behavior change recommendations related to their cannabis use. In their interviews, we specifically asked diverted youth to discuss if and how anticipated TECH app features might help them enact any of their personalized ECTG behavior change recommendations, to ensure the resulting app functioned as an adjunct to TAU.

The study’s principal investigator (SAH) conducted all interviews; she is a Non-Hispanic White, cis-gender female researcher with a doctoral degree in clinical psychology. Interactions between the interviewer and participants were limited to recruitment, consent/assent, and scheduling, followed by the one-time interview. Interviews were audio recorded, transcribed verbatim and checked for accuracy by two independent study team members, then de-identified. Formal field notes were not taken and transcripts were not returned to participants for review.

Participants provided basic demographics to allow for sample characterization. Diverted youth reported their recent behavioral health service usage on the Child and Adolescent Services Assessment (CASA; Burns et al., 1997; Farmer et al., 1994) and characterized their cannabis use on the Marijuana Use Questionnaire (MUQ; Cuttler & Spradlin, 2017; Metrik et al., 2009).

Caregiver and youth interviews were analyzed separately using a directed content analysis approach (Hsieh & Shannon, 2005). Analysis followed an a priori coding scheme derived from the BIT model (Mohr et al., 2014) constructs. The PI and two additional coders began by establishing definitions for each of the BIT model constructs, to form the initial codebook. The coding team independently applied the initial codebook to the same two transcripts, to refine a priori definitions and ensure consensus. Coding team members could also recommend additional, unanticipated codes; the team established definitions for these emergent codes to ensure consensus, resulting in the final version of the codebook. Two coders then independently reviewed all transcripts using the final version of the codebook. The coding team reconvened to discuss any independent codes and achieve consensus.

Quantitative measures were analyzed using percentages or means and standard deviations. For the five anticipated TECH app features that diverted youth rated on importance and selected their top three must-have features, we calculated the average importance rating and reported what percent of diverted youth endorsed a feature as being a must-have.

Two additional themes were identified during the coding process that were not related to the TECH app: diverted youth’s access to and use of technology and court-related concerns.

Prototype development began in November 2021 with a series of meetings between the research team and the software developer. We partnered with Mooseworks Software LLC, a US-based single-developer company that uses a library of pre-programmed features to build affordable mobile apps for a range of customers, including clinical researchers (e.g., Becker et al., 2022). Prior to launching the design phase, both parties engaged in institutional contracting and an information technology and security review process, which itself took approximately 6 months. In an initial 2-hour design meeting with Mooseworks, the research team described the TECH app’s clinical aims (i.e., helping youth diverted from JLS-involvement make a change in their cannabis use through interpersonal and intrapersonal mechanisms of behavior change) and summarized the qualitative results from interviewed youth and caregivers related to anticipated features. Key features that were selected for the prototype included (see Supplementary Table 1): a username and login, user settings, a newsfeed, goal setting, daily log and progress charts –both of which captured daily goals achieved, mood ratings, cannabis used, and money spent on cannabis—in-app notifications, a resources page with information about cannabis and 24 crisis lines, a leaderboard, activity-based badges, and a contact us feature to report malfunctions. Over the next month, the developer and the research team engaged in a series of meetings and email conversations to make decisions about characteristics and workflow of included features, referring back to the qualitative data frequently. As an example, in response to feedback from Phase 1 diverted youth that some teens may not be interested in the daily log feature and thus might be driven away by our proposed daily reminders to log their activities, we instead employed daily positive affirmations sourced from publicly available lists (e.g., Harra, 2018) to promote general app engagement rather than engagement with the daily log. Approximately three weeks after the initial meeting, the first iteration of the app prototype interface was available. From there, the research team and developer engaged in several months of iterative design and refinement of specific components.

Beginning in April 2022, the research team conducted several rounds of functionality testing with approximately 10 adult volunteers who were not involved in the design process, to ensure the software was working as intended. Then, the research team engaged in several rounds of a two-week long quality assurance process to test the app using fake user accounts. This process allowed the research team to populate the newsfeed prior to the launch of beta-testing and to confirm that the values users entered into the logs (e.g., user entered a ‘2’ in the daily log for ‘how many times did you use marijuana today?’) was consistent with the data presented back to the user in their progress charts and the metadata available on the administrator platform. In total, the developer “released” approximately 16 versions of the prototype for review and iterative refinement prior to the launch of Phase 2.

Recruitment and youth eligibility were consistent with Phase 1; caregivers did not participate in Phase 2. To ensure a sufficient user presence for the app’s interactive features, the research team waited to schedule baseline appointments until after at least 4 participants were available to be enrolled. Phase 2 youth attended a baseline appointment in which they completed online baseline surveys, the court’s TAU (i.e., the ECTG program; San Diego State University Research Foundation, 2009), and were then trained to use the prototype TECH app. Youth had access to the TECH app prototype for one month, after which they completed follow-up surveys.

Acceptability of the TECH app was captured two ways. First, diverted youth completed the Mobile Application Rating Scale - User Version (uMARS; Stoyanov et al., 2016) at follow-up. It is a publicly available and widely-used 20-item digital health quality assurance measure developed for and validated with youth. Youth rated the app on a series of item-specific 5-point Likert scales, where higher scores are better; uMARS subscales capture user-reported engagement, functionality, aesthetics, and information. Our target for app quality was average subscale scores of > 3. Second, diverted youth rated their satisfaction with the app on the 11-item Consumer Satisfaction Questionnaire (CSQ; McMahon & Forehand, 2003). Youth rated each question on a series of item-specific, 7-point Likert scales (e.g., 1 = very dissatisfied to 7 = very satisfied) and an average satisfaction score was calculated. The CSQ ended with an open-response question asking how the TECH app could be improved to help more than it has.

Due to the small sample, quantitative data are only presented in aggregate as percentages or means and standard deviations, to offer insights into the samples and preliminary feasibility and acceptability of the TECH app prototype.

This preliminary work must be considered in light of notable limitations. The TECH app was designed in partnership with a single, statewide family court. It was conceptualized as a digital adjunct that could sustain motivation to change among diverted youth who received a brief computerized intervention for cannabis use and help facilitate SU-related behavior change. Diversion programs can vary across jurisdictions and JLS settings, such that generalizability of the TECH app may be limited to settings that offer diverted youth SU services. Our sample was limited to a small number of youth and caregivers recruited entirely from our partner family court; as such, our findings solely reflect the perspectives of constituents and end users in a single-family court and region of the US. Relatedly, our samples had slightly higher prevalences of non-Hispanic White-identifying (i.e., 75% in this study, versus 62% court-wide) and female (i.e., 50% in this study, versus 29% court-wide) diverted youth than previously reported demographics of our partner court’s population (reported in Helseth et al., 2024). This may be an artifact of our study’s focus on cannabis use, the use of which has been shown to differ among various racial and ethnic groups (Substance Abuse and Mental Health Services Administration, 2024), or our small sample sizes (i.e., the difference in our sample’s rate of non-Hispanic White diverted youth and the court overall was by about 3 youth). Due to the emphasis on intervention content and user-centered design, rather than program implementation at the organizational level, we did not collect data from other key constituents like JLS staff or members of the court. Their guidance will be vital to future implementation and scale out efforts, if results are promising.

Digital interventions offer an accessible and scalable means of delivering clinical services with high-fidelity, particularly in high need, low resource settings that lack a clinically trained in-house workforce, which describes many JLS settings. Even so, the promise of digital approaches hinges on their ability to affect meaningful, real-world changes for youth who use them. Though still in its early stages, this work represents a small but important step forward in the broader effort to design and adapt digital health interventions (Bath et al., 2018) that can serve the unique behavioral health needs of youth in the JLS. A critical next step is to establish the efficacy and effectiveness of the TECH app as an early SU intervention tool for youth diverted from the JLS who use cannabis use; the pilot randomized trial of TECH is currently under way and will offer further insight into the clinical utility of digital interventions for youth in the JLS.