Introduction

The development of adolescents has always been an important population theme of the United Nations (United Nations 2012). Adolescents go through several changes and take more responsibilities when they become both physically and psychologically mature. To grow into healthy, caring and responsible adults, adolescents have to accomplish developmental tasks such as establishing personal identity, seeking independence from parental figures, developing interpersonal intimacy, recognizing sexual adequacy, developing self-esteem, and developing feelings of competence (Dolgin 2018; Erikson 1968; Fanos 1997; Pennington and Sharrott 1985). However, these developmental tasks of adolescents are not easily accomplished, and there is increased attention to the developmental problems among adolescents (Shek et al. 2011; Tonkin and Frappier 2003).

In Hong Kong, several health-related risk behaviors among adolescents have been identified, including adolescent substance abuse, youth delinquency, mental health problems (e.g., stress and suicidal ideation), internet addiction, school bullying, adolescent sexuality as well as material orientation issues (Chan and Wong 2015; Family Planning Association of Hong Kong 2017; Hong Kong Federation of Youth Groups 2007; Kwok and Shek 2010; Shek 2006, 2007, 2010; Shek and Yu 2016). Positive youth development programs may be a promising strategy for promoting adolescent wellbeing and preventing developmental problems (Catalano et al. 2012).

From an ecological perspective, adolescent developmental issues are inter-related and stem from similar individual and environmental conditions (Shek and Leung 2013). The commonality in risk factors or protective factors across a range of adolescent behavior problems suggests that prevention and positive youth development programs addressing multiple issues at the same time are efficient and effective (Catalano et al. 2012). Primary prevention and secondary prevention are two major types of intervention to prevent adolescent developmental problems. Primary prevention programs aim to prevent the occurrence of adolescent developmental problems which is often through targeting the entire population within schools and communities (Cornelius and Resseguie 2007; Fuchs et al. 2010). On the other hand, secondary prevention programs target adolescents who are at greater risk for developmental issues (Hale et al. 2014). School-based and community-based primary prevention programs focusing on adolescents in the general population as well as secondary prevention programs designed for at-risk adolescents have been found to be effective methods to reduce adolescent problem and risk behaviors, such as drug abuse, alcohol and tobacco use (Griffin and Botvin 2010).

Compared to programs that focus on prevention, the positive youth development (PYD) approach that focuses on adolescent psychosocial competences is recommended (Shek et al. 2011). There are several distinct features of this approach. First, it adopts an ecological perspective to understand individual and environmental factors and the interactions between them, which shape the development of adolescents. Based on the ecological model, protective factors are important assets that promote positive adolescent development (Shek 2016). Thus, the PYD approach focuses on psychosocial competences to enhance those protective factors, which is the second distinct feature (Shek et al. 2012a, b). Third, the PYD approach adopts a strength-based perspective, which is seen as a better way to avoid stigma and blaming in adolescents (Shek et al. 2011). Empirical support for the efficacy of the PYD approach has provided a solid foundation for further application of this approach in the Chinese context (e.g., Shek and Ma 2014b; Shek et al. 2012a, b; Shek and Sun 2010; Shek and Yu 2012).

To promote holistic development in high school students in Hong Kong, a PYD program entitled “Positive Adolescent Training through Holistic Social Programmes” (i.e., Project P.A.T.H.S.) for young people was introduced in 2005 (Shek and Wu 2016). Designed by Shek and his collaborators (Shek et al. 2007; Shek et al. 2012a, b; Shek and Sun 2012a), the program incorporated a wide range of PYD constructs proposed by Catalano and his colleagues (Catalano et al. 2004). These PYD attributes were commonly adopted in intervention approaches to promote positive development of adolescents (Catalano et al. 2012). For example, bonding is important for adolescent to develop a sense of security and become resilient in stressful situations (Lee and Lok 2012). Emotional competence is another meaningful PYD construct that is often related to adolescent subjective well-being (Lau and Wu 2012).

To meet the needs of different students, the Project P.A.T.H.S. provides both primary and secondary prevention programs (i.e., the Tier 1 Program and the Tier 2 Program). The Tier 1 Program is a primary prevention program for all high school students with the use of curricula-based programs focusing on various developmental outcomes (Ma et al. 2011). The students are expected to attend 10–20 h of curricula-based training in each school year. Developed by school social workers in collaboration with teachers, the Tier 2 Program is a secondary prevention program targeting students of at-risk groups. That is, students with special needs in academic and psychosocial areas were invited to join this tier. In general, about one-fifth of the students are program targets (Shek and Lee 2012).

Since its inception in 2005, the Project P.A.T.H.S. has been successfully implemented in the school context based on the evaluation findings using different strategies (Shek and Sun 2013c). For example, based on the objective outcome evaluation findings, students in the experimental schools showed much slower increase in problem behavior, such as delinquent behaviors and substance abuse (Shek and Yu 2012). Other evaluation studies such as qualitative evaluation and subjective outcome evaluation have been conducted, and the results demonstrated that students were satisfied with the program and there were positive changes in the related developmental outcomes (e.g., Shek and Sun 2013a, 2013b; Shek et al. 2017). Despite the above positive outcomes, little is known whether the impact varies by the contexts. Based on a systematic review of over 500 youth prevention programs, Durlak and DuPre (2008) suggested that provider characteristics (e.g., perceived benefits of the program, self-efficacy) is one of the factors affecting implementation process and outcomes of the program. It is particularly important to determine how well the programs are delivered and achieved in different contexts. In the spirit of nurturing positive youth development, more research in this area deserves our attention.

In 2013, a community-based extension phase of the project was launched (Shek and Wu 2016). The community-based design has several advantages. First, more junior high school students can benefit, especially students who have greater psychosocial needs. Second, the design is more flexible compared with the school-based design where some meaningful community programs, such as volunteer programs, can be included. Finally, community-based programs can be implemented at a flexible time. Using a pretest-posttest design, Ma and Shek (2017) found that adolescents reported positive changes in psychosocial and behavioral outcomes after participating PYD programs in community contexts.

Evaluation is an integral part of the P.A.T.H.S. Project. Similar to previous studies, multiple strategies have also been used in the evaluation of the community-based PYD program. The present study focused on the findings based on objective outcome evaluation using a one-group pretest-posttest design. In this research design, a single group of participants are pretested. Then researchers give treatment and collect posttest data on the same measure (Salkind 2010). If a significant difference is found in the scores between pretest and posttest, then treatment is considered as helpful.

In the present study, a one group pretest-posttest design was employed to identify the changes in different developmental outcomes among Chinese students who participated in a community-based Project P.A.T.H.S. We explored whether the changes in developmental outcomes differed from program types (i.e., Tier 1 program only, Tier 2 program only, both Tier 1 and Tier 2 programs). As Tier 1 Program was designed for the adolescent population and the Tier 2 Program was designed for adolescents with greater psychosocial needs (Shek and Sun 2013c), separate analyses were carried out in this study. Further to the report based on the first-year data (Ma and Shek 2017), this study attempted to examine the effectiveness of the project in the second year of implementation. In this paper, students’ PYD competences, life satisfaction and thriving were assessed. To further understand the impact of the program, we adopt the hierarchical approach by assessing the changes in four composite scores (i.e., secondary factor of PYD attributes) which were derived from multi-group confirmatory factor analyses (Shek and Ma 2010). This approach helps us understand the complex nature of service from participants’ perspective (Martínez Caro and Martínez García 2007) and was supported by empirical research (Lu et al. 2009).

Operationally, three hypotheses were examined in this study as follows:

  • Hypothesis 1: Compared with the pretest scores based on the PYD measures, the posttest scores would be significantly higher.

  • Hypothesis 2: Compared with the pretest scores based on the Satisfaction with Life Scale, the posttest scores would be significantly higher.

  • Hypothesis 3: Compared with the pretest scores based on the Thriving Scale, the posttest scores would be significantly higher.

Methods

Participants

The program participants were 17,544 junior high school students in 2014. To achieve diversity of study sample, the study consisted of 11 major districts in Kowloon (4 districts), Hong Kong Island (2 districts) and New Territories (5 districts). They were invited to complete the pretest assessment by filling in a questionnaire containing items on holistic youth development. At pretest, 14,414 students completed the questionnaire. Upon the completion of the program (posttest), the same questionnaire was distributed to the students and 13,937 students completed the survey. Pretest data were then matched with posttest data, resulting in 10,807 valid cases (success rate: 78%). The numbers of students participating in the Tier 1 Program only, the Tier 2 Program only, and the Tier 1 plus Tier 2 Program were 8278, 1139 and 1390, respectively (Table 1).

Table 1 Demographic information of the participants
Full size table

Among the 10,807 participants, 5618 (53.2%) were male students and 4939 (46.8%) were female students, and 250 did not report their gender when filling in the questionnaire. The mean age of the students was 12.99 (SD = 1.26) at pretest. The numbers of Grade 7, Grade 8, and Grade 8 students were 7261, 1465, and 1886, respectively. However, 195 students did not report their grade.

Procedure

The first round of the community-based project was implemented from January 2013 to December 2015 (i.e., a 3-year project) and 21 non-governmental organizations (NGOs) joined this PYD program (Ma and Shek 2017). Social workers from these NGOs were collaborated with school staff to implement this program among Chinese high school students. In each year, social workers and school teachers invited students to join this program. In this paper, we concentrate on the evaluation outcomes in the second year of program implementation. The findings based on 2013 (i.e., the first year of the program implementation) are published in elsewhere (Ma and Shek 2017). In the year 2014, 91 projects with pre-defined theme (31 projects in Tier 1 program, 31 projects Tier 2 program and 29 projects in two-tier program) were implemented at pretest stage. At posttest stage, 90 projects were implemented as one project withdrew from the Tier 2 Program (i.e., 31 projects in Tier 1 program, 30 projects Tier 2 program and 28 projects in two-tier program). Given the voluntary participation of the study, some NGOs encountered operational problems and did not collect data in pre- (January 2014) or post-test (February 2015).

Students’ developmental outcomes were assessed at two time points of the second-year program: before their participation in the program (pretest) and after their participation (posttest). On-site research assistants conducted data collection and provided instructions on how to fill in the questionnaire. The students were assured that the survey would be conducted anonymously and confidentially, and their participation in the study was voluntary. Informed consent was obtained from the participants and their schools and their parents prior to data collection. The study has been approved by the institutional review board.

Measures

A 61-item self-administered questionnaire was used in the research, containing measures of demographic information and holistic youth development.

Demographic Information

Students answered questions about their personal and family background, including their age, gender, grade, parents’ marital status, the community center they came from, etc. In addition, the questionnaire included an item assessing students’ perceived academic competence on a 5-point Likert scale.

Positive Youth Development

The Chinese Positive Youth Development Scale (CPYDS; Shek et al. 2007) was utilized to measure students’ developmental outcomes. Previous research demonstrated that this scale had sound psychometric properties (Shek and Ma 2010, 2012a, 2012b; Shek and Ma 2014a, b). The 44-item instrument included 15 subscales, including bonding, resilience, social competence, recognition for positive behavior, emotional competence, cognitive competence, behavioral competence, moral competence, self-determination, self-efficacy, clear and positive identity, beliefs in the future, prosocial involvement, prosocial norms, and spirituality. All subscales have three items, except the Self-Efficacy Subscale, which has two items. Regarding CPYDS, each respondent was asked to rate the items on a 6-point Likert scale with responses ranging from “strongly disagree” to “strongly agree”, except for the 3 items in the Spirituality Subscale, which were assessed by a 7-point Likert scale. Higher scores in the CPYDS reflect higher levels of psychological competence. The internal consistency of all subscales range between .70–.89. Based on the results of multi-group confirmatory factor analyses (Shek and Ma 2010), the 15 subscales were subsumed under four second-order factors, including general PYD qualities (GPYDQ), cognitive-behavioral competences (CBC), positive identity (PIT), and prosocial attributes (PA). The internal consistency of these composite scores range from .87–.97. Methodologically speaking, focus on the scores based on the second-order factors is a more parsimonious approach to understand the changes involved.

Life Satisfaction (LS)

The 5-item Satisfaction with Life Scale was utilized to assess students’ life satisfaction. Developed by Diener et al. (1985), the scale is widely used to assess one’s satisfaction with life, and has been translated into various languages. The Chinese version of the LS Scale has demonstrated good psychometric properties among Hong Kong students in previous research (Shek and Sun 2010; Sun and Shek 2010, 2013). A 6-point Likert scale was used to rate the items in LS scale. Higher scores indicate higher levels of LS. The internal consistency of this scale is .87.

Thriving (TH)

The 12-item Thriving (TH) Scale was used to assess the thriving process of students, which refers to students’ physical health, leadership, delay gratification, upholding personal values, overcoming adversities, and helping others (Scales et al. 2000). The translated version has been reviewed by several expert panels who were fluent in both English and Chinese. Furthermore, pilot study has been conducted to test the face validity of the scale. This scale has been used in past research (Ma and Shek 2017; Shek and Ma 2014a; Shek and Sun 2010). Participants were instructed to respond to the items in TH on a 6-point Likert-type scale. The internal consistency of this scale is .87.

Data Analysis

We used the multivariate general linear model (GLM) to test the changes of students participating in the community-based project. The interaction of time X outcome effect on multiple adolescent developmental outcomes was assessed based on the F ratio. To test the specific hypotheses, several GLMs targeting different dependent variables (the total CPYDS and composite scores, individual PYD outcomes, life satisfaction and thriving) were performed. In order to understand whether the changes of positive developmental outcomes vary by the tier programs, we also conducted separate analyses based on different programs (i.e., the Tier 1 Program only, the Tier 2 Program only, and the Tier 1 plus Tier 2 Program).

Results

First, we performed reliability analyses to test the internal consistency of the measures used in the present study. The values of Cronbach’s α and mean inter-item correlation coefficients can be regarded as good (see Tables 2, 3, 4, and 5).

Table 2 Overall changes in different indicators between pretest and posttest (whole sample)
Full size table
Table 3 Overall changes in different indicators between pretest and posttest (Tier 1 only, n = 8278)
Full size table
Table 4 Overall changes in different indicators between pretest and posttest (Tier 2 only, n = 1139)
Full size table
Table 5 Overall changes in different indicators between pretest and posttest (Tier 1 & 2, n = 1390)
Full size table

The Overall Program Effects

With respect to Hypothesis 1, results demonstrated a significant interaction between time and indicators based on the 15 subscales of CPYDS; F(14, 10,793) = 39.21, p < .001, partial eta squared = .05. Based on the omnibus F test, a significant interaction between time and second-order factors of CPYDS (i.e., GPYDQ, CBC, PIT, and PA) was found; F(3, 10,804) = 74.56, p < .001, partial eta squared = .02 (see Table 2). Results of the univariate analyses showed that the scores of primary and second-order factors of the CPYDS were significantly higher at posttest of the program with the effect size partial eta squared of different subscales ranging from .01 to .03. However, no significant changes in bonding, social competence and spirituality were found.

With respect to Hypotheses 2 and 3, we repeated the above analysis steps and found significant multivariate effect of time on life satisfaction and thriving; F(1, 10,806) = 207.88, p < .001, partial eta squared = .02. The results of the univariate analyses also demonstrated significant improvement on participants’ life satisfaction and thriving at posttest (ps < .001), though the change on thriving was very small.

Effects of the Tier 1 Program

Similar multivariate effects of time were shown in CPYDS scores (the primary-order factor: F(14, 8264) = 28.68, p < .001, partial eta squared = .05; the second-order factor: F(3, 8275) = 48.42, p < .001, partial eta squared = .02). As pointed out in Shek and Sun (2013c), the Tier 1 Program was developed for students in the general adolescent population. For the students who joined the Tier 1 Program only, we found significant increase in the scores in the primary and second-order factors of positive youth development outcomes, except for bonding, resilience, and prosocial norms, thus Hypothesis 1 was supported (see Table 3).

Significant multivariate effects of time and psychological outcomes (F(1, 8277) = 137.85, p < .001, partial eta squared = .02). Results based on univariate analyses also revealed that the students had significant increase in the scores in life satisfaction, but not in thriving. Therefore, only Hypothesis 2 was supported. It is noteworthy that the effect size was small (partial eta squared = .01, Table 3).

Effects of the Tier 2 Program

For Tier 2 Program, it was designed for students with greater psychosocial needs (Shek and Sun 2013c). For the students who joined the Tier 2 Program only, significant multivariate effects were found in all outcome measures, including all PYD measures (primary-order factor: F(14, 1125) = 6.20, p < .001, partial eta squared = .07 and second-order factor: F(3, 1136) = 14.16, p < .001, partial eta squared = .04). Results of univariate analyses showed that students had significant increase in the scores in all positive youth development outcomes with partial eta squared ranging from .01 to .09 (see Table 4). Similar results were found in life satisfaction and thriving (F(1, 1138) = 38.21, p < .001, partial eta squared = .03, Table 4). In general, all hypotheses were supported when testing the impact of Tier 2 program.

Effects of the Tier 1 plus Tier 2 Program

As the literature shows that higher dosage of intervention would have greater benefits for clients (Shek and Sun 2013c), it would be interesting to examine the combined effect of the Tier 1 and Tier 2 programs. Similar to the students who participated in Tier 2 program, those, who joined both primary and secondary prevention programs (i.e., Tier 1 plus Tier 2 Program), were benefited from the program. Significant multivariate effects of time were also found in all CPYDS attributes (primary-order factor: F(14, 1376) = 6.45, p < .001, partial eta squared = .06; second-order factor: F(3, 1387) = 16.74, p < .001, partial eta squared = .04). Significant increases in the scores in most of the PYD attributes were also found, except for bonding, social competence and spirituality (see Table 5). The positive changes were also found in life satisfaction and thriving (F(1, 1389) = 35.31, p < .001, partial eta squared = .03). In general, students, who participated in Tier 2 Program only and both tiers programs, reported more positive changes than those who participated in Tier 1 Program only.

Discussion

The study was conducted to evaluate the effectiveness of a community-based PYD program (Project P.A.T.H.S.) in Hong Kong. The positive outcomes of the program demonstrate that PYD approach is an effective strategy for enhancing protective factors for adolescent development. There are several strengths of the present study. First, we conducted the program with a large sample size. According to Catalano et al. (2012), most intervention programs addressing risks among adolescent provided services to only few hundred participants. Second, the validated Chinese Positive Youth Development Scale was used to measure different dimensions of positive youth development, which gave the full picture of adolescent mental health and other wellbeing outcomes (Shek and Ma 2010). Third, as several waves of data have been gathered since the community-based project was launched in 2013, the objective outcome evaluation findings can be replicated in different years and the credibility of the research can be accumulated.

Consistent with the first-year results of the project (Ma and Shek 2017; Shek et al. 2017), significant differences between pretest and posttest were found in the scores of CPYDS, Satisfaction with Life Scale and Thriving Scale. In particular, participants at posttest were found to perform better, which is consistent with Hypothesis 1–3. Besides the whole sample, further analyses focusing on participants in different tiers (i.e., the Tier 1 Program only, the Tier 2 Program only, and the Tier 1 plus Tier 2 Program) suggest that the project was effective. In short, the findings of this objective outcome evaluation provided support for the beneficial effects on adolescent development across multiple PYD and psychological outcomes.

The results are consistent with the findings in previous research of school-based P.A.T.H.S. Project. Shek and Ma (2012a) analyzed eight waves of longitudinal data collected from randomized trials utilizing objective outcome evaluation strategy, and found that students benefited from the programs in their psychological development trajectories. Furthermore, the findings echo the results of the subjective outcome evaluation of the community-based project. Shek 2016 found that students generally held positive views of program instructors, content, and overall effectiveness. In addition, program implementers (teachers and school social workers) also perceived the PYD programs to be beneficial to adolescent positive development (Shek 2012; Shek and Ma 2012b). Taken as a whole, the findings in the present study suggest that community-based project in Hong Kong is effective in promoting students’ PYD qualities, including a range of psychosocial competences which can promote adolescents’ mental health.

One unexpected result is that students in the Tier 2 Program reported more positive changes in the scores in all the PYD domains and psychological outcomes compared to students from different tier programs (i.e., Tier 1 program only and two tiers program). This may be related to the educational activities (e.g., overnight camping, volunteer services), which mostly occurred outside the traditional classroom settings. Perhaps, this targeted students learned better through experiential learning (Anderson-Butcher et al. 2013; de Anda 2001). It should be noted that participants in the Tier 2 Program got lower scores in most of the indicators compared with participants in the Tier 1 Program at pretest, thus their increases in terms of positive youth development were remarkable. In general, the present study demonstrated the benefits of the PYD program was not limited to the general youth population, but also to the target participants with special needs. More specifically, students, who joined Tier 2 program, reported more positive changes after participating the program. The findings suggest that the use of PDY programs is a promising approach to promote the well-being of adolescents with greater psychosocial needs. Because secondary prevention is an important strategy of prevention approach, it is suggested that we should screen for at-risk adolescents in practices and engage these adolescents more (Hale et al. 2014).

There are two important findings that deserve our attention. First, although the program had different effects on students’ developmental outcomes depending on the different scales in use, significant and positive program effects were observed for most indicators of PYD, life satisfaction and thriving. The findings are consistent with previous research that adolescents demonstrated better development after attending the PYD programs (Shek and Sun 2010, 2014). According to Shek (2016), adolescents joining PYD programs were less likely to engage in problem behaviors and more likely to promote positive outcomes for themselves, and the relationship was mediated by life satisfaction (Sun and Shek 2010; Sun and Shek 2013). Thus, this community-based program is a promising approach to promote PYD, increase protective factors and reduce risk factors.

Second, the results are consistent with the findings in previous research of school-based P.A.T.H.S. Project. Shek and Ma (2012a) analyzed eight waves of longitudinal data collected from randomized trials utilizing objective outcome evaluation strategy, and found that students benefited from the programs in their psychological development trajectories. Furthermore, the findings echo the results of the subjective outcome evaluation of the community-based project. Shek (2016) found that students generally held positive views of program instructors, content, and overall effectiveness. In addition, program implementers (teachers and school social workers) also perceived the PYD programs to be beneficial to adolescent positive development (Shek 2012; Shek and Ma 2012b).

Although the use of randomized control trials is often regarded as the “gold standard” in the evaluation of intervention programs, there are several difficulties utilizing RCT design in the present research. First, the expense is very high conducting RCTs. Second, the random allocation of schools and participants to intervention or control group may not be feasible (Shek and Sun 2012c). Thus, the present study adopted the one-group pretest-posttest design to evaluate program effectiveness. The one-group pretest-posttest design is commonly used in evaluating programs and program documentation (Leflore 2012; Issel 2013). This design answers not only what participants achieve at the end of intervention, but also how much they change during the participation in the intervention. Marlow (2010) regarded the one-group pretest-posttest as a useful design, which is certainly often used in program evaluation, especially when no comparison group is feasible. Thyer (2002) also argued that this design is useful in practice settings.

Researchers in the field of social work, public health, and education have commonly adopted the one-group pretest-posttest design to evaluate program effectiveness. For example, Siu et al. (2014) adopted this research design to examine the effectiveness of a secondary intervention program, and the results demonstrated a reduction in burnout and an improvement in work-related well-being among health care workers (n = 1034) in Hong Kong. Allen et al. (2014) investigated the effectiveness of a substance abuse prevention program among Latino youth using a one-group pretest-posttest design, and a change in parents’ perceptions (n = 83) regarding adolescent substance use was found. Fisak (2014) examined the effectiveness of an intervention program in preventing anxiety in preschool children (n = 20), and the initial findings were positive based on the pretest and posttest scores.

Using a one-group pretest-posttest design, Gao et al. (2016) found that the Active Videogames (AVGs) is an effective primary prevention program to improve underserved children’s classroom behaviors and academic effort (n = 95). Fagan et al. (2015) employed the same design to examine the effectiveness of a new parenting curriculum “Understanding Dad™” and found that mothers (n = 34) became more knowledgeable, confident, and skillful when engaging in co-parenting activities. In the area of social work education, a bridge course was found effective in enhancing MSW students’ knowledge and changing their attitudes towards LGB (lesbian, gay, and bisexual) people using a one-group, pretest-posttest design with nonrandom convenience sampling (n = 61; Wall and Rainford 2013). Case-Smith et al. (2012) evaluated the effectiveness of a handwriting and writing program using the pretest-posttest design and found this secondary prevention program to be useful for students (n = 36) who were at risk for handwriting and writing problems. Koller and Stuart (2016) evaluated 21 education intervention programs in Canadian schools based on pretest and posttest findings and concluded that the intervention was a promising approach in reducing negative and prejudicial attitudes of high school students towards others.

One notable advantage of this research design is that the one-group pretest-posttest is economical when there is a very large sample. This feature is important, since the Project P.A.T.H.S. is implemented on a large scale in Hong Kong. In addition, it is simple to derive the findings and it can facilitate the public to better understand research findings (Reichardt 2009). Engel and Schutt (2012) argued that the one-group pretest-posttest design could demonstrate whether improvement gained and how much change occurred at the end of the program, thus the design answered a variety of questions that social workers and social service agencies were interested in. Grinnell et al. (2015) also argued that the design was able to document the results of services. In conjunction with objective outcome evaluation (e.g., Shek and Ma 2012a; Shek and Yu 2012), subjective outcome evaluation (e.g., Shek and Sun 2012b, 2013b) and qualitative evaluation (e.g., Shek and Sun 2012a, d) findings, it can be concluded that the PYD program is a promising approach to promote the development of adolescents.

Some limitations of this evaluation study should be noted. The first limitation is that the effect sizes were small. Two reasons may explain the small effect sizes. First, it may be difficult to observe the immediate effects, since the program had been implemented for just one year. Second, a great proportion of the students were Tier 1 Program participants, and there may be only small increases in their developmental outcomes if they achieved high scores at pretest (i.e., high baseline scores). Nevertheless, given the small dosage in the Tier 1 Program (i.e., 10 h of training), the change can be regarded as impressive. Another limitation is that subgroup analyses were not performed in the study, because we put specific focus on assessing the general changes of students participating in the program. It would be meaningful to test if students’ gender, grade, or other factors contribute to the between-group differences in the future.

The limitations of using one-group pretest-posttest design should be clearly acknowledged. This study did not use a comparison group but comparing students’ developmental outcomes before and after participating in the program. The difference between pretest and posttest may be affected by other factors besides the treatment such as history, maturation, and testing effects which is a limitation of the one-group pretest-posttest design (Reichardt 2009). As suggested by Eckert (2000), scholars should carefully review the plausibility of each threat to internal validity. For examples, maturation may affect students’ changes from pretest to posttest. Some risk behaviors (e.g., alcohol use and teenage sexual intercourse and pregnancy) increase in adolescence and continue to increase into adulthood (Lerner and Galambos 1998). Thus, the effectiveness of an intervention program without the use of a control group may be underestimated in the evaluation process of adolescent risk prevention programs. Besides, students may gain positive experiences in participating in other youth enhancement programs (i.e., history effect). In addition, we compared the changes of positive youth qualities using the longitudinal self-report data and therefore, no causal relationship can be inferred. The absence of a control group should also be noted. Lastly, testing and regression to the mean may adversely affect the internal validity of the study, leading to an overestimation or underestimation of program effects. However, as argued by Thyer (2002), the pretest-posttest design could be useful in program evaluation. In some practical situations, the one-group pretest-posttest design is an appropriate research design providing tentative insights about an intervention.

Despite these limitations, the present study has important implications on the role of positive youth development programs. The study found that the Project P.A.T.H.S. is an important resource that can promote holistic development of adolescents in Hong Kong with the application of positive youth development approach. Enhancing protective factors for adolescents’ positive development is a promising direction for prevention programs to reduce adolescents’ risk behaviors and improve their wellbeing. While acknowledging its intrinsic limitations, the findings of the study demonstrated that a community-based youth programs is a feasible and viable approach. It extends the literature by showing the beneficial effects of a PYD program among Chinese adolescents. Given the existing findings are mostly conducted based on Western samples, the current study is considered to be a breakthrough in youth program research. We recommend that PYD components be included in future practices, especially for adolescents at greater risk for behavioral and psychosocial problems. In future study, more rigorous evaluation strategies for PYD programs are suggested.