Psychometric Properties of the Mindfulness in Teaching Scale in a Sample of Portuguese Teachers

Participants in this study were teachers from the public (83.4%), private (11.8%), and both (4.7%) sectors of the school system in Portugal (n = 863). They were mainly women (82.4%) with a mean age of 50.1 years (SD = 7.8, range = 22 to 67 years), and with a mean of 24.8 years teaching (SD = 8.9, range = 1 to 45 years). They taught in elementary school (21.4%), middle school (38.7%), and high school (39.9%), from all school regions of mainland Portugal, North (31.8%), Center (16.7%), Lisbon region (18.1%), Alentejo (6.8%), Algarve (4.9%), and islands, Azores (14.7%) and Madeira (6%); 1% worked in more than one region. Table 1 shows more details about the sample characteristics.

The study protocol, with all the questionnaires, was submitted and approved by the Ethics Committee of the Faculty of Psychology and Educational Sciences of the University of Coimbra (CEDI/FPCEUC:69/R_5). We had the authorization to disseminate the study from the Portuguese General Department of Education, and the Azores and Madeira Regional Education Departments. For inviting teachers’ participation, the study was then presented, by email, to the Head of the educational organizations with a link to a data collection website (LimeSurvey). The survey link was also shared on social networks, including teacher forums and Facebook pages. The online protocol presented, on the first page, the research objectives, inclusion criteria, and ethical issues concerning the study. Participants were also informed that their participation was voluntary and anonymous. Only teachers who agreed to the terms and conditions of the study protocol completed the survey comprised of questionnaires measuring job satisfaction, mindfulness in teaching, loneliness and positive solitude, self-compassion, positive and negative affect, dispositional mindfulness, and a socio-demographic questionnaire.

The Mindfulness in Teaching Scale (Frank et al., 2016) is a self-report measure comprised of 14 items organized in 2 dimensions: Teacher Intrapersonal Mindfulness (9 items; e.g., “I am often so busy thinking about other things that I am not really listening to my students”) and Teacher Interpersonal Mindfulness (5 items; e.g., “When I’m upset with my students, I notice how I am feeling before I take action”). Items are rated on a 5-point Likert scale ranging from 1 (Never true) to 5 (Always true). After reverse coding the items of the intrapersonal dimension, the total score of each dimension is obtained by summing the respective items, with higher scores indicating greater levels of Teacher Intrapersonal Mindfulness and Teacher Interpersonal Mindfulness. The Portuguese version of the MTS was developed after obtaining authorization from the authors of the original version to translate and validate the scale. First, to help improve the content validity, the original English version of the MTS was independently translated by two Portuguese experts on mindfulness, both in research and training and fluent in English. Then, the two translated versions were analyzed and discussed to get a single version that was subsequently translated back into English by an English-native speaker, fluent in Portuguese. Finally, the original and the back-translated versions were analyzed, with all items achieving semantic equivalence with the original ones. This final version was then pilot-tested with 12 teachers with similar characteristics to those of the target population to test face validity and identify possible doubts, both in the comprehensibility of the items and in the way of completion (cognitive debriefing); this final version was used in the study.

This instrument is an adaptation of the Portuguese version of the Satisfaction with Life Scale (Diener et al., 1985; Portuguese version: Simões, 1992) for teachers (Albuquerque et al., 2021a, b) and was used to assess teachers’ satisfaction with their job (e.g., “I am satisfied with my teacher life”). This 5 items questionnaire is rated on a 5-point Likert scale, ranging from 1 (I completely disagree) to 5 (I completely agree), with a higher total score indicating greater job satisfaction. Both in the original study and in the present study, the SWTPLS showed good reliability (α = 0.83) and (α = 0.89), respectively.

The Portuguese version of the MAASe (Brown & Ryan, 2003; Portuguese version: Gregório & Pinto-Gouveia, 2013) was used to assess teachers' mindfulness in their everyday life. The MAAS contains 15 items addressing cognitive, emotional, physical, interpersonal, and general domains with a single total score, rated on a 6-point Likert scale, ranging from 1 (Almost always) to 6 (Almost never), with higher scores indicating greater mindfulness (e.g., “I could be experiencing some emotion and not be conscious of it until some time later”). The original MAAS has shown good reliability (α = 0.87). Similarly, the Portuguese version of the MAAS has demonstrated excellent internal consistency (α = 0.90), and in the current study, was also excellent (α = 0.92).

The Positive and Negative Affect Schedule (Watson et al., 1988; Portuguese version: Simões, 1993) was used to measure teachers’ positive and negative affect. The version of the PANAS used in this study consists of the original 20 items, divided into two subscales, each with 10 items for positive affect (e.g., “interested”, “excited”, “active”) and 10 items for negative affect (e.g., “distressed”, “upset”, “irritable”). Items are rated on a 5-point Likert scale, ranging from 1 (Very slightly or not at all) to 5 (Extremely). The negative affect and the positive affect subscales are considered independent dimensions and rated separately. Higher values indicated higher negative or positive affect. The Portuguese version of the PANAS has shown good reliability in a previous study with teachers (Albuquerque et al., 2012), (PA, α = 0.82; NA, α = 0.85), and in the present study was good (PA, α = 0.89; NA, α = 0.89).

The Loneliness and Positive Solitude Scale (Chiodelli, 2021) was used to assess teachers’ feelings of loneliness or positive solitude when being alone. It is a bi-dimensional self-report measure consisting of 10 items: one 5-item subscale measuring loneliness (e.g., “Spending time with myself is unsatisfactory because I would like to be with other people”), and one 5-item subscale measuring positive solitude (e.g., “Spending time with myself helps me to look at my projects more creatively”). The scale is rated on a 5-point Likert scale, ranging from 1 (Never) to 5 (Always) with higher scores in the loneliness dimension indicating a greater aversion to being alone, and higher scores in the positive solitude dimension a greater perspective of being alone as something important and necessary. The LPSS revealed good reliability (loneliness, α = 0.79, positive solitude, α = 0.85) and construct validity in the original study, and in the present study reliability was good (loneliness, α = 0.80; positive solitude, α = 0.85).

Teachers' levels of self-compassion were assessed by the Portuguese version of the Self-Compassion Scale – short form (Raes et al., 2011; Portuguese version: Castilho et al., 2015). The SCS-SF has 12 items (e.g., “When I fail at something important to me I become consumed by feelings of inadequacy”) measuring six components of self-compassion (i.e., self-kindness, self-judgment, common humanity, isolation, mindfulness, and over-identification), that are rated on a 5-point Likert scale, ranging from 1 (Almost never) to 5 (Almost always). Self-compassion scores are obtained after reverse coding negative items and estimating the mean of the 12 items, with higher scores indicating higher self-compassion. The original scale of the SCS-SF has shown good psychometric properties, with good reliability (α = 0.86) and construct validity. The Portuguese version of the SCS-SF has shown good reliability (α = 0.89), temporal stability (r = 0.78), and convergent validity. In the present study, reliability was good (α = 0.89).

All analyses were performed with the software program JASP (v. 0.18.1.0) (https://​jasp-stats.​org). First, descriptive statistics (e.g., frequencies, mean, standard deviation) were used to characterize participants’ demographics and all variables assessed in this study, namely of the MTS-PT items. Skewness (Sk) and Kurtosis (Ku) for all ordinal and quantitative variables revealed an adequate approximation to the normality assumption (Sk <|3| and Ku <|8|, cf. Kline (2016).

Subsequently, a CFA was conducted to test the original MTS factorial structure: Factor 1: Teacher Intrapersonal Mindfulness (Items 1, 2, 3, 4, 5, 6, 7, 8, 9) and Factor 2: Teacher Interpersonal Mindfulness (Items 10, 11, 12, 13, 14). CFA models were fit using the Robust Diagonally Weighted Least Squares (RDWLS) estimator, which accounts for polychoric correlation matrices of Likert-type scales (Li, 2016). To determine model adjustment, multiple fit indices were used as indicated by Byrne (2016) and Kline (2016): the Chi-Square test (χ²; ideally non-significant); the Chi-Square Critical Ratio (χ²/df < 5, ideally < 3); the Comparative Fit Index (CFI) and the Tucker-Lewis Index (TLI), both above 0.95; and the Root Mean Square Error of Approximation (RMSEA < 0.09). The reliability values were computed by Cronbach’s alpha and McDonald's omega, following the recommendations of Murphy and Davidshofer (1988). The two competing models (Model 1 and Model 2) were compared using the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC), with preference being given to the model with the lowest AIC and BIC value (Kline, 2016).

We then conducted multiple-group CFA to determine whether the best fitting model was invariant across teachers’ sex (men versus women), teaching years (until 25 years versus more than 25 years) and educational levels taught (elementary school versus middle school versus high school). Both configural, metric and scalar invariance were tested. The existence of invariance was stablished by comparing the CFI and RMSEA indexes between the models, where differences less than 0.01 in CFI (ΔCFI ≥ 0.010) or less than 0.015 in RMSEA (ΔRMSEA ≤ 0.015), indicate invariance compared to the less-restrictive model (Chen, 2007).

Further, inferential statistical tests (t-tests, ANOVAs) were performed to determine whether the levels of the total scores of MTS-PT dimensions (intrapersonal and interpersonal mindfulness in teaching) were statistically different between groups, considering participating teachers’ sex, teaching years and education level taught. Independent sample t-tests were used between men and women teachers and between teachers with less experience (until 25 teaching years) and the ones with more experience (more than 25 teaching years). Cohen’s d was used to assess effect size, with values of 0.20, 0.50, and 0.80 representing small, medium, and large effects, respectively (Cohen, 1988). A one-way ANOVA was conducted to compare differences on the MTS-PT scores considering three educational levels taught by the participating teachers: elementary school, middle school, and high school. Eta squared was used as a measure of effect size, following Cohen’s (1988) criteria: small effect (η² < 0.01), medium effect (η² between 0.02 and 0.06) and large effect (η² > 0.14).

Finally, to examine the convergent and divergent validity of MTS-PT and external correlates, we computed Pearson’s correlation coefficients between the total scores obtained with the dimensions of the MTS-PT scale with teachers’ job satisfaction, self-compassion, positive and negative affect, loneliness and positive solitude, and dispositional mindfulness. Pearson’s correlation coefficients were interpreted according to Cohen's (1988) criteria, with 0.10 ≤ r < 0.30, 0.30 ≤ r < 0.50, and r ≥ 0.50, being considered small, medium, and large correlation, respectively.

In Table 2, we present the descriptive statistics of the MTS-PT items, with mean values between 3.42 (Item 14) and 4.32 (Item 11). Since the response range was between 1 and 5, mean values were above the theoretical center of the scale. The lowest means were observed in items 13 and 14, whose values came close to 3.5. Items 10 and 11 revealed a slight trend to skewness (−1.072 and −1.064, respectively) and kurtosis (1.174 and 1.719, respectively).

We conducted a CFA to examine the factor structure of the MTS-PT using the scores from both intrapersonal and interpersonal dimensions. Considering the original and previous validation studies of the MTS, two factor models for the internal structure of the MTS-PT were tested and compared: (a) Model 1 (1-factor model), and (b) Model 2 (2-related-factor model). As shown in Table 3, Model 1 reveals a less-than-desired fit: χ²/df value of 6.76, which is higher than 5, indicates a poor fit; the CFI and the TLI with values of 0.92 and 0.93, respectively, both below the cut-off of 0.95; only the RMSEA value of 0.075, which is lower than 0.09, presented an adequate fit. The next step was to test Model 2. This model achieved a good fit: χ²/df of 3.28, which is lower than 5, but still not achieving the ideal value below 3; the CFI and TLI values of 0.96 and 0.97, respectively, both above 0.95; and the RMSEA value of 0.051.

Then, the two competing models (Model 1 and Model 2) were compared. Based on the AIC and BIC values, Model 2 presents a better fit than Model 1 (AIC_Model2 = 27,776.467 < AIC_Model1 = 28,092.403) and (BIC_Model2 = 27,981.165 < BIC_Model1 = 28,292.340).

In Fig. 1, we present the path diagram of the 2-factor structure with a positive and moderate correlation between the 2 factors of 0.42, which suggests that despite their relationship, these 2 dimensions are relatively independent of each other. As demonstrated in Table 4, standardized weights ranged from 0.08 (Item 12) to 0.80 (Item 11), both from the interpersonal dimension. Item 12 was explained by the interpersonal dimension, but with a very low percentage of explained variance (0.08). We provide, in Supplementary Material, the fit of a model and the loadings of all items, if Item 12 is omitted from the scale.

We conducted multiple group CFA to examine whether the best fitting model for MTS-PT internal structure, the 2-related-factor model, was invariant across teachers’ sex (two levels: men and women), teaching years (two levels: until 25 years and more than 25 years) and educational levels taught (three levels: elementary school, middle school, and high school).

Goodness-of-fit indexes supported configural invariance of the model across sex (Table 5). Then, metric invariance or weak invariance was also tested and showed a good fit to the data and a minimal change on fit indices (ΔCFI = –0.005; ΔRMSEA = 0.003) in comparison with the configural invariance model. Scalar invariance or strong invariance was also supported, with a good model fit and a minimal change on fit indices (ΔCFI = 0; ΔRMSEA = –0.003) in comparison to the metric invariance model.

Concerning teaching years and education level taught by the participating teachers (Table 5), goodness-of-fit indexes also supported configural, metric and scalar invariance of the model.

We conducted an independent sample t-tests to examine whether the levels of the total scores of MTS-PT intrapersonal and interpersonal dimensions were statistically different between men (n = 152) and women (n = 711) teachers, and between teachers with up to 25 years of teaching experience (n = 459) and the ones with more than 25 years of teaching experience (n = 402). Results revealed no statistically significant differences between men and women teachers regarding both levels of intrapersonal and interpersonal mindfulness in teaching.

Concerning differences in levels of mindfulness in teaching between teachers with up to 25 years and teachers with more than 25 years teaching, results revealed a statistically significant but small difference (t[859] = –4.21, p < 0.05, Cohen’s d = –0.29) in the intrapersonal dimension, with higher values for more experienced teachers (M = 37.17, SD = 4.95) in comparison with less experienced ones (M = 35.67, SD = 5.42). No differences were found regarding the interpersonal dimension.

Finally, regarding levels of intrapersonal and interpersonal mindfulness in teaching across teacher’s educational levels taught (elementary school, n = 185; middle school, n = 334; and high school, n = 344), results revealed no statistically significant differences for the intrapersonal dimension. Regarding the interpersonal dimension, statistical significant but small differences were found (F_{(2, 860)} = 5.081, p < 0.05, η² = 0.01). Post hoc comparisons using Tukey HSD test indicated that the mean score for the interpersonal dimension of the elementary school teachers (M = 19.96, SD = 2.77) was higher than the mean score found both for middle school teachers (M = 19.33, SD = 2.83) and for high school teachers (M = 19.16, SD = 2.86). No differences were found between middle school teachers and high school teachers in respect to the interpersonal dimension.

We estimated scale reliability by computing both Cronbach's α and McDonald's ω, and inter-item correlations. The reliability for the intrapersonal subscale was good (α = 0.86, ω = 0.86), with an inter-item correlation of 0.409, and for the interpersonal subscale was acceptable (α = 0.61, ω = 0.61), with an inter-item correlation of 0.251. The elimination of Item 12 would marginally improve the internal consistency of the interpersonal subscale (α = 0.64, ω = 0.64).

The associations between MTS-PT dimensions and external correlates are presented in Table 6. Overall, we found that Teacher Intrapersonal Mindfulness subscale correlations were all statistically highly significant and in the small to large strength range. Specifically, results revealed a strong positive correlation between Teacher Intrapersonal Mindfulness and dispositional mindfulness (r = 0.54, p < 0.001), and moderate positive correlations with self-compassion (r = 0.43, p < 0.001) and positive affect (r = 0.33, p < 0.001), and a moderate negative correlation with negative affect (r = –0.39, p < 0.001); other correlations were weak. In contrast, Teacher Interpersonal Mindfulness correlations were weak, except for a moderate positive correlation with positive solitude (r = 0.30, p < 0.001).

This study has some limitations that should be addressed. First, the test-retest reliability of the MTS-PT was not carried out since the assessment protocol was only administered once. Examining the test-retest reliability of the MTS-PT should be considered in future studies. Second, since our study explored correlations between the MTS-PT and a unidimensional measure of dispositional mindfulness (MAAS; Brown & Ryan, 2003), upcoming studies could include measures capturing different facets of mindfulness (e.g., observing, describing, acting with awareness, non-judging, and non-reactivity – the Five Facet Mindfulness Questionnaire; Baer et al., 2008). These investigations may help to identify which specific facets of mindfulness are particularly relevant in the teaching context and provide pertinent information for the development of targeted MBI for teachers. Third, further investigations could examine the MTS-PT suitability among teachers working in Portuguese-speaking countries like Brazil and African countries. Fourth, given that Item 12 obtained poor statistical results, we suggest that future research investigates further this item statistics by reformulating or improving its translation. Fifth, longitudinal research designs linking mindfulness in teaching with teachers' and their students' pedagogical and developmental outcomes are warranted. In such studies, the differential analysis of the intrapersonal and interpersonal dimensions of mindfulness in teaching will be valuable to ascertain the specific effects of each dimension. Finally, we would like to recommend for future research to review and improve the construct of mindfulness in teaching from a theoretical point of view as well as the operationalization of the interpersonal dimension, for which it is very important to obtain qualitative data on the experience of these processes (Bergomi et al., 2013).

To conclude, the results of this study revealed that the MTS-PT is a psychometrically adequate self-report measure to evaluate mindfulness among Portuguese teachers. We hope that our research will lead to a widespread of the MTS-PT in the assessment of teachers’ mindfulness and contribute to future studies on mindfulness in education.