Adaptation and Validation the State Mindfulness Scale (SMS) to Physical Education in a Spanish Context

Abstract

The aim of this study is to validate and adapt to the Spanish context of Physical Education (PE) the State Mindfulness Scale (SMS). The study included 1428 students aged 13–19. In order to analyse the psychometric properties of the SMS, an exploratory factorial analysis was carried out to confirm the presence of two factors, a confirmatory factorial analysis that has offered support for both the structure of two factors. We also included an analysis of temporal stability, an analysis of invariance across gender, an analysis of internal consistency and an analysis of predictive validity through a t-test in which mental health (β = −0.28, p < 0.001) and physical or bodily experience (β = −0.16, p < 0.001) negatively predicted anxiety, respectively. The structure of the questionnaire was valid and reliable, gender-neutral and with adequate levels of temporal stability.

1. Introduction

Until now, studies in the field of physical education (PE) affirm that this science is in many cases a source of multiple physical, mental and social benefits for students [1]. However, for this to happen, it is necessary for students to get involved in PE classes, with the most likely being those who do so of their own free will [2]. According to the Self-Determination Theory, the most autonomous forms of motivation are intrinsic motivation, integrated and identified regulation. When students are intrinsically motivated, they tend to be active and participative due to the enjoyment and internal satisfaction derived from the physical, psychological and emotional sensations of the activities carried out in class [3,4]. On the other hand, more controlling forms of motivation are bad predictors of long-term adherence [5], such as participating in class to alleviate negative feelings (e.g., anxiety, shame) or to avoid something negative (e.g., grades, punishments). Mindfulness is a factor that can distinguish those students who are intrinsically motivated from those who are extrinsically motivated because it determines the degree to which students who are mentally involved in the class instead of paying attention to external distractors, irrelevant thoughts or anticipated outcomes while performing different activities [6,7]. However, there is hardly any scientific evidence that tries to measure and analyse in the Spanish context of PE the influence of mindfulness, even though its positive affect on the involvement, continuity or abandonment of PE is evident [8,9]. Thus, this study aims to show evidence of validity and adaptation to the Spanish context of the State Mindfulness Scale (SMS) for Physical Activity by Cox, Ullrich-French and French [10] as a fundamental instrument that could favour the positive affection towards the PE classes, and consequently, the benefits it produces for students.

Mindfulness is defined by Kabat-Zinn, Lipworth and Burney [11] as “consciousness that arises from paying attention, intentionally, to experience as it is presented in the moment, without judging it, evaluating it or reacting to it”. In the field of contemporary psychology, he makes use of mindfulness in order to increase consciousness and skilfully respond to mental processes that generate emotional discomfort and behaviours that turn out to be unadaptive. In this sense, emotional discomfort and/or worries can be “introduced” into our mind while we are performing a certain activity, distracting us from it, assuming a loss of concentration in the task.

Many studies have shown the effectiveness of mindfulness-based techniques, therapies and programs [12]. This state can be developed through the use of meditation, making us less reactive to what is happening now [13]. In this way, the person who practices mindfulness acquires a greater knowledge of himself/herself by accepting the thoughts, emotions and sensations experienced in situ, becoming aware of their volatility and transitory character, abandoning the pattern of judging them and considering them as elements with their own existence [14]. In the educational field, a study by Gallego, Aguilar-Parra, Cangas, Langer and Mañas [15] found that those students to whom a mindfulness program was applied throughout the academic year, reduced their levels of stress, depression and anxiety. Similarly, Kang, Choi and Ryu [16] obtained improvements in stress and anxiety levels, but not in depression, among a group of nursing students. However, the studies on mindfulness in the field of PE are still scarce, and in physical activity suggested that the association between mindfulness with respect to indicators of well-being (e.g., feeling of satisfaction, achievement and enjoyment), demonstrated the motivational relevance of being conscious at the moment [17].

The State Mindfulness Scale (SMS) by Tanay and Bernstein [18] was developed to address some of the existing deficiencies in the evaluation of mindfulness at both the physical and mental levels of the existing scales and questionnaires. Thus, Tanay and Bernstein [18] based on key concepts of mindfulness, such as self-regulation and attitude of acceptance, attention and curiosity developed the new scale. These new elements were thought to be totally integrated within each of the factors that compose the scale. In this way, they developed a scale composed by 21 items with a structure of two factors, being on the one hand, the mindfulness of mental health, and on the other hand, the mindfulness of physical or corporal experience, as well as a general factor of superior order, to reflect the global mindfulness. However, there is a lack of internal structure of validity to support the use of the questionnaire. The higher order model poses methodological problems in this case in statistical terms with serious limitations imposed on the model.

However, this same scale has been adapted to the French context by Cox, et al. [10]. Thus, in order to correct the existing deficiencies in the original questionnaire, eight new items were added and others from the original scale were modified. For the study, they used a sample of 199 subjects aged between 18–77 years with an average age of 34. An exploratory factorial analysis of the questionnaire was carried out eliminating those items with a load lower than 0.5 with respect to the factor to which it belongs and subsequently two factorial confirmatory analyses in order to validate the scale. In addition, a factorial confirmatory analysis of the higher order model was carried out where a one-dimensional mindfulness factor was represented, resulting in a scale composed of 12 items, equally distributed between each of the two factors.

Based on these precedents, the objective of this study was to analyse the psychometric properties of the adapted version of the SMS [10] to the Spanish context, in order to measure the mindfulness of PE students. On the one hand, it is expected that the fit indices of the exploratory and confirmatory factor analysis of the questionnaire are suitable for a two-factor, higher order model, and that these models are invariant with respect to gender. On the other hand, the different subscales are expected to present adequate validity indices through the analysis of internal consistency and adequate temporal stability. Finally, taking into account the results of previous studies that have negatively related mindfulness with respect to anxiety in the educational sphere [9,19], this same relationship is expected to be found in the sphere of PE classes, thus demonstrating the predictive validity of the scale.

2. Method

2.1. Participants

This study included 1428 secondary school students (711 men and 717 women) aged between 13 and 19 (M = 15.30; SD = 1.15) from the province of Almería (Spain). For the exploratory factorial analysis, 437 subjects (232 men and 205 women) with ages between 14 and 18 years old took part (M = 15.87; SD = 1.07). For the analysis of temporal stability, 157 subjects (82 men and 75 women) with ages between 14 and 16 years old took part (M = 15.1; SD = 0.67).

The sampling used was incidental non-probabilistic, based on those educational centres and students to which access was obtained.

2.2. Instruments

Mindfulness during PE. In order to measure mindfulness towards PE classes, the State Mindfulness Scale (SMS) by Cox, et al., [10] was validated and adapted. This questionnaire consists of 12 items equally distributed between two factors: mental health and physical or body experience (e.g., “I realized that thoughts come and go”). Subjects had to indicate their response by means of a Likert scale of 0 (not at all) to 4 (very much). In the original scale, an alpha of 0.87 was obtained in the mental health factor and 89 in body and physical experience (see, Appendix A).

Anxiety during PE. We used the factor with the same name from the instrument Achievement Motivation in Physical Education Test [20] using the Spanish version validated by Ruiz, Graupera, Contreras and Nishida [21]. This factor is made up of 15 items. The students had to respond by means of a Likert scale from 1 (totally disagree) to 5 (totally agree). The reliability of this factor in the original scale was 0.84.

2.3. Procedure

In order to validate the questionnaire in the Spanish context, the reverse translation strategy was used [22]. This process consists of the original questionnaire being translated into Spanish by a group of expert translators and then translated into the original language by another group. The goodness of the translation was judged according to the degree of coincidence with the original version. The obtained version was analysed by three experts [23] in PE and Mindfulness, in a way that it was guaranteed that the obtained items were well designed to measure the construct that wanted to be measured, without losing the original meaning.

Once the questionnaire was obtained, several educational centres in the province of Almería were contacted and informed of the objective of the research and asked for their collaboration. The subjects were required to have an authorization for their participation. Before administering the scale to all participants, it was completed by a small group of students (N = 14) to ensure the correct understanding of all items. The administration of the questionnaire was carried out under the supervision of an expert interviewer who was a member of the research group that explained and solved the doubts that arose when filling it out. The estimated time to complete the questionnaire was around 15 min.

2.4. Data Analysis

In order to determine the validity and reliability of the SMS in the Spanish context, the psychometric properties of the questionnaire were analysed. First, an exploratory factorial analysis (EFA) and then a confirmatory factorial analysis (CFA) were performed to test the factorial structure. Secondly, multi-group analysis was carried out to analyse gender invariance. Descriptive statistical analyses were then performed, and the reliability of the instrument was tested through the internal consistency analysis (Cronbach alpha) and a temporal stability analysis (intra-class correlation index, ICC). SPSS 24.0 and AMOS 21.0 statistical packages were used for data analysis.

Since the Mardia coefficient was found to be high (10,415) for the AFC, the maximum likelihood estimation method was used together with the bootstrapping procedure. The estimators were not affected by the lack of normality, so they were considered robust [24]. In order to accept or reject the model tested, a set of fit indices established by Hair, Black, Babin, Anderson and Tatham [25] were taken into consideration: χ²/df, CFI (Comparative Fit Index), TLI (Tucker Lewis Index), RMSEA (Root Mean Square Error of Approximation) plus its 90% confidence interval (IC), and SRMR (Standardized Root Mean Square Residual). Incremental indices (CFI, TLI) show a good fit with values equal to or greater than 0.95, while error indices (RMSEA and SRMR) are considered acceptable with values equal to or less than 0.06.

3. Results

3.1. Exploratory Factorial Analysis

Table 1. Correlations between each item and the total scale score. Reliability.

Items	Correlations Item-Test	Cronbach’s Alpha If the Item Is Deleted	Saturation Factor of Each Item with Its Factor
1	0.80 **	0.92	0.77 F1
2	0.82 **	0.92	0.76 F2
3	0.83 **	0.92	0.83 F1
4	0.81 **	0.92	0.80 F2
5	0.87 **	0.92	0.84 F1
6	0.88 **	0.92	0.80 F2
7	0.85 **	0.92	0.82 F1
8	0.83 **	0.92	0.81 F2
9	0.80 **	0.92	0.85 F1
10	0.81 **	0.92	0.79 F2
11	0.85 **	0.92	0.76 F1
12	0.89 **	0.92	0.83 F2

Note: F1 = Mental Health; F2 = Physical Experience; ** p < 0.001.

shows the correlations between each item and the total score of the scale that was in the overall range between 0.80 and 0.89. These results lead to the maintenance of all items, since the item-test correlation is higher than the cut-off point established at 0.30 [26]. Cronbach’s global Alpha was 0.92. In addition, exploratory factor analysis supports the existence of two factors, showing a saturation factor ranging from 0.76 to 0.85 for mental health and from 0.76 to 0.83 for physical experiences.

3.2. Confirmatory Factorial Analysis

The fit indices of the model tested (Figure 1) revealed appropriate fit indices: χ² (53. N = 1428) = 134.05, p < 0.001; χ²/df = 2.53; CFI = 0.97; TLI = 0.97; IFI = 0.97; RMSEA = 0.06 (90% CI = 0.047–0.073); SRMR = 0.035. Standardized regression weights ranged from 0.74 to 0.84 being statistically significant (p < 0.001). The correlation between the factors was 62 being statistically significant (p < 0.001).

For the higher order model the fit indices revealed appropriate fit indices: χ² (53. N = 1428) = 112.158, p < 0.001; χ²/df = 2.11; CFI = 0.98; TLI = 0.97; IFI = 0.98; RMSEA = 0.051 (90% CI = 0.038–0.064); SRMR = 0.029. There is a relationship between the higher order factor, called Mindfulness, with respect to mental health of 0.81 and physical experience of 0.76.

3.3. Discriminant Validity Analysis, Bivariate and Descriptive Correlations

The HTMT ratio of correlations between latent factors (

Table 2. Descriptive statistics, HTMT and bivariate correlations.

Factors	M	SD	Range	1	2	3
1. Mental Health	3.37	0.67	0–4		0.69 ***	−0.57 ***
2. Physical Experience	3.22	0.65	0–4	0.49		−0.45 ***
3. Anxiety and Stress	1.74	0.54	1–5

Note: The values under the diagonal correspond to the heterorrasge-monorrasge ratio (HTMT) between factors. *** p < 0.001.

) was 0.49, suggesting the existence of discriminant validity. In addition, Table 2 shows the correlation between the two factors being positive, which shows the clear reciprocity between both factors. In addition, the average score was higher for mental health than for physical experiences.

3.4. Analysis of Gender Invariance

In order to check whether the factor structure of the model is invariant to gender, a multi-group analysis was carried out.

Table 3. Gender Invariance Analysis.

Two Factors Model
Models	χ2	df	χ2/df	Δχ2	Δdf	CFI	TLI	IFI	RMSEA (IC 90%)	SRMR
Model 1	232.82	106	2.20			0.96	0.95	0.96	0.053 (0.044–0.062)	0.055
Model 2	246.06	116	2.12	13.24	10	0.96	0.95	0.96	0.051 (0.042–0.060)	0.057
Model 3	252.28	119	2.12	19.46	13	0.96	0.95	0.96	0.051 (0.042–0.060)	0.068
Model 4	306.42	131	2.34	73.60***	25	0.94	0.94	0.94	0.056 (0.048–0.064)	0.065
High Order Model
Models	χ2	df	χ2/df	Δχ2	Δdf	CFI	TLI	IFI	RMSEA (IC 90%)	SRMR
Model 1	193.07	106	1.82			0.96	0.96	0.96	0.062 (0.048–0.087)	0.048
Model 2	203.47	116	1.75	10.39	10	0.96	0.96	0.96	0.065 (0.053–0.089)	0.052
Model 3	205.88	117	1.76	12.81	11	0.95	0.95	0.95	0.070 (0.057–0.085)	0.053
Model 4	213.77	119	1.80	20.69 **	13	0.95	0.95	0.95	0.076 (0.069–0.086)	0.055
Model 5	232.51	125	1.86	77.36 ***	25	0.95	0.94	0.95	0.081 (0.071–0.092)	0.058
Model 6	270.43	131	2.06	99.55 ***	37	0.95	0.94	0.95	0.081 (0.071–0.091)	0.063

** p < 0.01; *** p < 0.01.

displays the various fit indices for each of the models. There were no significant differences between the unrestricted model (Model 1) and the mean weighted invariance model (Model 2) and Model 3 (structurally invariant covariance model). In turn, differences were found between Model 1 (unrestricted model) and Model 4 (invariant model of mean residuals). The lack of significant differences between Model 1 and Model 2 attained the minimum criteria for acceptance of invariance relative to gender [27,28].

In relation to the higher-order model, significant differences were not found between the unrestricted model (Model 1), the weighted mean invariance model (Model 2) and the invariant structural covariance model (Model 3). The results indicated significant differences among Model 1 and Model 4 (mean residual invariant model) as well as with Model 5 and 6. These results provide support as well for invariance across gender for the higher-order model.

3.5. Assessment of Internal Consistency and Temporal Stability

In order to obtain evidence of the scale’s reliability, an internal consistency analysis was performed through the Cronbach test at α where scores were satisfactory, being 0.91 for mental health and 0.90 for physical experiences. As for the analysis of temporal stability, the intra-class correlation coefficients (ICC) and their confidence intervals (CI) were calculated, giving a score of 0.89 (CI = 0.85–0.92) for mental health and 0.91 (CI = 0.87–0.94) for physical experiences.

3.6. Criteria Validity Analysis

In order to analyse the criteria validity of the Mindfulness questionnaire in the field of PE classes, a linear regression analysis was carried out (

Table 4. Linear regression analysis.

Variables	F	R2	β	t
	36.65	0.37 ***
Mental Health			−0.28	−5.75 ***
Physical Experience			−0.16	−3.25 ***

*** p < 0.001.

) by which anxiety was introduced as a dependent variable and mental health factors and physical experiences as an independent variable. The results showed that both factors significantly predict anxiety with a negative regression weight and providing an explained variance of 37%.

4. Discussion

The aim of this study is to adapt and validate the Cox, et al. [10] SMS to the context of PE during the secondary stage. To this end, this study sought to evaluate the psychometric properties of the instrument of the new instrument in the Spanish context by adding a higher-order factorial analysis to demonstrate that the two factors that make up the scale can be grouped into one.

The EFA results have shown the existence of two factors as well as the original scale. In addition, the CFA results of this new scale support the factor structure of two factors with positive correlations between them. The higher order model presents adequate fit indices which will allow the two factors to be grouped into one and allow researchers to simplify the statistical models in future studies where the sample size is reduced. The multi-group analysis showed that the structure of the questionnaire remained invariant with respect to gender, thus allowing the instrument to be used to analyse possible differences between boys and girls in the area of the PE previously identified by the research.

The results of the internal consistency analysis showed reliability values higher than 0.90, improving those obtained by the original questionnaire [10]. In addition, the analysis of temporal stability of the questionnaire showed that both were understood by the population in a similar way over time. On the other hand, the bivariate correlation between latent factors was below 0.85. This fact, with the existence of HTMT proportions in the correlations between the factors that integrated the instrument, were lower than 0.85 suggested that the factors included in this new version of the instrument had an adequate level of discriminant validity.

In accordance with the results of previous studies [9,29,30], the instrument has given evidence of criteria validity, since both mental health and physical experiences were negatively associated with anxiety. In this sense, a study conducted by Carsley, Heath, Gomez-Garibello and Mills [31] with high school students has shown that high levels of mindfulness reduce stress and anxiety levels and benefits their academic performance. Similarly, Langer, Schmidt, Aguilar-Parra, Cid and Magni [32] in their study with high school students showed how those students who participated in mindfulness therapies during the classes reduced their anxiety levels before exams and favored their performance. Finally, a similar study by Bluth, Campo, Pruteanu-Malinici, Reams, Mullarkey and Broderick [33] showed that those high school students who had high levels of mindfulness showed greater resistance to depression and stress. In this way, a full development of the mindfulness level helps the development of emotional and cognitive flexibility, which in turn implies an interaction with attention [34]. Thus, those techniques aimed at the development of mindfulness have a positive impact on emotional regulation and cognitive control, which helps to control the exacerbation of certain emotions, such as anxiety, improving student performance [30]. Thus, mindfulness acts as a predictor of general well-being through its positive associations with positive affection, autonomy, life satisfaction, self-esteem, and well-being and negative associations with respect to anxiety, depression, and negative affection [9].

However, future works will have to analyse in greater depth the robustness of the questionnaires with different populations and academic groups, with the aim of improving the tool available and analysing with greater criteria the mental processes present towards PA. In addition, future works could explore other traits, such as self-efficacy and self-compassion, which have been found to mediate and moderate the effectiveness of mindfulness. Finally, future studies should analyse the predictability of the Mindfulness State scale towards academic performance. Among the limitations of this study is the fact that students’ depression and anxiety were not previously evaluated.

5. Conclusions

In summary, the results of the present study support the usefulness of the new version of the SMS as an instrument that shows evidence of reliability and validity to measure the state of mindfulness in the Spanish context in the field of EF classes, in addition to being invariant with respect to gender. This instrument could contribute to a better understanding to analyse in more depth the mental processes and perceptions that students have during PE classes, positively affecting their affectivity towards these classes, which may encourage people in the future to include healthy habits of physical activity in their lives.

Furthermore, this study highlights the importance of implementing Mindfulness programs at the beginning of PE classes, since it would help the students to focus the experience on the “process” and not so much on the “result” of their actions, allowing them to reach the objectives set in that class session.