Introduction
Participation in social roles and activities (or social participation) is an important determinant of health [
1]. Specifically, participation restrictions are negatively associated with quality of life outcomes, social inclusion, and successful aging [
2‐
4]. In addition, higher levels of social participation have been found to protect against physical and mental illnesses and facilitate recovery from disease [
5]. The concept of participation was introduced by the World Health Organization as: “an individual’s ‘involvement in life situations’, where participation is defined in relation to an individual’s health condition, body functions and structures, activities and contextual factors” (International Classification of Functioning Disability and Health (ICF) [
6]). These life situations include interpersonal relationships, major life areas (e.g., employment), as well as recreation, leisure and community life [
6]. With the growing number of chronic conditions and longevity, optimizing opportunities for social participation is increasingly being called for [
7]. To be able to develop and evaluate interventions that can improve social participation, a measurement instrument is required that takes into account the diversity and dynamic nature of the construct social participation, applicable to individuals with and without (chronic) health conditions. However, social participation is difficult to measure as it involves a diversity of subdomains (such as work and socializing), which relevance may vary between individuals and over time [
8]. The meaning of participation often seems to depend on the purpose of a given clinical or research program [
9]. Furthermore, there is a lack of consensus on the definition of participation and available measures reflect the varying conceptual differences in constructs being measured [
10].
The Patient-Reported Outcomes Measurement Information System (PROMIS®) item bank v2.0 ‘Ability to Participate in Social Roles and Activities’ measures social participation across health conditions and settings [
11]. It was developed based on Item Response Theory (IRT) [
11]. An advantage of IRT is that subsets of items can be administered as short forms or as a Computerized Adaptive Test (CAT). In a CAT, the successive items are chosen based on given answers to previous items, enabling individuals only to respond to a minimal number of relevant items, making it less burdensome to fill in. Also, in IRT-based item banks, items can be removed or added without changing the underlying metric and the adapted item bank maintains comparable with scores using older versions of the measurement instrument [
12]. This is a significant advantage over measurement instruments based on classical test theory [
13], where making changes can have important consequences to the interpretation of the measurement instrument. Being able to make changes while maintaining comparability with older versions of the instrument is helpful for the measurement of social participation. For example, social innovations such as internet-enabled communication can have an impact on how restrictions in interpersonal relationships are experienced [
14].
An overview of the characteristics of the PROMIS® item bank ‘Ability to Participate in Social Roles and Activities’ is presented in Table
1. Validation of the item bank is ongoing and recommended through both qualitative and quantitative efforts [
15]. The item bank has been applied in different settings (e.g., rheumatoid arthritis [
16], heart failure [
17], and abdominal surgery [
18]). The item bank was also translated into Dutch-Flemish [
19]. Sufficient psychometric properties of this item bank were found in the Dutch general population [
20]. However, previous research into the meaning of participation [
8] indicated that the current item bank does not cover all subdomains that were found important by the general Dutch population, which may hamper the content validity and measurement precision of the item bank. Items were missed within the following ICF domains: domestic life, interpersonal relationships, economic life, recreation, community life, and social and civic life [
8]. Previous research supports that participation constitutes a variety of subdomains [
21‐
23]. Previous research has also found the PROMIS® item bank did not cover all subcategories of the ICF [
24]. Moreover, Tucker et al. concludes that mapping between PROMIS® and the ICF helps clarify measurement opportunities and that may lead to improved, comprehensive health outcome measures. Furthermore, the study by Terwee and colleagues found that a substantial number of adults scored the highest level of participation, indicating a ceiling effect [
20]. This means that when participating at a high level, an improvement in participation cannot be measured. The quality of the item bank may therefore benefit from adding items that cover all subdomains of participation meaningful to people with an without (chronic) health conditions, as well as from adding items that are relevant for people with a high level of participation. In order to address these issues, the purpose of the present study was to generate items to be added to the existing item bank and to evaluate the content validity of the proposed extension of the item bank.
Table 1
Characteristics of the PROMIS® item bank ‘Ability to Participate in Social Roles and Activities’
PROMIS domain | Social Health |
PROMIS subdomain | Social Function |
Developers | PROMIS® Social Health Workgroup [ 9, 11, 15] |
Target population | Healthy people, as well as those with a range of physical and mental health conditions |
Number of items | 35 items covering 6 ICF subdomains (i.e., household tasks, assisting others, informal social relationships, family relationships, work and employment, and socializing) [ 11] |
Wording | All 35 items are worded in terms of perceived restrictions, e.g., “I have trouble doing my regular daily work around the house” |
Response categories | 5-point Likert response scale (ranging from ‘never’ to ‘always’). Responses are reverse-coded, so that high scores represent a high level of participation |
Time frame | No time frame is included in the items |
Discussion
In the present study we aimed to generate items for subdomains of participation that are currently missing in the PROMIS® item bank v2.0 ‘Ability to Participate in Social Roles and Activities’ and evaluated their content validity. In total 17 items, covering 17 subdomains, were proposed to be added. Overall, the proposed items were perceived to be relevant and comprehensible, and the final item list was perceived to be comprehensive, as indicated by content experts and participants from a purposeful sample of people with and without (chronic) health conditions. The results therefore preliminarily support the addition of these items to the original item bank for further psychometric testing. With this qualitative study, a first step is taken towards an item bank that covers all (ICF) subdomains relevant to the general population, including items relevant for adults with a high level of participation. To our knowledge, we are the first to propose additional items to this item bank.
According to the content experts, participating in social media is an indication of an individual’s degree of social participation and more than just a way of interpersonal interaction, as the participation in social media is easy and passive and cannot always be replaced by other ways. This was supported by the participants as they found this subdomain relevant. Participation in social media was therefore added as an important subdomain of participation in social roles and activities. It has become an integral part of maintaining interpersonal relations [
14]. As explained in the introduction, the added value of IRT-based item banks is that items, such as (technological) developments/ social innovations that are important for social participation, can be added, and items that become less relevant over time can be removed. One may question whether social media is a way of maintaining interpersonal relations rather than a subdomain of participation itself. Most people are nowadays very active with digital resources and the use of social media is becoming an essential part of our communication/ social connectivity. Further research should indicate whether participation in social media is indeed a relevant and valuable addition to the item bank.
Methodological considerations
The distinction between functions, activities and participation is not always straightforward. They interact with each other, which is also depicted in World Health Organization’s ICF model [
6]. This ‘issue’ was also experienced in the expert discussion with regard to the stem formulation of the items. To illustrate, doing an activity is different from participating in an activity (e.g., doing a sport or participate in a sports club). This distinction was therefore considered during the development of the items. This was also one of the reasons to adhere to the stem formulations used in the original PROMIS® item bank, with the aim that the proposed items measure the same underlying construct. Moreover, it is known that a fixed formulation facilitates participants to complete a questionnaire [
31]. Additionally, the item formulations in the original Dutch item bank showed sufficient content validity [
19,
20].
We have identified additional subdomains and proposed new items that, based on our previous qualitative study and this study [
8], are considered part of the construct “participation in social roles and activities (or social participation)”. This may potentially improve the measurement of social participation in the Dutch general population. However, further testing should show whether the new items indeed improve the psychometric properties of the item bank and, thus, should be added to the item bank [
9]. To illustrate, the quantitative study of Hahn et al. showed that while the items were designed to measure the same construct, the IRT test results were not consistent with model expectations [
9]. In line with these results, not all of our new proposed items may fit the IRT model. It needs to be assessed whether the full set of items still measures one single construct, and IRT model fit and content validity should be balanced to make a decision on including new items in the item bank.
We have used the ICF as a framework to organize the subdomains of participation and as a reference to label and formulate newly generated items. In line with PROMIS®, the ICF concept of participation started from health and not from the concept of participation. The definition of participation in the ICF model is ‘involvement in a life situation’ [
6] and the definition of participation in the PROMIS® item bank is ‘the perceived ability to perform one’s usual social roles and activities’ [
9]. Previous research has found that the subcategories of the ICF appear to be more exhaustive than the PROMIS® item bank [
24]. The PROMIS® concept of role participation does not completely align with the ICF concept of participation. Based on previous work of Bruijning et al. and Elsman et al. on the development of participation questionnaires using the ICF model as starting point [
32,
33], we assume that maintaining romantic relationships, work and controlling finances are common-related social roles (being a partner and employee) and activities (working, controlling finances). Their questionnaires for (young) adults with vision impairment includes items on romantic relationships, managing finance/ allowance and work, and showed that these were unidimensional scales with sound psychometric properties [
32,
33]. However, if an item (subdomain) was not considered relevant in the current study, the item was removed. This is illustrated by the study of De Wind et al., in which it appeared that religion and spirituality (ICF code d930) was considered a subdomain of participation [
8]. However, based on the results of the current study there was insufficient evidence to propose an item on religion and spirituality to the PROMIS® item bank.
Another consideration is that although the ICF classification system provides a basis for identifying subdomains and levels of participation [
34], the translation of the description of the ICF category into item formulations sometimes made the items too complex, indicated by both the content experts and the interviewed adults. Therefore, for some of the items simplified versions of ICF’s category descriptions had to be formulated. For example, the item ‘I have trouble taking care of household and personal objects including animals, plants, and furniture’ (item 6, Supplemental Material 1) was simplified into ‘I have trouble taking care of my household’ (item 3, Supplemental Material 6). It may be questioned whether people give the same interpretation to the simplified item, and therefore whether the specific ICF domain is being measured.
The proposal of adding only one item per subdomain to the original item list was considered most appropriate for further testing, mainly because of feasibility reasons. Adding multiple items per subdomain for further testing may have had the potential advantage that the item with the best fit to the IRT model would end up in the final revised item bank. Furthermore, an adequate balance across content of the extended item bank should be further investigated, to retain a representative group of items in each subdomains in the final version of the item bank. Such ‘content balancing’ is preferred especially in an item bank, i.e., that subdomains are represented proportionally, preferably also in the CAT version [
35].
One of the strengths of PROMIS® item banks is the fact that they are applicable to everyone. Therefore, in the current item bank the question on participation in paid work is combined with a question on participation in unpaid work. However, the study by De Wind et al. found that paid work has a different meaning than unpaid work and should thus ideally be measured separately with the expense of not being relevant for everyone [
8]. Furthermore, both content experts and participants suggested that it might be advantageous if future study participants could indicate which subdomain of participation they find most relevant. Based on IRT methods, it would be possible to create subdomain-specific short forms. These short forms could still be scored on the same metric as the complete item bank or any other short form from the same item bank. Alternatively, some item banks (e.g., PROMIS® Sexual Function and Satisfaction item banks) include screener questions to select relevant items [
36]. With regard to social participation, both strategies should be further investigated in future work.
Strengths and limitations
The strength of this study is that we used qualitative methods to evaluate content validity and received direct input from experts and adults sampled as potential users of the item bank.
Some limitations to our study should be considered. The group of participants in which the proposed items were evaluated consisted of only 10 participants, which is small according to the PROMIS® guidelines [
25]. However, this does comply with the COSMIN guidelines [
27]. We may not have reached complete data saturation. However, as shown in Supplemental Material 5, limited new information emerged in the last interviews, including comments on the stem formulation of the items. The comments on the stem formulation were irrelevant as we adhered to PROMIS® stem formulations and the remaining comments were insufficient to schedule additional interviews in our opinion. In addition, we did not balance the sample for age and gender. Moreover, half of the participants included in the interviews experienced disabilities resulting from diabetes. Unfortunately, and despite great effort, a participant with low literacy was not found. Translation of our results to people with low literacy may therefore be limited. Another limitation is that the present study was conducted among Dutch speaking individuals, who participate in Dutch society. The proposed items should therefore not be added directly to other language-versions of the item bank. Whether our results apply to other languages, cultures, or settings should be subject of further research.
Implications/relevance
The present study provides new insights in the conceptualization and operationalization of the concept of participation in social roles and activities. Although the proposed item list cannot be added to the original item bank yet, we did show that the proposed items are considered relevant for measuring participation in social roles and activities by content experts and a sample of the targeted study populations.
Future directions
The added value of the proposed items now needs to be further tested in a large-scale field study, including IRT analyses. It is recommended to investigate whether the items improve the psychometric properties of the current PROMIS® v2.0 item bank, in terms of validity, measurement precision, and number of items required in a CAT. When studying the psychometric properties of the proposed items special care should be taken to verify whether the items still measure the same single construct, for example by comparing the results of confirmatory factor analysis and local dependence of the item bank with and without the proposed new items.
Acknowledgements
This study was funded by the Societal Participation and Health (SPH) program of the Amsterdam Public Health research institute, Amsterdam, The Netherlands (grant number not available). The authors also thank the physician assistant of the Department of Rehabilitation Medicine, Amsterdam UMC, location AMC, The Netherlands, and Guido Williams, Dimence group, Dimence foundation Specialized Assessment and Treatment Division, Department of Digital Mental Healthcare, Deventer, The Netherlands, for their help with recruiting participants. The authors also thank the content experts of SPH and participants for their participation in the group discussion/interview.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.