Figures
Abstract
Aim of the study was the development and validation of the psychometric properties of a six-item bi-factorial instrument for the assessment of social support (emotional and tangible support) with a population-based sample. A cross-sectional data set of N = 15,010 participants enrolled in the Gutenberg Health Study (GHS) in 2007–2012 was divided in two sub-samples. The GHS is a population-based, prospective, observational single-center cohort study in the Rhein-Main-Region in western Mid-Germany. The first sub-sample was used for scale development by performing an exploratory factor analysis. In order to test construct validity, confirmatory factor analyses were run to compare the extracted bi-factorial model with the one-factor solution. Reliability of the scales was indicated by calculating internal consistency. External validity was tested by investigating demographic characteristics health behavior, and distress using analysis of variance, Spearman and Pearson correlation analysis, and logistic regression analysis. Based on an exploratory factor analysis, a set of six items was extracted representing two independent factors. The two-factor structure of the Brief Social Support Scale (BS6) was confirmed by the results of the confirmatory factor analyses. Fit indices of the bi-factorial model were good and better compared to the one-factor solution. External validity was demonstrated for the BS6. The BS6 is a reliable and valid short scale that can be applied in social surveys due to its brevity to assess emotional and practical dimensions of social support.
Citation: Beutel ME, Brähler E, Wiltink J, Michal M, Klein EM, Jünger C, et al. (2017) Emotional and tangible social support in a German population-based sample: Development and validation of the Brief Social Support Scale (BS6). PLoS ONE 12(10): e0186516. https://doi.org/10.1371/journal.pone.0186516
Editor: Rüya-Daniela Kocalevent, University Medical Center Hamburg-Eppendorf, GERMANY
Received: September 16, 2016; Accepted: October 3, 2017; Published: October 12, 2017
Copyright: © 2017 Beutel et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The analysis presents clinical data of the Gutenberg Health Study (GHS), a large-scale population-based cohort with ongoing follow-up examinations. This project constitutes a major scientific effort with high methodological standards and detailed guidelines for analysis and publication to ensure scientific analyses of highest quality. Due to the Statutes and Rules of Procedure of the GHS, data are not made available for the scientific community outside the established and controlled workflows and algorithms. To meet the general idea of verification and reproducibility of scientific findings, we offer access to data at the local database in accordance with the ethics vote upon request at any time. The GHS steering committee convenes once a month and decides on access of internal and external researchers and use of the data and biomaterials based on a research proposal. Interested researchers make their requests to the Coordinating PI (Univ.-Prof. Dr. Philipp S. Wild, MSc) or the Administrative Head of the Gutenberg Health Study (Dr. Dirk Traiser). More information and contact details are available on the homepage of the GHS (www.gutenberghealthstudy.org).
Funding: The Gutenberg Health Study is funded by the government of Rhineland-Palatinate ("Stiftung Rheinland-Pfalz für Innovation“, contract AZ 961-386261/733), the research programs “Wissen schafft Zukunft” and “Center for Translational Vascular Biology (CTVB)” of the Johannes Gutenberg-University of Mainz, and its contract with Boehringer Ingelheim and PHILIPS Medical Systems, including an unrestricted grant for the Gutenberg Health Study.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The presence of social relationships has been associated with a great number of mental and physical health outcomes. Social ties have a beneficial influence on the maintenance of psychological well-being and health-promoting behavior [1]. Structural features of social integration comprise of characteristics such as marital status, living alone or the size of a person’s social network. Functional parameters usually refer to the perceived, respectively anticipated support from the social environment [2, 3]. Loneliness refers to a discrepancy between desired and available social relationships [4], a perceived sense of isolation [5], separateness or exclusion [6].
Following the seminal work of House et al. [7], a recent meta-analysis of 148 studies (totaling up to over 308.000 participants) found a 50% increased likelihood of survival in participants with stronger social ties taking into account initial health status, cause of death, and duration of follow-up. This was true for social integration and for social support, respectively its lack in loneliness [8]. Another meta-analysis [9] found that low social participation, less frequent social contact and more loneliness, but not size of the social network were statistically associated with incident dementia with similar rates of likelihood.
There are two models explaining the association of social relationships to health outcomes. The main effects model proposes beneficial cognitive, emotional, behavioral and biological influences of social relationships, such as providing stimulation and activity, stimulating brain activity, modeling health behavior, or promoting self-esteem or purpose in life by providing meaningful social roles. The buffering hypothesis proposes that social relationships provide emotional, informational or tangible resources that promote adaptation in case of stress or adversity [10, 11].
Many scales have been proposed to assess social support in the last decades, e.g. Duke-UNC Functional Social Support Questionnaire [12], Oslo Social Support Scale [13], Sarason’s Social Support Questionnaire [14], Interpersonal Support Evaluation List [15]. One established scale for measuring social support is the MOS (Medical Outcomes Study) Social Support Survey, which was developed based on responses of 2,987 chronically ill adults [16]. Of the 19 original items, one was dropped as it belonged to two scales. 18 items assess four dimensions of support: emotional-informational (emotional support and guidance or advice), tangible support (i.e. material aid or assistance), affectionate support (expressing love and affection), and positive social interaction (provided by individuals with whom it is fun to do things). A number of subsequent studies have also been mostly done on medically ill patients, e.g. American breast cancer patients [17] or Spanish outpatients from a primary care center [18]. Given its easy comprehensibility with a single sentence stem “If you needed it, how often is someone available…”, the instrument lends itself to use in a wide range of populations and age groups. Except for the Australian study of Holden et al. [19], population-based studies, however, have been scarce. One reason limiting its use may be the length of the scale. So far, a number of different scales have been developed, based on the initial item pool of the MOS trial with a range from 4 [20], 6 [19], 8 [17] and 12-item scales [20]. The numbers of factors also varies from 1 [18], 2 [17] to 4 [20]. Yet, even those papers endorsing more than one dimension usually used the total score of the respective scale for validation (e.g. [17, 19]), and no evidence has been presented for differential significance of the respective subscales.
Social support has been found to improve mental health and health-promoting behavior [1]. Most analyses identified the two dimensions of emotional-informational and tangible support [19–21]. There is some evidence that the distinction of both facets, emotional-informational and tangible support, are crucial [1, 22, 23], as affective support and material assistance can be provided by different environmental resources. Emotional-informational support mainly covers psychological components, particularly emotional and cognitive needs. Tangible support on the other hand, is based on practical support, such as material aid and behavioral assistance [16]. So far, an instrument accounting for the need of factorial differentiation of these two facets of social support is lacking. Especially the development of a bi-factorial questionnaire which can be used in population-based studies to collect representative data seems reasonable.
For this reason, the aim of this paper was to develop and validate a Brief Social Support Scale (BS6) based on the MOS Social Support Survey that reliably assesses emotional-informational as well as tangible support. The database was provided by the Gutenberg Health Study (GHS) which is a German population-based, prospective, observational single-center cohort study. To the best of our knowledge, this study is the first validating items of the MOS Social Support Survey including country specific data based on representative data collection. A factor-analytical approach was chosen to extract a bi-factorial brief version of the MOS Social Support Survey. In order to indicate external validity of the scales, socio-demographic data and health-related variables were analyzed which have been associated with facets of social-support [17, 24].
The following hypotheses were tested:
- There are two underlying independent factors representing aspects of social support, namely emotional-informational support and tangible support.
- Emotional-informational support and tangible support are associated with distress (depression, generalized anxiety, social phobia, panic disorder, suicidal ideation), physical and mental well-being, and health behavior (smoking, BMI, physical activity, alcohol consumption).
Methods
Study sample
We investigated cross-sectional data of N = 15,010 participants enrolled in GHS from April 2007 to April 2012 [25]. The GHS is a population-based, prospective, observational single-center cohort study in the Rhein-Main-Region in western Mid-Germany. The study protocol and study documents were approved by the local ethics committee of the Medical Chamber of Rhineland-Palatinate, Germany (reference no. 837.020.07; original vote: 22.3.2007, latest update: 20.10.2015). The primary aim of the study is to evaluate and improve cardiovascular risk stratification. The sample was drawn randomly from the local registry in the city of Mainz and the district of Mainz-Bingen. A multi-method approach was used for recruitment (see also [25]). For this purpose, an algorithm was developed defining time frames for establishing contact with the study participants. Overall, participants were contacted up to three times by postal means and up to 20 times by telephone. Participants who did not participate in the GHS study were asked to take part in a structured interview in order to gain information for non-responder analyses with regard to selection bias. The sample was stratified 1:1 for sex and residence and in equal strata for decades of age. Inclusion criteria were age 35 to 74 years and written informed consent. Persons with insufficient knowledge of German language, or physical and mental inability to participate were excluded. Based on the interim analysis 5.8% were excluded because of the exclusion criteria. The response rate (defined as the recruitment efficacy proportion, i.e. the number of persons with participation in or appointment for the baseline examination divided by the sum of number of persons with participation in or appointment for the baseline examination plus those with refusal and those who were not contactable) was 60.3%. Sociodemographic details of the sample are displayed in S1 Table.
Materials and assessment
The 5-hour baseline-examination in the study center comprised evaluation of prevalent classical cardiovascular risk factors and clinical variables, a computer-assisted personal interview, laboratory examinations from a venous blood sample, blood pressure and anthropometric measurements. In general, all examinations were performed according to standard operating procedures by certified medical technical assistants.
Questionnaires
The MOS Social Support Survey with 20 items was devised to assess social support based on the four dimensions of emotional-informational, tangible, affectionate support und positive social interaction. Based on samples of patients with medical diseases, Moser et al. [17] devised a brief scale with 8 items and two subscales (emotional and instrumental support, 4 items, each). GHS participants filled in part of the MOS assessing emotional-informational and tangible support (9 items). Items are rated from 1 = never, 2 = occasionally, 3 = mostly to 4 = always available. An additional item was assessed to estimate the number of persons the participants can count on in cases of emergency (no one to 3 and more).
Depression was measured by the Patient Health Questionnaire (PHQ-9), which quantifies the frequency of being bothered by each of the 9 diagnostic criteria of Major Depression over the past 2 weeks. Responses are summed to create a score between 0 and 27 points. A PHQ-9 sum score of ≥ 10 was used for the definition of caseness for depression yielding a sensitivity of 81% and a specificity of 82% for any depressive disorder [26].
Generalized anxiety was assessed with the two screening items of the short form of the GAD-7 (Generalized Anxiety Disorder [GAD]-7 Scale [27]). “Feeling nervous, anxious or on edge” and “Not being able to stop or control worrying”. Subjects can answer these items with 0 = “not at all”, 1 = “several days”, 2 = “over half the days”, and 3 = “nearly every day”. A sum score of 3 and more (range 0–6) out of these two items indicates generalized anxiety with good sensitivity (86%) and specificity (83%). Both the GAD-7 and its two core items (GAD-2) have been shown to perform well as a screening tool for all anxiety disorders [28].
Computer-assisted personal interview
During the computer-assisted personal interview participants were asked whether they had ever received the definite diagnosis of any depressive disorder (medical history of lifetime diagnosis of any depressive disorder, MH of Depression) by a physician. The socioeconomic status (SES) was defined according to Lampert’s and Kroll’s scores of SES with a range from 3 to 27 (3 indicates the lowest SES and 27 the highest SES) [29].
Statistical analysis
For the scale development, a factor-analytical approach was used. Item selection was based on exploratory factor analysis. Subsequently, a confirmatory factor analysis for testing the postulated bi-factorial model was conducted.
Descriptive analyses, correlation and regression analyses were performed to validate the subscales with external criteria. Variables were reported as absolute numbers, percentage or mean with standard deviation or medians with 25th and 75th percentiles as appropriate. For comparison of groups Cochran-Armitage Test or Jonckheere-Terpstra Test were performed. All reported p-values correspond to 2-tailed tests. As this part of the study is explorative, no adjustments for multiple testing have been done. P-values are given for descriptive reasons only. Due to the large number of tests, p-values should be interpreted with caution and in connection with effect estimates. Statistical analyses were performed using SAS for Windows 9.4 TS Level 1M1 (SAS Institute Inc.) Cary, NC, USA. For factor analyses, the software R 3.2.3 (package lavaan 0.5–22) was used.
Results
Factor analyses: Exploratory factor analysis and confirmatory factor analysis
Two sets of factor analyses were conducted after splitting the sample in two subsamples by randomization: An exploratory factor analysis with varimax rotation (n = 7,256) and a confirmatory factor analysis (n = 7,255). Table 1 and Fig 1 present the findings from the exploratory factor analysis with varimax rotation. As we were interested in developing a brief scale, we deleted three items with overlapping factor loadings (“to do something enjoyable with 38/.44“, “to have a good time with .56/.46”, “whose advice you really want .65/.39”). Based on 6 items (see Table 1), we identified two independent factors (r = .07) by means of exploratory factor analysis and referring to the scree-test (see Fig 1). Factor 1 explained 35% and factor 2 explained 34% of the variance. The first factor comprises items representing emotional-informational support, the second factor tangible support.
Reliability of the subscales indicated by Cronbach´s alpha was satisfactory: emotional-informational support α = .87, tangible support α = .86, overall α = .86. Model fit was evaluated by using following model fit indices: Chi-square statistic; the comparative-fit-index (CFI) and the Tucker-Lewis Index (TLI) to describe incremental fit; the root mean square error of approximation (RMSEA) was used as an absolute measure of fit. Values of TLI and CFI close to .95 or higher indicate a better fit. RMSEA should be 0.08 or smaller [30]. Confirmatory factor analyses yielded a good fit for the bi-factor-model (see Fig 2): RMSEA .030 (90% CI .023-.038), CFI = .996, TLI = .993, and SRMR = .029. In contrast, the one-factor-model reached an unacceptable fit: RMSEA .127 (90% CI .121-.134), CFI = .929, TLI = .882, and SRMR = .125.
Note: Confirmatory factor analysis with completely standardized solution.
Association of social support and its subscales with demographic characteristics health behavior, and distress
Table 2 gives an overview according to different degrees of social support. Sum scores of social support were stratified by quartiles into low (6–11), moderate (at least occasional support), high (at least mostly supported) to very high (maximum count, always supported).
The majority reported high (55%), followed by very high (27%) support. Fifteen percent reported moderate and only 3% low support. Age did not play a role, however, women reported lower support than men. Participants with low support were at a clear social disadvantage regarding partnership, children, socioeconomic status, lack of employment, and a migration background. They had by far the highest rates of cardiovascular risk (smoking, obesity, hyperlipidemia), physical disease (cardiovascular, diabetes). They had the highest rates of distress (depression, suicidal ideation, GAD, panic attacks, social phobia, depersonalization, sleep disorder, Type D personality) and the highest intake of analgetics, opioids and antidepressants and most frequently visited physicians.
Emotional-informational and tangible support as a predictor of mental disease
In multivariate analyses we predicted distress by the two subscales of social support. Even when we adjusted for age, sex, CVD, CVD risk, SES and number of persons in the household, both emotional-informational and tangible support were highly predictive of lower depression, suicidality, generalized anxiety, panic, social phobia, depersonalization, sleep disorder. For social anxiety and Type D personality, emotional support seems to be more important than practical support. When controlling for age, sex, CVD, CVD risk, SES, and number of persons in the household, for all listed mental problems, except for depersonalization, emotional-informational support appears to be a better predictor than tangible support. Overall, higher age and social status were associated with lower distress, whereas female sex and the presence of CVD were associated with higher distress. The results are displayed in Table 3.
Discussion
The postulated hypotheses were borne out by analyzing two independent population-based subsamples. First, we successfully derived a brief scale with only six items (see S2 Table). We could reproduce the factor structure of previous brief scales with two independent dimensions, emotional-informational support and tangible support assessed by three items each. Second, confirmatory factor analysis showed a very good fit for the bi-factorial model in comparison to the one-factor solution renderings support for the need of differentiation of both facets of social support. As summarized by [21], most of the studies using the MOS Social Support Survey show different factor structures depending on the chosen factor analytical approach. However, our findings based on a German population-based sample clearly indicate a two-factor structure regardless of the used methodological approach.
Different associations of some socio-demographic and health-related variables with emotional-informational and tangible support were observed, corroborating the assumption of differentiating effects of both facets of social support. People in a partnership are more likely to experience emotional-information support, whereas the experience of tangible support seems not to be correlated with having a partner. Type D personality and overall health seem to have a slightly stronger association with practical support. From a psychological perspective, the knowledge about the divergence of emotional and practical pathways might be crucial to provide adequate support. The relevance of emotional and practical support probably differs depending on situational variables and interindividual differences.
Nevertheless, similar to previous studies low social support was found generally in socially disadvantaged participants [31, 32]. Emotional-informational and tangible support were in sum consistently predictive of a better mental health, even when adjusting of determinants of distress such as female sex, low SES, and the presence of CVD. For social anxiety and Type D personality, emotional-informational support seems to be crucial. When adjusting for age, sex, CVD, CVD risk, SES, and number of persons in the household, emotional support appears to be more important to predict mental health. The results emphasize the practical implication of distinguishing between the emotional and practical dimension of social support.
However, one limitation of the study is the age range of the sample beginning with 35 years. Due to the lack of adolescents and adults in their twenties or early thirties in our sample, the full range of age related aspects with regard to dimensions of social support have not been investigated in this study. Especially the impact of perceived social support on personality development and health in critical stages of life while reaching adulthood needs further research with sound study designs.
Taking into consideration the strict time constraints to which large scale data collection are subject, extensive assessment tools are usually not applicable for these kind of survey research. The efficiency of this brief scale was demonstrated by presenting validation testing with a number of health related variables. Findings with external criteria were in line with results of other MOS Social Support Surveys (e.g. [16, 17, 19]). The generalizability of the study results is high because of the underlying population-based sample. With the BS6, we provide a reliable and valid short scale that can be applied in social surveys due to its brevity to assess emotional and practical dimensions of social support.
Supporting information
S1 Table. Sample characteristics based on sociodemographic variables.
https://doi.org/10.1371/journal.pone.0186516.s001
(DOCX)
References
- 1. Kawachi I, Berkman LF. Social ties and mental health. Journal of Urban Health. 2001;78(3):458–67. pmid:11564849
- 2. Fydrich TS Gert; Tydecks Stefan; Brähler Elmar. Fragebogen zur sozialen Unterstützung (F-SozU): Normierung der Kurzform (K-14) Zeitschrift für Medizinische Psychologie. 2009;18:pp. 43–8.
- 3. Kliem S, Mößle T, Rehbein F, Hellmann DF, Zenger M, Brähler E. A brief form of the Perceived Social Support Questionnaire (F-SozU) was developed, validated, and standardized. Journal of clinical epidemiology. 2015;68(5):551–62. http://dx.doi.org/10.1016/j.jclinepi.2014.11.003. pmid:25499982
- 4. Dahlberg L, McKee KJ. Correlates of social and emotional loneliness in older people: evidence from an English community study. Aging & mental health. 2014;18(4):504–14. Epub 2013/11/21. pmid:24251626; PubMed Central PMCID: PMCPMC3979439.
- 5. Friedler B, Crapser J, McCullough L. One is the deadliest number: the detrimental effects of social isolation on cerebrovascular diseases and cognition. Acta Neuropathologica. 2015;129(4):493–509. pmid:25537401
- 6. Hughes ME, Waite LJ, Hawkley LC, Cacioppo JT. A Short Scale for Measuring Loneliness in Large Surveys: Results From Two Population-Based Studies. Research on aging. 2004;26(6):655–72. Epub 2008/05/28. pmid:18504506; PubMed Central PMCID: PMCPMC2394670.
- 7. House JS, Landis KR, Umberson D. Social relationships and health. Science. 1988;241(4865):540–5. pmid:3399889
- 8. Holt-Lunstad J, Smith TB, Layton JB. Social relationships and mortality risk: a meta-analytic review. PLoS medicine. 2010;7(7):e1000316. Epub 2010/07/30. pmid:20668659; PubMed Central PMCID: PMCPMC2910600.
- 9. Kuiper JS, Zuidersma M, Oude Voshaar RC, Zuidema SU, van den Heuvel ER, Stolk RP, et al. Social relationships and risk of dementia: A systematic review and meta-analysis of longitudinal cohort studies. Ageing research reviews. 2015;22:39–57. Epub 2015/05/10. pmid:25956016.
- 10.
Cohen S, Gottlieb BH, Underwood L. Social relationships and health. In: Cohen S, Underwood L, Gottlieb BH, editors. Measuring and intervening in social support. New York: Oxford University Press; 2000. p. 3–25.
- 11. Cohen S, Wills TA. Stress, social support, and the buffering hypothesis. Psychological bulletin. 1985;98(2):310–57. Epub 1985/09/01. pmid:3901065.
- 12. Broadhead WE, Gehlbach SH, de Gruy FV, Kaplan BH. The Duke-UNC Functional Social Support Questionnaire: Measurement of Social Support in Family Medicine Patients. Medical care. 1988;26(7):709–23. pmid:3393031
- 13.
Dalgard OS. Community health profile: A tool for psychiatric prevention. In: Trent DR, Reed CA, editors. Promotion of mental health 5. Avebury: Aldershot; 1996. p. 395–402.
- 14. Sarason IG, Sarason BR, Shearin EN, Pierce GR. A brief measure of social support: Practical and theoretical implications. Journal of Social and Personal Relationships. 1987;4:497–510.
- 15. Cohen S, Hoberman H. Positive events and social supports as buffers of life change stress. Journal of Applied Social Psychology. 1983;13:99–125.
- 16. Sherbourne CD, Stewart AL. The MOS social support survey. Social Science & Medicine 1991;32(6):705–14.
- 17. Moser A, Stuck AE, Silliman RA, Ganz PA, Clough-Gorr KM. The eight-item modified Medical Outcomes Study Social Support Survey: psychometric evaluation showed excellent performance. Journal of clinical epidemiology. 2012;65(10):1107–16. Epub 2012/07/24. pmid:22818947; PubMed Central PMCID: PMCPMC4119888.
- 18. Gomez-Campelo P, Perez-Moreno EM, de Burgos-Lunar C, Bragado-Alvarez C, Jimenez-Garcia R, Salinero-Fort MA. Psychometric properties of the eight-item modified Medical Outcomes Study Social Support Survey based on Spanish outpatients. Qual Life Res. 2014;23(7):2073–8. Epub 2014/02/25. pmid:24563111.
- 19. Holden L LC, Hockey R, Ware RS, Dobson AJ. Validation of the MOS Social Support Survey 6-item (MOS-SSS-6) measure with two large population-based samples of Australian women. Qual Life Res. 2014;23(10):2849–53. pmid:24962651
- 20. Gjesfjeld CD, Greeno CG, Kim KH. A Confirmatory Factor Analysis of an Abbreviated Social Support Instrument: The MOS-SSS Research on Social Work Practice. 2008;18:231–7.
- 21. Yu Y, Shiu CS, Yang JP, Wang M, Simoni JM, Chen WT, et al. Factor analyses of a social support scale using two methods. Qual Life Res. 2015;24(4):787–94. Epub 2014/10/01. pmid:25267103.
- 22. Kendel F, Spaderna H, Sieverding M, Dunkel A, Lehmkuhl E, Hetzer R, et al. Eine deutsche Adaptation des ENRICHD Social Support Inventory (ESSI). Diagnostica. 2011;57(2):99–106.
- 23. Balaji AB, Claussen AH, Smith CD, Visser SN, Morales MJ, Perou R. Social Support Networks and Maternal Mental Health and Well-Being. Journal of Women´s Health. 2007;16(10). pmid:18062754
- 24. Uchino BNC, John T.; Kiecolt-Glaser , Janice K. The relationship between social support and physiological processes: A review with emphasis on underlying mechanisms and implications for health. Psychological bulletin. 1996;119(3):488–531. pmid:8668748
- 25. Wild P, Zeller T, Beutel M, Blettner M, Dugi K, Lackner K, et al. [The Gutenberg Health Study]. Bundesgesundheitsblatt, Gesundheitsforschung, Gesundheitsschutz. 2012;55(6–7):824–9. pmid:22736163
- 26. Kocalevent RD, Hinz A, Brahler E. Standardization of the depression screener patient health questionnaire (PHQ-9) in the general population. General hospital psychiatry. 2013;35(5):551–5. Epub 2013/05/15. pmid:23664569.
- 27. Lowe B, Decker O, Muller S, Brahler E, Schellberg D, Herzog W, et al. Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Medical care. 2008;46(3):266–74. Epub 2008/04/05. pmid:18388841.
- 28. Löwe B, Wahl I, Rose M, Spitzer K, Glaesmer H, Wingenfeld K, et al. A 4-item measure of depression and anxiety: Validation and standardization of the Patient Health Questionnaire-4 (PHQ-4) in the general population. Journal of Affective Disorders. 122(1–2):86–95. pmid:19616305
- 29. Lampert T, Kroll LE, Müters S, Stolzenberg H. Measurement of the socioeconomic status within the German Health Update 2009 (GEDA). Bundesgesundheitsblatt—Gesundheitsforschung—Gesundheitsschutz. 2013;56(1):131–43. pmid:23275948
- 30. Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research. 2003;8(2):23–74.
- 31. Ali SR, McWhirter EH, Chronister KM. Self-Efficacy and Vocational Outcome Expectations for Adolescents of Lower Socioeconomic Status: A Pilot Study. Journal of Career Assessment. 2005;13(1):40–58.
- 32. Melchiorre MG, Chiatti C, Lamura G, Torres-Gonzales F, Stankunas M, Lindert J, et al. Social support, socio-economic status, health and abuse among older people in seven European countries. PloS one. 2013;8(1):e54856. Epub 2013/02/06. pmid:23382989; PubMed Central PMCID: PMCPMC3559777.