Skip to main content
Top
Gepubliceerd in: Journal of Child and Family Studies 5/2022

Open Access 12-08-2021 | Original Paper

Reciprocal Relationships between Trajectories of Loneliness and Screen Media Use during Adolescence

Auteurs: David Lawrence, Simon C. Hunter, Rebecca Cunneen, Stephen J. Houghton, Corinne Zadow, Michael Rosenberg, Lisa Wood, Trevor Shilton

Gepubliceerd in: Journal of Child and Family Studies | Uitgave 5/2022

Abstract

Adolescence is the peak period for loneliness. Now a ubiquitous part of the adolescent landscape, electronic screens may provide avenues for ameliorating feelings of loneliness. Conversely, they may act as risk factors for the development of such feelings. Although cross-sectional studies to date have investigated the relationship between screen use and loneliness, longitudinal studies are needed if causal and directional associations are to be investigated. Utilising an accelerated longitudinal design and online survey we collected four waves of data from 1919 secondary school adolescents aged 10–15 years over two years. Random intercept cross-lagged panel models tested whether changes in five types of screen use (i.e., total screen time, social media use, gaming, passive screen use, and web use) are associated with changes in loneliness in the subsequent time-point, or changes in loneliness are associated with changes in screen use in the subsequent time-point. We found significant reciprocal associations between screen use and loneliness, with the strongest associations between social networking and electronic gaming and quality of friendships. These findings highlight that any significant increase in an adolescent’s screen use may be a potential indicator of changes in quality of friendships or feelings of isolation.
Opmerkingen

Supplementary information

The online version contains supplementary material available at https://​doi.​org/​10.​1007/​s10826-021-02066-3.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Loneliness is a complex construct that does not discriminate across the lifespan (Cacioppo et al., 2015), even manifesting among those surrounded by large numbers of others or with numerous contacts, followers, or friends on social media (Qualter et al., 2015). Defined as a subjective experience of lack of connectedness, which arises when one’s intimate and social needs are not adequately met in terms of quantity or quality of social relations (Heinrich & Gullone, 2006; Mellor et al., 2008; Rokach, 2011), loneliness is a multidimensional, negative, and painful experience.
Up to 80% of adolescents report feeling lonely at some time (Houghton et al., 2014; Qualter et al., 2015) while up to 22% can experience loneliness in a chronic form (van Dulmen & Goossens, 2013). However, its heterogeneous developmental course means levels of loneliness are not stable and not all adolescents follow the same pattern of loneliness over time (Qualter et al., 2013; Schinka et al., 2012). This is not surprising given the complex individual and social changes that take place during this developmental period (Qualter et al., 2015). Specifically, it is a time when individuals seek independence and begin to move away from family and home centred activities to develop closer ties with peers and peer groups (Chipuer & Pretty, 2000; Laursen & Hartl, 2013). For some, this period of transition away from the family can be very lonely and stressful, leading to feelings of heightened anxiety and depressive-like symptoms (Houghton et al., 2015).
The research evidence is unequivocal that loneliness can contribute to a constellation of physical and psychiatric dysfunctions across the life span, including for example, depression and suicide ideation (Cacioppo et al. 2006; Cacioppo et al., 2010; VanderWeele et al., 2011), parasuicide and self-harm (Lasgaard et al., 2011), suicidiality (Gallagher et al., 2014), eating and sleep disturbances (Cacioppo et al., 2000), and social anxiety (Cacioppo et al., 2014). As a result, loneliness is viewed as a potential antecedent of emerging mental health issues (Lim et al., 2016).
Screen-based activities, particularly social networking and video gaming, may provide avenues for young people to ameliorate symptoms of loneliness, or conversely they may act as risk factors for the development of such feelings, especially given the high levels of engagement with electronic devices that we currently observe (see Domoff et al., 2019). Indeed, most Australian adolescents spend several hours each day using the internet, social media, or playing electronic games (Houghton et al., 2015; Rikkers et al., 2016; Rosenberg et al., 2018). Despite high levels of social media use, particularly in young people, the current generation is lonelier than previous generations, and loneliness peaks in adolescence and young adulthood (Pittman & Reich, 2016). This may be because of the lack of intimacy afforded by social media platforms which affects the establishment of meaningful social connection (Pittman & Reich, 2016).
Clark et al’s 2018 interpersonal-connection-behaviours framework proposed that social networking can have a positive impact on wellbeing if it is used in a way that promotes connections (Clark et al., 2018), but it can negatively impact wellbeing when used to compensate for insufficient social networks (Clark et al., 2018). Nowland et al. (2018) proposed a similar model of loneliness and social media use that combines the displacement hypothesis (wherein time spent on social media can displace offline social activities) and the stimulation hypothesis (where people experiencing loneliness use social media to form new relationships) (Nowland et al., 2018).
The link between social media use (especially social networking) and loneliness in adults has been demonstrated (Primack et al., 2017; Shensa et al., 2016), with suggestions that this link might be bi-directional. That is, people experiencing higher levels of loneliness are attracted to social media as a less threatening way of establishing social connections (Skues et al., 2012), and that social media use may increase loneliness by displacing time that would otherwise have been used for more direct communications (Song et al., 2014). In addition, it has also been argued that social media promotes more superficial relationships with less feelings of affection or commitment, so while there may be a greater number of connections, these connections are less able to satisfy the user’s emotional needs (Huang, 2010). In these ways, social media use by young people may not be bi-directional, but rather associated with both positive and negative outcomes.
Video gaming has been a particular concern among families for some considerable time and some researchers have called on the video gaming industry to take greater responsibility in gamer protection (Griffiths and Pontes 2019). For many young people, video games may be used to access novel friendships and relationships (Mesch & Talmud, 2006). However, contradictory findings have been reported with regard to gaming. For example, using a cross-sectional design, Kowert et al. (2014) reported adolescents who spend more time playing games online have fewer and lower quality friends. Correspondingly, Lemmens et al. (2011) found reciprocal positive associations between loneliness and problematic (“pathological”) gaming in their two-year cross-lagged panel study involving 11 to 17 year olds (Lemmens et al., 2011). Similar findings were reported by Kaczmarek and Drążkowski (2014) concerning offline social support and gaming time, though they also noted that more gaming time is associated with higher levels of online social support for offline problems (Kaczmarek & Drążkowski, 2014). Others have failed to find any association (either cross-sectional or longitudinal) of perceived social support and social online video games (Domahidi et al., 2018) despite recruiting large samples (approximately 3500).
While much of the current literature has focussed on social media and electronic gaming, Erdoğan (2008) found that loneliness in adolescents was associated with increased internet usage (Erdoğan, 2008). With reference to longitudinal research, one recent study found reciprocal pathways between internet use and depressive symptoms (Salmela-Aro et al., 2016).
In summary, adolescent screen use is a concern for families (Domoff et al., 2019). However, cross-sectional studies that report associations between screen use and loneliness are limited in their ability to inform hypotheses on causal and directional associations. The present longitudinal study sought to address this by investigating the ways in which loneliness during adolescence is associated with the total time spent on electronic screens and four common screen-based media activities (social media use, gaming, passive screen use, and web use). The aims of the study were to (i) examine whether either loneliness or screen use in one time period predicted changes in the other variable in later time periods using a random intercept cross-lagged panel model, and (ii) determine if this association varied for different types of screen use activities. To achieve these aims we used data from a longitudinal study of screen use where both screen use and loneliness were assessed in four waves of data collection.

Method

Participants and settings

The total sample comprised 1919 adolescents (1085 males and 834 females). Three cohorts of adolescents were recruited at Time 1 from Grades 5 (10/11 years of age: 276 males, 247 females), 7 (12/13 years of age: 371 males, 298 females), and 9 (14/15 years of age: 289 males, 268 females). Data were collected via an online survey a further three times (over two years) at different times of the academic year to ensure coverage of screen use across all periods of the school year: Time 1, November/December; Time 2, August/September; Time 3, March/April; and Time 4 August/September. The attrition rates were below 6% between each time period of data collection. An analysis found no patterns in attrition rates between students. Table 1 shows the sample distribution.
Table 1
Sample distribution
 
N
%
Sex
Male
1085
57
Female
834
43
School grade (Time 1)
Grade 5
523
30
Grade 7
669
38
Grade 9
557
32
Geographical location (Time 1)
Urban
1350
77
Rural
399
23
Time period
1
1544
2
1703
3
1782
4
1591
Number of time periods completed by participantsa
1
167
9
2
46
2
3
573
30
4
1134
59
aNumber of waves completed by participants of the four waves of data used in this paper
To recruit the sample, 30 schools were initially selected for the study across the Perth metropolitan area and selected rural areas in Western Australia, with 25 schools agreeing to participate in the study. The sampled schools were selected to reflect the diversity of the sociodemographic characteristics of Perth schools. Where students moved schools over the course of the study, the new school was approached to participate in the study, which resulted in an increasing number of schools in the study. In particular, most students in the Year 5 cohort transitioned from primary school to high school in Year 7. By Time 4 there were 38 participating schools, including 34 state government schools (13 primary schools, 20 high schools and one combined K-12 school) and 4 non-government schools (K-12). Nine of the schools were located outside the Perth metropolitan area (Houghton et al., 2018).

Measures

Screen Based Media Use

The Screen Based Media Use Scale (SBMUS; Houghton et al., 2015) was administered online. Prior to completing the instrument, participants were told (in the survey) what ‘screens’ refer to and given visual examples of screen activities. Adolescents used an interactive slide bar to indicate their amount of screen use (including use both inside and outside of school) in hours and minutes, on both a typical weekday in the past week and on the previous weekend, from waking up until going to bed.
In addition, participants also completed the same information in four separate sections for specific screen-use relating to gaming (which included both online and offline gaming, and both solo and multiplayer games), social media use (including social networking sites such as Facebook, Kik, Tumblr, and Instagram, and instant messaging apps such as WhatsApp or Palingro), TV/passive screen use (watching TV and watching videos or movies) and searching the web (including visiting websites, online shopping, using Google, downloading music, sending or receiving emails, and doing school or homework on the web). Average daily screen use for each type of activity and for total use was calculated by summing the average weekday and weekend use, weighted in the ratio 5:2, and dividing by seven. Total time was reported separately from time spent on specific activities as young people often undertake several different screen use activities simultaneously (Cain et al., 2016; Jago et al., 2011). In this present study, we utilise these measures of screen use for specific activities to test whether each is uniquely associated with loneliness in adolescents.

Loneliness

The Perth A-Loneness scale (PALs; Houghton et al., 2014) is a 24-item self-report measure consisting of 4 correlated factors: friendship (i.e., having reliable, trustworthy supportive friends), isolation (i.e., having few friends or believing that there was no one around offering support), positive attitude to solitude (i.e., positive aspects and benefits of being alone such as relaxing, happiness) and negative attitude to solitude (i.e., negative aspects of being alone such as time dragging, unhappiness, isolation). The PALs has excellent model fit statistics (e.g., Houghton et al., 201420152016). Responses are given on a six point Likert scale: 1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = very often, 6 = always.

Procedure

The Human Research Ethics Committees of The University of Western Australia and the Western Australian Department of Education granted permission to conduct this research. Information sheets and consent forms were sent to parents of students explaining that involvement in the research consisted of multiple data gathering over approximately three school years. Informed consent was obtained from all individual participants included in the study. The SBMUS and PALs were subsequently completed by participants via an online survey during regular school hours. All participants were provided with a unique identification code, which allowed them to log on to the survey at each administration. To ensure that the correct code was used it was given to each participant immediately prior to each administration. This unique code also ensured that all information provided was confidential and that data could be linked across waves via these codes for the purposes of data analysis. The stringent monitoring of the survey administrations resulted in attrition rates being maintained below 6% per data collection period.
Although information about screen use and loneliness was collected at four time points the spacing between these time periods was not uniform. There was an average of 11 months between Time 1 and Time 2, six months between Time 3 and Time 3 and five months between Time 3 and Time 4.

Data Analysis

Preliminary analyses showed no cross-sectional or longitudinal association between loneliness and either positive or negative attitudes towards being alone, and so this study focussed on the Friendship and Isolation factors only and whether screen use is uniquely associated with either of these components. Estimates of reliability were satisfactory (e.g., Friendship α = 0.91 and Isolation α = 0.80; Houghton et al., 2016). The total scores for both the Friendship and Isolation factors were obtained by summing the six items measuring each factor, and so final scores could range from 6 to 36. Higher scores indicated better friendship quality on the friendship factor, and higher levels of isolation on the isolation factor.
The primary purpose of this analysis was to examine whether experiencing an increase in feelings of loneliness is associated with a later increase in the amount of time an adolescent spends using screens, or vice-versa. A Random Intercept Cross-Lagged Panel Model (RI-CLPM; Hamaker et al., 2015) was used to test whether changes in screen use are associated with changes in loneliness in the subsequent time-point, or changes in loneliness are associated with changes in screen use in the subsequent time-point. This model extends the traditional Cross-Lagged Panel Model (CLPM) by including random intercepts (in our case for both loneliness and screen use) to separately model within person changes over time and stable between-person differences. In a traditional cross-lagged panel model high levels of one factor at time one could be associated with high levels of a second factor at time two if there were a group of subjects who had persistently high levels of both factors at both time points. By introducing a random intercept term the RI-CLPM separately models stable deviations from the group mean. This means that the cross-lagged associations are a measure of whether change in an individual subject’s level of one factor from their individual average for that factor is associated with change in the second factor in the subsequent time-period.
The model fitted is shown in Fig. 1. The primary purpose of the model is to test if loneliness and screen use are associated with each other over time. This is shown in the central section of Fig. 1 highlighted by the dotted box. The autoregressive parameters α (the relationship between loneliness scores across time-points) and δ (the relationship between screen-use across time-points) control for the degree of stability in loneliness and screen use over time. The cross-lagged regression parameters β (extent to which change in loneliness is predicted by screen use at the previous time-point) and γ (extent to which change in screen use is predicted by loneliness at the previous time-point), are the indicators of the reciprocal relationship between loneliness and screen use. These are the parameters that are directly related to the hypotheses in this study.
We estimated the reciprocal relationships between either friendship or isolation and the five measures of screen use (total screen time, gaming, social media, passive screen use, or web browsing) by fitting ten separate models using MPlus software, Version 7.4 (Muthén & Muthén, 1998). Autoregressive correlations (α and δ) were specified as fixed across all time-points and the cross-lagged correlations were estimated freely for time 1 to time 2 (β1 and γ1) and constrained to be equal between time 2 to time 3 and time 3 to time 4 (β2 and γ2). Since there was 11 months between time 1 and time 2 measurements and only about 5 months from time 2 to time 3 and time 3 to time 4 we believed there could be different effects (or different ability to detect effects) between time 1 and time 2, compared to time 2 to time 3 and time 3 to time 4. We also fitted panel data models to test whether age or gender were associated with either friendship, isolation, or the five types of screen use.

Results

Descriptive statistics are pooled across all four time periods. In aggregate there were no changes in average friendship or screen use across time periods. The model reports on individual changes in these characteristics over time.

Loneliness and Screen Use

Adolescents reported an average of 3.8 h total screen time per day. For each of the screen activities, average times were 1.8 h of social media, 1.5 h of gaming, 2.5 h on passive screen use, and 1.9 h on web browsing. The average score of the friendship factor was 26.8 points (SD = 7.4). Half of all adolescents scored between 21 and 33 points (out of a possible 36) on the friendship factor and only 10% scored less than 17 points. The average score of the isolation factor was 10.9 points (SD = 6.0). Half of the adolescents scored between 6 and 13 points and 10% scored more than 18 points.

Gender and Age Differences

Females scored 2.32 points higher on the isolation factor for loneliness compared to males (95% CI: 0.02 to 4.61; p = 0.048). Females also reported more than one hour additional total screen time per day (1.28 h; 95% CI: 0.58 to 1.97; p < 0.001). However, they also spent almost one hour less surfing the web per day (−0.85; 95% CI: −1.52 to −0.18; p = 0.015), and almost one and a half hours less playing games on screens per day (−1.43; 95% CI: −2.13 to −0.73; p = 0.015). There were no other significant associations found between age or gender and the measures of loneliness or screen use used in this study.

Intra-Class Correlation Coefficients

The intra-correlation coefficient (ICC) for friendship from Model 1 (total screen time and friendship) was 0.402, indicating that 40% of the variation in friendship factor scores across the time-points was due to changes within individual adolescents. The amount of variation attributed to within individual differences for the isolation factor was about half as much as for the friendship factor. From Model 6 (total screen time and isolation) the ICC for isolation was 20.2%. The ICCs for the five types of screen use varied somewhat and were 49.7% for total screen time, 52.9% for social media use, 37.5% for gaming, 38.2% for passive screen use, and 38.9% for web use.

Autoregressive Parameters

As expected, there was reasonable degree of stability in the measures of friendship, isolation, and the five types of screen use over time, with a significant association between time points for each measure in most cases. Estimates of α and δ in each of the 10 models fitted are shown in Supplementary Table 1.

Cross-Lagged Path Coefficients

Table 2 shows estimated cross-lagged regression coefficients between the friendship factor subscale and each of the five screen use measures, while estimated coefficients for the models looking at the relationship between the isolation subscale and screen use are shown in Table 3. These tables report unstandardised model results. Standardised models are shown in Supplementary Tables 2 and 3.
Table 2
Random intercept cross-lagged panel data model—reciprocal effects estimates of friendship and screen use by type
 
β
SE
p
Model fit
Model 1: Total Screen Time
Friendship TIME1 → Screen use TIME21)
0.003
0.012
0.629
RMSEA = 0.035
Friendship TIME2 → Screen use TIME32)
−0.037
0.010
0.000
[0.027–0.043]
Friendship TIME3 → Screen use TIME42)
−0.037
0.010
0.000
χ2 = 83.887
Screen use TIME1 → Friendship TIME21)
0.037
0.117
0.749
p = 0.0000
Screen use TIME2 → Friendship TIME32)
−0.338
0.094
0.000
ICCFriendship = 0.402
Screen use TIME3 → Friendship TIME42)
−0.338
0.094
0.000
ICCScreen Use = 0.499
Model 2: Social Media
Friendship TIME1 → Social media TIME21)
0.017
0.011
0.143
RMSEA = 0.048
Friendship TIME2 → Social media TIME32)
−0.048
0.010
0.000
[0.040–0.056]
Friendship TIME3 → Social media TIME42)
−0.048
0.010
0.000
χ2 = 134.576
Social media TIME1 → Friendship TIME21)
0.016
0.131
0.905
p = 0.0000
Social media TIME2 → Friendship TIME32)
−0.442
0.102
0.000
ICCFriendship = 0.409
Social media TIME3 → Friendship TIME42)
−0.442
0.102
0.000
ICCSocial Media = 0.529
Model 3: Gaming
Friendship TIME1 → Gaming TIME21)
0.015
0.012
0.187
RMSEA = 0.053
Friendship TIME2 → Gaming TIME32)
0.018
0.010
0.062
[0.045–0.061]
Friendship TIME3 → Gaming TIME42)
0.018
0.010
0.062
χ2 = 158.054
Gaming TIME1 → Friendship TIME21)
0.034
0.116
0.771
p = 0.0000
Gaming TIME2 → Friendship TIME32)
0.159
0.098
0.103
ICCFriendship = 0.398
Gaming TIME3 → Friendship TIME42)
0.159
0.098
0.103
ICCGaming = 0.376
Model 4: Television/ Passive
Friendship TIME1 → Television TIME21)
0.015
0.013
0.263
RMSEA = 0.039
Friendship TIME2 → Television TIME32)
−0.022
0.009
0.017
[0.031–0.047]
Friendship TIME3 → Television TIME42)
−0.022
0.009
0.017
χ2 = 96.771
Television TIME1 → Friendship TIME21)
0.102
0.114
0.371
p = 0.0000
Television TIME2 → Friendship TIME32)
−0.237
0.091
0.009
ICCFriendship = 0.398
Television TIME3 → Friendship TIME42)
−0.237
0.091
0.009
ICCTelevision = 0.384
Model 5: Web use
Friendship TIME1 → Web use TIME21)
0.014
0.012
0.211
RMSEA = 0.038
Friendship TIME2 → Web use TIME32)
−0.023
0.008
0.006
[0.030–0.047]
Friendship TIME3 → Web use TIME42)
−0.023
0.008
0.006
χ2 = 95.453
Web use TIME1 → Friendship TIME21)
0.137
0.139
0.325
p = 0.0000
Web use TIME2 → Friendship TIME32)
−0.316
0.104
0.002
ICCFriendship = 0.401
Web use TIME3 → Friendship TIME42)
−0.316
0.104
0.002
ICCWeb Use = 0.392
Note: results shown in bold type p < 0.05
Change between Time 2 and Time 3 has been constrained to be equal to the change between Time 3 and Time 4
Table 3
Random intercept cross-lagged panel data model—reciprocal effects estimates of isolation and screen use by type
 
β
SE
p
Model fit
Model 6: Total Screen Time
Isolation TIME1 → Screen use TIME21)
0.014
0.013
0.280
RMSEA = 0.037
Isolation TIME2 →Screen use TIME32)
0.012
0.010
0.217
[0.029–0.045]
Isolation TIME3 → Screen use TIME42)
0.012
0.010
0.217
χ2 = 89.832
Screen use TIME1 → Isolation TIME21)
0.002
0.105
0.983
p = 0.0000
Screen use TIME2 → Isolation TIME32)
0.167
0.084
0.046
ICCIsolation = 0.202
Screen use TIME3 → Isolation TIME42)
0.167
0.084
0.046
ICCScreen Use = 0.494
Model 7: Social Media
Isolation TIME1 → Social media TIME21)
0.002
0.012
0.844
RMSEA = 0.051
Isolation TIME2 → Social media TIME32)
0.016
0.010
0.092
[0.043–0.059]
Isolation TIME3 → Social media TIME42)
0.016
0.010
0.092
χ2 = 147.056
Social media TIME1 → Isolation TIME21)
0.155
0.115
0.180
p = 0.0000
Social media TIME2 → Isolation TIME32)
0.354
0.089
0.000
ICCIsolation = 0.198
Social media TIME3 → Isolation TIME42)
0.354
0.089
0.000
ICCSocial Media = 0.529
Model 8: Gaming
Isolation TIME1 → Gaming TIME21)
0.025
0.013
0.045
RMSEA = 0.056
Isolation TIME2 → Gaming TIME32)
0.022
0.010
0.022
[0.048–0.064]
Isolation TIME3 → Gaming TIME42)
0.022
0.010
0.022
χ2 = 175.693
Gaming TIME1 → Isolation TIME21)
0.070
0.107
0.510
p = 0.0000
Gaming TIME2 → Isolation TIME32)
0.316
0.087
0.000
ICCIsolation = 0.198
Gaming TIME3 → Isolation TIME42)
0.316
0.087
0.000
ICCGaming = 0.374
Model 9: Television/ Passive
Isolation TIME1 → Television TIME21)
−0.035
0.014
0.013
RMSEA = 0.036
Isolation TIME2 → Television TIME32)
0.011
0.009
0.251
[0.028–0.044]
Isolation TIME3 → Television TIME42)
0.011
0.009
0.251
χ2 = 87.269
Television TIME1 → Isolation TIME21)
0.035
0.102
0.730
p = 0.0000
Television TIME2 → Isolation TIME32)
0.146
0.080
0.066
ICCIsolation = 0.200
Television TIME3 → Isolation TIME42)
0.146
0.080
0.066
ICCTelevision = 0.380
Model 10: Web use
Isolation TIME1 → Web use TIME21)
0.003
0.012
0.836
RMSEA = 0.039
Isolation TIME2 → Web use TIME32)
0.015
0.008
0.067
[0.031–0.048]
Isolation TIME3 → Web use TIME42)
0.015
0.008
0.067
χ2 = 98.900
Web use TIME1 → Isolation TIME21)
0.018
0.125
0.888
p = 0.0000
Web use TIME2 → Isolation TIME32)
0.228
0.094
0.015
ICCIsolation = 0.196
Web use TIME3 → Isolation TIME42)
0.228
0.094
0.015
ICCWeb Use = 0.385
Note: results shown in bold type p < 0.05
Change between Time 2 and Time 3 has been constrained to be equal to the change between Time 3 and Time 4
Across eight of the ten models, β1 (loneliness at time 2 predicted by screen use at time 1) and γ1 (screen use at time 2 predicted by loneliness at time 1) were estimated as not significantly different from zero. We hypothesise that this was due to the large measurement gap of 11 months between time 1 and time 2.
For four of the five models with friendship quality as the measure of loneliness, the cross-lagged coefficients β2 (friendship predicted by screen use at the previous time-point) and γ2 (screen use predicted by friendship at the previous time-point) were significantly different than zero. An increase of one point in the friendship factor between time 2 and time 3 or time 3 and time 4 was associated with a decrease of 0.04 h in total screen time, 0.05 h of social media screen use, 0.02 h of passive screen use and 0.02 h of web browsing screen use. Changes in the friendship factor were not associated with hours of gaming screen use at the next time point. Conversely, increases in all types of screen use, except for gaming screen use, were associated with decreases in the friendship factor: −0.34 points for each hour of total screen time, −0.44 for social media use, −0.24 for passive screen use, and −0.32 for web browsing screen use.
For the remaining five models with isolation as the measure of loneliness, the results were not uniform across the five types of screen use. An increase of one point in the isolation factor between time 1 and time 2 was associated with an increase of 0.03 h of gaming screen use and a decrease of −0.03 h of passive screen use only. An increase of one point in the isolation factor between time 2 and time 3 or time 3 and time 4 was associated with an increase of 0.02 h of gaming screen use, but was not associated with total screen time, social media screen use, passive screen use or web browsing screen use.
Increases in all types of screen use, except for passive screen use, between time 2 and time 3 or time 3 and time 4 were associated with increases in isolation: 0.17 points for each hour total screen time, 0.35 for social media use, 0.32 for gaming screen use, and 0.23 for web browsing screen use.
We investigated the possibility of differing results between metropolitan and rural schools by running separate models for each area, but found no appreciable differences between areas.

Discussion

This study found complex relationships in the association between the amounts of time spent on screen activities and self-reported levels of loneliness. Across extended periods of 11 months, there was limited evidence of an association between any form of screen use and either of the loneliness factors (i.e., friendship and isolation). In contrast, across shorter intervals of around five months there were reciprocal relationships evidenced between the friendship factor and four of the indices of screen use assessed. Specifically, higher friendship factor scores (i.e., higher quality friendships) were associated with lower later screen time, and higher screen time estimates were associated with lower later friendship factor scores. A similar pattern was evidenced for isolation loneliness scores. Specifically, higher isolation loneliness scores were associated with increases in later gaming and passive screen time estimates; in contrast, all forms of screen use except passive screen time were associated with subsequent increases in isolation loneliness scores. Kraut et al. (2002) also found that over a two to three year period the relationship between times spent communicating online and perceptions of increased loneliness all but disappeared. Some studies have reported loneliness predicts problematic social media use and vice versa (for a review see O’Day & Heimberg, 2021), while others have not been able to conclude any causal effects (e.g., Ando et al., 2008). It is clear that the relationship between loneliness and social media involvement among adolescents is multifaceted and complex (Smith et al., 2021) and potential causal pathways may include, for example, shyness, social support and social capital (Song et al., 2014). As shown here, the cross-lagged panel model fitted is useful for expanding our understanding of the nature of the relationship between related factors. The existence of time-lagged relationships add weight to hypotheses of causal association, and are particularly useful for testing causal theories where experimental studies would be practically challenging or ethically inappropriate.
The results are important for families who regularly express concern about the amount of time their children spend on screens (Domoff et al., 2019). Specifically, increases in friendship quality are followed by a subsequent decrease in the amount of time spent on screen activities and vice versa. It is evident from the present study that quality of friendships appear to matter for adolescents (Cacioppo et al., 2014) which is in line with Asher and Paquett (2003) and Masi et al. (2011) who argued that having at least one quality friend is particularly important. However, it is important to note that there are other solo activities in which adolescents typically indulge, such as reading, listening to music, and doing art that might increase in association with loneliness. Adolescents experiencing feelings of loneliness might simply choose screen based activities over other solo activities as a way of filling in time because it’s easier. The design of our study and its findings prevent any determination of whether screen use holds a special place in the lives of lonely adolescents or whether it is just another solo activity of choice.
While significant associations were found in both directions, they were relatively modest in size. A decrease in the friendship factor score of 1 point would be associated with a later increase in screen use of about 2.2 min per day, while an increase in screen use of an average 60 min per day would be associated with a later decrease in the friendship factor score of less than half a point. This suggests the magnitude of the association between friendships and subsequent screen use is larger than the reverse association between screen use and subsequent friendships score. Thus, young people who perceive higher quality friendships are less likely to later spend time on screen time activities, but that time spent on screens has little impact on friendship.
From a real world perspective, a reduction of 1 point in the friendship factor is somewhat unlikely, while a 2.2 min increase in screen use would be difficult to detect. An interesting avenue for future research would be to examine whether adolescents with higher quality friendships spend less time on screen based activities because they are engaged in more ‘real life’ social activities together.
In the present study, loneliness was assessed at unequally spaced time periods, allowing us to examine whether the duration between measurement periods affects the observed cross-lagged associations. Suppose that the relationship between exposure to some factor and change in some subsequent outcome is conceived to have a latency period and a persistence period. That is, we assume that the change in the second factor will occur after some latency period following change in the first factor, and this change in the second factor will persist for a certain period of time. The time delay between measurement points in a cross-lagged model is related to assumptions that we make about these latency and persistence periods. No cross-lagged associations were detected between time 1 to time 2 for screen use and loneliness in 8 of the 10 models fitted in this study, suggesting that 11 months is too great a lag to observe any effects between screen use and loneliness. However, since significant effects were found between times 2 and 3, and between 3 and 4 (both lags of 5-6 months), this suggests that the impact of a change in screen use on friendships, and vice versa, may occur in a relatively short space of time. It also suggests that the impact may not be sustained over a period as long as a year.
As a sensitivity analysis, we also fitted a model with three time points (Time periods 1, 2 and 4 in Fig. 1) where each time point was separated by on average 11-12 months. No significant cross-lagged effects were found in this model. This suggests that over the longer period of 11-12 months it is likely that further changes in feelings of loneliness or levels of screen use occurred and have largely changed prior to the subsequent measure. This would imply that if a decline in friendship quality is followed by an increase in time using screens, this increase is not typically sustained for as long as a year, and screen use is likely to revert back to normal levels within that time. Similarly if an increase in screen use activity has a subsequent negative impact on friendship quality, this decline is not likely to persist for as long as a year. While we conceptualise loneliness and screen use as relatively stable constructs, it is possible that a pathological change in either induced by an increase in the other may persist on average for more than five months but less than one year.

Limitations of the Study

This study recruited a large cohort, obtained multiple measurement points with unequal spacing in time, and employed robust measures of screen use and friendships. The random intercept cross-lagged panel model of Hamaker et al. (2015) is also a stronger test of potential causal associations than the traditional CLPM. However, time spent on screen use activities and loneliness were based on adolescent’s retrospective self-report only. While the optimal recommended strategy is to use two or more sources (Antshel et al., 2012), very little research has examined the concordance between parents and adolescents estimation of specific health related behaviours, such as screen use (Berge et al., 2016; Wood et al., 2019). Less parental supervision, a perceived social desirability to underestimate time, and increased use of screen media by adolescents on unmonitored personal devices such as smartphones make it very difficult for parents to accurately report screen use (Wood et al., 2019). Moreover, because the data collected on screen use included the entire waking day, it was only the adolescents themselves who could supply this information. Thus, self-report was seen as an effective means of obtaining an accurate insight into the subjective dispositions (such as screen use and loneliness) that can be difficult to obtain from third parties such as parents (Baldwin & Dadds, 2007; Houghton et al., 2013).
Nevertheless, self-reports are susceptible to poor recall (Barnett et al., 2018) and using retrospective memory reports gives rise to a potential confound. Specifically, those students who had more interactions with friends could have a bias toward recalling those friendship-related memories over screen-time memories, thereby leading to the underreporting of screen time. In comparison, students who have fewer friend interactions may be more accurate in their reporting of screen time. Future research may wish to use more objective measures such as smartphones or wearable cameras (Kelly et al., 2013), the latter now being more affordable, smaller and proficient in capturing more data (Doherty et al., 2013; Doherty et al., 2012; Gurrin et al., 2013). Using wearable cameras in adolescent behaviour research has increased over the past 10 years (see Everson et al., 2019) such that the more accurate assessment of adolescents’ screen use in the various social contexts in which it occurs is now possible (Doherty et al., 2011; Loveday et al., 2015).
The current study did not distinguish between the use of screens at school and outside of school, and it is likely that some internet use and some passive use of screens would occur within the school environment. Furthermore, there was no differentiation between screen use for educational and/or recreational purposes within school and home contexts, and electronic gaming online with friends versus other types of electronic games were not separated.

Conclusions

In a longitudinal study with four waves of data, cross-lagged associations were found between screen use and loneliness. The strongest associations were found between quality of friendships and both time spent social networking and electronic gaming, two screen use activities especially concerning to families. These associations were found in models with a 5-6 month time interval between measurement points, but were not observed in models with an 11-12 month time interval between measurement points. Thus, the impact of changes in screen use on subsequent loneliness may be sustained for 6 months or more but is unlikely to be sustained for periods of a year or longer. Based on these findings, educators, parents and allied health professionals should be mindful that substantial increases in the amount of time adolescents spend on screens might be a potential indicator of changes in quality of friendship or isolation.

Supplementary information

The online version contains supplementary material available at https://​doi.​org/​10.​1007/​s10826-021-02066-3.

Acknowledgements

We are indebted to the Department of Education (Western Australia) for providing access to schools. We acknowledge Karen Martin of the University of Western Australia who contributed to the development of the study. This study was funded by a grant through the Western Australian Health Promotion Foundation (Healthway, #22951).

Author contributions

SJH, SCH and DL designed the study. CZ, MR, LW and TR contributed to the initial design and conceptualisation for the research grant application. RC, SCH and DL conducted data analyses. SCH, SJH and DL drafted the paper. RC, LW and TR collaborated on the writing of the paper.

Compliance with Ethical Standards

Conflict of Interest

The authors declare no competing interests.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee of the University of Genoa and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study. Parents and adolescents completed consent forms, and verbal assent was also obtained from adolescents.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://​creativecommons.​org/​licenses/​by/​4.​0/​.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Onze productaanbevelingen

BSL Psychologie Totaal

Met BSL Psychologie Totaal blijf je als professional steeds op de hoogte van de nieuwste ontwikkelingen binnen jouw vak. Met het online abonnement heb je toegang tot een groot aantal boeken, protocollen, vaktijdschriften en e-learnings op het gebied van psychologie en psychiatrie. Zo kun je op je gemak en wanneer het jou het beste uitkomt verdiepen in jouw vakgebied.

BSL Academy Accare GGZ collective

Bijlagen

Supplementary information

Literatuur
go back to reference Barnett, T. A., Kelly, A. S., & Young, D. R., et al. (2018). Sedentary behaviors in today’s youth: approaches to the prevention and management of childhood obesity: A scientific statement from the American Heart Association. Circulation, 138, 142–159.CrossRef Barnett, T. A., Kelly, A. S., & Young, D. R., et al. (2018). Sedentary behaviors in today’s youth: approaches to the prevention and management of childhood obesity: A scientific statement from the American Heart Association. Circulation, 138, 142–159.CrossRef
go back to reference Berge, J. M., MacLehose, R. F., Meyer, C., Didericksen, K., Loth, K. A., & Neumark-Sztainer, D. (2016). He said, she said: examining parental concordance on home environment factors and adolescent health behaviors and weight status. Journal of the Academy of Nutrition and Dietetics, 116(1), 46–60.CrossRef Berge, J. M., MacLehose, R. F., Meyer, C., Didericksen, K., Loth, K. A., & Neumark-Sztainer, D. (2016). He said, she said: examining parental concordance on home environment factors and adolescent health behaviors and weight status. Journal of the Academy of Nutrition and Dietetics, 116(1), 46–60.CrossRef
go back to reference Cacioppo, J. T., Ernst, J. M., Burleson, M. H., McClintock, M. K., Malarkey, W. B., Hawkley, L. C., & Spiegel, D. (2000). Lonely traits and concomitant physiological processes: the MacArthur social neuroscience studies. International Journal of Psychophysiology, 35(2-3), 143–154.CrossRef Cacioppo, J. T., Ernst, J. M., Burleson, M. H., McClintock, M. K., Malarkey, W. B., Hawkley, L. C., & Spiegel, D. (2000). Lonely traits and concomitant physiological processes: the MacArthur social neuroscience studies. International Journal of Psychophysiology, 35(2-3), 143–154.CrossRef
go back to reference Cacioppo, J. T., Hawkley, L. C., Ernst, J. M., Burleson, M., Berntson, G. G., Nouriani, B., & Spiegel, D. (2006). Loneliness within a nomological net: an evolutionary perspective. Journal of Research in Personality, 40(6), 1054–1085.CrossRef Cacioppo, J. T., Hawkley, L. C., Ernst, J. M., Burleson, M., Berntson, G. G., Nouriani, B., & Spiegel, D. (2006). Loneliness within a nomological net: an evolutionary perspective. Journal of Research in Personality, 40(6), 1054–1085.CrossRef
go back to reference Cain, M. S., Leonard, J. A., Gabrieli, J. D., & Finn, A. S. (2016). Media multitasking in adolescence. Psychonomic Bulletin & Review, 23(6), 1932–1941.CrossRef Cain, M. S., Leonard, J. A., Gabrieli, J. D., & Finn, A. S. (2016). Media multitasking in adolescence. Psychonomic Bulletin & Review, 23(6), 1932–1941.CrossRef
go back to reference Doherty, A. R., Hodges, S. E., King, A. C., Smeaton, A. F., Berry, E., Moulin, C. J., & Foster, C. (2013). Wearable cameras in health: the state of the art and future possibilities. American Journal of Preventive Medicine, 44(3), 320–323.CrossRef Doherty, A. R., Hodges, S. E., King, A. C., Smeaton, A. F., Berry, E., Moulin, C. J., & Foster, C. (2013). Wearable cameras in health: the state of the art and future possibilities. American Journal of Preventive Medicine, 44(3), 320–323.CrossRef
go back to reference Doherty, A. R., Kelly, P., Kerr, J., Marshall, S., Oliver, M., Badland, H., & Foster, C. (2012). Use of wearable cameras to assess population physical activity behaviours: an observational study. The Lancet, 380, S35.CrossRef Doherty, A. R., Kelly, P., Kerr, J., Marshall, S., Oliver, M., Badland, H., & Foster, C. (2012). Use of wearable cameras to assess population physical activity behaviours: an observational study. The Lancet, 380, S35.CrossRef
go back to reference Doherty, A. R., Moulin, C. J., & Smeaton, A. F. (2011). Automatically assisting human memory: a SenseCam browser. Memory, 19(7), 785–795.CrossRef Doherty, A. R., Moulin, C. J., & Smeaton, A. F. (2011). Automatically assisting human memory: a SenseCam browser. Memory, 19(7), 785–795.CrossRef
go back to reference Domahidi, E., Breuer, J., Kowert, R., Festl, R., & Quandt, T. (2018). A longitudinal analysis of gaming-and non-gaming-related friendships and social support among social online game players. Media Psychology, 21(2), 288–307.CrossRef Domahidi, E., Breuer, J., Kowert, R., Festl, R., & Quandt, T. (2018). A longitudinal analysis of gaming-and non-gaming-related friendships and social support among social online game players. Media Psychology, 21(2), 288–307.CrossRef
go back to reference Erdoğan, Y. (2008). Exploring the Relationships among Internet Usage, Internet Attitudes and Loneliness of Turkish Adolescents. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 2, 4. Erdoğan, Y. (2008). Exploring the Relationships among Internet Usage, Internet Attitudes and Loneliness of Turkish Adolescents. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 2, 4.
go back to reference Everson, B., Mackintosh, K. A., McNarry, M. A., Todd, C., & Stratton, G. (2019). Can wearable cameras be used to validate school-aged children’s lifestyle behaviours? Children, 6(2), 20.CrossRef Everson, B., Mackintosh, K. A., McNarry, M. A., Todd, C., & Stratton, G. (2019). Can wearable cameras be used to validate school-aged children’s lifestyle behaviours? Children, 6(2), 20.CrossRef
go back to reference Gurrin, C., Qiu, Z., Hughes, M., Caprani, N., Doherty, A. R., Hodges, S. E., & Smeaton, A. F. (2013). The smartphone as a platform for wearable cameras in health research. American Journal of Preventive Medicine, 44(3), 308–313.CrossRef Gurrin, C., Qiu, Z., Hughes, M., Caprani, N., Doherty, A. R., Hodges, S. E., & Smeaton, A. F. (2013). The smartphone as a platform for wearable cameras in health research. American Journal of Preventive Medicine, 44(3), 308–313.CrossRef
go back to reference Houghton, S., Hattie, J., Carroll, A., Wood, L., & Baffour, B. (2016). It hurts to be lonely! Loneliness and positive mental wellbeing in Australian rural and urban adolescents. Journal of Psychologists and Counsellors in Schools, 26, 52–67. https://doi.org/10.1017/jgc.2016.CrossRef Houghton, S., Hattie, J., Carroll, A., Wood, L., & Baffour, B. (2016). It hurts to be lonely! Loneliness and positive mental wellbeing in Australian rural and urban adolescents. Journal of Psychologists and Counsellors in Schools, 26, 52–67. https://​doi.​org/​10.​1017/​jgc.​2016.CrossRef
go back to reference Jago, R., Sebire, S. J., Gorely, T., Cillero, I. H., & Biddle, S. J. H. (2011). “I’m on it 24/7 at the moment”: A qualitative examination of multi-screen viewing behaviours among UK 10-11-year olds. International Journal of Behavioural Nutrition and Physical Activity, 8, 85 https://doi.org/10.1186/1479-5868-8-85.CrossRef Jago, R., Sebire, S. J., Gorely, T., Cillero, I. H., & Biddle, S. J. H. (2011). “I’m on it 24/7 at the moment”: A qualitative examination of multi-screen viewing behaviours among UK 10-11-year olds. International Journal of Behavioural Nutrition and Physical Activity, 8, 85 https://​doi.​org/​10.​1186/​1479-5868-8-85.CrossRef
go back to reference Kelly, P., Marshall, S. J., Badland, H., Kerr, J., Oliver, M., Doherty, A. R. & Foster, C. (2013). An ethical framework for automated, wearable cameras in health behavior research. American journal of preventive medicine, 44(3), 314–319.CrossRef Kelly, P., Marshall, S. J., Badland, H., Kerr, J., Oliver, M., Doherty, A. R. & Foster, C. (2013). An ethical framework for automated, wearable cameras in health behavior research. American journal of preventive medicine, 44(3), 314–319.CrossRef
go back to reference Laursen, B., & Hartl, A. C. (2013). Understanding loneliness during adolescence: Developmental changes that increase the risk of perceived social isolation. Journal of Adolescence, 36(6), 1261–1268.CrossRef Laursen, B., & Hartl, A. C. (2013). Understanding loneliness during adolescence: Developmental changes that increase the risk of perceived social isolation. Journal of Adolescence, 36(6), 1261–1268.CrossRef
go back to reference Loveday, A., Sherar, L. B., Sanders, J. P., Sanderson, P. W., & Esliger, D. W. (2015). Technologies that assess the location of physical activity and sedentary behavior: a systematic review. Journal of medical Internet research, 17(8), e192.CrossRef Loveday, A., Sherar, L. B., Sanders, J. P., Sanderson, P. W., & Esliger, D. W. (2015). Technologies that assess the location of physical activity and sedentary behavior: a systematic review. Journal of medical Internet research, 17(8), e192.CrossRef
go back to reference Mesch, G. S., & Talmud, I. (2006). Online friendship formation, communication channels, and social closeness. International Journal of Internet Science, 1(1), 29–44. Mesch, G. S., & Talmud, I. (2006). Online friendship formation, communication channels, and social closeness. International Journal of Internet Science, 1(1), 29–44.
go back to reference Muthén, L. K., & Muthén, B. O. (1998). Mplus User’s Guide. 7th ed. Los Angeles, CA: Muthén & Muthén. 2012. Muthén, L. K., & Muthén, B. O. (1998). Mplus User’s Guide. 7th ed. Los Angeles, CA: Muthén & Muthén. 2012.
go back to reference O’Day, E. B., & Heimberg, R. G. (2021). Social media use, social anxiety, and loneliness: A systematic review. Computers in Human Behavior Reports, 3, 100070. O’Day, E. B., & Heimberg, R. G. (2021). Social media use, social anxiety, and loneliness: A systematic review. Computers in Human Behavior Reports, 3, 100070.
go back to reference Qualter, P., Brown, S. L., Rotenberg, K. J., Vanhalst, J., Harris, R. A., Goossens, L., Bangee, M., & Munn, P. (2013). Trajectories of loneliness during childhood and adolescence: predictors and health outcomes. Journal of Adolescence, 36, 1283–93. j.adolescence.2013.01.005.CrossRef Qualter, P., Brown, S. L., Rotenberg, K. J., Vanhalst, J., Harris, R. A., Goossens, L., Bangee, M., & Munn, P. (2013). Trajectories of loneliness during childhood and adolescence: predictors and health outcomes. Journal of Adolescence, 36, 1283–93. j.adolescence.2013.01.005.CrossRef
go back to reference Rokach, A. (2011). From loneliness to belonging: a review. Psychology Journal, 8(2), 70–81. Rokach, A. (2011). From loneliness to belonging: a review. Psychology Journal, 8(2), 70–81.
go back to reference Rosenberg, M., Houghton, S., Hunter, S. C., Zadow, C., Shilton, T., Wood, L., & Lawrence, D. (2018). A latent growth curve model to estimate electronic screen use patterns amongst adolescents aged 10 to 17 years. BMC Public Health, 18(1), 332.CrossRef Rosenberg, M., Houghton, S., Hunter, S. C., Zadow, C., Shilton, T., Wood, L., & Lawrence, D. (2018). A latent growth curve model to estimate electronic screen use patterns amongst adolescents aged 10 to 17 years. BMC Public Health, 18(1), 332.CrossRef
go back to reference Salmela-Aro, K., Upadyaya, K., Hakkarainen, K., Lonka, K., & Alho, K. (2016). The dark side of internet use: two longitudinal studies of excessive internet use, depressive symptoms, school burnout and engagement among Finnish early and late adolescents. Journal of Youth and Adolescence, 46, 343–357. https://doi.org/10.1007/s10964-016-0494-2.CrossRefPubMed Salmela-Aro, K., Upadyaya, K., Hakkarainen, K., Lonka, K., & Alho, K. (2016). The dark side of internet use: two longitudinal studies of excessive internet use, depressive symptoms, school burnout and engagement among Finnish early and late adolescents. Journal of Youth and Adolescence, 46, 343–357. https://​doi.​org/​10.​1007/​s10964-016-0494-2.CrossRefPubMed
go back to reference Smith, D., Leonis, T. & Anandavalli, S. (2021). Belonging and loneliness in cyberspace: impacts of social media on adolescents’ well-being. Australian Journal of Psychology, 73(1), 1–12.CrossRef Smith, D., Leonis, T. & Anandavalli, S. (2021). Belonging and loneliness in cyberspace: impacts of social media on adolescents’ well-being. Australian Journal of Psychology, 73(1), 1–12.CrossRef
go back to reference Wood, C. T., Skinner, A. C., Brown, J. D., Brown, C. L., Howard, J. B., Steiner, M. J., & Perrin, E. M. (2019). Concordance of child and parent reports of children’s screen media use. Academic Pediatrics, 19(5), 529–533.CrossRef Wood, C. T., Skinner, A. C., Brown, J. D., Brown, C. L., Howard, J. B., Steiner, M. J., & Perrin, E. M. (2019). Concordance of child and parent reports of children’s screen media use. Academic Pediatrics, 19(5), 529–533.CrossRef
Metagegevens
Titel
Reciprocal Relationships between Trajectories of Loneliness and Screen Media Use during Adolescence
Auteurs
David Lawrence
Simon C. Hunter
Rebecca Cunneen
Stephen J. Houghton
Corinne Zadow
Michael Rosenberg
Lisa Wood
Trevor Shilton
Publicatiedatum
12-08-2021
Uitgeverij
Springer US
Gepubliceerd in
Journal of Child and Family Studies / Uitgave 5/2022
Print ISSN: 1062-1024
Elektronisch ISSN: 1573-2843
DOI
https://doi.org/10.1007/s10826-021-02066-3

Andere artikelen Uitgave 5/2022

Journal of Child and Family Studies 5/2022 Naar de uitgave