Measurement Properties of the Suicidal Behaviour Questionnaire-Revised in Autistic Adults

Adults diagnosed with Autism Spectrum Conditions (ASC; hereafter autistic adults) are significantly more likely to report suicidal thoughts and suicidal behaviours (Cassidy et al. 2014, 2018c; Hedley and Uljarević 2018; Zahid and Upthegrove 2017) and to die by suicide (Hirvikoski et al. 2016; Kirby et al. 2019) compared to the general population. However, suicidality in autism is poorly understood and under researched (Cassidy and Rodgers 2017). In particular, there are few studies exploring why autistic people are more likely to contemplate and attempt suicide than the general population, to inform suicide prevention strategies for this group (Cassidy in press; Cassidy and Rodgers 2017; Hedley and Uljarević 2018). Addressing this crucial knowledge gap will require further research, but a key barrier is lack of validated research tools available to accurately capture suicidal thoughts and suicidal behaviours in autistic adults (Cassidy et al. 2018a; Hedley and Uljarević 2018). Therefore, it is unknown whether tools developed for, and validated in the general population operate similarly for autistic adults, or whether these tools need to be adapted for this group. This study therefore aimed to explore the appropriateness ¹ and measurement properties of a widely used and validated suicidality assessment tool originally developed for the general population, in autistic adults. This will in turn inform potential adaptations to better capture suicidal thoughts and suicidal behaviours in autistic adults in future research.

A previous systematic review showed that despite a growing number of studies exploring suicidality in autistic adults, none had yet used a tool with evidence of validity in this population, and no suicidality assessment tool had yet been developed or validated for autistic adults (Cassidy et al. 2018a). However, the review identified moderate-strong evidence in support of internal consistency, structural validity, and criterion validity for the Suicidal Behaviours Questionnaire-Revised (SBQ-R; Osman et al. 2001) in suicidality research in the general population (Cassidy et al. 2018a). The review found that the SBQ-R had been used in a number of research studies exploring suicidal thoughts and suicidal behaviours in the general population, but had not yet been extensively used or validated in clinical settings, or in psychiatric samples (Cassidy et al. 2018a). Importantly, the SBQ-R while brief and free to use had comparable quality of evidence for internal consistency, structural validity and criterion validity compared to other longer self-report and interview tools [i.e. the Beck Scale for Suicidal Ideation (BSS; Beck et al. 1988), and the Columbia Suicide Severity Rating Scale (C-SSRS; Posner et al. 2011)] that carry a high financial cost and are thus expensive to use in research (Cassidy et al. 2018a). The SBQ-R was therefore recommended as a validated suicidality assessment tool for use in general population research (Cassidy et al. 2018a). Hence, the SBQ-R could be a promising candidate tool to begin exploring the appropriateness and measurement properties of a suicidality assessment tool developed for and frequently used in research in the general population, in autistic adults (Cassidy et al. 2018a).

The SBQ-R is a four-item self-report questionnaire, assessing presence of lifetime suicidal thoughts and suicidal behaviours (item one), frequency of suicidal thoughts over the past year (item two), communication of threat of suicide attempt to others (item three), and likelihood of attempting suicide someday in the future (item four) (Osman et al. 2001). The characteristics of ASC could affect the measurement properties of this tool in autistic people in comparison to general population adults. For example, autistic adults often have difficulties in remembering what happened to them in the past (autobiographical memory) and imagining what will happen to them in the future (Crane et al. 2013; Lind and Bowler 2010). This could lead to difficulties particularly with item four (likelihood of future suicide attempt), and possibly items one and two (lifetime suicidality and frequency of suicidal thoughts over the past year). Autistic adults could also have difficulties in communicating their suicidality to others (item three) due to difficulties in identifying and describing their thoughts and feelings (termed Alexythymia; Bird et al. 2010), and differences in communication style (APA 2013) which is difficult for neurotypical people to interpret (Alkhaldi et al. 2019; Jaswal and Akhtar 2019; Mitchell et al. 2019; Sheppard et al. 2016). In addition to difficulties in communication, autistic adults may also have reduced opportunities to tell others about their suicidality, due to increased chance of being socially isolated (Hedley et al. 2018), and increased barriers to accessing appropriate diagnosis, treatment and support from mental health services (Au-Yeung et al. 2018; Camm-Crosbie et al. 2018; Crane et al. 2019).

Autistic people may thus interpret and respond differently to the SBQ-R items originally developed for the general population, which could be particularly problematic for research. For example, autistic people may tend to report reduced communication of suicide threat to others for reasons attributable to difficulties characteristic of autism, rather than reduced suicidality per se. This measurement difference could undermine the predictive power of this item in capturing suicidality in autistic adults compared to the general population. Such measurement differences could therefore result in reduced size of correlations between suicidality and other variables of interest in autistic people, lower internal consistency, and different factor structure, compared to the general population. Hence, it may not be valid to use tools such as the SBQ-R developed for the general population, to compare rates of suicidality between autistic and general population adults, as any group differences found could (at least in part) be attributed to measurement differences, rather than true differences in the variable (suicidality) under study.

Previous research has rarely explored the measurement properties of tools between groups (Cassidy et al. 2018a, b). This is highly problematic, as it is crucial when comparing mean scores on a tool between groups that both groups attribute the same meaning to the items, and all items measure the same construct (Byrne 2004). To explore this, measurement invariance analysis can quantitatively compare the structural equivalence of a tool between different groups or at different time points (Byrne 2004). Cognitive interviews (Willis and Artino 2013) can subsequently be used to explore how people within a group tend to interpret items in a tool, to check that items are interpreted as they were originally intended to capture the latent construct being measured, identify any problems in interpretation and inform changes to improve the relevance and clarity of items to a particular group. Given that no previous research has yet explored how autistic adults interpret and respond to suicidality assessment tools developed for and used in the general population, we use these two methods in combination to answer the following research questions:

Cognitive interviews ensure that the target population interpret and respond to questions as the researchers intend, a crucial part of establishing content validity for a tool (Willis et al. 2005). Lack of evidence for measurement invariance, would suggest that groups do not attribute the same meaning to the items. However, measurement invariance analysis does not specify how the items are interpreted differently by a group, to help interpret differences in item level scores between groups, or inform adaptations to the SBQ-R.

The current study aimed to explore the appropriateness and measurement properties of a widely used suicidality assessment tool validated for use in general population research, the SBQ-R (Osman et al. 2001) in autistic adults. Despite a growing body of research showing increased risk of suicidal thoughts and suicidal behaviours in autistic adults (Cassidy et al. 2014, 2018c), there is no suicidality assessment tool yet validated for this group (Cassidy et al. 2018a; Hedley and Uljarević 2018). The cognitive style and experiences of autistic people may affect the interpretation and thus validity of suicidality assessment tools such as the SBQ-R (Cassidy et al. 2018a). For example, communication differences (Alkhaldi et al. 2019; Jaswal and Akhtar 2019; Mitchell et al. 2019; Sheppard et al. 2016), lack of social connections (Cassidy in press; Cassidy et al. 2019; Hedley et al. 2017; Orsmond et al. 2013; Pelton and Cassidy 2017; Pelton et al. in press) and alexithymia (Bird et al.. 2010) could all result in reduced endorsement of communicating suicide threat to others, without necessarily indicating decreased experience of suicidality (Cassidy et al. 2018a). Difficulties in abstract future thinking in autistic people (Crane et al. 2013; Lind and Bowler 2010) could result in difficulties when rating one’s likelihood of attempting suicide ‘someday’ in the future (Cassidy et al. 2018a).

Consistent with our hypotheses, results suggest that the SBQ-R does not operate in the same way in autistic and general population adults. First, there was evidence for metric non-invariance for item four (likelihood of a future suicide attempt), with a significantly higher factor loading compared to the general population. This suggests that asking about likelihood of a future suicide attempt ‘someday’ appears to be more strongly associated with the underlying construct (suicidality) in the autistic compared to the general population group. There was also evidence of measurement non-invariance for item three (communication of threat of suicide attempt), with a lower factor loading in the autistic compared to the general population group. This suggests that asking about whether a person has told anyone else about their suicidality is less strongly associated with the underlying construct (suicidality) in the autistic compared to the general population group. Results thus suggest that autistic and general population adults attribute different meaning to these items of the SBQ-R, meaning that scores cannot be compared between these groups, or interpreted in line with the current clinical cut-off identified in the general population.

Cognitive interviews explored how autistic adults interpreted and responded to each item of the SBQ-R, to help interpret results from the above measurement invariance analysis. Many autistic adults reported the SBQ-R questions were difficult to interpret and respond to, were not autism relevant, and did not capture their experience of suicidality as autistic people. Item one did not capture serious, intense suicidal thoughts that occur in absence of a plan that could lead to spontaneous suicide attempts when lethal means of self-harm are available in the moment. Item two did not sufficiently capture the full range of frequency, duration and intensity of suicidal thoughts in the past year. Item three was considered irrelevant by many autistic people, given lack of social connections and opportunities to tell others about their suicidal intent. Item four was considered the hardest question to answer given the ambiguity of the term ‘someday’ and difficulty for many autistic adults for ‘predicting the future’.

Results from the cognitive interviews are consistent with the findings from the measurement invariance analysis, and provide important context for interpreting item level measurement differences between autistic and general population adults. For example, autistic people may lack social connections and opportunities to disclose suicide intent to others, but still experience suicidal intent. Therefore, this item may not be as strongly associated with other suicidality items (lifetime, current and future suicidal thoughts and behaviours), as in the general population. This is reflected in the lower factor loading for item three (communication of suicidal intent) in the autistic compared to the general population. Whereas for item four (likelihood of attempting suicide someday in the future) despite the difficulty in answering an ‘impossible’ future question, and having to choose a response in a ‘grey area’, this item is nevertheless more strongly correlated with the other suicidality items in autistic people compared to the general population. This is reflected in the significantly higher factor loading for item four in the autistic compared to the general population group.

These findings provide important and novel insights into the potentially unique nature of suicidality in autism, and how to adapt current tools to more accurately identify suicidality in this group. For example, many participants described their experiences of attempting suicide in the absence of a plan when lethal means presented themselves in a moment of crisis, and difficulty in understanding the concept of a suicide plan. Future research must further explore whether the phenomenology of suicide attempts in autism is different to the general population. For example, whether suicide attempts without a plan are more common in autistic people, or more driven by availability of access to lethal means of self-harm.

In adapting the SBQ-R, it will be important to not only consider results from our research, but results from studies that have adapted survey tools for autistic adults. Similar to the findings in our study, autistic adults tend to report difficulties with complex language, imprecise response options, lack of autism relevant items, and inappropriate or insensitive language (e.g. Nicolaidis et al. 2020). Therefore, it will be important to avoid multi-clause questions, include questions about the intensity and frequency of suicidal thoughts without a plan evident, and provide a more concrete alternative to gauge future suicide intent. Communication of suicidal thoughts and behaviours to others although difficult for autistic people due to lack of social connection and difficulty in social and communication skills, nevertheless appears to be a potentially important indicator and correlate of the underlying construct of suicidality in this group (although not as strong compared to the general population). Rather than excluding such items, it will be important clinically to explore not only whom the autistic person has disclosed their suicidality to (e.g. online, to a friend, or healthcare provider etc.), but also why the person may not have wanted or been able to disclose their suicidality to others (e.g. lack of support or contact with services, social isolation, difficulty in communicating one’s feelings, fear etc.). Probing the context and reasons for communicating suicidal thoughts and behaviours to others will be important to inform treatment, support and suicide safety planning for autistic people experiencing suicidal thoughts and behaviours.

The current study has a number of strengths and limitations. A key strength was the participatory approach to the study. Feedback from autistic people and those who support them ensured that the online survey, and cognitive interview schedule, were appropriate and accessible to autistic adults who took part, and comprehensively explored how autistic adults interpreted and responded to the items of the SBQ-R. A further strength was the mixed methods approach, which allowed us to explore the structural equivalence of a tool validated for the general population in comparison to autistic adults, and explore in more depth the root causes of any measurement non-invariance. Limitations of the current study are that the results are only relevant to autistic adults without intellectual disability (ID), who were diagnosed in adulthood—a particularly high-risk group for suicide (Cassidy et al. 2014, 2018c; Hirvikoski et al. 2016). It will be important to explore how tools can be adapted for autistic children, and autistic people with ID, where self-injurious behaviour are common (Minshawi et al. 2014) and it is unclear whether this is indicative of suicidality. Only 40% of the autistic group in the current study was male, which is lower than in the wider autistic population (Dworzynski et al. 2012). Autistic females without co-occurring intellectual disability are also more at risk of dying by suicide than non-autistic females (Hirvikoski et al. 2016; Kirby et al. 2019). This could limit the generalisability of our results to the wider autistic population. However, in the current study both groups had a similar gender ratio, meaning that measurement differences between the autistic and general population groups are not attributable to differences in gender. Given possible differences in suicide risk between autistic men and women, future research exploring the measurement properties of adapted autism specific suicidality assessment tools should test for measurement invariance between autistic men and women. However, this was beyond the scope of the current study. The sample size used in the cognitive interviews was in line with recommendations (Willis et al. 2005) and the researchers agreed that saturation point (where no new information emerged from further interviews) was reached.

Research has suggested that suicidality assessment tools on the whole are poor predictors of future suicide attempts, many perform worse than patient self-report or clinician opinion, and may therefore be a waste of valuable resources (Quinlivan et al. 2016, 2017). In light of such evidence, our focus on exploring the appropriateness and measurement properties of such an assessment tool in autistic people could be questioned. However, validity of tools vary according to context in which they are applied and the purpose they are used for (Kamphaus and Frick 2005). Our previous systematic review showed that the SBQ-R had strong evidence in support of its measurement properties specifically for use in research, for example to distinguish sub-groups of people who have attempted suicide from people who have not attempted suicide (Cassidy et al. 2018a). Hence, our results are most relevant for research, and future research will need to ascertain the best methods of assessing risk of future suicide attempts in autistic people. Further, given the lack of any validated tools or data regarding how autistic people may interpret questions attempting to probe suicidality, in light of the significantly increased risk of death by suicide in this group (Hirvikoski et al. 2016; Kirby et al. 2019), it is imperative to obtain these data to inform more accurate and useful questions for use in research and clinical practice. It is also important to highlight that such tools should form a start point, as part of a full psycho-social assessment when assessing suicide risk, and not be relied upon in isolation to inform any clinical judgment, particularly regarding access to treatment or support in any group.

Results from the current study suggest that a suicidality assessment tool widely used in the general population, the SBQ-R, cannot be directly compared with autistic people in research or used in clinical practice without adaptation for this group. In light of our results from the current study, our group is adapting the SBQ-R in partnership with autistic adults, with the aim of better capturing suicidality in this group. Our research joins an important call to action to explore suicidality in autism (Cassidy and Rodgers 2017), and develop new validated tools which more accurately capture the unique presentation of mental health problems and suicidality in this group for use in future research and clinical practice (Cassidy et al. 2018a, b; Wigham and McConachie 2014).