Developing a proxy version of the Adult social care outcome toolkit (ASCOT)

Abstract

Background

Social care-related quality of life is a key outcome indicator used in the evaluation of social care interventions and policy. It is not, however, always possible to collect quality of life data by self-report even with adaptations for people with cognitive or communication impairments.

A new proxy-report version of the Adult Social Care Outcomes Toolkit (ASCOT) measure of social care-related quality of life was developed to address the issues of wider inclusion of people with cognitive or communication difficulties who may otherwise be systematically excluded. The development of the proxy-report ASCOT questionnaire was informed by literature review and earlier work that identified the key issues and challenges associated with proxy-reported outcomes.

Methods

To evaluate the acceptability and content validity of the ASCOT-Proxy, qualitative cognitive interviews were conducted with unpaid carers or care workers of people with cognitive or communication impairments. The proxy respondents were invited to ‘think aloud’ while completing the questionnaire. Follow-up probes were asked to elicit further detail of the respondent’s comprehension of the format, layout and content of each item and also how they weighed up the options to formulate a response.

Results

A total of 25 unpaid carers and care workers participated in three iterative rounds of cognitive interviews. The findings indicate that the items were well-understood and the concepts were consistent with the item definitions for the standard self-completion version of ASCOT with minor modifications to the draft ASCOT-Proxy. The ASCOT-Proxy allows respondents to rate the proxy-proxy and proxy-patient perspectives, which improved the acceptability of proxy report.

Conclusions

A new proxy-report version of ASCOT was developed with evidence of its qualitative content validity and acceptability. The ASCOT-Proxy is ready for empirical testing of its suitability for data collection as a self-completion and/or interview questionnaire, and also evaluation of its psychometric properties.

Background

In the UK, social care refers to long-term care services that aim to maintain the quality of life of adults with long-term health conditions or ageing-related support needs (for example, home care, day care or residential care). The increased demand for social care due to ageing populations in Europe combined, in some countries, with the trend to reduce public spending has contributed to an interest in the evaluation of the effectiveness and cost-effectiveness of social care based on the outcomes of people who use services [1]. The shift towards outcomes-based social care policy and administration in the UK has also been shaped by narratives of personalisation, increased choice and control for service users, and wider accountability and transparency in the use of public funds [2, 3].

To evaluate the effectiveness of social care based on personal outcomes rather than outputs, it is important to define the objectives of social care and to develop an appropriate instrument that reflects these objectives. Social care may be broadly conceptualised as services that compensate for the effect of impairments on overall wellbeing or quality of life [4,5,6,7]. Therefore, the evaluation of social care support requires the consideration of a broad range of quality of life attributes beyond health that are sensitive to the person-centred, compensatory activity of social care. It has been recognised that condition-specific instruments may not be suitable to assess the broader impact of social care support that is accessed by adults with a wide range of needs and also that generic health-related quality of life measures may not be sensitive enough to capture the outcomes of social care [8].

The construct of social care-related quality of life (SCRQoL) has been proposed as the basis for developing instruments to measure social care outcomes [7]. SCRQoL reflects aspects of quality of life that are important to people who use social care services and may also be conceptualised as the target of the compensatory activity of social care support [9, 10]. The Adult Social Care Outcomes Toolkit (ASCOT) instrument is a preference-weighted measure based on the construct of social care-related quality of life [10,11,12]. The psychometric properties of the ASCOT self-completion questionnaire (ASCOT-SCT4) have been established in samples of English, Dutch and Australian older adults [12,13,14]. The instrument has been recognised as a suitable outcome measure for the evaluation of adult social care services [13, 15] and has been used in local and national social care data collections and evaluation studies in England to inform policy strategy, commissioning and care practice [2, 16,17,18].

If individual quality of life is used evaluate the effectiveness of interventions and policy of social care in this way, a key challenge is how to collect quality of life data from people with cognitive or communication difficulties, who are unable to answer on their own behalf even with support, alternative formats or communication aids, so to avoid the issues of sampling bias and systematic exclusion from ‘having a voice’ [19,20,21]. In the evaluation of health care interventions using patient-reported outcome or experience measures, a widely-used method is the collection of data from someone who answers on behalf of the individual whose quality of life is to be assessed (by ‘proxy’). Despite its widespread use, it has been argued that data collection by proxy-report should only be used as a last resort when other methods are not possible because it takes away the individual’s opportunity to express their views [22, 23]. While the standard in quality of life measurement is self-report whenever possible, it is recognised that proxy-report is preferable to the systematic exclusion of individuals who are unable to self-report based primarily on the principles of equity and inclusion, as well as the potential methodological issues associated with missing data and bias [19, 20].

The aim of the study was to develop a new social care-related quality of life measure based on the ASCOT-SCT4 that could be used in circumstances where the individual is unable to self-report and other methods of eliciting outcomes information are not feasible. This article outlines the second phase of work that sought to apply the findings identified in the first phase of the project [24] to develop the content and format of the new instrument and to evaluate its content validity and acceptability.

Methods

This qualitative study aimed to develop a proxy-report version of the self-completion ASCOT-SCT4 with adequate content validity and acceptability to proxy respondents [10]. Content validity is a measurement property that assesses whether questionnaire items reflect the perspective of the population of interest (in this case, proxy respondents for adults who use social care services) and also whether the questionnaire format, wording and instructions are relevant, understandable and acceptable [25]. The study involved an initial phase to develop items and the questionnaire layout based on an earlier phase of research [21, 24] followed by cognitive interviews to refine the questionnaire and establish its content validity [25,26,27,28].

Questionnaire development

While recognising that proxy-report is not equivalent to self-report, the questionnaire development sought to draw upon the same principles and concepts as the ASCOT-SCT4 [10, 12]. The eight attributes captured by the ASCOT-SCT4 are: Control over daily life; Occupation (‘doing things I value and enjoy’); Social participation and involvement; Personal safety; Personal cleanliness and comfort; Food and drink; Accommodation cleanliness and comfort; and Dignity. The instrument has four response options for each item to distinguish between the ideal state, no needs, some needs and high-level needs in relation to each quality of life attribute [10] (see Table 1).

Table 1 ASCOT response levels (adapted from Netten et al. [10])

The key challenges of developing a proxy-report instrument were explored in an earlier phase of research, which involved a rapid literature review, focus groups and in-depth one-to-one interviews [21, 24]. In the English Adult Social Care Survey, the majority of proxy respondents are family or friends who live with the respondent or elsewhere and proxy-report by care staff is relatively infrequent (5.9% of proxy respondents in the 2014/15 ASCS) [29]. It should be recognised that some people may have no or limited contact with family or friends, so care staff may be called upon to act as proxy respondents, especially if they have frequent contact with individuals and in-depth knowledge of the person’s needs and preferences through their practice of care (for example, with support staff who work intensively one-to-one or with small groups of adults with learning disabilities). While some studies have found that proxy-reported quality of life varies by ‘type’ of proxy, for example, health or social care professionals compared to family [20, 30,31,32,33], other studies have not found significant differences by type of proxy [34,35,36]. There is also evidence that this may be accounted for by differences in proximity, intimacy and frequency of contact between the proxy and the individual rather than the ‘type’ of proxy per se [37, 38]. Therefore, in this study and also the earlier phase of research, we considered both care support staff and also family or friends as potential proxy respondents.

The key challenges in proxy response identified in this earlier phase of research were: (1) care workers’ resistance to the idea of acting as a proxy respondent; (2) the perceived difficulty of judging care recipients’ internal subjective state (i.e. how they ‘think’ and ‘feel’); and (3) proxy respondents’ wanted to express that their own response (as a proxy) differed from how they thought the care recipient would respond based on the proxy respondents’ judgement of the care recipients’ perspective [24]. Proxy respondents also commented that they found it difficult to differentiate between the top response options of the ‘ideal state’ and ‘no needs’ [24], whereas earlier work on self-report had found that the inclusion of the ‘ideal state’ supported comprehension and judgement of response [10] (see Table 1). Since the ‘ideal state’ represents that an individual’s needs and preferences are met to his/her desired level [10], it was hypothesised that this was a proxy-specific issue related to the proxy perspective adopted in formulating a response.

In response to the issues summarised above, the authors with advice from the ASCOT development team at the University of Kent (www.pssru.ac.uk/ascot), explored four significant variations in the format and content of the draft proxy version of the questionnaire. First, the questionnaire was formatted to allow the collection of proxy-ratings of quality of life for each attribute from the two proxy perspectives identified by Pickard and Knight [39]. The proxy-proxy perspective represents the proxy’s view based on their own preferences and values, while the proxy-patient perspective asks the proxy to answer based on the proxy’s best attempt at a reconstruction of the individual’s internal mental state based on their knowledge of the individual and their preferences. It was hypothesised that allowing proxy respondents to express both proxy perspectives may improve the acceptability of a proxy-report instrument, especially for paid carers. Furthermore, the specification of the proxy perspective within the questionnaire was anticipated to reduce the potential for bias that may arise if different types of proxy respondents (i.e. paid or unpaid carers), without clear instructions as to which proxy response to use, systematically refer to different proxy perspectives in order to formulate their response to the questionnaire.

The second modification was to include a comments box after each question. In the first phase of work, care staff reported that they would feel more comfortable with acting as a proxy if they had the opportunity to add further detail or explanation of their responses, especially if quality of life had been rated as some or high-level needs [24]. The inclusion of the comments boxes aimed to improve the acceptability of the questionnaire, especially for care workers.

Third, during the development of the original ASCOT-SCT4, it was found that some respondents understood the Dignity item to refer to the impact of having help on their self-esteem and self-perception, rather than the effect of the way they are supported or treated by care staff [10]. Based on this evidence, an additional item was developed for the ASCOT-SCT4 to allow respondents to express any difficulties they have with coming to terms with needing help [10]. This item is not considered in scoring the ASCOT-SCT4. In the development of the proxy-report questionnaire, by contrast, it was initially decided to include only one Dignity item because of the additional complexity of asking respondents to adopt two different proxy perspectives.

Finally, based on the proxy respondent’s report that they found it difficult to judge how the individual thinks or feels [24], the wording of some items was modified in the draft questionnaire to be more ‘objective’ (version 1.0): for example, Occupation was revised to refer to what the person ‘wants to do’ rather than ‘values and enjoys’, Social participation referred to any type of social activity rather than specifically to social interaction with ‘people s/he likes’ and the items for Personal safety and Personal comfort and cleanliness were modified to refer to ‘being’ rather than ‘feeling’ safe or clean and comfortable. In the first phase of interviews with the version 1.0 questionnaire, we sought to explore whether the use of more ‘objective’ criteria would improve the acceptability of the proxy-report questionnaire. The research team were, however, also aware that these modifications were in tension with the underlying conceptual basis of the ASCOT measure to capture social care-related quality of life with reference to the individual’s preferences, values and attitudes.

A summary of the process of the initial development and refinement of the questionnaire is shown in Fig. 1.

Fig. 1

Flow diagram of the process of cognitive testing

Recruitment of participants

Participants were recruited purposively via three social care provider organisations across three local authorities in South East England. The study inclusion criteria were adult care workers or unpaid family carers (aged 18 years or over) in regular contact with an adult who uses social care services and who would be unable to answer the ASCOT-SCT4 on his/her own behalf due to cognitive and/or communication impairments. Potential participants were sent an invitation letter and information sheet by the care provider, which explained the purpose of the research. Those who were interested in participating were invited to contact the research team to arrange an interview and to answer any questions about the purpose or nature of the research. A total of 25 proxy respondents (13 care workers, 12 family carers) were recruited and completed a cognitive interview.

The study was approved by the national social care research ethics committee in England (reference: 13/IEC08/0020). Written informed consent was obtained from all participants prior to interview.

Cognitive interviews

All interviews were conducted between June 2015 and May 2016 by three trained qualitative interviewers (SR, GC, JC), two of whom had prior experience of cognitive interviews in the development of self-report questionnaires (SR, JC). The interviews lasted between 40 and 75 min and took place at a location convenient for the respondent – usually, in a private office or meeting room in the workplace or at the respondent’s home. The proxy respondents were asked to complete the ASCOT questionnaire using a think-aloud method of cognitive interviewing with concurrent follow-up probes [26]. Think-aloud requires the respondent to speak aloud their thoughts as they read the instructions and complete each item of the questionnaire. Following this, the interviewers used verbal probing to further explore the comprehension, clarity and relevance of each item and its response options (for example, ‘why did you choose this response rather than the level above/below?’) [26] (see Table 2 for further examples). The probes were used flexibly by the interviewers to explore the respondents’ understanding and ability to respond to the questions, as well as to identify any potential issues with the acceptability of item wording, format or layout. For some items, alternative versions were presented to assess the proxy respondents’ preferences and to evaluate whether the different content or format would affect the respondent’s comprehension, judgement or response. After the cognitive interview, the respondents were asked to reflect on their overall experience of completing the questionnaire with a focus on clarity and the acceptability of answering the questionnaire as a proxy respondent (see Table 3).

Table 2 Examples of cognitive interview probes

Table 3 Interview guide

Qualitative analysis

The interviews were audio-recorded and transcribed for analysis. The written interview transcripts were analysed using NVivo version 10 software [40]. While it is possible to informally analyse data from cognitive interviews to identify and resolve issues with item wording, layout or format, the use of systematic thematic analysis may provide a more rigorous way of analysing think-aloud cognitive interviews ([26] p.157). The researchers agreed on an initial analytical coding structure based on the framework of cognitive interviews as a technique to explore the understanding and acceptability of questionnaire item wording, format and layout (see Table 4) ([26] pp.164–7). The data was coded by two researchers (SR, GC). The coding structure was iteratively refined based on themes and subthemes that emerged from the data, which allowed the identification of key issues and insights.

Table 4 Coding tree

Iterative questionnaire refinement

The questionnaire was refined through an iterative process of cognitive interviewing ([26] p.146). Each round of interviews had a minimum of six and a maximum of twelve participants, which is within the guidelines suggested by Willis [26] (p.138). The researchers met during the fieldwork to review the data, to discuss any emerging themes, and to agree on any modifications to the questionnaire.

After six interviews with questionnaire version 1.0, the data were analysed to identify any issues with comprehension, response or acceptability that would justify modification of the questionnaire at that point. The edits were applied to generate version 2.0 of the questionnaire, which was tested in the next round of interviews (n = 7). The second interim analysis considered whether any further changes were required based on the interview data. The questionnaire with these changes is referred to as version 3.0, which was then tested in the final set of interviews (n = 12). During this final set of interviews, the research team agreed that we had reached saturation (i.e. no new concepts or issues emerged). At this stage, the third and final analysis was conducted of this final set of interviews, both separately and also alongside the two earlier phases.

A summary of the iterative development process is shown in Fig. 1.

Results

Sample characteristics

The demographic characteristics of the sample are shown in Table 5. Of the 25 respondents, 60% were female. The sample comprised care staff (n = 13) or family carers (n = 12). Half of the family carers responded on behalf of their spouse or partner (n = 6). The other carers responded on behalf of other family members (their adult children, parents or siblings). The majority of proxy respondents reported that the care recipient’s primary support reason was an intellectual disability (n = 9) or autism (n = 7). The remaining interviews were conducted with proxies who spoke on behalf of someone with dementia or another long-term condition that affected cognitive ability, comprehension or ability to respond.

Table 5 Characteristics of the cognitive interview participants (n = 25)

Cognitive interviews

With the iterative modifications outlined in detail below, the questionnaire items were found to be understandable and acceptable to care staff and family carers asked to act as proxy respondents. A summary of the findings in relation to the understanding of concepts and acceptability of each item to proxy respondents is presented in Table 6.

Table 6 Ease of understanding and acceptability of the ASCOT-Proxy items

Interim analysis and review (1)

Based on analysis of the first round of interviews, the modifications outlined in Table 7 were applied to the draft questionnaire. For Occupation and Social participation, it was found that the wording of the original ASCOT-SCT4 item or response options were preferred to the version 1.0 proxy questionnaire because they were perceived to be more person-centred (“looks at the individual”). One formal carer, for example, compared the two versions of the Occupation item, ‘what s/he values and enjoys’ (original ASCOT-SCT4) to ‘what s/he wants to do’ (version 1.0), as follows:

“I think that’s worded better… because it’s talking about her values and her enjoyment… When we’re looking at the people we support we’re looking at their values and what they like to do, their likes and dislikes - so I just think it’s easier to answer”. [GC_FC_01]

Table 7 Modifications based on respondent feedback (questionnaire version 1.0)

Because of this, the items of Occupation, Social participation, Personal safety and Personal comfort and cleanliness were modified to correspond to the original ASCOT-SCT4 item wording to improve acceptability and comprehension (see Table 7).

Based on the earlier research that explored care staff and carers’ views on the ASCOT-SCT4 questionnaire, the version 1.0 proxy questionnaire sought to avoid the use of ‘enough’ because of potential issues with comprehension or acceptability [24]. The first round of interviews identified that respondents found the alternative wording used in version 1.0 (‘It’s ok’) to be too informal and imprecise:

“I just don’t like how that looks, just remove the ‘it’s ok’… it’s too vague”. [SR_FC_02]

The first round of interviews explored whether the adaptation of the Personal safety and Personal comfort and cleanliness items to be more ‘objective’ (i.e. being rather than feeling safe) would improve acceptability of the measure to proxy respondents [24]. Interestingly, despite the finding of the earlier phase of work that some respondents were uncomfortable with rating the cared-for person’s internal subjective state, the respondents spontaneously answered the questions by attempting to construct the cared-for person’s subjective perspective based on their observations and knowledge of the individual. This was prompted by the questionnaire format of asking respondents to rate the two different proxy perspectives (proxy-proxy and proxy-patient).

“It took me a little bit longer to answer this one I thought, I don’t know whether you thought exactly the same but because obviously I’m trying to think how this individual is thinking, and also I'm thinking about what she does, and is that because she doesn’t feel 100 percent safe, and what does safe mean to her as well”. [GC_FC_01]

Based on this finding, the Personal safety and Personal comfort and cleanliness were adapted to correspond to the original ASCOT-SCT4, which refers to feeling rather than being safe or clean and presentable. In subsequent rounds of cognitive interviews, the respondents were able to understand and respond to these two modified items with no evidence of any issues with acceptability.

Interim analysis and review (2)

Based on advice from the ASCOT development team, the researchers asked respondents in the second round of cognitive interviews to compare two versions of the questionnaire (see Table 8). Cognitive interviews with a carer version of the ASCOT, the ASCOT-Carer, had identified that some respondents were not able to understand the word ‘adequate’ used in some response options [41]. The aim of comparing the two versions in this study, which used either ‘adequate’ or ‘enough’ in the response options for six of the eight items (see Table 8), was to further test the acceptability and comprehension of ‘adequate’ and also to determine whether the alternative item wording (‘enough’) would affect comprehension, judgement or response in the context of a proxy-report questionnaire. For each of the six items, the respondents were asked to respond to the version using ‘adequate’ before being asked to review the version using ‘enough’. The interviewer probed to explore whether the respondent preferred one version to the other, and also whether the different wording affected the respondent’s comprehension, judgement or response.

Table 8 Questionnaire version 2.0 modifications to response options

There was no clear preference between the two versions. Two respondents showed a consistent preference for ‘adequate’ [GC_FC_05; SR_FC_03], while three respondents preferred ‘enough’ [GC_FC_04; GC_FC_06; SR_IC_01; SR_IC_02] and one respondent indicated a preference for ‘adequate’ or ‘enough’ that varied by item [SR_IC_03]. The findings indicate that ‘adequate’ was understood by some respondents to be an objective baseline related to social services’ or other external agencies’ standards, while ‘enough’ implied a subjective standard.

“If you’re social services… they’ve got very different opinions of what’s adequate to bring a child up in a house. Most of us would say definitely not, but because there’s levels, there’s only concerns on certain levels, isn’t there? So adequate makes me feel that that’s what it’s looking at, whether it’s adequate. Whereas this is more about--, if you’re asking my opinion then it’s whether it’s enough for me”. [SR_IC_01]

While the intention of the ASCOT tools is to ask respondents to rate quality of life from a subjective perspective rather than an objective standard, it was decided to use the term ‘adequate’ rather than ‘enough’ to maintain comparability between the ASCOT-SCT4 and the proxy-report version.

Proxy perspectives

Pickard and Knight’s (2005) conceptual framework of proxy response was used to develop proxy questionnaire items that asked: (1) respondents to rate their own view (proxy-proxy perspective); and (2) what they thought the care recipient would say (proxy-patient perspective). This dual response approach was found to be acceptable to care worker and family carer proxies. The format allowed the respondents to express differences between their opinion and their understanding of the care recipient’s perspective. These differences were explained by reference to different personal preferences influenced, for example, by social or cultural norms or health condition-related factors:

“Her generation don’t do all this changing sheets every day malarkey and they don’t believe in so much washing. That’s just alien to them. And she used to say, “That’s fine, it doesn’t need a wash”. But to me it would be like reeking and I’d be like, “No, you must”. [SR_IC_10]

While the results suggest that the two proxy perspectives improved the acceptability of the questionnaire, some respondents were confused by the questionnaire layout with two columns that correspond to the two proxy perspectives (see Fig. 2). In the first two rounds of interviews, four of the thirteen proxy respondents answered the first question (Food and drink) by ticking multiple boxes in each column or writing yes/no to indicate whether each statement applied or not. They were, however, able to correctly complete the remainder of the questionnaire once they had realised that the format required one tick per column. Based on these findings, the questionnaire for version 3.0 was re-formatted to include ‘please tick one box’ above the response boxes (Fig. 2).

Fig. 2

Questionnaire format

In the third round of interviews (n = 12), two family carers were initially confused by the layout. The interviewers had been briefed to allow the respondent to continue the questionnaire without prompts to highlight the questionnaire instructions to tick one box per column. In one of the two cases ([SR_IC_05]), the respondent spontaneously realised that their initial response with multiple ticks had been incorrect and amended their response. There were no issues with the remaining questions. In the other case ([SR_IC_07]), the respondent directly asked the interviewer whether they needed to tick multiple boxes in each column. Once they had seen the instruction, with a prompt from the interviewer, the respondent understood that only one tick per column was required to indicate their rating:

“Tick, ‘Please tick one box for each,’ oh, ‘Tick one box for each column.’ Oh I see, so [sigh], so it’s just the top one then isn’t it?” [SR_IC_07]

Dignity

The ASCOT Dignity item aims to capture the positive and negative psychological effects of formal support and care on the service user’s personal sense of significance [10]. In this study, it was found that the Dignity item was generally understood to relate to the effect of care workers' working practices and interpersonal interactions on the cared-for person’s sense-of-self. The interviews often focussed on interpersonal interactions associated with personal care:

“I would say dignity is like--, if someone is having a bath and you’re bathing them, like for instance they get out of the bath you turn away or you give them towels so they can hide themselves so they’ve, you know, privacy”. [SR_FC_02]

Other respondents, however, reflected on the wider implications of care practice and the quality of interpersonal interactions with care staff on the individual’s self-perception and dignity:

“I would say the quality of care is what results in dignity and respect… If the person who is going in to support them ultimately has that person’s--, as top most priority their needs, they will always think better of themselves at the end of that”. [JC_FC_03]

In one interview with a family carer, although the carer initially answered the question in relation to paid care, the boundary between the support given by family and paid carers became blurred:

“I would probably tick the top one and say he thinks and feels better about himself because we always do, we always praise him up. We always tell him he looks good”. [SR_IC_03]

To address this issue, the question format was modified to bold and italicize the reference in the question to paid care.

As observed in the development of the ASCOT-SCT4 [10], although less frequently, three proxy respondents (care worker (1), family carer (2)) spoke of the effect of needing help on the individual’s sense-of-self.

“Have I ever met anyone who has been quite positive and upbeat? I don’t think I ever have. I think if I did they most probably wouldn’t want our support … it’s the very act of receiving care that may make them feel undermined”. [JC_FC_03]

Importantly, however, the ‘think aloud’ indicated that the three proxy respondents made their final judgement and response based on the way in which the care recipient was treated by care workers, rather than the effect of needing help on the individual’s sense-of-self.

Discussion

The aim of this research was to develop a proxy-report measure of social care-related quality of life and establish its qualitative content validity through cognitive interviews [26]. This qualitative research showed that proxy respondents were generally able to respond to an adapted version of the ASCOT-SCT4 instrument designed to capture social care-related quality of life. The acceptability of the questionnaire was improved by the use of the two proxy perspectives proposed by Pickard and Knight [39] to allow respondents to express their opinion (proxy-proxy perspective) and distinguish this from their view of the care recipient’s own perspective (proxy-patient perspective). The findings of this study indicate that proxy respondents found it acceptable to adopt both proxy-proxy and proxy-patient perspectives and that they were able to understand and respond to the items based on both perspectives.

While an earlier phase of work found that care workers were hesitant to provide proxy judgements of individual care recipient quality of life especially when questions related to subjective concepts such as feeling safe rather than more objective judgements, such as being safe [24], this qualitative study indicates a high-level of acceptability of judging subjective states using the two proxy perspective conceptual framework [39]. Furthermore, the first round of interviews indicated that the item modifications that made the items less subjective in description (i.e. being rather than feeling safe) were less acceptable than items framed around the individual’s subjective perspective when proxy respondents were given a questionnaire based on dual proxy-patient and proxy-proxy perspective report. However, some proxy respondents still had difficulty in projecting themselves into the individual’s internal state when adopting the proxy-patient perspective, especially when the proxy respondent had to rely on ambiguous external cues and behaviours to formulate their response.

While the two proxy perspective methodology has improved the acceptability of proxy report of social care-related quality of life in the eight ASCOT-SCT4 domains [24], there remains the question of which source of information to use if the aggregate data is to be used for policy, administration and planning of social care services. The proxy-report framework proposed by Pickard and Knight [39] was developed in the context of health-related quality of life alongside a differentiation between the intra-rater gap between proxy perspectives and also the inter-rater gap between self-rated and proxy-rated quality of life. Indeed, there is an extensive literature that compares self-report and proxy-report, of which only a minority of studies have found no significant difference between self-report and proxy-report [42,43,44] or that proxy-report tends to overestimate quality of life compared to self-report [45]. The majority of studies that directly compare self-report and proxy-report have found an underestimation of quality of life by proxy respondents compared to patient self-report (for example, [20, 34, 38, 46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65]. While it is unclear whether these differences represent a systematic bias in proxy-report that may be extrapolated to cases where proxy-report is the only means of estimating an individual’s quality of life [19], it is evident that proxy-reported quality of life cannot simply be used interchangeably as a substitute for self-reported quality of life.

The two proxy perspectives may be conceptualised as two separate sources of information based on the proxy respondent’s mental construct of the care recipient’s likely internal state informed by their knowledge of the individual (proxy-patient perspective) or the proxy’s judgement based on their internal standards, preferences and attitudes (proxy-proxy perspective) [39]. Based on this view, it has been proposed that in the absence of a clear justification of one over the other, it would be informative to gather ratings based on both perspectives [21, 39]. When collecting data from both proxy perspectives, there remains the judgement as to how to use the two sets of data in analysing, reporting and applying the data in policy-making and service improvement. As such, while the qualitative analysis presented in this paper has shown that proxy respondents are able to rate both proxy perspectives and that this improves the acceptability of the questionnaire, further work is required to establish the psychometric properties of the data collected from both proxy perspectives and to explore how best to handle, analyse and apply the data.

The study has some limitations. First, the first phase of cognitive interviews included only care workers who supported adults with intellectual disabilities in a supported living context. In the subsequent two rounds of interviews, we purposively sampled a mixture of care workers and family carers because of evidence that different types of proxy respondent may respond differently to proxy-report questionnaires [66, 67]. We also sought to include carers or care staff who supported adults with other support needs and who lived in different support contexts to ensure that the questionnaire was feasible and acceptable to a range of proxy respondents. Second, the testing of the final version indicated that not all respondents were immediately able to understand the layout of the questionnaire. While there was no evidence of comprehension issues related to the two proxy perspectives, some respondents assumed that they had to tick all boxes in each column that applied rather than tick one box to indicate which statement best applies. This issue persisted even with a modification to include the instruction to ‘please tick one box for each column’ above the response options (see Fig. 2). Although it was found that respondents were eventually able to work out how to record their response to the questions, a pilot self-completion survey would be required to explore whether proxy respondents would be able to work through this without the presence of an interviewer in the context of a paper-based self-completion survey.

Conclusions

In conclusion, it was found that the item wording, format and layout of the ASCOT-Proxy questionnaire was understandable and acceptable to care workers and family carers invited to act as proxy respondents. Respondents’ comprehension of items corresponded to the construct definitions captured by the items in the self-report version of ASCOT-SCT4. This was improved by modifications to the ASCOT-Proxy to minimise differences in item wording between the ASCOT-Proxy and ASCOT-SCT4, whilst maintaining the acceptability of the instrument. The next step to evaluate the feasibility and psychometric properties of the new instrument in the context of self-completion surveys is justified.