Plain English summary
Introduction
Methods
Experimental design for the selection of SWEMWBS states
Valuation platform
Interview process
“Could you tell me more about how easy/difficult completing this time trade-off task was?” |
“You told me that you felt confused about determining the indifferent point for some of these 8 trade-off tasks/choosing between this pair of mental well-being profiles, could you tell me more about it?” |
“What thoughts came to mind when you were making trade-offs between different mental well-being states/making a choice between this pair of mental well-being profile?” |
“Were the practice tasks useful for you? How?” |
“Did you think the instructions for the practice tasks clear for you?” |
“Could you summarise the factors were you considering when deciding the indifferent point?” |
“Did you find the number of valuation tasks (i.e. 8 trade-off tasks) manageable for you?” |
“Could you tell me how easy/how difficult of completing these 8 valuation tasks were in general? |
“Was the feedback slide useful for you?” |
“Could you summarise the factors were you considering when deciding the most preferred option between pairs of mental well-being profile?” |
“Did you find the number of valuation tasks (i.e. 8 tasks) manageable for you?” |
“Could you tell me how easy/how difficult of completing these 8 valuation tasks were in general?” |
“Did you think the first part of the interview (i.e. to make trade-off between choices of imaginable life) is easier or more difficult than the second part of the interview (i.e. to look at pairs of mental well-being profiles and choose the one you prefer)? Or did you feel roughly the same for both parts? Were they still manageable for you?” |
“Was the total number of valuation tasks in this interview (i.e. 8 trade-off tasks and 8 choice tasks between pairs of mental well-being profile) manageable to you?” |
“Would you prefer to have both parts of the interview or would you prefer only either one of them?” |
“Do you have any final overall feedback or comments of this interview?” |
Data analysis
Results
Characteristics | Number of participants |
---|---|
Gender | |
Male | 5 |
Female | 9 |
Age | |
18–30 | 3 |
31–40 | 5 |
41–50 | 2 |
51–60 | 2 |
> 60 | 2 |
Highest education level attained | |
GCSE | 1 |
O-Level | 2 |
A-Level | 2 |
Undergraduate | 4 |
Postgraduate | |
Master | 2 |
PhD | 3 |
Ethnicity | |
White | 12 |
Asian/Asian British | 1 |
Arab | 1 |
Occupation | |
Administrator/Manager/Coordinator | 6 |
Researcher | 3 |
Student | 1 |
Cleaner | 1 |
Retired | 3 |
SWEMWBS score | |
25 or less | 2 |
26–30 | 10 |
31–35 | 2 |
Mean score | 27.64 |
Theme 1: Format and structure
Inappropriate examples
“This is a really tough one… because I'm 67 and I don't really care about job applications”. (Female, 67)
Confusion on scenario completion
The DCE exercise simply required participants to click on the preferred option between two scenarios and no option selection problems were identified.“The scale is portrayed in a manner that my mind doesn't work. I find it quite strange to… delete and workup to equate a matching valuation”. (Male, 32)
Improvement of presentation layout
Theme 2: Items and levels
Contradiction in levels
“often deal with problems well despite the fact that you can’t think clearly now, that is strange. And you can rarely make up your mind, now this does not make sense. I mean how can I only think clearly some of the time and I can't make my mind up about anything, but I can deal with problems well often!” (Female, 67)
Non-linear effects of levels
“It's like a sort of a diminishing return... when you go from none of the time to rarely, it is a big jump. But then rarely to some of the time is still quite a big jump. Then some of the time to often is a smaller jump. Then from often to all of the time... it reduces....?” (Male, 32)
Inferiority of top levels
“I really struggled with... the whole concept of FMWB, because FMWB as described... is too perfect. I don't believe it and I don't like it... I'm a human being, I have ups and downs, that's quite normal and healthy. And it would be really unhealthy to be in this perfect state of MWB all of the time because... what's life about?” (Female, 67)
Theme 3: decision heuristics
Lexicographic ordering
“They might instinctively [be] going towards option B… just because you're relaxed, you've got people close to you…” (Male, 32)
Interpretation of levels
Four participants chose a preferred state with a higher level-sum score by counting the number of occurrences of each level in a state.“I would go for B because I think A seems more extreme like none none, and then all all, whereas B is... you know only got one all and one none. So it's sort of more middle of the road”. (Female, 29)
Personal and external factors
“Possibly I don't make up my mind about things, I'll leave things to her (i.e. his wife)…” (Male, 28)
Availability heuristic
Four participants used an analogy to illustrate the meaning of a state.“I have a brother-in-law... … who had a stroke when he was... late forties… … so I think this might kind of almost describe him. Because emotively he's still there, but physically... he’s not... able to do anything and mentally, he's not able to be doing... very much.” (Female, 67)
“Not able to make up your own mind at all...... again that’s a bit like... being in a prison or institutionalised or something if you can't ever make any decisions for yourself...” (Male, 32)
Rejection of unimaginable states
“Sometimes I was choosing the other one, not necessarily because I preferred it, but because I rejected one. It's like I just don't believe that.” (Female, 67)
Theme 4: valuation feasibility
However, all participants found the interviews manageable and the C-TTO and DCE tasks complementary. Participants also acknowledged the importance of the C-TTO practice tasks to relieve uncertainties from mere description of instructions and recognise their standard and position on time preference. The C-TTO and DCE tasks were beneficial and allowed them to reflect on life and their personal preferences.“It was tough... but... doable... in terms of... used quite a brainpower... it's just you're trying to hold a lot of things in your mind at the same time as you've got the profile of attributes on the left and then the profile on the right, and then is just trying to weight those up simultaneously.” (Male, 32)
Theme 5: valuation outcome
Failure to reach the C-TTO indifference point
Non-trading effects
“I would be happy with either of those, because none of them are particularly... gonna make you sad, are they? Okay, it’s not... all of the time, but I don’t think life in general is like that... ...” (Female, 51)
Discussion
Issue identified | Related section | Proposed modification |
---|---|---|
Inappropriate C-TTO practice examples | 3.1.1 | One additional version of practice example related to physical health and relationship |
Confusion about the time trade-off procedure | 3.1.2 | More detailed explanations of the instructions Slowing the instructing speed Encouraging participants to raise questions Clarification of practice states before completion More step-by-step trade-off demonstrations |
Visual difficulty in differentiating the states within the C-TTO feedback module | 3.1.3 | Guidance to enhance the readability of the states line-by-line will be provided |
Incomprehensible combinations of levels of attribute | 3.2.1 | The selection of experimental design choice sets with potential uncommonly reported states could be avoided |
The exhibition of lexicographic ordering | 3.3.1 | Participants will be instructed to consider all attributes within the allocated states |
The existence of preference heterogeneity | 3.3.3 | Advanced modelling techniques with the inclusion of covariates and interaction terms could be applied |
Visualisation of states from a third party perspective | 3.3.4 | Participants will be told by the instruction to imagine themselves being in the allocated states |
Promising manageability of the number of tasks | 3.4 | The number of tasks for each of the C-TTO and DCE parts will be increased from 8 to 10 (i.e. 10 C-TTO and 10 DCE tasks) |