Introduction
Method
Development narrative
First PDSA cycle (April 2014–April 2015)
Plan: initial preparations
Cycle: study | N | Sex | Characteristics |
---|---|---|---|
Cycle 1: stakeholder interviews (patients) | 18 | 39% male | Years of experience as a patient: - 7 (6–10) - 6 (11–20) - 4 (21–30) - 1 (31–40) Proportion inpatient vs. outpatient: - 38% inpatient, 62% outpatient |
Cycle 1: stakeholder interviews (clinicians) | 37 | 32% male | Years in practice: - 3 (< 5 years) - 19 (6–15 years) - 15 (16–30 years) |
Cycle 1: initial factor structure | 550 | 26% male | 98% Heterosexual Level of Education: - 102 No University - 167 1–3 Years University - 168 University - 123 Masters - 14 Doctoral 23 in Current Mental Health Treatment |
Cycle 2: branching logic design and testing And Cycle 3: re-evaluating items and measure structure | 794 | 22% male | 98% Heterosexual Level of Education: - 149 No University - 223 1–3 Years University - 222 University - 170 Masters - 30 Doctoral 36 in Current Mental Health Treatment |
Cycle 3: re-evaluating items and measure structure (clinical sample) | 222 | 33% male | 96% Heterosexual Level of Education: - 113 No University - 39 Vocational School - 48 University - 6 Masters Clinical Setting for data collection: - 41 Inpatient - 171 Outpatient |
Cycle 4: re-evaluating dynamic structure | 323 | 30% male | No other demographics provided for this clinical sample |
Do: initial item development
NORSE 1.0 Scale | Items | Norse 2.0 Scale | Items |
---|---|---|---|
Attachment | 4 | – | – |
Avoidance | 6 | Situational avoidance | 3 |
– | – | Social avoidance | 3 |
Connectedness | 6 | Social safety | 6 |
Demoralization | 5 | Hopelessness | 5 |
Eating problems | 6 | Eating problems | 5 |
Emotional distancing | 2 | Internal avoidance | 5 |
Hurtful rumination | 5 | Worry | 3 |
Hypervigilance | 3 | Trauma reaction | 4 |
Perfectionism-control | 6 | Control | 4 |
Pressure from negative affect | 6 | Sad affect | 4 |
Psychosis | 3 | – | – |
Relational distress | 5 | Irritability | 3 |
Resilience and personal coping | 10 | General functioning | 3 |
Social role functioning | 2 | ||
Somatic anxiety | 2 | Somatic anxiety | 5 |
Substance use | 4 | Substance use | 4 |
– | - | Substance recovery | 4 |
Suicide risk | 4 | Suicide risk | 4 |
- | Self-criticism | 7 | |
- | Cognitive problems | 6 | |
- | Readiness for recovery | 3 | |
- | Recovery environment | 5 | |
Scale items | 79 | Branching-Logic Scale items | 86 |
Single item | 2 | Single Item | 5 |
Alliance | 4 | Alliance | 4 |
Expressed needs in therapy | 4 | Expressed needs in therapy | 5 |
Medication needs | 2 | Medication needs | 2 |
Total items | 91 | Total items | 102 |
Study: measure structure and utility
Act: initial measure finalization and deployment
Second PDSA cycle (May 2015–August 2015)
Plan: dynamic assessment methodology
Do: measurement changes
Study: adaptation evaluation
Act: branching logic fine tuning and digital deployment
Third PDSA cycle (September 2015–September 2017)
Plan: re-evaluating items
Decision to emphasize item response theory (IRT)
Qualitative interview format
Do: data collection
Study: quantitative and qualitative findings
Quantitative results
Qualitative results
Branching logic measure reduction
Act: re-deployment of the new measure
Fourth PDSA cycle (September 2017–June 2018)
Plan: re-evaluating branching logic
Do: collecting PRN data
Study: methods for trigger item and thresholds
Subscale | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | Accuracy (%) | % triggering |
---|---|---|---|---|---|---|
Cognitive problems | 77 | 94 | 92 | 82 | 86 | 40 |
Control | 61 | 90 | 85 | 70 | 76 | 34 |
Eating | 72 | 95 | 89 | 85 | 86 | 29 |
General functioning | 66 | 85 | 83 | 70 | 75 | 24 |
Hopelessness | 78 | 88 | 86 | 81 | 83 | 43 |
Internal avoidance | 67 | 92 | 88 | 77 | 81 | 33 |
Irritability | 48 | 98 | 90 | 85 | 86 | 13 |
Readiness for recovery | 66 | 92 | 89 | 74 | 79 | 35 |
Recovery environment | 71 | 82 | 69 | 84 | 78 | 33 |
Sad affect | 86 | 89 | 87 | 87 | 87 | 47 |
Self-criticism | 73 | 89 | 82 | 83 | 82 | 36 |
Situational avoidance | 49 | 98 | 96 | 66 | 73 | 25 |
Social avoidance | 71 | 96 | 95 | 79 | 84 | 35 |
Social safety | 66 | 85 | 84 | 98 | 75 | 42 |
Somatic anxiety | 79 | 83 | 81 | 81 | 81 | 46 |
Substance recovery | 84 | 99 | 97 | 95 | 95 | 19 |
Substance use | 84 | 99 | 97 | 95 | 95 | 19 |
Suicide | 74 | 93 | 86 | 86 | 86 | 31 |
Trauma | 61 | 95 | 91 | 76 | 90 | 29 |
Worry | 82 | 91 | 93 | 77 | 85 | 52 |