Development and psychometric validation of a patient-reported outcome measure of recurrent urinary tract infection impact: the Recurrent UTI Impact Questionnaire

An online cross-sectional survey of adult females experiencing rUTI (N = 1983, see Table 1 for demographic characteristics) was conducted to collect qualitative data about the impact of living with this condition. Visitors to a website hosted by Live UTI Free (https://liveutifree.com), a UTI patient advocacy and research organisation, were invited to complete the survey. Inclusion criteria comprised a minimum age of 18 years old and meeting the diagnostic criteria for rUTI (≥ 2 UTI in 6 months, or ≥ 3 UTIs in 12 months) based on self-report [1, 18]. Informed e-consent was obtained prior to survey completion and participants could withdraw at any point.

Most participants were from the USA (56.3%, n = 1,116), UK (19.3%, n = 383), Canada (5.30%, n = 105), and Australia (4.54%, n = 90). Participant ages were broadly distributed across the sample, with 56.7% (n = 1,126) aged between 18 and 39 years old and 43.2% (n = 857) aged 40 years old or above. Most participants reported that their UTI episodes last between 1 and 5 days long (57.8%, n = 1,146) and that they have 1 to 3 months between infections (45.6%, n = 904). Over a third of participants had experienced more than 15 UTIs (38.6%, n = 766).

After providing consent, participants answered questions about their demographic characteristics and clinical history with rUTI, including frequency and duration of symptoms. Next, participants were asked to qualitatively describe their experience of rUTI via an open-text box.

Framework analysis of the qualitative data and a thorough literature review were conducted to produce a conceptual framework of rUTI impact (see Online Resource 1 for data handling strategy and see Online Resource 2 for summary of literature consulted) [28, 30, 31]. After data familiarisation, the first author developed an initial thematic framework of psychosocial components based on factors identified a priori from existing literature and emergent issues raised by participants [31]. The entire research team reviewed this collaboratively in conjunction with the dataset and literature review findings, making agreed revisions. The first author then indexed and annotated the full dataset according to this framework, and patterns were charted and mapped to group the data into defined concepts (see Online Resource 3 for thematic framework) [31]. Secondary coding and triangulation were undertaken by the last author. Both coders hold advanced postgraduate training in research methods and extensive qualitative experience (see Stage III). Disagreements were resolved in discussion with the wider research team and in close consultation with the data. Data saturation was achieved. Each overarching framework component was selected for exploration as a preliminary subscale in the RUTIIQ, and initial items were drafted for each according to the relevant framework subcomponents.

A Delphi methodology was employed to evaluate the content validity of the initial questionnaire from the perspective of clinicians with rUTI expertise (see Online Resource 4 for all items tested at this stage) [32]. Two rounds of anonymous surveys took place to build towards an expert consensus about the relevance and clarity of the questionnaire items [32].

Thirty-seven expert clinicians were invited to take part (n = 22 female, n = 15 male), of whom 15 were successfully recruited (n = 12 female, n = 3 male), meeting sample size recommendations (see Table 1 for demographic characteristics) [26, 32]. Purposive recruitment with snowball sampling was applied to obtain a heterogeneous sample with an equal proportion of clinicians working in primary and secondary care to reduce the risk of bias [33, 34]. Inclusion criteria comprised those currently working as either a general health practitioner or specialist doctor/nurse practitioner within urology or an allied discipline.

The expert clinicians were aged between 32 and 64 years old (M = 46.8, SD = 9.24) and their experience in treating rUTI ranged from 2 to 30 years (M = 13.2, SD = 7.95). Approximately half of the expert clinicians were general practitioners (53.3%, n = 8), and half were specialist doctors or nurse practitioners (46.7%, n = 7). Specialists practised within urology (71.4%, n = 5) and urogynaecology (28.6%, n = 2). The participants practised in the USA (53.3%, n = 8), UK (40.0%, n = 6), and Canada (6.67%, n = 1). Eighty percent retention was achieved in Round 2, supported by regular personalised email reminders.

RUTIIQ items were presented using an online survey tool (REDCap; https://​www.​project-redcap.​org). In Round 1, the clinicians were asked to rate each for relevance and clarity using a 7-point scale (0 = not at all relevant/clear, 6 = highly relevant/clear) and provide qualitative comments about comprehensiveness and comprehensibility [26, 32]. In Round 2, each RUTIIQ item was presented alongside the median relevance and clarity ratings and anonymised qualitative feedback from Round 1. The expert clinicians either retained or updated their original ratings and provided further qualitative feedback.

Median ratings from Round 2 were calculated and analysed in conjunction with qualitative feedback. Content validity indices for items (I-CVI) were computed by dividing the number of experts who scored an item’s relevance/clarity as at least 4 out of 6 by the total number of experts [35]. A threshold of 0.75 was specified a priori as the minimum I-CVI required to indicate acceptable consensus of content validity, with a minimum median score of 4 [32, 34].

To evaluate how the RUTIIQ may be mentally processed and where problems may arise, one-to-one semi-structured interviews were conducted with rUTI patients using Microsoft Teams [36]. Cognitive debriefing techniques were used to encourage participants to think aloud as they answered the RUTIIQ questions (see Online Resource 5 for all items tested at this stage) [36]. Interviews took place in two phases, with interim iterative refinements validated in the second phase [37].

All cognitive interviews were conducted by the first author to ensure homogeneity in interview style and to facilitate rapport and participant comfort [28, 36]. The interviewer had advanced postgraduate training in conducting and analysing qualitative interviews, and ongoing training and quality monitoring were provided by the last author who has extensive expertise as a Chartered Health Psychologist and academic qualitative researcher. To facilitate transcription, all interviews were audio-recorded. Full ethical approval was granted by the School of Psychology and Clinical Language Sciences Research Ethics Committee, University of Reading (project reference: 2021-043-KF). Anonymised recordings were securely stored on the University of Reading server, only accessible to the research team. Data were retained for five years from the study’s completion and destroyed sooner if requested. APA guidelines for ethical conduct were maintained at all times [38].

A clinically and demographically diverse sample of 28 adults experiencing rUTI was purposively recruited (see Table 1 for demographic characteristics). To reduce risk of selection bias and sampling error, a large sample representative of the different subgroups that make up the rUTI patient cohort was sought via a broad recruitment strategy with two possible sources: (1) via people signed up to receive newsletters and research notifications from a key stakeholder group: Live UTI Free (https://liveutifree.com), and (2) via other UTI-related online sources, such as support groups. Interested participants were encouraged to share the study information on social media. Recruitment was incentivised using a £25 online shopping voucher prize draw for one random winner, aiming to reduce dropout and attrition bias [39].

Inclusion criteria comprised a minimum age of 18 years old, native or advanced fluency in English, and meeting the diagnostic criteria for rUTI based on self-report [1, 18]. Participants who reported a current diagnosis of interstitial cystitis, were using urinary catheterisation, or were pregnant were excluded. Seventy-three potential participants completed a screening survey without exclusion, from which as diverse a sample as possible was selected via maximum variation sampling (see Online Resource 6) [40]. A minimum sample size of 25 participants was sought to achieve confidence that all possible problems with the questionnaire had been identified [26, 41]. With a final sample of 28 interviewed participants (n = 18 in the first interview phase, and n = 10 in the second interview phase), sampling adequacy was reached.

The final sample had an age range of 18 to 82 years old (M = 46.8, SD = 16.9), and comprised 92.9% females (n = 26) and 7.14% males (n = 2), with one participant describing themselves as non-binary and assigned female at birth. Participants resided in 10 countries, predominantly the USA (32.1%, n = 9), the UK (25.0%, n = 7), and Canada (14.3%, n = 4). The median number of UTI episodes in the past 6 and 12 months was 4 (IQR = 4) and 7 (IQR = 8), respectively. Years of UTI symptoms ranged from 1 to 65 (M = 17.3, SD = 14.5) and years of UTI impact to QoL ranged from 1 to 60 (M = 8.90, SD = 12.0).

Participants provided e-consent after reviewing the study’s ethical considerations. The interviewer presented the RUTIIQ to participants using a ‘screen-share’ function and invited them to think aloud their thought processes whilst deciding their answer for each question, allowing evaluation of the measure’s overall comprehensiveness and comprehensibility [26, 28, 37]. Informed by Willis’ guidance on planning and conducting cognitive interviews for instrument development [36, 37], a topic guide (Online Resource 7) was designed and employed to semi-structure each interview, including scripted probes to encourage elaboration and clarification [37]. Questions about the scale and response options, the time taken to answer questions, and the questionnaire layout and formatting were also asked [36]. Anonymised field notes and written summaries were prepared throughout and following each interview to build richness and support transcript interpretation [37].

Anonymised verbatim interview transcripts of the audio recordings were created using speech-to-text transcription software (Otter; https://​otter.​ai/​), with errors manually corrected. Predefined codes from the Question Appraisal System (QAS-99) [42], designed to support the systematic assessment of questionnaire items and identification of potential problems, were used to support question feature coding. The QAS-99 training manual was applied to create a coding system for the dataset, systematically evaluating every participant response to each RUTIIQ item in relation to question features (for example, comprehension of technical terms). Initial coding was conducted by the first author, with triangulation undertaken by the last author. This assessment was supported by verbatim participant quotes and interviewer field notes [37].

Weekly wider research team meetings were held, allowing for in-depth analysis of transcripts, drawing together a variety of perspectives from expertise including experience in academic mixed-methods research, clinical practice, and patient advocacy. If there were any uncertainties found in Phase 1, the decision was taken to retain the item in Phase 2 and gain further feedback. A third version of the RUTIIQ was created after Phase 1 for assessment in Phase 2, with the same process undertaken at the end of Phase 2 to create a fourth version for pilot testing.

To collect data for psychometric testing of the RUTIIQ and final item reduction, a two-part cross-sectional survey of adults experiencing rUTI was conducted online. Participants completed the same procedure twice to facilitate test–retest analysis [26‐28]: (1) at baseline, and (2) 24 h later (Test–Retest Assessment). A maximum time period of 48 h to complete the Test–Retest Assessment was applied following the minimum satisfactory test–retest window advocated by Streiner, Norman & Cairney [43]. This prioritised within-episode stability and reduced the possibility of a separate rUTI episode altering reporting of rUTI symptom impact [44].

In addition to completing the RUTIIQ, participants completed existing validated measures relating to each RUTIIQ subscale to facilitate construct validity [27, 28]. It was hypothesised a priori that there would be moderate to strong correlations (Spearman’s ρ > 0.50) between the RUTIIQ subscale scores and existing instruments measuring constructs related to the RUTIIQ subscales (‘concurrent measures’) [27, 28].

A sample of 240 adults meeting the diagnostic criteria for rUTI completed the Baseline Assessment, of whom 106 (44.2%) completed the Test–Retest Assessment (see Table 1 for demographic characteristics, and Online Resource 8 for sampling and recruitment strategy). Participants were recruited using the same recruitment channels used in Stage III, also applying the same inclusion and exclusion criteria. At least 210 participants were required to complete the Baseline Assessment to perform final item reduction via exploratory factor analysis with at least 5 participants per questionnaire item (RUTIIQ = 42 items before final item reduction). Sampling adequacy was exceeded.

The final sample (N = 240) was aged between 18 and 84 years old (M = 45.0, SD = 17.3), and comprised 97.9% females (n = 235) and 2.08% males (n = 5), with two participants describing themselves as non-binary and assigned female at birth. Twenty-four countries were sampled, with most participants residing in the USA (38.8%, n = 93), the UK (32.1%, n = 77), Australia (7.50%, n = 18), and Canada (6.67%, n = 16).

Approximately half of the participants (56.3%, n = 135) reported taking antibiotics at the time of participation, either to treat a current UTI, prevent new UTIs, and/or for other indications. Approximately three-quarters (76.3%, n = 183) reported managing their rUTI with non-antibiotic treatment including natural remedies or supplements. More than three-quarters of participants (77.5%, n = 186) reported experiencing persistent lower urinary tract symptoms for at least the past three months, with the remainder reporting symptoms which occur on an episodic basis. The mean number of episodes of symptoms reported in the past six months was 6.81 (SD = 24.3), and the mean in the past year was 13.9 (SD = 48.4).

In the Baseline Assessment, participants provided e-consent via REDCap after reviewing the study’s ethical considerations. Eligible participants who were not excluded during a screening questionnaire proceeded to complete the RUTIIQ and the following six concurrent measures: the Patient Health Questionnaire 9 (PHQ-9) to assess symptoms of depression [45], the Generalized Anxiety Disorder 7 (GAD-7) to assess levels of anxiety [46], the University of Los Angeles Loneliness Scale Version 3 to explore feelings of loneliness and social isolation [47], the Work Productivity and Activity Impairment Questionnaire for Specific Health Problems to evaluate the health-related impact on carrying out work and daily activities [48], the Female Sexual Distress Scale–Revised (FSDS-R) to measure sexual dysfunction [49], and the Patient Satisfaction Questionnaire 18 to assess patient-reported satisfaction with healthcare [50]. The Test–Retest Assessment included the same instruments.

After preparing the data for analysis (see Online Resource 9 for data handling strategy [51‐60]), summed scores were calculated for each RUTIIQ subscale and each concurrent measure. Exploratory factor analysis was conducted to perform final item reduction and determine the latent factor structure (structural validity) of the RUTIIQ [26, 28, 53]. Further psychometric analyses of the RUTIIQ, comprising test–retest reliability, internal consistency, and construct validity, were conducted and compared to gold-standard recommendations [27, 28]. Linear regression analyses were also performed to examine any measurement invariance in RUTIIQ scores (for example, related to sociodemographic differences). The Automated Readability Index, a readability measure known to be especially applicable to non-narrative text such as questionnaires [58], was computed to estimate the literacy level required for comprehension of the RUTIIQ.

Bartlett’s Test of Sphericity was statistically significant (p < 0.001), indicating the absence of multicollinearity [53]. The Kaiser–Meyer–Olkin Measure of Sampling Adequacy estimate was high at 0.89, confirming the suitability of the data for exploratory factor analysis [53]. All extracted communalities were greater than 0.40 except for item A5 (impact on diet), which also did not load above 0.40 on any initial factors and was therefore removed. All other items met these thresholds for communalities and factor loadings. Items demonstrating multiple cross-loadings (B1: impaired close relationships; B3: impaired social activities; B7: worrying about being a burden to others) were removed.

The final five-factor structure comprises the following factors: ‘patient satisfaction’, ‘work and activity interference’, ‘social wellbeing’, ‘personal wellbeing’, and ‘sexual wellbeing’ (see Table 2), together accounting for 73.8% of the total variance in scores. They represent a strong fit for the data, clearly distinguishing between the RUTIIQ subscales. The final version of the RUTIIQ consists of 30 items in total (see Table 3; the full questionnaire is available in Online Resource 10).

The final 30-item RUTIIQ consists of five subscales (see Table 3): personal wellbeing (4 items), social impact (5 items), work and activity interference (7 items), sexual wellbeing (4 items), and patient satisfaction (10 items). All five sections utilise an 11-point Likert scale ranging from 0 (‘strongly disagree’) to 10 (‘strongly agree’). For the first four subscales, greater scores indicate greater impact to QoL. For the patient satisfaction subscale, greater scores indicate greater patient satisfaction with UTI-related medical care.

Observed RUTIIQ subscale scores highlighted the breadth in patient experiences, with scores spanning the full possible range for all subscales except for the sexual wellbeing subscale, for which all participants reported at least some level of impact (see Table 4 for RUTIIQ descriptive statistics, see Online Resource 11 for concurrent measure descriptive statistics). The average PHQ-9 and GAD-7 scores indicated moderate depression (M = 11.5, SD = 7.22) [45], and mild to moderate anxiety (M = 9.23, SD = 6.21) [46], respectively. Participants typically indicated sexual distress considerably beyond the ‘normal’ FSDS-R range of 0–10 (M = 30.3, SD = 13.7) [49].

The Automated Readability Index for the RUTIIQ is 6.2, indicating suitability for people with a reading age of 11 years old or above [58]. Internal consistency (Cronbach’s α), test–retest reliability (ICC), and construct validity (Spearman’s ρ) were moderate to strong for all RUTIIQ subscales (see Table 4). All subscale psychometric statistics surpassed the relevant gold-standard recommendations except the sexual wellbeing subscale, which still achieved moderate performance [27, 28].

Linear regression analyses indicated the broad sociocultural applicability of the RUTIIQ, indicating no statistically significant group differences in scores in terms of ethnicity, country of residence, relationship status, or level of fluency in English (p > 0.05). Younger respondents typically reported greater rUTI impact than older respondents across all subscales except sexual wellbeing (p < 0.01, see Online Resource 12). Female participants felt that they experienced greater rUTI impact in personal wellbeing, work and activity interference, and sexual wellbeing than male participants (p < 0.05, see Online Resource 13). However, further testing with males is required to examine this difference.