Content validation of a daily patient-reported outcome measure for assessing symptoms in patients with Small Intestinal Bacterial Overgrowth

The initial study phase comprised of a TLR, clinician interviews, and brief concept elicitation (CE) patient interviews to explore salient concepts in SIBO for draft SSM development.

Two TLR searches (using combination of controlled and free-text terms [11]) were conducted in Medline, Embase, and PsycInfo to identify English-language publications from January 2014 to January 2019 (conducted in January 2019) comprising of adult SIBO patients (≥ 18 years) that reported on symptoms, impacts, functioning and/or HRQoL related to disease burden (Table 2), and included a PRO measure utilised for SIBO patients (Table 3). Case reports, letters, and editorials were excluded (Table 4). Based on clinician inputs (who are also co-authors), the definition of SIBO has evolved over time and as such older literature may not capture the current thinking about the disease. For example, the first North American consensus on how to diagnosis SIBO utilizing methane-based breath testing was published by Rezaie et al. in May 2017 [2]. Further, the American College of Gastroenterology SIBO Clinical Guidelines by Pimentel M, et al. were published in February 2020 which shapes the current thinking of the disease [9]. Hence, the TLR was designed to identify the most recent 5 years of data i.e. January 2014 to January 2019. These results were used to inform the clinician interview guide and provide insights into potential symptoms and impacts to include in the first draft development of the SSM. A follow up search in May 2022 was conducted to ensure most updated and relevant literature published from January 2019 to May 2022 was captured.

In-person, 60-min CE interviews were conducted with 8 patients and 3 clinicians, to gather an initial understanding of SIBO disease experience. The interview guide for both patients and clinicians included questions on experiences/knowledge of SIBO, key signs and symptoms, treatment experience and impacts on HRQoL and functioning. The data from these initial CE interviews, along with the TLR and clinician interviews, was used to develop the first version of the SSM to be tested in cognitive interviews (CI) conducted in stages 2 and 3. Further, findings from the TLR and clinical expert interviews informed the development of semi-structured patient interview guides for the CE interviews in stage 2.

In stage 2, patient interviews were virtual, lasted 90-min, and included an abbreviated CE portion and a CI portion on the initial draft SSM. The purpose of the additional semi-structured CE interviews was to strengthen rationale for inclusion of each symptom in the SSM, as well as ensure comprehensiveness (i.e. saturation) of the items. Saturation allows evaluation of thematic saturation and is defined as the point at which no new concepts emerge [12]. CI interviews in this stage were conducted iteratively in three “waves” (of the total 4 interview waves) to allow for refinement of the draft SSM between interview waves. Following each wave, transcripts were qualitatively analysed; and each SSM item evaluated for clarity, applicability of response options, patient interpretation, and relevance. These analyses informed any necessary revisions to the SSM instructions, format, and items between waves, with changes tested to achieve a content-valid version of the SSM.

After stage 2, FDA feedback was sought on the SSM from the Clinical Outcomes Assessment Committee and the Office of Immunology and Inflammation, Division of Gastroenterology. The FDA provided their feedback as a Written Responses Only.

Stage 3 was conducted following FDA feedback on the SSM. Interviews in stage 3 (wave 4) were virtual, lasted 60-min, and were designed to confirm the content validity of the SSM in terms of patient understanding, ease of completion, and the relevance of the items/response options to the patients.

The TLR involved two searches (conducted in January 2019 and repeated in May 2022), one identifying qualitative research (search 1) and one identifying PROs measuring SIBO symptoms (search 2). Search 1 resulted in 66 records but screening did not identify any qualitative research studies in SIBO patients. However, SIBO has been proposed to be common in IBS patients with reported prevalence ranging from 10 to 70% [16]. As symptoms related to SIBO are somewhat similar to IBS symptoms, a second search was conducted to identify qualitative studies in IBS, yielding 69 references of which 5 articles were selected for data extraction (Supplementary Fig. 1). Symptoms reported included bowel symptoms, abdominal pain, abdominal discomfort, gas related symptoms, and other GI symptoms such as nausea, and non-GI symptoms such as fatigue.

Search 2 yielded 319 records after duplicates were removed based on the search criteria in Table 3. Upon review of the abstracts, 206 records were excluded. The remaining 113 records were selected for full text review, and 100 were selected for critical review (Supplementary Fig. 2). This literature review identified commonly used PROs such as the Bristol Stool Form Scale (BSFS) and the Patient Assessment of Upper Gastrointestinal Disorders-Symptom Severity Index; no SIBO-specific PROs were identified.

Three gastroenterologists with extensive clinical experience in diagnosing and managing SIBO patients were interviewed. All three gastroenterologists reported that bloating and distention are the most frequently reported and most bothersome symptoms (to patients) and the most important SIBO symptoms to treat. The gastroenterologists defined distension as the physical manifestation (i.e. enlarged abdomen) of excess gas, and bloating as the sensation of gassiness. They viewed bloating and distension as distinct concepts that may or may not co-occur when patients are describing their SIBO experience. They further noted that how patients report their symptoms depends on their baseline perception of “normal” as it relates to bloating and distension, thus making it difficult to think of a standardized way to describe these symptoms. Other symptoms reported by the gastroenterologists included diarrhea, constipation, dyspepsia, abdominal pain, discomfort, weight fluctuations, reflux, brain fog, fatigue, joint pain, and rectal symptoms.

CE interviews were conducted both before the development of the draft SSM (Stage 1), and during refinement of the SSM (Stage 2). The first stage of CE interviews lasted 60 min and included 8 patients (mean age 46.2 years [range: 25–61 years], 7 females, 6 blacks; Table 5). All 8 patients spontaneously reported bloating as a key symptom (Table 6). Additional key symptoms that were reported by at least half of the sample and were identified as the initial core set of symptoms in the SSM included abdominal distention, gassiness, abdominal pain, appetite changes, fatigue, constipation, weight changes, and abdominal discomfort. Within these 8 patients, the following symptoms were reported as most bothersome: bloating (N = 4), distension (N = 2), abdominal pain (N = 2), diarrhea (N = 1), and fatigue (N = 1). Patients generally described their symptoms as varying throughout the day, and day-by-day. For example, 7 patients reported that bloating occurs daily or is constant and fluctuates throughout the day.

CE interviews in Stage 2 were abbreviated due to inclusion of CI discussed further below and were conducted with 15 additional patients. For this stage, the mean age was 54.5 years (range 34–77 years) with 8 males and majority of white patients (N = 10) (Table 5). Combining stage 1 and stage 2 data, our sample of CE interviews includes a total of 23 SIBO patients. There were 4 new symptom concepts identified in the second stage: heartburn/acid reflux, joint pain, concentration, and shortness of breath (Table 6). However, each of these symptoms was reported by 1–3 patients and did not appear prevalent among the patient cohort as whole, nor were any of them clearly attributed to SIBO specifically. Table 6 shows the complete saturation grid from CE interviews, indicating that concept saturation was reached in terms of condition-specific symptoms by the 17^th interview. Further, Table 7 shows descriptors patients used to define symptoms.

As previously noted, a total of 4 waves, each wave representing a modified SSM, were conducted within Stages 2 and 3 (three waves in Stage 2 and one wave in Stage 3). Conducting the CIs in waves allowed for the refinement of the measure as more patient data were collected. The evolution of the SSM is described below and as a condensed item tracking matrix (Supplementary Table 1).

Following patient feedback from waves 1 and 2 and the clinicians’ inputs, wave 3 included the addition of a belching, appetite loss, and meal skipping item, for a total of 13 items. One clinician suggested adding the item on meal skipping, not intended for inclusion in the total SSM score but to examine in further psychometric work. In addition, the item on bowel movements (not intended to be included in the final score) serving as a gating item for subsequent diarrhea item, was included based on qualitative interviews and clinician inputs.

Most patients (N = 5) noted the questionnaire was relevant and accurately outlines how patients experience SIBO. The remaining patients (N = 3) noted the questionnaire was precise, simple, and relevant. Most patients (N = 14) were able to define the symptom of bloating, however, to help distinguish from the distension item, the wording was changed to “feeling of bloating”. Although there was some conceptual overlap between abdominal distension and bloating, we found enough support from both clinician and patient feedback to retain the two as separate items, until further psychometric evaluation. To help patients distinguish between these two symptoms, a definition of abdominal distention was added in this wave. All patients in wave 1 and 2 understood flatulence, however it was not clear how the severity of flatulence was determined. In wave 3, the flatulence item was revised so that the lower end of the scale represented “normal” flatulence.

Following FDA feedback, minor edits were made to the SSM, namely the flatulence and appetite loss items were modified to ask about worst levels as opposed to severity. For ease of understanding, the fatigue item was reworded to physical tiredness. Stage 3 comprised of the last SSM version (wave 4 of interviews), tested with 12 additional patients (mean age 37.5 years [range: 19–56 years], 6 females, and majority (N = 10) white) (Table 5). One patient in this wave expressed intellectual difficulties with understanding and interpreting the measure, most likely because of illiteracy. Therefore, 13 interviews were conducted, and one patient’s transcript was excluded from CI analysis. For defining diarrhea, the SSM references the BSFS stool consistency Types 6 and 7 [17, 18]. In wave 4, the diarrhea item was simplified, to include two relevant forms (types 6 and 7) of the BSFS, and the appetite item was re-worded to appetite loss rather than appetite, for clarity. Overall, patients in this wave found the questionnaire to be a suitable instrument to evaluate SIBO symptoms. Based on results from wave 4, no additional items were added and no additional changes were made.

Version 1.0 of the SSM emerging from this process is an 11-item (13 items including meal skipping and number of bowel movements which are not included in total score) PRO instrument developed in accordance with the FDA guidance for developing new, content valid measures [10]. The novel SSM assesses severity of the key symptoms associated with SIBO. The SSM, which measures overall symptom severity as a single construct, assesses the severity of bloating, abdominal distention, abdominal discomfort, abdominal pain, flatulence, physical tiredness, nausea, diarrhea, constipation, appetite loss, and belching. With the exception of diarrhea, which is assessed as a daily frequency, response options range from 0 to 10, where 0 is the absence of the symptom and 10 is the worst possible experience of the symptom. Following CI, patients shared their SIBO experience changes on a daily basis, and that a daily recall period was appropriate. Using both CE and CI techniques, a content valid measure was developed. Psychometric testing to establish additional measurement properties in future studies will confirm reliability, construct validity, scoring metrics, and the ability to detect change for the SSM, allowing for final revisions to content that optimize measure performance for clinical trials use.