Introduction

Patients with borderline personality disorder (BPD) are among the highest utilizers of psychiatric services within public sector settings due to repeated usage of these services.1–3 Patients with BPD have been found to receive greater amounts of treatment than patients with depression and those with other personality disorders, and patients with BPD were more likely to have received every type of psychosocial treatment except self-help groups compared to depressed patients.2 The criteria for a patient to be considered a high utilizer may differ based on the health system; however, high utilizer status is often determined based on the utilization rate of acute psychiatric services such as visits to psychiatric emergency services and inpatient admissions and hospitalizations. Previous studies indicate that 9–40% of high utilizers have been diagnosed with BPD.4–9 Interestingly, the highest 40% prevalence rates came from studies focused specifically on the highest utilizers of psychiatric hospitals.4,7 For individuals with BPD who demonstrate continued need for and use of inpatient and crisis psychiatric services, it is questionable whether these services are effective in preventing further suicidal or crisis behaviors.9,10

Patients with BPD who are high utilizers of acute inpatient psychiatric services are a particularly important subgroup of public sector patients for both clinicians and mental health care planners. Evidence-based outpatient treatments for BPD have been shown in randomized controlled trials to be effective at reducing suicidal behavior and increasing functioning for individuals with BPD.11–17 Since the greatest potential improvement and cost savings for evidence-based outpatient treatment may be among high inpatient utilizers, a more detailed understanding on the prevalence of BPD in this population, as well as within the general public mental health population, is important.

Previous research has shown BPD to have a prevalence of about 2–6% of the general population, 9–10% of psychiatric outpatients, and 20% of psychiatric inpatients, but little is known about prevalence among patients receiving publically funded mental health services.18–21 Oldham and Skodol19 examined the prevalence of BPD across a public mental health system and found a rate of 1.9% in 129,286 publicly funded patients in New York State based on computer records.19 Within a public behavioral health system in California, patients with BPD comprised 8% of the publicly funded patient population, and these patients utilized 20% of all mental health services and 26% of all involuntary detentions based on clinical records.20 Given that the general population rates of BPD are as high as 6%, it seems very likely that diagnoses included in clinic records underestimate the true prevalence of BPD within public mental health systems.21 Other research on high utilization and BPD has taken place within one or a small number of hospitals or mental health centers (and therefore may well reflect the admission criteria or other idiosyncrasies of that agency) rather than within a designated mental health system such as a county or state.22,23

Several studies have noted the poor diagnostic agreement between structured interviews and clinicians in “real-world” settings.24,25 In the first study to compare research and clinical diagnosis of BPD, Zimmerman and Mattia26 found that BPD was much more likely to be diagnosed when a structured interview was used.26 Similarly, a study comparing a semi-structured interview based on a Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV) checklist versus clinical interview versus a self-report measure found that only 42% of participants were diagnosed with BPD by all three methods and that clinical interview diagnosed fewer BPD subjects than semi-structured.27 By contrast, a study in Barcelona, Spain, found BPD was diagnosed 30% less when using semi-structured interviews than with clinical evaluation.28 A review of the literature concluded that convergent validity for BPD diagnosis increases as the structure of the assessment increases.29 However, there is a lack of empirical data about the concordance of clinical interviews and structured interviews in more severe populations with BPD such as those in the publicly funded system and high utilizers.

The goals of this study were to examine the prevalence of BPD among publicly funded psychiatric outpatients across King County, a large urban county in Washington State that includes Seattle, and to determine the concordance of diagnosis of BPD in the clinical record with a standardized research diagnosis for those with and without a history of high utilization of inpatient psychiatric hospitalization.

Methods

Participants

The study’s initial sample was comprised of English- and Spanish-speaking adults (18–60 years old) who were enrolled in one of the participating mental health centers in the King County Regional Support Network (KCRSN) who had received at least one outpatient mental health service in the previous 90-day period. Regional Support Networks are the Washington State designated health systems which administer all publicly funded mental health services in a given locality. All Medicaid-eligible patients with a disabling mental disorder were eligible to enroll in KCRSN at the time of the study. KCRSN functions through a total of 17 mental health centers, 15 of which agreed to participate in the study (representing 91% of KCRSN clients).

To conduct this study, the KCRSN Management Information System (MIS) drew a random sample of individuals meeting selection criteria, oversampling high utilizers. In order to protect the privacy of these KCRSN clients, each participating agency was sent a list of the clients selected from that agency by the KCRSN MIS. The agencies then sent a letter from the study to each individual inviting him or her to participate under a cover letter from the agency endorsing the study. Individuals provided consent to the research team in one of two ways: either by return of a postcard or by not calling their agency to refuse participation. Recruitment changed from the more active “sending in a card” to the more passive “not calling to refuse” due to a low response rate with the former approach. In the latter approach, agencies gave lists of non-refusing individuals and their contact information to the research staff.

Correct contact information was provided by agencies for 481 individuals; however, 18 individuals could not be reached. The remaining 463 were contacted and asked to complete a screening interview. Of those contacted, 38% (N = 174) refused to participate either on the phone or at the screening interview. Thus, screening was completed on 289 individuals. The screening criterion was a positive score on three or more items of the Personality Diagnostic Questionnaire—Revised (PDQ-R) which has very high sensitivity for BPD.30–32 Of those screened, 71% (N = 206) met this criterion and 29% (N = 83) did not. A standardized diagnosis interview (i.e., Personality Disorders Examination for BPD) was then conducted for those meeting the screening criterion for BPD.33 Three quarters (76%; N = 156) scheduled and completed the structured interview. Thus, a final sample of 239 individuals had a known BPD diagnostic status: those screened who did not meet criteria (N = 83) plus those who screened positive and completed the structured interview (N = 156). Identifying information for the final sample was provided to KCRSN MIS who, in turn, provided the diagnoses of clinical record for these individuals. Due to missing identifier information from county agencies, it was only possible to uniquely match 160 of the 239 (66.9%) study participants to the KCRSN MIS. This formed the final sample.

High utilization of inpatient mental health services is defined in Washington State as three or more inpatient psychiatric hospitalizations in the past 2 years or any hospitalization of 30 days or longer in the past 2 years. High utilizers were expected to have particularly high prevalence of BPD based on the literature (9–40%) and were therefore oversampled such that all KCRSN high utilizers were included in the selection sample. The final sample of 160 included 133 non-high utilizers and 26 high utilizers. Of the 133 participants who were non-high utilizers, 16.9% had a severe primary Axis I diagnosis in the clinical record of schizophrenia; 37.9% had either unipolar, bipolar, or major depression; 16.9% had an anxiety disorder; 18.5% had a substance use disorder; and 7.3% had another type of diagnosis such as eating and dissociative disorders or no diagnosis recorded yet. Among the 26 participants who were high utilizers, 8.3% had a diagnosis of schizophrenia; 29.1% had either unipolar, bipolar, or major depression; 16.7% had an anxiety disorder; 41.7% had a substance use disorder; and 4.2% had another type of diagnosis such as eating and dissociative disorders or no diagnosis recorded yet.

This sample (N = 160) was approximately half female (42%). The majority of the sample was Caucasian (79%). Other ethnic groups included African-American (15%), Hispanic (3%), Native American or Alaskan Native (2%), Middle Eastern (0.5%), and Pacific Islander (0.5%). Asian patients were underrepresented because the study was limited to persons who spoke English or Spanish. Otherwise, this sample was comparable in gender and ethnic diversity to the full sample, the random sample by the county, and the county client population overall.

Participants were screened for BPD by research staff (a master’s level clinician and undergraduate psychology students supervised by this clinician and the PI). All structured interviews were conducted by the first author or a master’s level clinician, both trained on the Personality Disorder Examination (PDE) by its author.33 All research staff were trained by the PI to work with patients with a wide range of psychiatric disorders following a structured protocol. For their participation, participants were paid $5 at the screening interview and, if screened positive, an additional $15 for the PDE assessment. All contact, consent, and interview procedures and changes were approved by university and county institutional review boards.

Measures and sources of information

Personality Diagnostic Questionnaire—Revised

This brief questionnaire, containing items representing criteria for all DSM-III-R personality disorders, was used to provide an initial screening of participants for BPD. It has excellent reliability and validity as a screening measure although results in a high false-positive rate.32,34–36

Personality Disorders Examination

Participants were administered the BPD items of the PDE to determine if they met criteria for BPD diagnosis based on this structured research interview. The PDE was selected as it was the World Health Organization assessment for personality disorders in international studies. Interrater reliability for BPD on the PDE has been found to be from .73 to .89 and temporal stability from .56 to .84, clearly in the acceptable region.33,37 Validation for BPD diagnosis with the SCID-II was found by Oldham and colleagues.19

King County Regional Support Network MIS database

The KCRSN MIS is a database containing clinical and treatment information for all publicly funded mental health clients in King County. This database is designed for administrative reporting. A client may have up to four Axis I and Axis II clinical diagnoses within this database.

Data analysis

Research diagnosis of BPD was defined on PDE, as with the DSM-IV, as meeting five out of nine diagnostic criteria. A diagnosis of BPD anywhere in the computerized clinical record was considered a definite clinical diagnosis of BPD. All clinical diagnoses were considered definite as the KCRSN MIS does not have an option for rule outs or probable diagnoses. A sensitivity and specificity analysis was conducted to determine the sensitivity, specificity, and positive and negative predictive values of the diagnosis in the clinical record compared to the research diagnosis as the standard.

Results

BPD prevalence among randomly selected non-high utilizers

Among the non-high utilizers with records matched in the MIS, there were a total of 133 participants (83.1% of the final sample). Of these patients, 9.7% (N = 13) met the full research criteria for BPD and 4.5% (N = 6) met criteria in clinical MIS records. As seen in Table 1, only 1.5% of participants were diagnosed with BPD by both methods. In addition, 3% of the non-high utilizers who were negative by research diagnosis had a positive diagnosis of BPD in the medical record. Using both criteria, 87% of the sample were negative for BPD. Using the research diagnosis as the gold standard, the sensitivity of the clinical record diagnosis was 15% and the specificity was 97%. The positive predictive value of the clinical diagnosis for the non-high utilizer group was 33, indicating that it was 33% likely that patients testing positive for BPD in their clinical diagnosis actually have the diagnosis as determined by the PDE. The negative predictive value for the clinical diagnosis for the non-high utilizer group was 91; therefore, 91% of patients with no clinical diagnosis of BPD truly did not meet criteria for the disorder based on the research diagnosis.

Table 1 Relationship between cases diagnosed by structured interview and by clinical record for the non-high utilizer group (N = 133)
Full size table

BPD prevalence among high utilizers

Twenty-six participants (16.3% of the final sample with records matched in the MIS) were high utilizers by Washington State definition. Of these high utilizers, 42.3% (N = 11) met the full research criteria for BPD and 19.2% (N = 5) met criteria by clinical MIS record.

As seen in Table 2, examining just the high utilizers (N = 26), a rate of 45.4% of participants with a research diagnosis of BPD were also diagnosed as BPD in the clinical record. Research diagnosis of BPD was found in 100% of those with a diagnosis of BPD in the clinical record. In addition, none of the participants without a research diagnosis had a clinical diagnosis of BPD. Using both criteria, there was agreement on 19.2% of participants meeting the BPD criteria and 57.7% not meeting criteria. For the high utilizer group, the sensitivity of the clinical record diagnosis (compared to the gold standard of the research diagnosis) was 45% and specificity was 100% (there was never an incorrect clinical diagnosis of BPD among the high utilizers). The positive predictive value of the clinical diagnosis for the high utilizer group was 100, indicating that it was 100% likely that patients meeting criteria for BPD in their clinical diagnosis actually have the diagnosis as determined by the PDE. The negative predictive value for the clinical diagnosis for the high utilizer group was 71.4, suggesting that 71.4% of patients with no clinical diagnosis of BPD truly do not meet criteria for the disorder based on the research diagnosis.

Table 2 Relationship between cases diagnosed by structured interview and by clinical record for the high utilizer group (N = 26)
Full size table

Discussion

The results from this pilot study indicate that the research diagnosis prevalence of BPD was 9.7% among non-high utilizers of inpatient psychiatric services, on the low end of previous prevalence estimates in outpatient care based on research diagnosis.1,2 By contrast, 4.5% of the sample were diagnosed with BPD in the clinical record—higher than the New York State sample of 1.9% and considerably lower than the sample within a public behavioral health system in California of 8%.19,20 Only 1.5% of the sample was diagnosed with BPD by both methods. Thus, while the specificity and negative predictive value of the clinical record were high, the sensitivity and positive predictive value of the clinical record diagnosis were quite low, and the clinical record greatly underestimated the prevalence of BPD in this sample.

For those who were high utilizers of inpatient psychiatric services, the research diagnosis prevalence was much higher (42.3%) though only 19.2% were diagnosed with BPD in the clinical record. The clinical record diagnosis of BPD was found to have a sensitivity of 45% which indicates that less than half of the patients who had the research diagnosis of BPD did not have that diagnosis in the clinical record. However, the positive predictive value and specificity of 100 indicated that everyone diagnosed with BPD in the clinical record also met criteria by research diagnosis and none of the individuals without BPD in the clinical record met research criteria. However, the negative predictive value of 71%—with the sensitivity of 45%—indicates that the absence of the diagnosis in the medical record should not imply the absence of the diagnosis for high utilizers. There was more agreement between clinical and research diagnoses in this population suggesting that the presence of crisis behaviors may trigger the inclusion of a BPD diagnosis in the clinical record. Additionally, high utilization may be a proxy for clinical severity which may be associated with a clearer diagnostic picture.

As hypothesized, there was poor agreement between the research and clinical diagnoses, which is consistent with previous studies that have noted the poor diagnostic agreement between structured interviews and clinicians in “real-world” settings.24–28 As seen in the studies by Zimmerman and Mattia26 and Sansone et al.27, the clinical system appeared to have many false negatives and fewer false positives in BPD diagnosis relative to research interview.26,27 This is the opposite of what was found by Andion et al.28 in the Barcelona, Spain, sample.28 This may be due to the system requirement for Axis I but not Axis II disorders, a desire to avoid stigmatizing patients with a diagnosis with negative connotations or a bias against diagnoses for which treatment is not readily available. However, the underdiagnosis of BPD is important because those with undiagnosed BPD may appear to have treatment-refractory depression or other mood or anxiety disorders and appear untreatable when in fact they might respond if given treatment specific to BPD. This study also supports previous research suggesting the need for more structured and standardized methods of BPD diagnosis by public mental health clinicians and systems.26,29

This pilot study had several limitations. There were problems in matching the research cases with the clinical MIS record resulting in a smaller subgroup with both diagnoses. There are concerns about the representativeness of the sample based on diagnosis due to the fact that high utilizers were oversampled; however, the sample was representative in terms of gender and ethnicity. Another limitation was the use of the computerized clinical record as a measure of clinical diagnosis. This record allows up to four diagnoses, but it may be that an examination of each mental health center’s medical record would reveal diagnoses or “rule outs” which were not coded into the MIS system. The validity of the clinical diagnosis of BPD in the MIS is not known and is likely to be a highly conservative measure of diagnostic prevalence. The study examined the concordance between research and clinical diagnoses within one public behavioral health system; therefore, results might not be generalizable to other public or private sector settings.

Implications for Behavioral Health

Results indicate that the county’s clinical record of BPD indicates adequate specificity (those without BPD are not diagnosed with it) but such low sensitivity that most individuals with BPD will be missed. This has several health services implications. First, the research diagnosis indicates that there is twice the need for services for BPD than the county would have reason to suspect from its own data. At 10% of the non-high utilizer patient population and almost half the high utilizer population, BPD is clearly prevalent enough to indicate the need for services for this disorder.

Results of the current study should be interpreted given the context of the health system within King County, as the available treatment options for outpatient care are unique to this system; therefore, results might not be generalizable to all public behavioral health systems. “Standard outpatient care” in King County’s public mental health system, as in many others, is based on assertive community treatment, which was designed for patients with schizophrenia or other chronically disabling psychotic or mood disorders as opposed to patients with BPD.38,39 Dialectical behavior therapy (DBT) is the most extensively studied of all approaches to BPD, and several randomized controlled trials have demonstrated the efficacy of DBT.17,40,41 However, there are several other treatments that have empirical support for the treatment of BPD, including mentalization-based treatment, schema-focused therapy, transference-focused therapy, and general psychiatric management.11–14,42–44 The evidence-based treatments for patients with BPD differ significantly from the model based on assertive community treatment in that (a) they are staffed by trained psychotherapists, not case managers; (b) the majority of the treatment is individual psychotherapy (and in DBT skills training groups) instead of coordination of pharmacotherapy, psychoeducational and supportive group psychotherapy, and social services such as housing, finances, and vocational rehabilitation; (c) most appointments are in the office as opposed to on the phone and in the community; and (d) appointments are at regularly scheduled weekly times as opposed to flexibly in response to the changing needs of the patient. Thus, if typical mental health center treatment is mismatched for BPD patients, high utilization of inpatient services is not surprising, especially when suicide or crisis behaviors are so common.2,3 Better documentation of the true prevalence of BPD in public mental health is critical to the public system taking seriously this poor match of care for individuals with BPD.

It is concluded that BPD is markedly underdiagnosed in this public mental health population and that the reliability of BPD diagnosis among providers in a public mental health system is increased based on the patients’ utilization rate of inpatient hospitalizations. Further, since effective outpatient treatment has been developed for these expensive BPD high utilizers, such major under-recognition has significant implications for the outcomes of outpatient mental health services. Other studies are needed to better assess both the prevalence of individuals with borderline personality disorder in mental health centers and whether they are receiving adequate care.