Adherence to mental health guidelines by Dutch occupational physicians

Abstract

Background In 2000, the Dutch Association of Occupational Physicians published a national guideline for the management of employees with mental health problems.

Objectives To examine predictors of adherence to this guideline by Dutch occupational physicians (OPs).

Methods Using the Theory of Planned Behaviour, a questionnaire was developed about self-reported guideline adherence of OPs and possible predictors of this behaviour. A total of 165 OPs were approached to complete the questionnaire and registration forms of first consultations of workers with mental health problems. Performance indicators based on the guideline were developed to calculate performance rates of guideline adherence by OPs.

Results Eighty of 165 (48%) OPs approached completed the questionnaire. Fifty-six OPs returned one or more registration forms, totalling 344 consultations. On a five-point Likert scale, ranging from never (1) to always (5), the mean score on self-reported guideline adherence was 2.35, compared to a mean score of 4.06 on the intention to comply with the guideline. The mean performance rate of OPs ranging from 0 to 2 was 1.27 on diagnosis and 0.60 on guidance. No relation was found between self-reported guideline adherence and performance rates. Self-reported guideline adherence correlated significantly with perceived behaviour control (r = 0.48, P < 0.05), subjective norms (r = 0.33, P < 0.05) and positive job stress (r = 0.35, P < 0.05).

Conclusions Guideline adherence by Dutch OPs lags behind its acceptance. Further implementation efforts need to focus on diminishing barriers and enhancing social norms of OPs to work according to the guideline.

Introduction

Since 1999, the Dutch Association of Occupational Physicians (NVAB) has been developing and disseminating evidence-based practice guidelines, as they are one of the most promising and effective tools for improving the quality of occupational health care [1–3]. In 2000, a national guideline was published regarding the management of employees with mental health problems [4,5]. A main reason for addressing this topic in a practice guideline was that mental health problems account for one-third of all disability benefits in the Netherlands. The majority of these employees are at risk for chronic disability, while only suffering from minor reversible psychiatric disorders [6,7]. As Dutch employees are required to visit their occupational physician (OP) for rehabilitation purposes, the Dutch OP has an opportunity to detect people at risk and influence recovery [5,7]. The current practice guideline gives OPs recommendations on rehabilitation activities which are successful in facilitating return to work [8–10]. This suggests that rigorous implementation of the guideline could improve the effectiveness of occupational rehabilitation among workers with mental health problems.

After its publication in 2000, the guideline was sent to all members of the NVAB and several courses emerged to inform OPs about the content of the guideline. However, since 1994 Dutch occupational health services (OHS) have been independent profit-driven enterprises that sell their services to companies in contracts that often only allow for a minimum level of services. This changing work environment has created a difficult position for OPs, as patients, employers and the management of OHS put pressure on them with different interests. This is reflected by a relatively high level of negative job stress among Dutch OPs, and the perception that they are not able to deliver the necessary quality of care [11].

The conditions discussed above show the practical relevance of an evaluation of the actual state of guideline adherence and factors that can be changed in the implementation process. Theoretically, this is relevant while no research is known about predicting guideline adherence in occupational medicine. In implementation research guideline adherence has been examined by using the Theory of Planned Behaviour (TPB), which predicts the occurrence of a specific behaviour provided that the behaviour is intentional [12,13]. Figure 1 shows the TPB model and represents three variables which theoretically predict intention to perform behaviour. To predict whether a person intends to do something, we need to know whether the person is in favour of doing it (attitude); how much the person feels social pressure to do it (subjective norm) and whether the person feels in control of the action in question (perceived behavioural control). The intentions, together with perceived behaviour control, are precursors of the behaviour. These manifest variables should be related to appropriate sets of salient behavioural, normative and control beliefs about the behaviour.

Guideline adherence of an OP is an example of intentional behaviour which can be measured by asking OPs how they assess their own guideline adherence (self-report) [14,15]. A theoretically new and more objective way to do this is by registration of their consultation behaviour and use of performance indicators (PIs) to monitor guideline adherence [8,16].

In addition to the variables in the TPB, there is reason to examine negative job stress of OPs as a possible predictor of guideline adherence, as OPs are at risk of negative job stress [17]. Positive job stress can affect performance as well and therefore job stress is added to the theoretical framework of the TPB for guideline adherence in Figure 1 [18].

In summary, the aim of this study is three-fold: (i) to examine guideline adherence of OPs and possible predictors of adherence by applying the theoretical framework of the TPB; (ii) to assess guideline adherence in medical practice of OPs by means of newly developed PIs and compare this to their self-reported guideline adherence and (iii) to evaluate job stress of Dutch OPs as a possible predictor of guideline adherence.

Methods

One hundred and sixty-five OPs were approached to participate in a cross-sectional study between April and October 2001. They were working for 11 units, belonging to six major OHS, in which a central key figure encouraged OPs to participate in the study. To each OP, a questionnaire and 15 patient registration forms were sent with the request to complete and return them within a month. OPs were asked to complete one registration form for each employee with a mental health problem after their first consultation. Face validity and feasibility of the forms were tested in daily practice by five OPs in a small pilot. During the registration period, OPs were not informed about specific PIs and corresponding criteria used in the study. Returned registration forms were only included in the study if the OP diagnosed a psychological disorder into one of the four categories provided by the guideline.

The questionnaire (92 items) was used to collect data on demographic characteristics of the OP and their experience with the guideline (22 items), on self-reported guideline adherence and its possible predictors according to the TPB (40 items) and on job stress of the OP (30 items).

To transform the items of the TPB (Figure 1) for guideline adherence, Ajzen's most extensive coverage of applying TPB was used [12]. One example of the used items, the number of items and the internal consistency of the constructs are shown in Table 1. Some items could only be answered if the respondent had some knowledge of the guideline content. In those cases, the option ‘no opinion’ was added. For each construct a mean score was calculated.

Guideline adherence was measured in the questionnaire by two items: ‘Do you use the guideline in the management of clients?’ and ‘How often do you apply the guideline in practice?’. Intention was measured by asking OPs to answer three questions about their plan, desire and prediction to start or keep using the guideline.

To measure attitudes, OPs were asked to score in seven evaluative dimensions on the item: ‘I consider working according to the guideline meaningful/intelligent/fine/skilful/good/necessary/important’ [12]. Beliefs were measured by five items on expectations about guideline adherence on quality of care, transparency of professional behaviour, evidence-based practice and job satisfaction.

Subjective norms of five items about whether OPs thought their manager, colleagues, employees and employers expected them to comply with the guideline. Additionally, normative beliefs about their motivation in general to comply with the wishes of these different individuals in their working environment.

Perceived behaviour control represents the OPs' perceptions of their ability to perform a given behaviour and is determined by control beliefs about the presence of factors that may facilitate or impede performance of the behaviour. Respondents were asked about different factors such as time, training, work organization, contract with employer and their complications on guideline adherence.

Job stress was measured by using the Utrecht burnout scale (UBOS), and a three-factor structure of engagement, the Utrecht engagement scale (UBES) [19,20]. The UBOS measures three negative job stress factors: emotional exhaustion, cynicism (in relation to others or work) and professional efficacy. The UBES measures three positive job stress dimensions: vigour, dedication and absorption. Each dimension consists of five items, describing a state of mind which refers to the work situation (Table 1). One must fill in how often this state of mind is experienced on a seven-point Likert scale ranging from never (0) to daily (6). For each job stress dimension, a mean score was calculated. Internal reliability of each dimension was satisfactory (α > 0.75), except the one representing absorption (α = 0.65). Factor analysis showed that professional efficacy in the UBOS measured the same as the positive job stress dimensions in the UBES.

Registration forms (47 items) were developed to gather more objective data on guideline adherence in medical practice of the OP. The registration forms were constructed in such a way that information of two central elements of the guideline was gathered.

For each PI, criteria were developed based on if–then logic [16]. Table 3 shows the indicators and their criteria. These criteria depended on the diagnostic classification by the OP into one of the categories of the guideline. On each of the four PIs, one point was scored if the criteria of the guideline were met according to logical decision rules designed by the researchers. For each consultation with a different patient, an OP could thus score two points on diagnosis and two points on management, resulting in a total score between 0 (no compliance with the guideline) and four points (maximum compliance). For each OP, the mean score of PIs was calculated by dividing the sum of the scores of the registration forms by the number of completed registration forms.

The first step of the data analysis included descriptive statistics. The second step was to calculate the objective measured performance rates (registration forms) for each OP and to relate them to the scores on self-reported guideline adherence (questionnaire) by means of Pearson correlation. The third step was testing the constructs of the TPB as predictors of subjective measured guideline adherence by means of Pearson correlation. A stepwise regression analysis was done, first for guideline adherence on intention and perceived behaviour control and second for intention on ‘attitude’, ‘subjective norm’ and ‘perceived behaviour control’. The last step was a regression analysis for guideline adherence on job stress and, if this one was significant, for intention and perceived behaviour control on job stress to measure their expected mediating effects.

Results

Eighty of the 165 OPs returned the questionnaire corresponding to a response rate of 48%. Of the respondents, 60% were male, mean age was 43 years and on average respondents had worked as an OP for 9 years. Mean hours of working was 33 h/week and included 40 patient consultations, which corresponded to 60% of their working time. Fifty-eight per cent were registered as OPs and 29% were in training. Since its publication, 68% of the respondents had followed courses or seminars on mental health problems. Of this group, 80% had spent less and 20% had spent >17 h on this type of continuing medical education.

Table 2 shows the mean scores on the different constructs of the TPB and job stress. Self-reported guideline adherence of 80 OPs showed a mean score of 2.35 on the two items. The mean scores on intention and attitude were relatively high compared to the mean scores on subjective norm and perceived behaviour control. The scores on the dimensions of positive job stress are relatively high compared to the scores on negative job stress.

Fifty-six of the 80 participating OPs (70%) returned 344 registration forms of consultations in which a mental health problem was classified, thus making it possible to calculate PIs. These 56 OPs had a significantly higher score on NVAB membership (F = 27.6, P < 0.01), absorption (F = 0.02, P < 0.05), attitude (F = 8.5, P < 0.01) and intention (F = 8.2, P < 0.05), than the 24 OPs who did not return any registration forms.

The median number of returned registration forms was 10 (range 1–15). On 225 (65%) of the registration forms the diagnosis adjustment disorder was made (27% distress, 29% nervous breakdown, 9% burnout), on 19% depression, on 9% anxiety disorder and on 7% remaining psychiatric disorder. Table 4 shows the results on the different PIs on guideline adherence in practice (on the left side in the table) and the score for each registration form. The mean score on the accuracy of the diagnosis was twice as high as the mean score on quality of management. More than 80% of the registration forms scored one or two points in a range of 0 to 4.

Scores for each OP on self-reported guideline adherence, measured by the questionnaire, and the performance rates of the registered guideline adherence showed no significant correlation (r = −0.05; accuracy of diagnosis r = −0.22; quality of management r = 0.12). No significant relationships were found between the various predictors and guideline adherence measured by the performance rates.

Figure 1 shows the pair-wise Pearson correlations between the various predictors in the theoretical model of planned behaviour and job stress regarding self-reported guideline adherence. Most of the expected relationships according to the TPB showed a significant correlation. No significant correlation was found between guideline adherence and intention, intention and subjective norms and between intention and perceived behaviour control. A significant positive correlation was found between the mean score of the four dimensions of positive job stress and guideline adherence (r = 0.35, P < 0.05).

Table 5 shows the results of the stepwise regression analysis, executed to examine the causality of the expected relationships in TPB. In the left column, the predicting constructs are mentioned. The third column shows that the regression coefficients for intention on attitude are significant. The right column shows that the regression coefficient for guideline adherence on perceived behaviour control is significant.

Table 5 also shows the results of the pair-wise correlations in the TPB between the different determinants of the TPB of guideline adherence and job stress. In addition to the expectations of the TPB, significant relationships were found between guideline adherence and attitude and between guideline adherence and subjective norms.

Significant positive correlations were found between the dimensions of positive job stress and guideline adherence. Negative job stress was significantly negatively correlated with perceived behaviour control. Additionally, a mediating effect of perceived behaviour control in the relationship between the positive job stress factors professional efficacy, vigour, dedication and guideline adherence was found.

Discussion

The present study found that 1 year after the publication of national guidelines on the management of mental health problems (in 2000), Dutch OPs had a positive attitude towards the guidelines and in general intended to apply them in practice but compliance with the guidelines appeared to be minimal. In most consultations, OPs did not seem to work in accordance with the recommendations of the guidelines. No relationship existed between guideline adherence measured by the questionnaire and independently measured PIs on diagnosis and management.

This study shows that Ajzen's theoretical model of Planned Behaviour is useful in assessing and explaining the actual state of guideline implementation. The constructs of the model were consistent, most expected relationships of the model were confirmed and the constructs gave a clear indication of possible barriers in the implementation process.

The theoretical value of this model of behaviour change seems limited, because ‘intention’, the central element of the model, had no significant relationship with the studied behaviour. Possibly, the ASES model, in which barriers and knowledge are expected to moderate the relationship between intention and ‘behaviour’ might open new perspectives [21,22].

The results of this study may have been biased by the low response rate, which may be due to negative expectations of a time-consuming registration procedure. Participating OPs, however, never complained of this aspect of the study and on the contrary, they commented positively on the structuring effects of registering consultations. This response bias may therefore have led to positive selection. OPs who experienced positive job stress and experienced control in their work may have been more willing to participate in the study. This confounding effect may even have been stronger for the 56 OPs who returned the registration forms. These OPs scored significantly higher on the intention to apply the guideline. This may have influenced the external validity of this study, so caution is needed in generalizing the results.

Surprisingly, no relationship was found between guideline adherence measured by the questionnaires and performance based on the registered consultations, although both showed a minimal amount of guideline adherence. The limited number of items used to measure self-reported guideline adherence may have been a source of bias. The data which were collected on guideline adherence by registration forms were limited to central elements of the guideline and probably did not fully represent the consultation behaviour in practice. The criteria used to calculate performance rates may have been too rigid, especially for the registered quality of management. This may have caused a lack of internal and external validity. At the time of our study, other researchers developed and validated different PIs to measure quality of occupational rehabilitation for workers with mental health problems [8,10,23]. The PIs developed in this study have been useful tools in the further development of indicators to measure implementation and effects of guidelines in the field of occupational or public mental health [24,25].

Apart from these methodological considerations, the results of this study show that the implementation process of the guideline is not completed and should be continued, elaborated and improved in the near future. This study gives reason to be optimistic if new interventions are multifaceted, personalized and aimed at the actual barriers perceived by OPs in applying the guidelines. Such implementation strategies have been proven to be effective [26,27]. Interventions for further implementation should focus on

• (i) improving working conditions and offering personal education for OPs to improve their perceived control in guideline adherence;

• (ii) creating a social norm enhancing atmosphere: public attention and reminders on guideline adherence by OHS, professional debate in local consensus groups [27] and

• (iii) audit of medical files for monitoring guideline adherence by means of indicators.

Besides adapting the practice of OPs on the guideline, the content of the guideline itself should be critically reappraised. Clear action lists for each diagnosis, sharply defined referral criteria, attention for collaborating professions and a better layout could facilitate the application of the guideline. At the moment, a new committee is working on a revision of the guideline in the Netherlands, but in other countries such as the United Kingdom initiatives emerge as well [28]. It is important that future research is carried out to prove that implementation of a guideline leads to better outcomes in terms of quality of care and return to work.

Conflicts of interest

None declared.

References