Development and Validation of an Attitudinal-Profiling Tool for Patients With Asthma

Asthma is a common chronic respiratory disease. As of 2014, it was estimated to have afflicted 334 million people worldwide.1 This was an increase of 100 million new sufferers within a decade from 2004 when there were 234.9 million sufferers. In that same year, the South-east Asia and Western Pacific regions accounted for 45.5% (45.7 and 61.2 million, respectively) of all sufferers.2 The chronic nature of the disease and the large number of sufferers can present a major economic and social burden for countries in these regions.3 Asian studies of patients with asthma, which were conducted through face-to-face or phone interviews, addressed the burden of disease on the patient and their reactions to exacerbations caused by the disease.4, 5, 6, 7 The 2014 REcognise Asthma and LInk to Symptoms and Experience (REALISE) Asia8 survey of patients with asthma, instead, focused on their perceptions of asthma control, and attitudes toward the disease and its treatment in an accessible, real-life adult Asian population. It showed that many patients perceived their asthma to be controlled and not serious despite the presence of symptoms and exacerbations. A second REALISE Asia survey of physicians who treated patients with asthma showed that physicians also over-estimated the level of control among their patients.9, 10 Physician-patient follow-up consultations were short (average <10 minutes) and usually verbal, with limited use of printed material. Real-life studies also showed that asthma control, as defined by the Global Initiative for Asthma (GINA)11 criteria, was poor and highlighted a complex interaction of disease, doctor, patient and environment-related factors.8, 12, 13, 14 As a result, the optimal levels of asthma control achieved in clinical studies could not be replicated in real-life studies.15

Some tools are available to assist the physician in assessing the current level of asthma control and quality of life of the patient include the Asthma Control Test (ACT)16 and Asthma Control Questionnaire (ACQ).17 However, the ability of a patient to cope with a chronic disease, such as asthma, is influenced not only by its symptoms but also his personality profile and behavioral characteristics.18, 19, 20 Thus, physicians should consider the patient's psychological well-being and attitudes toward his/her disease besides treating the clinical symptoms of asthma as these can have an important bearing on the eventual treatment outcome.18, 19, 21, 22 Indeed, Axelsson et al.23 have shown that personality traits can influence how asthma patients adhere to asthma treatment and report their asthma control and quality of life. Unfortunately, tools that can help physicians identify the attitudinal profile of their patients and the treatment approach appropriate to this profile are not readily available. Jones et al.24 proposed a tool that can be used by physicians to compute the risk of having uncontrolled asthma based on patient attitudes toward medical professionals and the treatment for their asthma. However, to our knowledge, the development of a tool that can segment the population into distinct attitudinal profiles and assist in personalizing and tailoring management approaches has not been previously attempted for asthma patients.

In our REALISE Asia patient study (the results of which have been published8), an online survey was conducted among eligible respondents from validated25 consumer panels in China, Hong Kong Special Administrative Region, Indonesia, South Korea, Malaysia, Philippines, Singapore, and Taiwan who used social media. The survey focused on patients' perceptions of asthma control, and their attitudes toward the disease and its treatment in an accessible, real-life adult Asian population. The attitudinal data, collected from 2,467 respondents aged 18-50 years who suffered from the disease, were analyzed and used to identify attitudinal control groupings or 'clusters' for patients with asthma. A factor analysis (also published26) was used to summarise responses to 27 attitude-related question items from the survey questionnaire into a set of 9 attitudinal factors, and established the linkage between these 9 factors and the extent to which they are correlated. Then, through a cluster analysis, these respondents were grouped into clusters based on their ratings against these 9 factors. Each cluster described patients who were similar in their attitudes toward asthma but distinct from those in the other clusters. The result was 5 attitudinal clusters: Cluster 1 (Well-adjusted and at least partly controlled), Cluster 2 (In denial about symptoms), Cluster 3 (Adrift and poorly controlled), Cluster 4 (Tolerating with poor control), and Cluster 5 (Worried with multiple symptoms) which encompassed the whole spectrum of patient attitudes toward their asthma condition (Fig. 1).26

Fig. 1
Patient attitudinal clusters.

• Click for larger image
• Download as PowerPoint slide

This manuscript describes the development and validation of a profiling tool that will identify which 1 of the 5 attitudinal clusters identified above, a patient with asthma belongs. The knowledge of the cluster that a patient belongs to is expected to help healthcare professionals tailor the treatment approach to improve its outcome. At the time of this manuscript, there is an on-going study by an international panel of clinicians and allied health professionals working from Europe and Asia to generate consensus asthma management recommendations for each of these attitudinal clusters.27

We adopted a 2-step methodology: (1) to develop a Profiling Tool using a minimal list of question items from the REALISE Asia survey questionnaire that will accurately assign a patient to the correct attitudinal cluster, and (2) to validate this Profiling Tool using a split-sample procedure to confirm that its predictive accuracy can be achieved in actual practice.

Development phase

Subjects

The REALISE Asia patient survey was done through email invites to a random sample of individuals from validated consumer panels between December 2013 and March 2014 asking them to complete a 30-minute online questionnaire.25 The survey included individuals aged 18-50 years who were diagnosed with asthma, given ≥2 asthma-related prescriptions in the past 2 years (to ensure that individuals have active asthma) and had used social media. The following respondents were excluded from the survey: (1) those who were employed or had relatives employed in the healthcare, pharmaceutical, advertising or market research industries; and (2) those who had taken part in healthcare-related market research in the past 3 months. The survey flowchart is reproduced and shown in Fig. 2.8

Fig. 2
REALISE Asia patient survey flowchart.

• Click for larger image
• Download as PowerPoint slide

Item generation

The initial question set for the Profiling Tool consisted of the 27 attitude-related questions from the survey questionnaire which were used for the factor analysis to determine the 5 patient attitudinal clusters26 (Supplementary Material, highlighted in blue, green, and yellow). In the survey, respondents were asked on their degree of agreement, i.e., 'strongly disagree,' 'disagree,' 'tend to agree,' and 'strongly agree,' to various statements about their asthma. However, we decided to dichotomize (agree/disagree) their responses to take into account the possible variation in response styles between countries due to their cultural differences.

Item reduction

We set a range of 4-12 items as a viable number of questions that patients would be willing to answer in the Profiling Tool, which would also accurately identify a patient's cluster membership. As the number of combinations of 4 to 12 items that can be selected from the 27 attitude-related questions was very large (>120 million), we used a multinomial logistic regression technique to exclude question items that were deemed to be poorer predictors of cluster membership on the basis of their Wald statistics (<35) to reduce this number. Starting from question items with the highest Wald statistics, the smallest subset was drawn such that it contained at least 1 item from each of the 9 attitudinal factors which was used to obtain the attitudinal clusters.26 The selected subset consisted of 14 question items (namely, Q2: a3, a6, a9; Q34a2; Q39: a2, a6; Q46: a1, a2; and Q48: a2, a3, a5, a6, a7, a11; Supplementary Material, highlighted in green and yellow).

Discriminant function analysis (DFA) was used to determine whether this selected subset of question items (predictors) was effective in predicting the cluster membership of a given case.28 The output from the DFA was a set of equations; 1 for each cluster. Each equation contained the predictors and had a weight associated with each of them. The equation with the highest score for a case's set of predictor values was that with the highest probability of being the cluster that the case belonged to. Besides the 14 question items which were to be answered by the patient, the level of asthma control (i.e., well-controlled, partly controlled, and uncontrolled) using the GINA-based Helping Asthma in Real-life Patients initiative (iHARP)29 criteria was added to provide an input from the physician. Three combinations using the question items and the iHARP classification for the level of asthma control were considered: (1) including the iHARP classification as a single co-variate; (2) using the individual iHARP items as co-variates; and (3) excluding the iHARP classification altogether. We ran the DFA to test several formulations for each of these combinations, each time choosing 12 or fewer question items from the 14 items available to satisfy our desired range of 4-12 items. The objective was to find the formulation with the lowest misclassification rate or conversely, the highest accuracy rate i.e., the predicted cluster compared to the attitudinal cluster of the case which was known at the onset. We applied a rule-of-thumb of 75%30, 31 as the minimum accuracy to find the best formulations.

Most (62.3%) of the respondents were in the 26-40 years age groups. Both sexes were almost equally represented, with males forming 53.7% of the population. At the country level, the percentage of males ranged from a low of 41.1% for Malaysia to a high of 60.0% for Korea. They also accessed social media frequently with 85.0% of them using it 1 or more times daily. About 1 in 5 (17.8%) of the population had controlled asthma using the GINA-defined control status.11 Fig. 3 shows the demographic, socio-economic, and disease profiles of the respondents by country and for the population as a whole. These profiles were similar across the countries except for the reported household occupation in China where there was a higher proportion of respondents in the intermediate managerial category. For the distribution of the attitudinal clusters, Clusters 1 and 4 had the highest (28.9% and 29.0%, respectively) representation, while cluster 5 the least (11.3%). Fig. 4 shows the cluster distribution pattern for respondents in each of the surveyed countries and the total population. These cluster profiles were similar to each other.

Fig. 3
Demographic, socio-economic, and disease profiles by country.

• Click for larger image
• Download as PowerPoint slide

Fig. 4
Attitudinal cluster profile by country.

• Click for larger image
• Download as PowerPoint slide

Development phase

Fig. 5 shows the accuracy rates for the DFA model which used the full sample of 2,400 cases from the REALISE Asia survey and had complete data for the question items of the Profiling Tool. The set of formulations using the iHARP classification as a single co-variate produced solutions with the best accuracy rates ranging from 70.1% to 77.5% depending on the number of question items used. The 11-predictor variables model (Fig. 5, circled brown); consisting of 10 question items (Supplementary Material, highlighted in yellow), and iHARP classification as a single variable was selected as the formulation that had the highest accuracy (77.2%, benchmark: 75.0%) while using a minimal number of question items. The question items in this formulation were applied to the Profiling Tool in the same sequence as in the original survey questionnaire so as to retain the order effect among the questions.

Fig. 5
Predictive accuracy rates corresponding to the number of predictor variables for iHARP variable combinations.

• Click for larger image
• Download as PowerPoint slide

Final adjustments

We re-arranged the order of the questions by bringing forward the 'stressful life' question item (Q46a2) as it is related to the respondent's general situation rather than concerning their asthma specifically or its treatment and by grouping the inhaler-related questions together (Q39a6, Q48a2-3). We excluded Q39a2 (My doctor doesn't understand my asthma) as we felt that it was a leading question on the patient-doctor relationship and replaced it with Q2a3 (I have no time to think about my health as other things are more important). The final list of questions was marginally less efficient (76.2% vs. 77.2%) (Table 1).

Table 1
List of questions in the Profiling Tool

• Click for larger image
• Click for full table
• Download as Excel file

Validation phase

Fig. 6 shows the predictive accuracies of the 100 validation runs, ordered independently by the estimation and the validation sub-samples, from the most to the least accurate. The blue line shows the accuracy rates of the estimation sub-samples, each with an error bar (also in blue) denoting the 95% confidence intervals for that accuracy rate. The accuracy rates ranged from 74.8% to 78.7% (average 76.7%) with reference to the original attitudinal cluster of each case. The ordered accuracy rates for the validation sub-samples, which ranged from 72.3% to 78.3% (average 75.3%), were about 1.4 percentage points lower compared to the ordered rates for estimation sub-samples (Fig. 6, red line). The 76.2% accuracy line (Fig. 6, green line) of the Profiling Tool lies within the 95% confidence intervals of all accuracy rates except for 1 estimation and 7 validation sub-samples.

Fig. 6
Ranked accuracy rates of estimation and validation sub-samples.

• Click for larger image
• Download as PowerPoint slide

The scatter plot of the accuracy rates of the estimation and validation sub-samples for the 100 validation runs shows an inverse relationship whereby the more accurate the estimation model gets, the less accurate the validation sub-sample estimates becomes (Fig. 7). The point where the accuracy rates of the validation and estimation sub-samples are approximately equal was 76.0% (Fig. 7, dashed lines). Table 2 shows the accuracy rates by cluster for the Profiling Tool and the estimation and validation sub-samples. Cluster 1 had the highest accuracy rates (90.7%-91.1%). The accuracy rates for the other clusters ranged from 66.3% to 73.4%.

Fig. 7
Scatter plot of accuracy rates of validation sub-samples to estimation sub-samples.

• Click for larger image
• Download as PowerPoint slide

Table 2
Accuracy rates of the Profiling Tool, validation sub-sample, and estimation sub-sample by cluster

• Click for larger image
• Click for full table
• Download as Excel file

This study was the first attempt to develop a patient attitudinal-profiling tool that can help physicians tailor the asthma management plan for their patients at the point-of-care. Besides the physician assessment of the disease condition, the patient's attitude toward the disease was concurrently evaluated as it has an important influence on his/her adherence to the management plan and treatment outcome.33 The Profiling Tool contained 10 simple 'agree/disagree' questions for self-completion by the patient, and a question on the level of asthma control for completion by the attending physician. We believed that the small number of questions and simple response structure (agree/disagree) would make the tool easy for patients to use.

We recognized that there could be differences in the socio-economic profiles of respondents from the countries in the survey that may limit the effectiveness of the tool. Such differences can be expected of a survey spanning a number of countries. However, the profile plots for a number of socio-economic and disease factors showed that they were, by and large, similar across the surveyed countries and the total population. There were no indications of bias for these factors in the respondent population of any country. The cluster representation of the respondents in each of the surveyed country matched the total population. We believe that respondents who had access to social media may be similar across the surveyed countries, especially those from the 26-40 years age groups and who used such media frequently, and contributed to the uniformity seen in these profiles. This may place a limitation on the use of the tool beyond the tech-savvy population of the urban centers.

The validation results showed that the discriminant functions for the Profiling Tool were robust and accurate in assigning a case to one of the 5 attitudinal clusters. Although the predictive model from each of the estimation sub-samples used a smaller number of cases (about 1,200 cases) compared to the Profiling Tool (2,400 cases), its accuracy rate in assigning the cases in the validation sub-sample was as good as the tool. These validation sub-samples were sufficiently large and could optimally detect differences in the predictive accuracy of the estimation models. However, there were variations in the accuracy rates (67.8%-90.7%) at the cluster level, Cluster 5 (worried with multiple symptoms) being the lowest and Cluster 1 (well-adjusted and at least partly controlled) the highest. The well-adjusted patients were apparently more consistent in their responses. While the tool was most effective in identifying the well-adjusted patients, it remained sufficiently effective for the other clusters. The lowest accuracy rate of 67.8% exceeded the 20.0% level (i.e., 1 in 5 clusters) that would be expected by chance. Importantly, the tool could identify patients in the most urgent need of attention i.e., those who were tolerating (Cluster 4, 70.6%) or were in denial (Cluster 2, 69.5%) of their poor asthma condition, in 7 cases out of 10.

An inherent limitation to our method was how to ensure that the sample from which we used for the Profiling Tool, which came from a population aged 18 to 50 years with access to social media, was not peculiar or different from future samples. In particular, the management of asthma in the elderly presents its own challenges.34 The parameters derived by the DFA maximized the distinctions between the clusters (i.e., 'patient attitudinal types'). The key factor for the stability of these parameters was that they were not specific to our sample but were also applicable to other samples. In our survey, the cross-validation technique used to address this issue by randomly and repeatedly drawing the sample cases from patient populations in all the 8 countries/area into the 2 sub-samples proved that the derived parameters were not susceptible to the patient composition i.e., country, age, gender, or disease status in the sample. Had some data points of a sub-sample been peculiar, they would affect the tool's predictability and lower its accuracy rate. Low accuracy rates would have meant that our Profiling Tool could not be generalized to other samples. Our results showed that this was not the case. The accuracy rates of the estimation and validation sub-samples (76.7% and 75.3% respectively) were comparable to the accuracy of the discriminant functions operating in the Profiling Tool (76.2%) in assigning the cases to the correct clusters. On average, the tool correctly fitted 3 in every 4 cases: an accuracy rate exceeded the expected 20.0% level and met our 75.0% benchmark target.30, 31 This reinforced our view that the discriminant functions used in the Profiling Tool were generalizable beyond our population of 2,400 cases.

The relationship between the accuracy rates of the estimation and validation sub-samples exhibited an inverse relationship whereby the more accurate the estimation model, the less accurate the validation sub-sample model estimate. The 76.0% point where the 2 accuracy rates were equal was similar to the accuracy rate of 76.2% of the tool, which suggested that the discriminant functions for the Profiling Tool have not been over-fitted. Furthermore, the difference between the accuracy rates of estimation and validation sub-samples of 1.4 percentage-points provided an upper-bound estimate for the degree of over-fitting that we can expect using the original 2,400 cases to estimate the discriminant functions as the larger sample (2,400 vs. 1,200) would have generated a smaller confidence interval.

Our results showed that our Profiling Tool could identify which attitudinal cluster a patient with asthma belongs to and enable the physician to make appropriate asthma management decisions in practice.

REALISE Asia patient survey questionnaire.

Attitudinal questions used for Profiling Tool highlighted (27 attitudinal questions were highlighted in blue, green and yellow. Of the 14 question items used for the discriminant function analysis to identify the predictive model, the 10 questions items which were selected and included in the Profiling Tool are highlighted in yellow, and the 4 excluded are highlighted in blue).