Assessing Frailty with the Tilburg Frailty Indicator (TFI): A Review of Reliability and Validity

Introduction

The Tilburg Frailty Instrument (TFI) is an instrument for assessing frailty in community-dwelling older people. It has been developed as a self-report instrument in that older people have to complete the TFI themselves.¹ The TFI is based on the following conceptual definition of frailty: ‘frailty is a dynamic state affecting an individual who experiences losses in one or more domains of human functioning (physical, psychological and social), which is caused by the influence of a range of variables and increases the risk of adverse outcomes’.² Both the conceptual definition of frailty and the TFI derived from it consider frailty as a multidimensional concept, including physical, psychological and social functioning of older people, and emphasize the importance of an integral approach to human functioning. The World Health Organization recommends this holistic approach to take care of frail, older people.³ Paying sole attention to physical frailty can lead to fragmentation of care^2,4 and possibly to a reduction of the quality of care and a decrease in the experienced quality of life in frail older people. According to Gilardi et al, a multidimensional approach to frailty can be more effective to plan and implement care services, as well as establish prevention programs for frail older people.⁵

The TFI contains two parts: part A, on 10 determinants of frailty, and part B, on 15 components of frailty.¹ The determinants are sex, age, marital status, education, income, ethnicity, lifestyle, life events, multimorbidity and living environment. The components of frailty refer to physical frailty (eight), psychological frailty (four) and social frailty (three). Physical frailty includes physically unhealthy, unexplained weight loss, difficulty walking, difficulty maintaining balance, poor hearing, poor vision, lack of strength in the hands and physical tiredness. Psychological frailty consists of the components of memory problems, feeling down, feeling nervous or anxious and unable to cope with problems. Social frailty includes living alone, lack of social relations and lack of social support. The total score of the TFI is 0–15, with a score ranging from 0 to 8 for physical frailty, 0 to 4 for psychological frailty and 0 to 4 for social frailty. Higher scores refer to greater frailty, as older persons with a total TFI score ≥5 are considered to be frail.¹

Originally, the TFI was developed in the Netherlands by Gobbens et al and based on an extensive literature search and opinions of an international group of frailty experts, including geriatricians, gerontologists, nurses and psychologists.⁶ Their first study examined the psychometric properties of the TFI in two Dutch samples of community-dwelling persons aged 75 years and older.¹ Subsequently, the TFI was translated into several languages, including Brazilian Portuguese,⁷ Danish,⁸ Italian,⁹ Portuguese,¹⁰ Polish,¹¹ German,¹² Chinese,¹³ Spanish,¹⁴ and Turkish.¹⁵ Until now, two systematic reviews and one narrative review have been published, indicating that the TFI is very suitable for assessing frailty among community-dwelling older people.^5,16,17 According to Pialoux et al, both the TFI¹ and the SHARE Frailty Index¹⁸ are potentially suitable for screening frailty in older people in primary care settings.¹⁶ Sutton et al concluded that the TFI has the most robust evidence of reliability and validity of 38 frailty assessment instruments, including frequently used instruments, such as the Phenotype of Frailty¹⁹ and the Frailty Index.¹⁷ In addition, the narrative review by Gilardi et al identified the TFI as the best screening instrument to use in public health because it was the only one of the selected instruments (including the Phenotype of Frailty,¹⁹ Vulnerable Elders Survey,²⁰ Frailty Index,²¹ and the SHARE Frailty Index)¹⁸ with three features: a multidimensional structure, quick and easy to use, and an accurate risk prediction of adverse outcomes of frailty.⁵ In addition, De Witte et al used the TFI as a gold standard for the validation of their instrument, The Comprehensive Frailty Assessment Instrument.²²

Since the TFI was developed approximately 10 years ago,¹ and many studies into its psychometric properties have been carried out since then,¹⁷ this study aims to provide a review of the main findings regarding this issue. We present the reliability and validity of these studies.

Methods

Literature Search

We conducted a literature search in the PubMed and CINAHL databases on May 30, 2020, using “Tilburg Frailty Indicator AND psychometric properties”, “TFI AND psychometric properties”, “Tilburg Frailty Indicator AND validity”, “TFI AND validity”, “Tilburg Frailty Indicator AND reliability”, and “TFI AND reliability”. An inclusion criterion was the use of the entire TFI, part B, referring to the 15 components. No restrictions were placed on language or year of publication. The studies were screened and selected for inclusion by the first author. In total, 27 studies were selected for the purpose of this review.

Reliability and Validity

Reliability

Four types of reliability were distinguished: internal consistency, test-retest, inter-rater reliability, and parallel forms reliability. Internal consistency refers to consistency across items of measurement. Statistical techniques used for this purpose were Cronbach’s alpha, Kuder–Richardson Formula 20 (KR-20) and item correlations. Cronbach’s alpha and KR-20 values >0.70 were considered acceptable.^23,24 The higher the item correlations, the better the internal consistency of the measurement instrument.

Test-retest indicates consistency among time (stability). Correlations, simple agreement, kappa (chance-corrected agreement) and intraclass correlation coefficient (ICC) were used to determine test–retest reliability. The higher the correlation, simple agreement, kappa value and ICC, the higher the concordance between the two assessments will be. The correlation coefficient was evaluated using the classification of Callegari-Jacques (weak, <0.30; moderate, 0.30–0.60; strong, 0.60–0.90; very strong, ≥0.90).²⁵ For the interpretation of the Kappa value, we used the Landis and Koch evaluation (absent, <0.10; weak, 0.10–0.20; fair, 0.21–0.40; moderate, 0.41–0.60; substantial, 0.61–0.80; nearly perfect, 0.81–1.00).²⁶ The ICC was evaluated using the guideline provided by Koo and Li (poor <0.50; moderate, 0.50–0.75; good, 0.75–0.90; excellent >0.90).²⁷

Inter-rater reliability concerns consistency across different researchers. Frequently used techniques to establish inter-rater reliability are Kappa and ICC.

The fourth and final type of reliability, parallel forms reliability, assesses the correlation between two equivalent versions of a measurement instrument. High correlation between the two instruments indicates high forms reliability.

Validity

Six types of validity can be distinguished: criterion, construct, content, face, structural, and known-group validity. Criterion validity concerns the relation between the score on a measurement instrument and some external criterion.²⁸ If the measurement instrument corresponds to a criterion assessed simultaneously, the validity is considered concurrent. If the measurement instrument forecasts a criterion value in the future, the validity is labeled predictive.²⁸ Criterion validity can be checked by determining a correlation coefficient and conducting receiver operating characteristics (ROC) curve analyses and calculating the Area Under the Curve (AUC). An AUC <0.7, 0.7–0.8, 0.8–0.9, and ≥0.9, is considered poor (no discrimination), acceptable, excellent, and outstanding, respectively.²⁹

Construct validity refers to the simultaneous process of measure and theory validation.³⁰ Convergent and divergent (discriminant) validities constitute construct validity. Convergent validity involves the degree by which two measures of constructs should be or are related. By contrast, divergent validity indicates whether measures that should be unrelated, are unrelated.³¹ Correlation tests are performed to establish convergent and divergent validity and thus, construct validity.

Content validity concerns the extent in which a measurement instrument includes all necessary components of the construct to be assessed.³² According to Burns and Grove, content validity is obtained from literature, representatives of the population concerned and experts.³³ A measurement instrument has face validity if it appears to assess what it is supposed to assess and that it will work.³⁴ As well as when determining content validity, representatives of the population concerned and experts can be involved; however, establishing face validity is more informal, compared to content validity.

Structural validity refers to the extent to which an instrument covers the hypothetical dimension of a construct.³² According to Souza et al, factorial analysis and structural equation modeling are the appropriate statistical techniques to assess structural validity.³⁵ Finally, known-group validity involves an instrument’s ability to make a distinction between groups. Group differences can be determined using a chi-square test and t-test. When describing the findings on the reliability and validity of the TFI in the included studies, the classification described above has been used foremost.

Results

Characteristics of the Studies

Table 1 presents the characteristics of the 27 included studies. The first studies were performed in 2010.^1,36 Thirteen studies were carried out in the Netherlands,^1,36–47 of which two were part of a large study that also collected data in other European countries.^46,47 Three studies were exclusively conducted in Denmark,^8,48,49 and two in Poland^11,50 and Brazil.^7,51 By far, most of the studies were conducted among community-dwelling older people. Two exceptions were studies including older Danish people admitted to a hospital⁴⁹ and one study evaluating older Dutch residents of assisted living facilities.³⁹ Two-thirds of the studies used a cross-sectional design, two of which were qualitative studies conducted in Denmark,^8,48 and the other studies were characterized by a longitudinal design. The sample size varied from 14 to 27,527 people.^46,48 The most commonly used age groups were people aged ≥65 years (eight studies) and aged ≥70 years (seven studies). One study only showed the mean age of the sample.⁸ The highest mean age was observed among Dutch people residing at assisted living facilities (84.8 years).³⁹

Table 1 General Characteristics of the Studies Included

Using the original TFI cut-off point of 5,¹ prevalence figures concerning the general population of community-dwelling older people ranged from 12.4% to 47.1% in samples of Chinese and Dutch individuals.^1,13 Specifically, prevalence was higher among Turkish people admitted to a geriatrics outpatient clinic (63.6%),¹⁵ residents of assisted living facilities (76.5%)³⁹ and among physical pre-frail and frail community-dwelling older people (64.8%).⁴³ In the latter group, the frailty status of the participants was first assessed by the Phenotype of Frailty.¹⁹ Among community-dwelling older people, the prevalence of frailty was highest in a sample of Portuguese people (54.8%), while the cut-off point was 6.¹⁰ It should be noted that seven studies did not present a prevalence figure of frailty.^{8,14,41,42,44,47,48} A Dutch sample including 479/484 participants and a Dutch sample consisting of 2420 participants were used in three^1,41,42 and two studies,^43,44 respectively. Moreover, two other Dutch studies partially used the same sample.^36,37 Additional details are displayed in Table 1.

Reliability of the TFI

In total,16 studies report the reliability of the TFI. The four types of reliability observed were internal consistency, test–retest, inter-rater and parallel forms reliability. Fifteen studies determined the internal consistency reliability and one study failed to do so (see Table 2).⁴⁶ The Cronbach’s alpha for the TFI total was 0.66 (lowest)⁹ to 0.80 (highest),⁴⁵ whereas the KR-20, calculated in three studies was 0.69, 0.70 and 0.78.^10,14,52 Eight studies also present the Cronbach’s alpha for physical, psychological and social frailty, ranging from 0.57 to 0.79,^7,9 0.37 to 0.63,^1,50 and 0.25 to 0.59,^13,50 respectively. The lowest and highest values of the KR-20, with regard to physical, psychological, and social frailty, were 0.64¹⁴ and 0.75,¹⁰ 0.48¹⁰ and 0.58,¹⁴ and 0.22¹⁴ and 0.49,¹⁰ respectively. Six studies examined the internal consistency reliability of the TFI using corrected item-total correlations.^{9,12,15,36,50,52}

Table 2 Internal Consistency Reliability of the TFI

Test–retest reliability was observed in nine studies, using Pearson correlations,^1,7,10 Kappa,^7,10,14,50 simple agreement,^7,10,14 and ICC^12,13,15,52 (see Table 3). Pearson correlations, with regard to frailty total were 0.88,⁷ 0.90,¹ and 0.91,¹⁰ using a period less than 3 weeks. The Pearson correlation coefficient was 0.79 for a 1-year period.¹ The correlation coefficients, with respect to the frailty domains, are detailed in Table 3. Using Kappa, the level of agreement varied greatly at item level.^7,10,14,50 Obviously, the level of agreement was higher when the simple agreement technique was used^7,10,14 (displayed in Table 3). For frailty total, the ICC ranged from 0.86 to 0.99^15,52 in two studies involving a follow-up period of 1 week.

Table 3 Test–Retest Reliability, Inter-Rater Reliability, and Parallel Forms Reliability of the TFI

Table 3 also shows the inter-rater and parallel forms reliability of the TFI. Inter-rater reliability was identified in only one study in which, on the same day, two observers came to almost perfect agreement (ICC = 0.99).¹⁵ Finally, in two studies, parallel forms reliability was determined.^13,46 One of these studies examined the agreement between the TFI (frailty total) and other validated frailty instruments. The highest agreement existed with the Frailty Index (FI) and the Comprehensive Geriatric Assessment and less agreement was found with the Edmonton Frail Scale and the Frail scale.⁴⁶ The other study used Kappa to establish the level of agreement between TFI items and alternative measures resulting in low and high levels of agreement (0.12 for hearing problems and 1.00 for living alone).¹³

Validity of the TFI

Criterion Validity

Tables 4–6 show an overview of the validity of the TFI, which includes 24 of the 27 studies. Criterion validity was the most frequently presented type of validity, with concurrent and predictive characteristics reported in 10 and 9 studies, respectively (see Table 4). Concurrent validity was determined using different techniques: correlations, AUC and regression analyses. Frequently occurring adverse outcomes of interest were lower quality of life,^{1,10,39,40,45} disability,^{1,9,10,13,39,47} and an increase in health-care utilization.^1,9,10,13,39 Using different instruments for assessing quality of life (WHOQOL-BREF,⁵³ WHOQOL-OLD⁵⁴ and EUROHIS-QOL)⁵⁵, four included studies demonstrated that higher scores on the TFI were correlated with lower quality of life.^1,10,40,45 Regarding disability and referring to limitations in performing activities of daily living (ADL) and/or instrumental activities of daily living (IADL), the AUCs were excellent in three studies,^1,9,47 and acceptable for ADL and poor for IADL in two studies.^10,13 Many different indicators of health-care utilization were used, eg, visits to a general practitioner, hospitalization and receiving nursing care. In most studies, the findings were poor^9,10,13; however, Gobbens et al observed an excellent AUC for reporting personal care, and acceptable AUCs for reporting nursing and informal care.¹ Three studies determined the discriminating ability of the TFI to identify frailty with other validated frailty measures using the AUC: the Groningen Frailty Indicator (GFI),¹⁰ the phenotype of frailty,^10,13 Survey of Health, Ageing and Retirement in Europe-Frailty Instrument (SHARE-FI),⁴⁷ and the Frailty Index (FI).¹³ The AUCs for GFI, SHARE-FI and the FI were excellent; however, the AUCs for the phenotype of frailty were not unequivocal. Two studies examined the correlations between the TFI and other frailty measures: the GFI,³⁶ Sherbrooke Postal Questionnaire (SPQ),³⁶ and the Phenotype of Frailty.¹⁴ The correlations between the TFI and GFI, SPQ and phenotype of frailty were 0.76, 0.42, and 0.49, respectively.^14,36

Table 4 Criterion Validity of the TFI

Table 5 Construct Validity of the TFI

Table 6 Content Validity, Face Validity, Structural Validity, and Known-Groups Validity of the TFI

The predictive validity was established using regression analyses and AUC. The prediction period ranged from 6 months⁴⁹ to 5 years.⁴⁶ Most studies only used a period of 1 or 2 years.^{37,38,42–44,51} In particular, adverse outcomes of interest were disability,^{37,38,42–44,51} mortality,^{37,43,46,49,51} increased health-care utilization^{37,38,42,43,51} and lower quality of life.^41,42 Disability was predicted by the TFI; however, Gobbens et al^38,42 indicated that the predictive value was excellent, while Op Het Veld et al concluded that it was poor,⁴³ presenting a positive predictive value of 42.6% and a negative predictive value of 75.2%.⁴⁴ In four studies, the findings concerning mortality were at least acceptable; only Op Het Veld et al qualified the predictive value of the TFI as poor.⁴³ As with the determination of concurrent validity, many different indicators of health-care utilization were used as outcome variables, resulting in findings that were not unanimous. For instance, the TFI predicted hospitalization in a sample of 430 Dutch people ≥70 years (OR = 2.59, 95% CI = 1.36–4.90),³⁷ in comparison to a poor AUC in a sample of 2420 Dutch people ≥65 years.⁴³ Both studies that aimed to assess the predictive value of the TFI for quality of life provided evidence that the TFI predicts lower quality of life using a follow-up period of 1, 2 and 4 years.^41,42

Construct Validity

The reviewed studies frequently determined the criterion validity of the TFI, as well as the construct validity. Ten studies determined the construct validity of the TFI,^{1,7,9,10,12–14,45,47,52} with nine of these studies found to address the issue of convergent and divergent validity, with exception of the study by Renne and Gobbens (see Table 5).⁴⁵ Most of the studies found the expected correlations between total frailty, domains, items, and alternative measures, while two studies observed similar correlations between the psychological and social domains of the TFI and alternative psychological.^10,13 Moreover, the Spanish study demonstrated a stronger correlation between social frailty and IADL, assessed with the Lawton scale, than between social frailty and physical frailty, which was expected.¹⁴ Finally, in the Brazilian study, the item coping was not correlated as expected.⁷

Content and Face Validity

Table 6 presents the content, face, structural and known-groups validity of the TFI. Three studies determined the content validity of the TFI,^1,46,48 showing that the TFI contains the majority of important frailty items. Based on interviews with community-dwelling older people, Andreasen et al argue that items referring sleep quality, pain, spirituality, and meaningful activities should be included in the TFI.⁴⁸

Face-validity was established in only two studies.^1,8 In the first study, the TFI was checked by participants at geriatric meetings and in the second study, conducted in Denmark, after translating the TFI, a pretest was performed by cognitive interviewing concluding that the TFI could be further tested in practice.

Structural Validity and Known-Groups Validity

Both structural validity and known-groups validity were only determined by Vrotsou et al (see Table 6).¹⁴ Fit indexes of a second-order model of three factors (frailty domains) were acceptable and the TFI differentiated well between frail and non-frail people, as defined by the Gérontopôle Frailty Screening Tool (GFST)⁵⁶ and the Short Physical Performance Battery (SPPB).⁵⁷

Discussion

The TFI is a questionnaire that is increasingly used to determine frailty in older people. Since the introduction of the TFI in 2010, many studies on its psychometric properties have been conducted. In this study, we aimed to present a review of findings regarding the reliability and validity of the TFI. The present study is the first to assess these psychometric properties of the TFI.

The literature search performed through May 30, 2020, showed that 27 studies reported on the psychometric properties of the TFI, as related to reliability, validity or both. Most of the studies (n = 25) were focused on community-dwelling older people. Since the TFI was developed in the Netherlands,¹ it is not surprising that 13 of the included studies were conducted in that country and there appeared to be large differences in the prevalence of frailty. The lowest and highest prevalence figures were 12.4%¹³ and 76.5%,³⁹ respectively. Our review shows higher prevalence figures of frailty are closely related to greater age. This finding is supported by a systematic review containing cross‐sectional data from community‐based cohorts.⁵⁸ In addition, we found higher prevalence figures among people residing in settings other than the community, eg, acutely admitted patients,⁴⁹ residents of assisted living facilities,³⁹ and people admitted to a geriatrics outpatient clinic.¹⁵ Zhang et al emphasized that the mean score on the TFI is influenced by the country of residence of the participants.⁴⁷ For example, the mean score for people with an average age of 75.3 years from Greece was 5.80, while this score was 4.25 in a sample of Dutch people with an average age of 81.5 years.⁴⁷ The country of residence of the participants may also be the possible explanation for the high prevalence among older Portuguese people (54.8%);¹⁰ this prevalence is exceptionally high if we consider that the TFI cut-off point of 6¹⁰ was taken, not the established cut-off point of 5.¹

Our study shows that the reliability of the TFI has been comprehensively assessed; 15 and nine studies examined the internal consistency and test–retest reliability, respectively. In many cases, the reliability of the TFI, reflected by Cronbach’s alpha, was >0.70, which indicates satisfactory reliability.²⁴ The internal consistency of the individual domains of the TFI was worse, particularly for the psychological and social domains, which can be explained by the fact that these domains only contain four and three items. Test–retest reliability, reflected by correlations (>0.60) and ICC (>0.75), was good.^25,27 Moreover, kappa coefficients showed a substantial or nearly perfect level of agreement concerning many individual TFI items. The two other types of reliability, inter-rater and parallel forms, have only been examined to a limited extent.^13,15,46

The validity of the TFI has been established in 24 studies. In many studies focusing on criterion validity (concurrent and predictive), adverse outcomes of interest were disability, increased health-care utilization, lower quality of life and mortality. Regarding disability, studies predominantly show results that are excellent,^1,9,38,42,47 with AUCs >0.80.²⁹ The findings pertaining to an increase in health-care utilization present a less unambiguous picture and seem to depend strongly on the indicator used (eg, personal care, informal care, visits to a general practitioner or hospitalization). The TFI, however, clearly has a poor association with visits to a general practitioner^1,9,42 and hospitalization.^{1,13,37–39,42,43} In all six studies using quality of life as the outcome, the TFI showed good associations with lower quality of life, independent of the quality of life instrument that has been used.^{1,10,40–42,45} Moreover, the TFI predicted mortality, with only Op Het Veld et al⁴³ qualifying the predicted value for this outcome as low. It should be noted that follow-up periods were short, consisting of 1 or 2 years, except the study by Theou et al⁴⁶ that included a follow-up of 5 years. Unfortunately, Theou et al did not use the original TFI for assessing frailty.⁴⁶ Therefore, we recommend determining the prediction value of the original TFI for mortality using a follow-up period >5 years. Finally, concerning criterion validity, the TFI showed discriminating ability in regards to GFI,¹⁰ SHARE-FI,⁴⁷ and FI,¹³ reflected by excellent AUCs (>0.80).

Construct validity of the TFI, distinguishing convergent and divergent validity, was established in nine studies. In many cases, the domains and components of the TFI correlated as expected with alternative measures providing evidence for good construct validity. Content and face validity were only determined in three^1,46,48 and two studies,^1,8 respectively. Two of these studies were qualitative in nature,^8,48 which seems ideally suited to establish these types of validity. In our opinion, to validating an instrument like the TFI, the involvement of preferably older people, is necessary, whom, from their perspective, can indicate what needs to be questioned in the context of frailty. Only one study assessed the structural and known groups validity.¹⁴ These findings were satisfactory.

Some limitations of our study should be noted. First, different instruments and questions have been used to assess disability, eg, Groningen Activity Restriction Scale (GARS)⁵⁹ and Katz scale,⁶⁰ and indicators of health-care utilization. Differences concerning variables controlled in regression analyses also exist, which could have influenced the findings. Secondly, many of the included studies aimed to determine the reliability and validity of the TFI were conducted in samples of community-dwelling older people. More studies are needed in order to establish good reliability and validity of the TFI in other samples of older people (eg, residents of assisted living facilities, nursing homes, mental health institutions and hospitalized patients). Thirdly, many data on reliability and validity were available and it was impossible to present all these data. A selection based on the most relevant data for the present study was therefore utilized and the individual studies were referred to for more detailed information. Finally, the majority of studies have been performed in Europe and especially in the Netherlands, with less in Brazil,^7,51 Saudi Arabia,⁵² and China.¹³ The determination of the psychometric properties of the TFI in the other continents of the world, such as Africa, North America and Australia, is recommended.

The TFI is specifically designed for screening of multidimensional frailty among community-dwelling older people; our review provided much evidence that the TFI is ideally suited to that target group. Well-known determinants of frailty, assessed with the TFI, are higher age, being a woman, and a low socio-economic status, expressed by low-educational and low-income level.^36,61,62 To prevent frailty, it is recommended to start screening among people that meet those criteria. Based on this screening, primary health-care professionals (eg, general practitioners, nurses, physiotherapists, occupational therapists) can determine, preferably in a multidisciplinary consultation, whether these people need a comprehensive geriatric assessment (CGA) and a care needs assessment. In addition, the findings of the screening of frailty using the TFI can provide primary health-care professionals a first direction to the interventions that should be conducted next. Evidence of beneficial effects of multidomain interventions compared to unidomain interventions on frailty status or score is limited but increasing.⁶³

In conclusion, our literature search revealed 27 studies examining the reliability and/or validity of the TFI. Many studies showed that the internal consistency and test–retest reliability are noteworthy, as well as the criterion and construct validities. In contrast, the association of the TFI with some indicators of health-care utilization can be indicated as poor (eg, visits to a general practitioner or hospitalization). Knowing that population aging is occurring all over the world, the availability of the TFI is critical. In addition to the qualification of its psychometric properties as good, it is well known that the TFI is a user-friendly instrument for assessing frailty. The findings of this assessment can support health-care professionals in selecting interventions to reduce frailty and delay its adverse outcomes, such as disability and lower quality of life.