Abstract

In the last decades, several instruments have been used to evaluate the impact of oral health problems on the oral health-related quality of life (OHRQoL) of individuals. However, some instruments lack thorough methodological validation or present conceptual differences that hinder comparisons with instruments. Thus, it can be difficult to clinicians and researchers to select a questionnaire that accurately reflect what are really meaningful to individuals. This short communication aimed to discuss the importance of use an appropriate checklist to select an instrument with a good methodological quality. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to provide tools for evidence-based instrument selection. The COSMIN checklist comprises ten boxes that evaluate whether a study meets the standard for good methodological quality and two additional boxes to meet studies that use the Item Response Theory method and general requirements for results generalization, resulting in four steps to be followed. In this way, it is required at least some expertise in psychometrics or clinimetrics to a wide-ranging use of this checklist. The COSMIN applications include its use to ensure the standardization of cross-cultural adaptations and safer comparisons between measurement studies and evaluation of methodological quality of systematic reviews of measurement properties. Also, it can be used by students when training about measurement properties and by editors and reviewers when revising manuscripts on this topic. The popularization of COSMIN checklist is therefore necessary to improve the selection and evaluation of health measurement instruments.

Key-Words:
checklist; methods; oral health; quality of life; surveys and questionnaires

Resumo

Nas últimas décadas, vários instrumentos tem sido utilizados para avaliar o impacto dos problemas de saúde bucal na qualidade de vida relacionada à saúde bucal (OHRQoL) dos indivíduos. No entanto, alguns instrumentos não possuem validação metodológica completa ou apresentam diferenças conceituais que dificultam as comparações com instrumentos. Assim, pode ser difícil para clínicos e pesquisadores selecionar um questionário que reflita com precisão o que é realmente significativo para os indivíduos. Esta comunicação rápida teve como objetivo discutir a importância de usar um checklist apropriado para selecionar um instrumento com boa qualidade metodológica. O COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist foi desenvolvido para fornecer ferramentas para a seleção de instrumentos baseados em evidências. O COSMIN checklist compreende dez quadros que avaliam se um estudo atende o padrão para uma boa qualidade metodológica e dois quadros adicionais para atender a estudos que usam o método Item Response Theory e os requisitos gerais para a generalização de resultados, resultando em quatro etapas a serem seguidas. Desta forma, é necessário pelo menos alguma experiência em psicometria ou clinimetria para um amplo uso deste checklist. As aplicações do COSMIN incluem seu uso para garantir a padronização de adaptações transculturais e comparações mais seguras entre estudos de mensuração e avaliação de qualidade metodológica de revisões sistemáticas de propriedades de mensuração. Além disso, ele pode ser usado por estudantes ao treinar sobre propriedades de mensuração e por editores e revisores ao revisar manuscritos sobre este tópico. A divulgação do COSMIN checklist é, portanto, necessária para melhorar a seleção e avaliação dos instrumentos de medição da saúde.

Introduction

Subjective health measurement instruments play a key role in public health, clinical practice and decision-making processes for health services organization ¹1 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford, PW; Knol, DL; et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010;19:539-549.^,22 Berke, EM; Choudhury, T; Ali, S; Rabbi, M. Objective measurement of sociability and activity: mobile sensing in the community. Ann Fam Med 2011;9:344-350.. In Dentistry, several instruments have been developed and used aiming to assess the impact that oral diseases or even poor oral health has on daily functioning, well-being or oral health-related quality of life (OHRQoL) of individuals and families ³3 Locker, D; Allen, F. What do measures of ‘oral health-related quality of life’ measure? Community Dent Oral Epidemiol 2007;35:401-411.. These data are a useful source of information for planning and organizing oral health care services, directed to the studied population ⁴4 Martins Júnior, PA; Vieira-Andrade ,RG; Corrêa-Faria, P; Oliveira-Ferreira, F; Marques, LS; Ramos-Jorge ML. Impact of early childhood caries on the oral health-related quality of life of preschool children and their parents. Caries Res 2013;47:211-218..

Most of these instruments have been cross-culturally adapted and validated to be used in a plenty of countries ⁵5 Slade, GD; Spencer, AJ. Development and evaluation of the Oral Health Impact Profile. Community Dent Health 1994;11:3-11.^,66 Jokovic, A; Locker, D; Stephens, M; Kenny, D; Tompson, B; Guyatt, G. Validity and reliability of a questionnaire for measuring child oral-health-related quality of life. J Dent Res 2002;81:459-463.^,77 Martins Júnior, PA; Ramos-Jorge, J; Paiva, SM; Marque, LS; Ramos-Jorge, ML. Validations of the Brazilian version of the Early Childhood Oral Health Impact Scale (ECOHIS). Cad Saude Publica 2012;28:367-374.. Nevertheless, some instruments do not follow rigorous methodological validation or cannot be compared among themselves, because of their conceptual or methodological differences ⁸8 Gilchrist, F; Rodd, H; Deery, C; Marshman, Z. Assessment of the quality of measures of child oral health-related quality of life. BMC Oral Health 2014;14:40.. Thus, it is difficult to determine whether the questions accurately reflect what are really meaningful to individuals ³3 Locker, D; Allen, F. What do measures of ‘oral health-related quality of life’ measure? Community Dent Oral Epidemiol 2007;35:401-411.^,88 Gilchrist, F; Rodd, H; Deery, C; Marshman, Z. Assessment of the quality of measures of child oral health-related quality of life. BMC Oral Health 2014;14:40.. Besides that, instruments differ in qualitative attributes and in measurement properties. Given these concerns, clinicians and researchers are in needing of more defined criteria for choosing a questionnaire to meet specific purposes ¹1 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford, PW; Knol, DL; et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010;19:539-549.^,99 Mokkink, LB; Prinsen, CA; Bouter, LM; Vet, HC; Terwee, CB. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther 2016;20:105-113.. So, how to select an instrument with a good methodological quality?

The COSMIN Checklist

The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed in an international Delphi study with the aim to provide tools for evidence-based instrument selection ¹1 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford, PW; Knol, DL; et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010;19:539-549. (available at http://www.cosmin.nl/).

The COSMIN checklist comprises 12 boxes: Ten boxes evaluate whether a study meets the standard for good methodological quality. The first nine boxes are related to studies on measurement properties (A=Internal consistency, B=Reliability, C=Measurement error, D=Content validity, E=Structural validity, F=Hypothesis-testing, G=Cross-cultural validity, H=Criterion validity, I=Responsiveness) and the last box contains standards for studies on interpretability of Health-Related Patient-Reported Outcomes (HR-PRO) instruments (J=Interpretability). Two additional boxes were included to meet studies that use the Item Response Theory method (IRT box) and general requirements for results generalization (Generalizability box). Four steps should be taken to complete the COSMIN checklist ¹1 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford, PW; Knol, DL; et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010;19:539-549.^,99 Mokkink, LB; Prinsen, CA; Bouter, LM; Vet, HC; Terwee, CB. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther 2016;20:105-113.^,1010 Mokkink, LB; Terwee, CB; Knol, DL; Stratford, PW; Alonso, J; Patrick ,DL; et al. Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments. BMC Med Res Methodol 2006;6:2. (Fig. 1).

Figure 1
Four steps for completing the COSMIN checklist. Reproduced with permission from Dr. Caroline Terwee (http://www.cosmin.nl/cosmin_checklist.html).

In the COSMIN checklist, three measurement properties are relevant ¹¹11 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford ,PW; Knol, DL; et al. COSMIN checklist manual. 2012;1-56. Available from: Available from: http://www.cosmin.nl/images/upload/files/COSMIN%20checklist%20manual%20v9.pdf . Latest access 20 Oct 2017.
http://www.cosmin.nl/images/upload/files... :

Reliability: is the degree to which the instrument is free of measurement errors. It is divided in internal consistency, reliability and measurement error. Internal consistency is the degree of relationships between items. Reliability refers to the proportion of the total variance of the instrument, due to the true differences between the patients. And measurement errors are the random and systematic errors in patient scores that are not attributed to true changes in the construct to be measured ⁹9 Mokkink, LB; Prinsen, CA; Bouter, LM; Vet, HC; Terwee, CB. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther 2016;20:105-113.^,1111 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford ,PW; Knol, DL; et al. COSMIN checklist manual. 2012;1-56. Available from: Available from: http://www.cosmin.nl/images/upload/files/COSMIN%20checklist%20manual%20v9.pdf . Latest access 20 Oct 2017.
http://www.cosmin.nl/images/upload/files... ;

Validity: is the ability of the instrument to measure what it has proposed, that is, to measure the HR-PRO. Content validity is the degree to which the content of an HR-PRO instrument reflects the proper construction to be measured; within this construct validity, the face validity can be analyzed as the degree to which the items of the instrument are reflexes of the construct. Construct validity is divided into structural validity, hypothesis-testing, and cross-cultural validity; these items refers to how much the instrument scores are referring to the hypotheses, to the reflection of the dimensionality of the construct and to a performance similar to the original PRO instrument. Finally, criterion validity is how much the scores reflect the “gold standard” ⁹9 Mokkink, LB; Prinsen, CA; Bouter, LM; Vet, HC; Terwee, CB. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther 2016;20:105-113.^,1111 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford ,PW; Knol, DL; et al. COSMIN checklist manual. 2012;1-56. Available from: Available from: http://www.cosmin.nl/images/upload/files/COSMIN%20checklist%20manual%20v9.pdf . Latest access 20 Oct 2017.
http://www.cosmin.nl/images/upload/files... ;

Responsiveness: it is the ability of the instrument to detect changes over time in the construct to be measured ⁹9 Mokkink, LB; Prinsen, CA; Bouter, LM; Vet, HC; Terwee, CB. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther 2016;20:105-113.^,1111 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford ,PW; Knol, DL; et al. COSMIN checklist manual. 2012;1-56. Available from: Available from: http://www.cosmin.nl/images/upload/files/COSMIN%20checklist%20manual%20v9.pdf . Latest access 20 Oct 2017.
http://www.cosmin.nl/images/upload/files... .

Besides that, the COSMIN checklist presents interpretability domain as the qualitative interpretations due to the scores or score changes of an instrument. Interpretability is not considered a measurement property, but it is evaluated as an important characteristic of a measurement instrument.

COSMIN Aplications

Most of specific OHRQoL instruments were developed and cross-culturally validated all around the world prior to specific checklists publication ⁵5 Slade, GD; Spencer, AJ. Development and evaluation of the Oral Health Impact Profile. Community Dent Health 1994;11:3-11.^,66 Jokovic, A; Locker, D; Stephens, M; Kenny, D; Tompson, B; Guyatt, G. Validity and reliability of a questionnaire for measuring child oral-health-related quality of life. J Dent Res 2002;81:459-463.^,1212 Adulyanon, S; Vourapukjaru, J; Sheiham, A. Oral impacts affecting daily performance in a low dental disease Thai population. Community Dent Oral Epidemiol 1996;24:385-389.^,1313 Pahel, BT; Rozier, RG; Slade GD. Parental perceptions of children’s oral health: the Early Childhood Oral Health Impact Scale (ECOHIS). Health Qual Life Outcomes. 2007;5:6.. In this way, the COSMIN checklist was developed to try to standardize measurement studies and scientific articles with this scenario. This same purpose must be taken into account in cross-cultural adaptation and validation of instrument studies, which publications have exponentially increased in the last years ¹⁴14 Naik, A; John, MT; Kohli, N; Self, K; Flynn, P. Validation of the English-language version of 5-item Oral Health Impact Profile. J Prosthodont Res 2016;60:85-91.^,1515 Abreu-Placeres, N; Garrido, LE; Féliz-Matos, LE. Cross-cultural validation of the Scale of Oral Health-Related Outcomes for 5-year-old-children with a low-income sample from the Dominican Republic. J Int Soc Prev Community Dent 2017;7:84-89.. The advantage of cross-cultural adaptation is that it allows the collection of comparable information in different populations with different cultures and languages ¹⁶16 Junkes, MC; Fraiz, FC; Sardenberg, F; Lee, JY; Paiva, SM; Ferreira, FM. Validity and reliability of the Brazilian version of the Rapid Estimate of Adult Literacy in Dentistry: BREALD-30. PLoS One 2015;10:e0131600.. Furthermore, the use of COSMIN checklist could ensure the standardization of cross-cultural adaptations and safer comparisons between outcomes. It can be suggested that future development and validation studies should follow checklists criteria such as COSMIN to help guiding the development, validation and reporting of instrument outcomes ⁸8 Gilchrist, F; Rodd, H; Deery, C; Marshman, Z. Assessment of the quality of measures of child oral health-related quality of life. BMC Oral Health 2014;14:40.^,99 Mokkink, LB; Prinsen, CA; Bouter, LM; Vet, HC; Terwee, CB. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther 2016;20:105-113.. Also, undergraduate and post-graduate students can take advantage of COSMIN checklist when studying and learning about measurement properties ¹¹11 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford ,PW; Knol, DL; et al. COSMIN checklist manual. 2012;1-56. Available from: Available from: http://www.cosmin.nl/images/upload/files/COSMIN%20checklist%20manual%20v9.pdf . Latest access 20 Oct 2017.
http://www.cosmin.nl/images/upload/files... .

Another potential application of the COSMIN checklist is to evaluate the methodological quality of systematic reviews of measurement properties, allowing verifying the quality of the studies and the risks of biases ¹1 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford, PW; Knol, DL; et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010;19:539-549.^,1717 Terwee, CB; Mokkink, LB; Knol, DL; Ostelo RW; Bouter, LM; de Vet, HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res 2012;21:651-657.. However, the use of COSMIN in systematic reviews is still not frequent ⁸8 Gilchrist, F; Rodd, H; Deery, C; Marshman, Z. Assessment of the quality of measures of child oral health-related quality of life. BMC Oral Health 2014;14:40., which represents a rich field to be explored. Although not yet required by most journals as other checklists such as PRISMA ¹⁸18 Liberati, A; Altman, D; Tetzlaff, J; Mulrow, C; Gøtzsche, PC; Ioannidis,JP; et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. J Clin Epidemiol 2009;62:e1-e34. and CONSORT ¹⁹19 Schulz, KF; Altman, DG; Moher, D. CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ2010;340:c332., COSMIN checklist can be a new tool to help editors, publishers and reviewers for evaluating manuscripts. Thus, scientific journals would have standards for evaluating the methodological quality of studies on the measurement properties of health measurement instruments.

It is noteworthy that applying the COSMIN checklist requires at least some expertise in psychometrics or clinimetrics. This is necessary because of the complexity of the methodological principles found in the development and evaluation of these measures. The use of COSMIN can reduce the risk of biased ratings of the quality of these measures ¹¹11 Mokkink, LB; Terwee, CB; Patrick, DL; Alonso, J; Stratford ,PW; Knol, DL; et al. COSMIN checklist manual. 2012;1-56. Available from: Available from: http://www.cosmin.nl/images/upload/files/COSMIN%20checklist%20manual%20v9.pdf . Latest access 20 Oct 2017.
http://www.cosmin.nl/images/upload/files... . Finally, popularizing the application of checklists such as COSMIN is therefore necessary and should occur as quickly as possible while the theme is on the rise and cross-cultural adaptations are taking place.

Acknowledgements

The authors declares no conflict of interest. This study was supported by the Brazilian Cordination of Higher Education, Ministry of Education (CAPES), the Research Foundation of the State of Minas Gerais (FAPEMIG) and the National Council for Scientific and Technological Development (CNPq). The authors wish to express our gratitude to all COSMIN group for kindly accept our request to reproduce the figure utilized in this manuscript.

References

Publication Dates

History