Original ArticleAHRQ Series Paper 3: Identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the Effective Health-Care program
Introduction
Globally, people are struggling with the reality of limited resources to address the breadth of health and health care needs. Evidence has been recognized as the “new anchor for medical decisions” [1], and many consider systematic reviews to be the best source of information for making clinical and health policy decisions [2]. These translational research products rigorously summarize existing research studies so that health and health care decisions by practitioners, policy makers, and patients are more evidence based. Yet, dollars for research—whether for systematic reviews, trials, or observational studies—are constrained, and are likely to be into the future. Effective prioritization is clearly necessary to identify the most important topics for synthesized research investment that may help the US health care system realize powerful and meaningful improvements in health status.
This article discusses the identification, selection, and refinement of topics for comparative effectiveness systematic reviews within the Agency for Healthcare Research and Quality's (AHRQ) Effective Health Care (EHC) program, which has been described in more detail elsewhere [3]. Briefly, AHRQ's EHC program was authorized in 2003 by the US Congress to conduct and support research on outcomes, comparative clinical effectiveness, and appropriateness of pharmaceuticals, devices, and health care services. This program uses the AHRQ Evidence-based Practice Center (EPC) program, with 14 designated centers throughout North America that conduct comparative effectiveness systematic reviews, among other research products of the program. AHRQ has designated a Scientific Resource Center (SRC) currently housed at the Oregon EPC to support the EHC program as a whole, with specific responsibilities including assisting AHRQ with all aspects of research topic development (see Fig. 1), providing scientific and technical support for systematic reviews and outcomes research, and collaborating with EHC stakeholder and program partners.
Selecting and developing good topics for research is not a simple process. Researchers' success depends, in large part, on their ability to identify meaningful questions, whereas funding agencies continually seek to maximize the return on their investment by funding research on important, answerable questions relevant to significant portions of priority populations. Although some have criticized how well funders have actually achieved these results [4], there is little guidance for successfully developing a research program that generates the type of evidence necessary to improve the public's health.
Section snippets
Guiding principles for identifying and selecting comparative effectiveness research topics in AHRQ's EHC program
To derive guiding principles for selecting important comparative effectiveness systematic review topics, we considered what others have done when trying to select priority topics for any health care–related activity. Over the last 18 years, the Institute of Medicine (IOM) and selected others have explored priority setting models and approaches [5], [6], [7], [8], [9], [10]. Across a diverse set of international health- and health care–related activities, including the development of guidelines
Processes for identifying and selecting comparative effectiveness systematic reviews in AHRQ's EHC program
As illustrated in Fig. 2, the current EHC program processes aim to allow the consistent, broadly focused development of a portfolio of relevant comparative effectiveness systematic reviews. These processes are more heavily focused now on engaging stakeholders than in the initial years of the EHC program, particularly during topic identification, but throughout the processes of research development and dissemination within the EHC program. We are using new and existing publicity avenues to
Principles and processes for refining selected topics for comparative effectiveness systematic review in AHRQ's EHC program
Once topics are selected for comparative effectiveness systematic review, they are further focused into research questions for systematic review. This process is designed to ensure that the research review results in a product that meets the needs of stakeholders. Key questions should reflect the uncertainty that decision makers, patients, clinicians, and others may have about the topic. Key questions guide the entire systematic review process, from the formulation of comprehensive search
Challenges in identifying, selecting, and refining topics for comparative effectiveness systematic reviews in AHRQ's EHC program
One of the main challenges we face as we move forward is to ensure that we have engaged the most important perspectives since, due to issues of timeliness and cost, we cannot engage all types of stakeholders at each step for every topic. Our goal is to continue to develop a system that fairly represents the range of interests of all stakeholders across all aspects of the program (Fig. 2), yet that results in timely and clear reports that are useful to decision makers and other audiences. For
References (24)
- et al.
AHRQ Series Paper 1: Comparing medical interventions: AHRQ and the Effective Health-Care Program
J Clin Epidemiol
(2010) Evidence-based medicine: a unified approach
Health Aff (Millwood)
(2005)- et al.
Systematic reviews: time to address clinical and policy relevance as well as methodological rigor
Ann Intern Med
(2007) - et al.
The relation between funding by the National Institutes of Health and the burden of disease
N Engl J Med
(1999) - et al.
Setting priorities and selecting topics for clinical practice guidelines
CMAJ
(1995) National priorities for the assessment of clinical conditions and medical technologies: report of a pilot study
(1990)Setting priorities for health technology assessment: a model process
(1992)Setting priorities for clinical practice guidelines
(1995)Priority areas for national action: transforming health care quality
(2003)Knowing what works in health care: a roadmap for the nation
(2008)
Setting priorities in health care organizations: criteria, processes, and parameters of success
BMC Health Serv Res
Improving the use of research evidence in guideline development: 2. Priority setting
Health Res Policy Syst
Cited by (74)
An in-person and technology-implemented holistic health promotion program for older Black adults in low-income communities
2024, Journal of the National Medical AssociationRevising the Academy's Research Priorities: Methods of the Research Priorities and Strategies Development Task Force, 2017-2019
2021, Journal of the Academy of Nutrition and DieteticsFacilitation of risk assessment with evidence-based methods – A framework for use of systematic mapping and systematic reviews in determining hazard, developing toxicity values, and characterizing uncertainty
2020, Regulatory Toxicology and PharmacologyCitation Excerpt :In practice, this combination of formats is likely to be applied to different outcomes and endpoints (e.g., cancer and specific cancer types) and potentially to different populations or exposure scenarios. *Develop and document primary research question(s) in PECO (Population, Exposure, Comparator/Control, Outcome) format (Morgan et al., 2018a; Woodruff and Sutton, 2014) as well as supporting or other key questions related to the PECO (OHAT, 2015b; Whitlock et al., 2010). PECOTS (Population, Exposure, Comparator, Outcome, Timing, Setting) may also be applicable in some scenarios.
A common framework of steps and criteria for prioritizing topics for evidence syntheses: a systematic review
2020, Journal of Clinical EpidemiologyCitation Excerpt :Of these, four described how the criteria were developed and agreed on [14,29,32,36]. Two provided further guidance on the use of the criteria: Whitlock et al., prepared issue briefs addressing program prioritization criteria, whereas Akl et al. developed a user manual with signaling questions for each criterion [14,32]. The studies included a mean of eight prioritization criteria (range: 2–22), with a total of 106 mentions of criteria.
Systematic reviews and metaanalyses of air pollution epidemiological studies
2020, Traffic-Related Air PollutionAHRQ EPC Series on Improving Translation of Evidence into Practice for the Learning Health System: Introduction
2019, Joint Commission Journal on Quality and Patient Safety