Skip to main content

Advertisement

SpringerLink
Log in

The Contextualized Technology Adaptation Process (CTAP): Optimizing Health Information Technology to Improve Mental Health Systems

  • Original Article
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Health information technologies have become a central fixture in the mental healthcare landscape, but few frameworks exist to guide their adaptation to novel settings. This paper introduces the contextualized technology adaptation process (CTAP) and presents data collected during Phase 1 of its application to measurement feedback system development in school mental health. The CTAP is built on models of human-centered design and implementation science and incorporates repeated mixed methods assessments to guide the design of technologies to ensure high compatibility with a destination setting. CTAP phases include: (1) Contextual evaluation, (2) Evaluation of the unadapted technology, (3) Trialing and evaluation of the adapted technology, (4) Refinement and larger-scale implementation, and (5) Sustainment through ongoing evaluation and system revision. Qualitative findings from school-based practitioner focus groups are presented, which provided information for CTAP Phase 1, contextual evaluation, surrounding education sector clinicians’ workflows, types of technologies currently available, and influences on technology use. Discussion focuses on how findings will inform subsequent CTAP phases, as well as their implications for future technology adaptation across content domains and service sectors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23.

    Article  PubMed  PubMed Central  Google Scholar 

  • Aarons, G. A., Green, A. E., Palinkas, L. A., Self-Brown, S., Whitaker, D. J., Lutzker, J. R., & Chaffin, M. J. (2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7(32), 1–9.

    Google Scholar 

  • Atkinson, N. L. (2007). Developing a questionnaire to measure perceived attributes of eHealth innovations. American Journal of Health Behavior, 31(6), 612–621.

    Article  PubMed  Google Scholar 

  • Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 24(6), 574–594. doi:10.1080/10447310802205776.

    Article  Google Scholar 

  • Becker, E. M., & Jensen-Doss, A. (2013). Computer-assisted therapies: Examination of therapist-level barriers to their use. Behavior Therapy, 44(4), 614–624.

    Article  PubMed  Google Scholar 

  • Bickman, L. (2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 47(10), 1114.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services, 62, 1423–1429.

    Article  PubMed  Google Scholar 

  • Bickman, L., Kelley, S. D., & Athay, M. (2012). The technology of measurement feedback systems. Couple and Family Psychology: Review and Practice, 1, 274–284.

    Article  Google Scholar 

  • Borntrager, C., & Lyon, A. R. (2015). Client progress monitoring and feedback in school-based mental health. Cognitive & Behavioral Practice, 22, 74–86.

    Article  Google Scholar 

  • Brooke, J. (1996). SUS-A quick and dirty usability scale. In P. W. Jordan, B. Thomas, I. L. McClelland, & B. Weerdmeester (Eds.), Usability evaluation in industry (pp. 189–194). Bristol, PA: Taylor & Francis Inc.

    Google Scholar 

  • Butler, K. A., Haselkorn, M., Bahrami, A., & Schroder, K. (2011). Introducing the MATH method and toolsuite for evidence‐based HIT. In Paper presented at the 2nd Annual AMA/IEEE EMBS Medical Technology Conference, Boston, MA. Published online at https://www.uthouston.edu/dotAsset/fcc91d1b-3a16-495b-9809-37e2108ed5e2.pdf.

  • Chambers, D., Glasgow, R., & Stange, K. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implement Science, 8(1), 117.

    Article  Google Scholar 

  • Connors, E. H., Arora, P., Curtis, L., & Stephan, S. H. (2015). Evidence-based assessment in school mental health. Cognitive and Behavioral Practice, 22, 60–73.

    Article  Google Scholar 

  • Cook, J. M., Biyanova, T., & Coyne, J. C. (2009). Barriers to adoption of new treatments: An internet study of practicing community psychotherapists. Administration and Policy in Mental Health and Mental Health Services Research, 36(2), 83–90.

    Article  PubMed  PubMed Central  Google Scholar 

  • Courage, C., & Baxter, K. (2005). Understanding your users: A practical guide to user requirements methods, tools, and techniques. San Francisco, CA: Morgan Kaufmann.

    Google Scholar 

  • Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Science, 4(1), 50.

    Article  Google Scholar 

  • Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.

  • Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982–1003.

    Article  Google Scholar 

  • De Jong, M., & Van Der Geest, T. (2000). Characterizing web heuristics. Technical Communication, 47(3), 311–326.

    Google Scholar 

  • DeSantis, L., & Ugarriza, D. N. (2000). The concept of theme as used in qualitative nursing research. Western Journal of Nursing Research, 22(3), 351–372.

    Article  CAS  PubMed  Google Scholar 

  • Diaper, D., & Stanton, N. (Eds.). (2003). The handbook of task analysis for human-computer interaction. Boca Raton: CRC Press.

    Google Scholar 

  • Farmer, E. M., Burns, B. J., Phillips, S. D., Angold, A., & Costello, E. J. (2003). Pathways into and through mental health services for children and adolescents. Psychiatric Services, 54(1), 60–66.

    Article  PubMed  Google Scholar 

  • Few, S. (2006). Information dashboard design: The effective visual communication of data. O’Reilly.

  • Flanagan, M. E., Saleem, J. J., Millitello, L. G., Russ, A. L., & Doebbeling, B. N. (2013). Paper-and computer-based workarounds to electronic health record use at three benchmark institutions. Journal of the American Medical Informatics Association, 20(e1), e59–e66.

    Article  PubMed  PubMed Central  Google Scholar 

  • Foster, S., Rollefson, M., Doksum, T., Noonan, D., Robinson, G., & Teich, J. (2005). School Mental Health Services in the United States, 2002–2003 (DHHS Pub. No. (SMA) 05–4068). Rockville, MD: Center for Mental Health Services, Substance Abuse and Mental Health Services Administration.

  • Friese, S. (2012). ATLAS.ti 7 user manual. Berlin: ATLAS. ti Scientific Software Development GmbH.

    Google Scholar 

  • Furukawa, M. F., King, J., Patel, V., Hsiao, C.-J., Adler-Milstein, J., & Jha, A. K. (2014). Despite substantial progress in EHR adoption, health information exchange and patient engagement remain low in office settings. Health Affairs, 33(9), 1672–1679.

    Article  PubMed  Google Scholar 

  • Gance-Cleveland, B., & Yousey, Y. (2005). Benefits of a school-based health center in a preschool. Clinical Nursing Research, 14(4), 327–342.

    Article  PubMed  Google Scholar 

  • Glasgow, R. E., Phillips, S. M., & Sanchez, M. A. (2013). Implementation science approaches for integrating eHealth research into practice and policy. International Journal of Medical Informatics, 83, e1–e11.

    Article  PubMed  Google Scholar 

  • Glasgow, R. E., Kessler, R. S., Ory, M. G., Roby, D., Gorin, S. S., & Krist, A. (2014). Conducting rapid, relevant research: Lessons learned from the my own health report project. American Journal of Preventive Medicine, 47(2), 212–219.

    Article  PubMed  PubMed Central  Google Scholar 

  • González, M. P., Lorés, J., & Granollers, A. (2008). Enhancing usability testing through datamining techniques: A novel approach to detecting usability problem patterns for a context of use. Information and Software Technology, 50(6), 547–568.

    Article  Google Scholar 

  • Grossman, T., Fitzmaurice, G., & Attar, R. (2009). A survey of software learnability: Metrics, methodologies and guidelines. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 649–658.

  • Hackos, J. T., & Redish, J. (1998). User and task analysis for interface design. New York, NY: Wiley.

    Google Scholar 

  • Health Information Technology for Economic and Clinical Health Act of 2009, Title XIII of Division A and Title IV of Division B of the American Recovery and Reinvestment Act of 2009 (ARRA), Pub. L. No. 111-5, 123 Stat. 226 (Feb 17, 2009), codified at 42 U.S.C. §§300jj et seq.; §§17901 et seq.

  • Heeks, R. (2006). Health information systems: Failure, success and improvisation. International Journal of Medical Informatics, 75(2), 125–137.

    Article  PubMed  Google Scholar 

  • Hill, C. E., Thompson, B. J., & Williams, E. N. (1997). A guide to conducting consensual qualitative research. The counseling Psychologist, 25(4), 517–572.

    Article  Google Scholar 

  • Hill, C. E., Knox, S., Thompson, B. J., Nutt Williams, E., & Hess, S. A. (2005). Consensual qualitative research: An update. Journal of Counseling Psychology, 52, 196–205.

    Article  Google Scholar 

  • Holden, R. J., & Karsh, B. T. (2010). The technology acceptance model: its past and its future in health care. Journal of Biomedical Informatics, 43(1), 159–172.

    Article  PubMed  PubMed Central  Google Scholar 

  • Holtzblatt, K., Wendell, J. B., & Wood, S. (2004). Rapid contextual design: A how-to guide to key techniques for user-centered design. San Francisco: Elsevier.

    Google Scholar 

  • Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies, 64(2), 79–102. doi:10.1016/j.ijhcs.2005.06.002.

    Article  Google Scholar 

  • Hornbæk, K., & Law, E. L. C. (2007). Meta-analysis of correlations among usability measures. In Proceedings of the SIGCHI conference on Human factors in computing systems.

  • Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288.

    Article  PubMed  Google Scholar 

  • International Standards Organization. (1998). Ergonomic requirements for office work with visual display terminals (VDTs)—Part 11: Guidance on usability. International Organization for Standardization, 9241–11.

  • International Standards Organization. (2010). Ergonomics of human-system interaction—Part 2010: Human centered design for interactive systems. International Organization for Standardization.

  • Kokkonen, E. W., Davis, S. A., Lin, H. C., Dabade, T. S., Feldman, S. R., & Fleischer, A. B., Jr. (2013). Use of electronic medical records differs by specialty and office settings. Journal of the American Medical Informatics Association, 20(1), 33–38.

    Article  Google Scholar 

  • Krug, S. (2014). Usability testing on 10 cents a day (pp. 110–141). Don’t make me think: A common sense approach to web usability, revisited.

  • Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice, 10, 288–301.

    Google Scholar 

  • Lewis, J. R. (1994). Sample size for usability studies: Additional considerations. Human Factors, 36, 368–378.

    CAS  PubMed  Google Scholar 

  • Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57–78.

    Article  Google Scholar 

  • Lewis, J. R. (2002). Psychometric evaluation of the PSSUQ using data from five years of usability studies. International Journal of Human-Computer Interaction, 14(3–4), 463–488.

    Article  Google Scholar 

  • Lyon, A. R., & Lewis, C. C. Designing health information technologies for uptake: Development and implementation of measurement feedback systems in mental health service delivery. Introduction to the special section. Administration and Policy in Mental Health and Mental Health Services Research (this issue).

  • Lyon, A. R., Borntrager, C., Nakamura, B., & Higa-McMillan, C. (2013). From distal to proximal: Routine educational data monitoring in school-based mental health. Advances in School Mental Health Promotion, 6(4), 263–279.

    Article  Google Scholar 

  • Lyon, A. R., Dorsey, S., Pullmann, M., Silbaugh-Cowdin, J., & Berliner, L. (2015). Clinician use of standardized assessments following a common elements psychotherapy training and consultation program. Administration and Policy in Mental Health and Mental Health Services Research, 42, 47–60.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lyon, A. R., Ludwig, K., Knaster Wasse, J., Bergstrom, A., Hendrix, E., & McCauley, E. Determinants and functions of standardized assessment use among school mental health clinicians: A mixed methods evaluation. Administration and Policy in Mental Health and Mental Health Services Research (in press).

  • McLellan, S., Muddimer, A., & Peres, S. C. (2012). The effect of experience on system usability scale ratings. Journal of Usability Studies, 7(2), 56–67.

    Google Scholar 

  • Michel-Verkerke, M. B., & Spil, T. A. M. (2008). The USE IT-adoption-model to predict and evaluate adoption of information and communication technology in healthcare. Methods of Information in Medicine, 47(3), 260–269.

    Google Scholar 

  • Mohr, D. C., Burns, M. N., Schueller, S. M., Clarke, G., & Klinkman, M. (2013). Behavioral intervention technologies: Evidence review and recommendations for future research in mental health. General Hospital Psychiatry, 35(4), 332–338.

    Article  PubMed  PubMed Central  Google Scholar 

  • Norman, D. A., & Draper, S. W. (Eds.). (1986). User centered system design: New perspectives on human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Owens, J., Lyon, A. R., Brandt, N. E., Maisa Warner, M., Nadeem, E., Spiel, C., & Wagner, M. (2014). Implementation science in school mental health: Key constructs and a proposed research agenda. School Mental Health, 6, 99–111.

    Article  PubMed  PubMed Central  Google Scholar 

  • Palinkas, L. A., Aarons, G. A., Horwitz, S., Chamberlain, P., Hurlburt, M., & Landsverk, J. (2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 44–53.

    Article  PubMed  PubMed Central  Google Scholar 

  • Patient Protection and Affordable Care Act of 2010, Pub. L. No. 111-148, § 6301, 124 Stat. 727 (2010).

  • Pringle, B., Chambers, D., & Wang, P. S. (2010). Toward enough of the best for all: Research to transform the efficacy, quality, and reach of mental health care for youth. Administration and Policy in Mental Health and Mental Health Services Research, 37(1), 191–196.

    Article  PubMed  Google Scholar 

  • Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38, 65–76.

    Article  PubMed  PubMed Central  Google Scholar 

  • Rausch, T., & Leigh Jackson, J. (2007). Using clinical workflows to improve medical device/system development. In High Confidence Medical Devices, Software, and Systems and Medical Device Plug-and-Play Interoperability, 2007. HCMDSS-MDPnP. Joint Workshop on IEEE (pp. 133–134).

  • Rogers, E. M. (2003). Diffusions of innovations (5th ed.). New York, NY: Free Press.

    Google Scholar 

  • Rosenbaum, S. (1989). Usability evaluations versus usability testing: When and why? IEEE Transactions on Professional Communication, 32(4), 210–216.

    Article  Google Scholar 

  • Rubin, J., & Chisnell, D. (2008). Handbook of usability testing: how to plan, design, and conduct effective tests. Indianapolis, IN: Wiley.

    Google Scholar 

  • Sauro, J. (2011). A practical guide to the sytem usability scalel background, benchmarks & best practices. New York: CreateSpace Independent Publishing Platform.

    Google Scholar 

  • Shehabuddeen, N. T. M. H., & Probert, D. R. (2004). Excavating the technology landscape: Deploying technology intelligence to detect early warning signals. Proceedings of the IEEE Engineering Management Society, 1, 332–336.

    Google Scholar 

  • Shekelle, P., Morton, S. C., & Keeler, E. B. (2006). Costs and benefits of health information technology. Rockville, MD: Agency for Healthcare Research and Quality.

    Google Scholar 

  • Tabak, R. G., Khoong, E. C., Chambers, D., & Brownson, R. C. (2013). Models in dissemination and implementation research: Useful tools in public health services and systems research. Frontiers in Public Health Services and Systems Research, 2(1), 8.

    Google Scholar 

  • Tullis, T., & Albert, B. (2013). Measuring the user experience: Collecting, analyzing, and presenting usability metrics (2nd ed.). Burlington, MA: Morgan Kaufmann.

    Google Scholar 

  • Tullis, T. S., & Stetson, J. N. (2004). A comparison of questionnaires for assessing website usability. In Usability Professional Association Conference, Minneapolis, MN.

  • Turner, C. W., Lewis, J. R., & Nielsen, J. (2006). Determining usability test sample size. International Encyclopedia of Ergonomics and Human Factors, 3, 3084–3088.

    Google Scholar 

  • Unützer, J., Katon, W., Williams, J. W., Jr, Callahan, C. M., Harpole, L., Hunkeler, E. M., & Langston, C. A. (2001). Improving primary care for depression in late life: the design of a multicenter randomized trial. Medical Care, 39(8), 785–799.

    Article  PubMed  Google Scholar 

  • Unützer, J., Katon, W., Callahan, C. M., Williams, J. W., Jr, Hunkeler, E., Harpole, L., & Impact Investigators. (2002). Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA, 288(22), 2836–2845.

    Article  PubMed  Google Scholar 

  • Unützer, J., Chan, Y. F., Hafer, E., Knaster, J., Shields, A., Powers, D., & Veith, R. C. (2012). Quality improvement with pay-for-performance incentives in integrated behavioral health care. American Journal of Public Health, 102(6), e41–e45.

    Article  PubMed  PubMed Central  Google Scholar 

  • U.S. Department of Health and Human Services, Office of the National Coordinator. (2014a). Health IT Glossary. Retrieved Oct 5, 2014, from http://www.healthit.gov/unintendedconsequences/content/glossary.html.

  • U.S. Department of Health and Human Services, Office of the National Coordinator. (2014b). About the Blue Button Initiative. Retrieved Oct 5, 2014, from http://www.healthit.gov/patients-families/blue-button/about-blue-button.

  • Vredenburg, K., Isensee, S., Righi, C., & Design, U. C. (2001). User centered design: An integrated approach. Englewood Cliffs: Prentice Hall.

    Google Scholar 

  • Walker, J., Pan, E., Johnston, D., Alder-Milstein, J., Bates, D. W., & Middleton, B. (2005). The value of health care information exchange and interoperability. Health Affairs, W5, 10–18.

    Google Scholar 

  • Walker, S. C., Kerns, S. E., Lyon, A. R., Bruns, E. J., & Cosgrove, T. J. (2010). Impact of school-based health center use on academic outcomes. Journal of Adolescent Health, 46(3), 251–257.

    Article  PubMed  Google Scholar 

  • Williams, J. W., Katon, W., Lin, E. H., Nöel, P. H., Worchel, J., Cornell, J., & Unützer, J. (2004). The effectiveness of depression care management on diabetes-related outcomes in older patients. Annals of Internal Medicine, 140(12), 1015–1024.

    Article  PubMed  Google Scholar 

  • Wolpert, M., Curtis-Tyler, K., & Edbrooke-Childs, J. A qualitative exploration of patient and clinician views on patient reported outcome measures in child mental health and diabetes services. Administration and Policy in Mental Health and Mental Health Services Research (in press).

  • Zhou, R. (2007). How to quantify user experience: fuzzy comprehensive evaluation model based on summative usability testing. In N. Aykin (Ed.), Usability and Internationalization. Global and local user interfaces (pp. 564–573). Heidelberg: Springer.

    Chapter  Google Scholar 

Download references

Acknowledgments

This publication was made possible in part by funding from Grant Number K08 MH095939, awarded to the first author from the National Institute of Mental Health (NIMH). The authors would also like to thank the school-based mental health provider participants, Seattle Children’s Hospital, and Public Health – Seattle and King County for their support of this project. Dr. Lyon is an investigator with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis; through an award from the National Institute of Mental Health (R25 MH080916) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (QUERI).

Author information

Authors and Affiliations

  1. Department of Psychiatry and Behavioral Sciences, University of Washington, 6200 NE 74th St., Suite 100, Seattle, WA, 98115, USA

    Aaron R. Lyon, Kristy Ludwig, Eric J. Bruns, Jürgen Unützer & Elizabeth McCauley

  2. Public Health – Seattle and King County, Seattle, USA

    Jessica Knaster Wasse

  3. Department of Human Centered Design and Engineering, University of Washington, Seattle, USA

    Mark Zachry

Authors
  1. Aaron R. Lyon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jessica Knaster Wasse
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kristy Ludwig
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mark Zachry
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eric J. Bruns
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jürgen Unützer
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Elizabeth McCauley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aaron R. Lyon.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lyon, A.R., Wasse, J.K., Ludwig, K. et al. The Contextualized Technology Adaptation Process (CTAP): Optimizing Health Information Technology to Improve Mental Health Systems. Adm Policy Ment Health 43, 394–409 (2016). https://doi.org/10.1007/s10488-015-0637-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-015-0637-x

Keywords

Search

Navigation