User participation in requirements elicitation comparing focus group interviews and usability tests for eliciting usability requirements for medical equipment: a case study
Introduction
User interfaces of medical equipment must have high usability in order to create prerequisites for safe and effective equipment operation, installation, and maintenance (Sawyer, 1997). Poorly designed human–machine interfaces of medical equipment increase the risks for human error (Hyman, 1994; Obradovich and Woods, 1996), as well as for incidents and accidents in medical care. Although some manufacturers of medical equipment have already integrated human factors principles in their products, there is still a need for more widespread applications (Sawyer, 1997). For instance Obradovich and Woods (1996, p. 591) state that “The challenge for the human factors community is to steer the development of medical technology toward more user-centred approaches in all facets of medical care”.
This paper is based on the experiences made in a technology procurement project in which a specific aim was to achieve improved usability of medical equipment. The origin of the study was that the medical engineers in one of the larger university hospitals in Sweden had become aware that the technical performance of available medical equipment did not differ to any great extent but that the usability qualities of the user interfaces did. The engineers believed that if the usability of user interfaces was improved, incidents and accidents could be reduced as could the required time to learn how to use new equipment. In order to improve the handling qualities of the equipment, the engineers saw a need to specify usability requirements for the new equipment to be purchased. However, specifying these usability requirements required the end-users to be involved as they were considered to be the ‘experts’ on interaction and use. In earlier projects, the end-users had been involved but then mainly in the latter stages of the purchase process, i.e. when a few equipment alternatives were tested in clinical use. In the new project, methods had to be developed whereby end-user participation could be achieved already in the initial stages of the purchase process, i.e. when the specification of requirements was formulated.
The involvement of end-users in development and implementation of new technology is a key issue in user-centred approaches such as Participatory Ergonomics (e.g., Imada, 1991). The overall goal of this approach is to encourage and support work force participation in the analysis, redesign and evaluation of their own tasks, workplaces, and tools by applying different participative methods and techniques. This notion of user participation formed the overall approach of the study described in this paper. However, true participation requires a dialogue between the actors involved, in the specific case a dialogue between the medical engineers and the end-users. Earlier research has suggested that in order for the dialogue to develop, certain preconditions have to be formed including a common frame of reference in terms of knowledge and experiences as well as increased awareness and critical thinking (Garmer et al., 1995). Consequently, the methods used in the project had to increase the end-users as well as the medical engineers awareness of different usability problems and, as a consequence, their critical thinking.
Most often a participatory activity involves a group of individuals and the methods available and used are similar to those used in any group activity (cf. Kourinka, 2001). Thus, feasible methods in order to include end-users in development processes are, e.g., brainstorming, interviews, observations, and questionnaires (e.g., Haines and Wilson, 1998).
In order to elicit and specify user requirements for technical equipment, focus group interviews (Caplan, 1990) and usability tests (Nielsen, 1993) are possible options. Both methods are accepted methods for involving end-users in the identification and specification of user requirements but the respective methods have slightly different foci. Usability tests, for instance, focus on identifying (usability) problems associated with the user interface by allowing individual users to carry out specific tasks interacting with said interface. During the usability test, observations and ‘verbal protocols’ (e.g., Nielsen, 1993) are important tools for identifying the problems and their causes while complementary interviews and questionnaires are used to collect, e.g., the participating users’ ideas for improvements. Focus group interviews, on the other hand, allow groups of individuals to bring forward and discuss, e.g., problems and different solutions to those problems while being guided by a ‘moderator’ (Caplan, 1990). The discussion may be supported by different stimuli or mediating objects (Karlsson, 1996) but reflections on, e.g., usability problems are most often reflections in retrospect.
This paper describes the experiences from a study in which two hospitals in Sweden, a larger university hospital (A) and a smaller hospital (B), were to purchase a number of new ventilators. Ventilators are used in order to keep up or support artificial respiration and are mainly used at Intensive Care Units (ICU). It was therefore relevant for the two hospitals to draw up specifications of user requirements for a ventilator user interface and within this context try out new methods and approaches which could strengthen end-user participation. Two types of focus groups were tried: one based only on work experience; the other based on work experience in addition to results from and experiences of usability tests.
The paper addresses the following questions: What method elicited the most relevant response in terms of user problems and user requirements? Were there any differences between the methods in terms of the number and the character of usability aspects identified? Did the approaches contribute to increased awareness and critical thinking regarding usability issues among the end-users and medical engineer? Did the methods contribute to an improved dialogue between the end-users and the medical engineer?
Section snippets
Introduction and general description
Similar activities were carried out at the two hospitals, (A) and (B). Altogether, the study entailed 23 nurses from three different ICUs; two units from hospital A and one unit from hospital B. The nurses, who all volunteered to participate in the study, had worked with ventilators at ICUs for between 1.5 and 9 years. Nurses were chosen as participants as they were considered the most frequent users of ventilators. Different hospital units were chosen in order to obtain input to the
Evaluation
One of the aims of the case study described in this paper was to assess the respective procedures and methods in terms of their efficiency for identifying usability aspects and user requirements. In order to do so, the tape recordings from the focus group interviews were transcribed and encoded into categories. In this manner, usability aspects related to the ventilators could be identified. In order to distinguish the different aspects, all comments and statements were grouped together: first,
Evaluation
A second aim in the study was to assess the procedure as a method for increased user participation in the process of specifying requirements.
Conclusions and concluding remarks
In this case study, initial questions concerned what method elicited the most relevant response in terms of user problems and user requirements and if there were any differences between the methods in terms of the number and the character of usability aspects identified? The experiences from the case study are that no single method but rather the combination of methods worked as a fruitful way to detect the final list of usability aspects. For instance, the usability tests were important in
Acknowledgements
The authors would like to express their gratitude to the staff at the Sahlgrenska University Hospital and at the Borås Hospital, Sweden. We would also like to thank the anonymous reviewer(s) for useful comments on an earlier version of this paper.
References (25)
- et al.
Ergonomic development workco-education as a support for user participation at a car assembly plan. A case study
Journal of Applied Ergonomics
(1995) - et al.
Application of usability testing to the development of medical equipment. Usability testing of a frequently used infusion pump and a new user interface developed with a Human Factors approach
International Journal of Industrial Ergonomics
(2002) Categorical Data Analysis
(1990)Practical Statistics for Medical Research
(1991)Human activity approach to user interfaces
Journal of Human–Computer Interaction
(1989)Using focus group methodology for ergonomics design
Ergonomics
(1990)- et al.
Nonparametric Statistical Inference
(1992) - Garmer, K., 2002. Participatory Ergonomics in Practice: An Evaluation of Three Methods for Supporting End-User...
- et al.
Arguing for the need of triangulation and iteration when designing medical equipment. Nature of requirements on medical equipment derived from a Human Factors approach and from usability tests
Journal of Clinical Monitoring and Computing
(2002) - et al.
The voice of the customer
Marketing Science
(1993)
Errors in the use of medical equipment
Cited by (58)
A function combined baby stroller design method developed by fusing Kano, QFD and FAST methodologies
2020, International Journal of Industrial ErgonomicsCitation Excerpt :Researchers have made some progress in the areas of user requirements analysis and vehicle function analysis. Kano (1984) proposed the user requirements model; Garmer et al. (2004) put forward a user requirements gathering method which requires the involvement of users; Kumar (2014) proposed a method that adopts user behavior as the handle to efficiently investigate user requirements; Rashid (2010) et al. have developed a system that simulates customer answers in accordance with Kano model. Fazlollahtabar (2012) et al. proposed e-learning users' satisfaction attributes having noticeable impacts on enhancing instruction paradigm with Kano model.
The beating heart of newborn resuscitation
2019, ResuscitationErgonomic analysis of washing machines for elderly people: A focus group-based study
2018, International Journal of Industrial ErgonomicsInvolving users in a ship bridge re-design process using scenarios and mock-up models
2016, International Journal of Industrial ErgonomicsHazard Assessment and Remediation Tool for Simulation-Based Healthcare Facility Design Testing
2024, Health Environments Research and Design Journal