Introduction

Since the latter part of the 1990’s, the English speaking medical education research community has been engaged in a debate regarding the types of research that should take precedence (Albert, 2004). Several researchers and journal editors have expressed opinions on this subject, trying to define what form research in medical education should take and what role it should play (Bligh, 2003; Bligh and Parsell, 1999; Colliver, 2003; Dauphinée, 1998; Pirrie, 2000; Reeves et al., 2006; Whitcomb, 2002). Three questions are at the heart of this debate:

  1. 1.

    Must research in medical education maintain a heavily applied orientation (responding to the needs of faculties of medicine in the education of doctors) or should it be more open to theoretical questions?

  2. 2.

    To what audience should researchers target their work: clinician educators, university administrators, or the community of university researchers at large?

  3. 3.

    Could other, less frequently used methods such as qualitative research methods produce results that are as legitimate as quantitative methods?

This debate comes at a moment when the English-speaking world of medical education research is experiencing tremendous growth (Bordages, 2000). Evidence of this expansion includes the launch of 2 new journals in 1989 and 1996 (Teaching and Learning in Medicine and Advances in Health Sciences Education); the growth, beginning in 1999, from 6 to 12 issues published annually by the journal Medical Education; the creation in 1992 of a new section for posters at the annual meeting of Association of American Medical Colleges Research in Medical Education Section; and the creation by the journal Academic Medicine in 1990 of a supplement in which the paper presentations given at the Research in Medical Education conference are published.

With this growth, an increasing number of researchers from diverse disciplines (psychology, communication, kinesiology, education, etc) are entering the field and are bringing with them the research models and practices as well as definitions of “success” that are current in their own disciplines. These models value, among other things, the publication of articles in scientific journals, obtaining research grants, training students for higher degrees, and developing a program of research over the longer term. Thus, the expansion and resulting inclusion of these diverse individuals into the medical education community has added new voices and perspectives to the debate.

In order to shed light on this debate — and to reflect upon its implications for the practice of research, we have undertaken interviews with “influential figures” from the English-speaking community of medical education research. The interviews were designed to determine the judgments and opinions of participants on the current practices of research and to understand their conception of what form that research should take. Our intention in initiating this study was not to take part in the debate, nor to make any judgment about the merit of the different types of research, but to clarify the dimensions of the debate, and thereby contribute to the community’s reflections regarding the future development of medical education research.

Theoretical Framework

At a theoretical level, our study draws on the concept of “field” developed by sociologist Pierre Bourdieu (1991; 1993a; 2004). For Bourdieu, a field is a “space” encompassing both the production, circulation and appropriation of material goods or symbols (as for example in the fields of literature, art, science, or economy), as well as the power relations between players (individuals, institutions or social groups). Each field is characterized by its own game. For example, in the field of art, the game is the “legitimate definition” of art (what makes art “authentic” and what are the dimensions by which to assess “quality”). In the scientific field, the game is the definition of legitimate science (of “good” science). In each field the competition for predominance of one definition over competing definitions as the recognized model of excellence in the field results in a struggle between players as each tries to promote a definition that places value on their own products and their own ways of doing things. The ultimate currency in this struggle is the acquisition of prestige, the power to influence the activities within the field itself.

Various social fields are structured in a bipolar fashion. With regard to the scientific field, recent studies (Albert, 2003, 2004) have shown that the structure consists of, at one end, a “pole of production for producers” (PP pole; see Figure 1) and at the other, a “pole of production for users” (PU pole). The researchers that are concentrated around the PP pole argue that scientific production should, first and foremost, aim for the advancement of knowledge and respond to strict academic criteria of excellence. Production for peers and its evaluation by these peers therefore constitutes, in their eyes, the most important means of acquiring scientific legitimacy. This definition of legitimacy is reflected in the attribution of value to certain types of production, such as articles published in peer-reviewed journals, refereed presentations, and research projects funded by peer-reviewed grants. Consistent with this conception of scientific practice, these researchers would like to see the scientific field develop and maintain its own autonomy in the face of demands arising from external sources. More concretely, the right to participate in the field and its debates depend exclusively, for them, on the scientific competence of those wishing to participate, and excludes those who would wish to exercise influence on the basis of economic, political, administrative or other power.

Figure 1
figure 1

The two poles of research in the field of medical education research.

Full size image

At the other extreme of the continuum (the PU pole) are found researchers who, for reasons of relevance and utility, would like to see the field receptive to external influences. These researchers support the production of research that aims to address the problems and needs of the users of knowledge. For members of the field at this pole, studies targeted at issues from outside the community of scientists have a legitimacy equal to or greater than those that are immediately relevant only to the community of scientists themselves. In addition, in contrast to their colleagues at the PP pole, these researchers argue that evaluation by peers does not constitute the only means by which work can acquire scientific legitimacy. The utility of the research, its capacity to identify solutions to a real world problem, and its potential conversion into an innovation can equally constitute criteria of excellence. Further, addressing forms of production, documents such as reports destined for public organizations or administrators or documents that lead to new programs of education constitute equally legitimate forms of production.

It is important to recognize that the 2 poles do not constitute 2 distinct camps with clearly defined boundaries into which all researchers can be categorized. Rather, members of the community are distributed along a continuum between these 2 poles. Most researchers are involved in scientific activities which, in varying degrees, involve some production destined for producers and others destined for users. Further, the majority hold a definition of scientific legitimacy that leans to one or the other of the two poles but does not conform with one alone.

Methodology

Purposive sampling was utilized to select participants (Kuzel, 1999). The primary criterion for inclusion was that these participants be “influential figures” in the field of medical education research. More concretely, the participants targeted were those who have played a sufficiently prestigious role that their views and actions (within the framework of their professional roles), exert an influence on the community’s research practices. These criteria were operationalized by selecting participants who occupied diverse positions and key functions in the field of medical education research: editors and members of editorial boards for scientific journals; directors of research units; chairs and committee members of scientific meetings; and leaders of regulatory bodies in medical education. The number of respondents interviewed (N = 23) was determined using the method of saturation; a method arising from a grounded theory approach (Creswell, 1998; Strauss and Corbin, 1990). Consistent with this method, new participants were added to our sample until the variety of opinions and judgments expressed was exhausted, at which point saturation was deemed to have been achieved. The interviews, lasting approximately 1-h, were conducted in person or by telephone with respondents in Canada, the United States and Europe (Table 1). The interview guide addressed 3 principal themes:

  1. 1.

    The strengths and weaknesses of research in medical education.

  2. 2.

    The role of research in medical education.

  3. 3.

    The usefulness of theory in relationship to the development of knowledge in medical education.

Table 1 Main characteristics of sample
Full size table

The respondents were also invited to address any issue they felt was important but was not included in the interview guide.

The analysis of interviews was undertaken in 2 stages. Vertical analysis was first undertaken for each of the interviews in order to identify categories of perceptions and judgments of the influential figures. A transverse analysis followed in order to identify the recurrent views and opinions across all interviews as well as any discordant opinions (the latter were relatively few). The principal investigator and a research assistant undertook the analysis of interviews. Both of them read the individual interviews and compared their analysis and coding. When there were differences of interpretation, these were resolved through discussion until a consensus was obtained.

Results

The results are presented in 3 parts. The first presents the opinions of influential figures related to research in medical education. The second concentrates on their views of factors that influence the production of knowledge. The last addresses the means that they consider relevant for raising the level of quality of research. We will discuss the results with reference to Bourdieu’s concept of field.

Opinions on research in medical education

The analysis of interviews reveals that a large majority of influential figures feel that research in medical education remains of insufficient quality despite progress realized over the course of the last 2 decades. Four major reasons were given in interviews for this judgment:

  1. 1.

    Studies are often repetitive; and insufficiently contextualized with regard to the literature in medical education;

  2. 2.

    The knowledge of researchers in the area of theory is limited;

  3. 3.

    The works are opportunistic;

  4. 4.

    The education of researchers in science, and more specifically in social science has gaps.

As we will see from their comments, respondents tend to be more on the production for producers pole.

The repetitive character of studies

For several influential figures, the insufficient quality of work in medical education is explained in part by the fact that researchers have a limited knowledge of the literature in their own field. This limited understanding has two important consequences: first, researchers have a tendency to repeat studies that have already been undertaken on similar subjects, and second, certain researchers have difficulties in justifying the originality of their work and showing how it contributes to the advancement of knowledge. The two following quotations illustrate the opinion of the majority of influential figures about the repetitive nature of studies:

A large number of works only reproduce what has already been done. These works do not add new knowledge. In my opinion there are certain questions that researchers should ask themselves before undertaking a research project, for example: ‘What is the state of knowledge of the subject I want to study?’ ‘Why is it important to undertake this research?’ ‘What new knowledge will this provide?’ For several researchers the posing of these questions already constitutes a challenge.

The second quotation addresses the “decontextualized” and fragmented character of work:

One of the limits of many research projects undertaken is the fact that they do not locate themselves in any context: there is no analysis which allows one to situate the study and there is no review of the literature which makes it possible to relate it to existing knowledge on the subject. Several researchers give the impression of working in a black box. This makes it difficult to advance knowledge in medical education. These studies, it seems to me, have difficulty in making a contribution in this regard.

Gaps in theoretical knowledge

A second judgment emerging from the analysis of the interviews is the observation that a number of studies in medical education contain gaps in theory. Several influential figures felt that these gaps interfere with development of research for 2 main reasons: on one hand they interfere with the creation of an integrated body of knowledge related to important problems, and on the other hand they limit the understanding of the factors and causes underlying observed phenomena. According to a number of respondents, the fact that researchers are unfamiliar with existing theories restricts their analysis of a phenomenon to a descriptive level.

Related to this observation, several influential figures felt that although research in medical education should have a utilitarian dimension, the research agenda is currently too strongly subordinated to the demands of educators and administrators. This subordination acts as a brake on the development of works of a theoretical nature or that target more fundamental questions. According to these individuals, the current production seems to be going in several disparate directions without being tied to any organizing principals or identifiable themes. The following excerpt illustrates this point of view:

While research in medical education must have a practical aspect, it is absolutely essential that it rests on fundamental theories because if not it becomes superficial and uninteresting. This is particularly true for research that aims to resolve problems. In fact, if there is no theory permitting the understanding of fundamental processes, how is it possible to predict and control for the effectiveness of interventions? If research in medical education continues to be only a free service for medical schools it risks becoming a functional activity as opposed to a true intellectual activity.

Opportunistic works

A third concerning characteristic of medical education research that was expressed by participants was the observation that a certain number of works in medical education are essentially opportunistic, that they arise from random situations or circumstances. The following commentary illustrates this position:

It is not unusual for researchers to undertake a research project only because there is a group of students available — this becomes their cohort — and a new course is given to them. The research consists of evaluating that course.

With little orientation toward longer-term knowledge building efforts, this type of research tends to remain on a small scale and focus on questions that are of greatest interest to the institution where they take place. According to our participants, these studies are difficult to generalize to all faculties of medicine and therefore are of questionable value as evidence for the development of other training programs.

Research in medical education has not addressed the truly “big questions”. The sampling is limited and the projects are always short-term. For example, there are no longitudinal studies addressing multiple faculties of medicine. The projects are often developed as a result of the available resources rather in relationships to issues that are worth studying.

Gaps in research training

A final element arising from the interviews touches on the education of researchers in the field. Several influential figures believe that certain researchers have gaps in their own education regarding research generally, and more specifically regarding research in the social sciences. A recurrent theme on this subject was the observation that certain researchers tend to use methods taken from clinical research, and in particular randomized trials, in the domain of social sciences, without questioning the implications of such a transfer. This practice has detrimental effects on the quality of research, in particular on qualitative research; which requires specific methodological competencies. The participants attributed this tendency to transpose methodologies from clinical research into medical education research to a lack of education of the researchers themselves. The following quotation illustrates the point-of-view of several respondents in this regard:

Several researchers in medical education do beginner research. They have been educated in medicine and some of them do clinical research. When they begin to become interested in medical education, often they have not had the necessary preparation to lead research projects. To make up for this gap, they have a tendency to transfer their understanding from the field of clinical research into the domain of medical education research.

Factors influencing research practices

Three principal factors were identified by our influential figures as explaining why research in medical education had been weak in these ways. These factors relate to the conditions of the researchers’ working environment, budgetary constraints in medical education research and the dominant conception of research in the medical environment.

In terms of working conditions, it was suggested that a key problem involves the fact that research units give priority to providing consultation service to their faculties of medicine. According to participants to our study, the primary role of these centers currently involves responding to the needs of medical schools in areas of medical training, contributing to the improvement of teaching, and supporting clinicians who wish to undertake research activities in education. This mandate constitutes an obstacle to the development of long-term theoretical work. The research undertaken in these units aims primarily to find solutions to problems or to evaluate the effects of an intervention, leaving little place for more theoretical work. The following quotation illustrates the point-of-view of several respondents in this regard:

Research in medical education is very applied because it is first and foremost seen as a tool of service for faculties of medicine. In fact, the primary reason for which [the faculty] supports centers for research in medical education is in order to respond to their needs in terms of medical education. Given these high expectations, researchers hardly have the time to distance themselves from the mandate given to them.

Further, a certain number of our participants argued that the medical community itself seems less receptive to works of a theoretical nature. According to these individuals, clinician educators have a vision of research that is primarily utilitarian and have a limited interest in the theoretical or fundamental dimension of scientific research. This disinterest also becomes an obstacle to the diversification of research in medical education. The following excerpt illustrates this perception:

The reason that theory has little importance in medical education comes from the fact the doctors give it little credibility. Doctors are not interested in listening to your theory of power or of influence for example. Certainly there are exceptions but for most part this is the reality. What they want to know is how they can make something more effective and cost less. They are not interested in understating the complexities of research, what they want are results. This practical attitude is generalized in the medical environment and it significantly influences research.

The second factor arising from the interviews to explain the gaps in theory and the opportunistic nature of the work is the lack of financial support for medical education research. Several influential figures underscored the fact that there are almost no granting agencies or programs of grants dedicated to support research in medical education. This absence requires researchers to rely on internal resources at their institution in order to undertake their research projects. Because these sources are generally limited and irregular, it is difficult for them to elaborate research programs rather than small-scale projects.

A third identified factor arises from the dominant conception of research in the medical environment in which medical education research is published. According to certain participants, this dominant conception is strongly inspired by a biomedical model of sciences which gives priority to a type of research practice and to publication formats that seem to be incompatible with the development and discussion of theoretical questions. This conception comes to life in the rules for editing articles, which, in certain cases, are perceived to be particularly constraining. The argument of one respondent illustrates the opinion of several influential figures on this subject:

Articles are always very short, extremely structured and predictable. What counts is methodology and results. The background, the discussion and the conclusion have a secondary importance.

Given that the biomedical model is the dominant model in the field of medical education research, as underlined by certain respondents, researchers have difficulty making other choices. If they want to receive recognition from their peers, they must conform. This model is valued by the majority of journals and constitutes the legitimate way of doing research and of reporting results. The journal Advances in Health Sciences Education was seen to differentiate itself somewhat from the other journals because it imposes no length restrictions on authors.

Two means to improve research in medical education

Finally, recognizing these problems and their sources, our participants identified two principal means to improve research in medical education. The first relates to intensifying the collaboration between holders of PhD’s and clinicians, given that the competencies of each group are complementary. While MD’s, as a result of their clinical educator role, are best placed to identify key practical questions to study, the PhD’s, because of their academic training and theoretical backgrounds, have the required competencies to enrich those practical questions with theoretical perspectives and contribute effectively to the development and enactment of the associated research projects. As one participant stated:

Research in medical education necessarily has to be done in collaboration. Many PhDs doing research in this domain don’t have any medical training and don’t do any medical training. Thus, it’s crucial for them to team up with physicians, because physicians are in a way the specialists when it comes to the specification of the research problem. PhDs’ contribution lies in the fact that they can set the problem in a theoretical context and frame it in a way that it contributes to the advancement of knowledge.

The second means envisioned by the influential figures to improve medical education research involved further diversifying the disciplines engaged in research in this area, with the goal of generating new questions and enlarging the perspectives on research. Several respondents indicated that they would value an approach to medical education research that gives a larger place to the social sciences. One participant stressed that research in medical education “need sociologists, anthropologists, economics, and historians, etc. to study the problems related to the training of physicians through innovative angles.”

Discussion/Conclusion

If we interpret these results in light of the bipolar model of the scientific field as described in Figure 1, what becomes apparent from our interviews is that most of the influential figures we identified tend to think of the improvement of medical education research in terms of a movement toward the scientific practices characterized by the production for producers pole (PP Pole). That is, many of their judgments on research and their reflections on the means to increase its quality would seem to suggest that they wish to see the redefinition of what is scientifically legitimate (that is a redefinition of “good” science) toward one that gives priority to the development of work aimed at the community of researchers and at the advancement of theoretical knowledge. In this sense, this majority subset of influential figures appears to desire greater autonomy (or self determination) for the field of medical education research and, hence, a reduction in the sphere of influence exerted by the needs of faculties of medicine in orienting research questions in the field.

This apparent desire to see research evolve from the PU Pole to the PP Pole is not unique to the field of medical education. Rather, it is a phenomenon associated with the process of establishing and increasing autonomy in most scientific fields. In this sense it is a sociological regularity. Work in sociology and history that has examined the disciplines of physics (Gingras, 1991), economics (Albert, 2003) and the science of education (Fournier et al., 1988) has shown that the development of a scientific field moves in tandem with the creation of a space for the production and circulation of knowledge which is under the control of researchers and which functions according to the rules of the academic world.

Nevertheless, although the judgments of the majority of our influential figures suggest that they favor the construction of such a field of medical education research, it is clear that at the same time this support does not go to the point that they wish to break away from utilitarian research. In fact, the influential figures seem to wish that researchers could benefit from a growing freedom in terms of the choice and approach of objects of research and, at the same time, underscore the collaboration with clinicians in order to better respond to the needs of the faculties of medicine. Thus, perhaps a better interpretation of the position of these influential figures might be a desire for a better equilibrium between the PP and PU poles than to achieve a true autonomy for the field.

Why do the influential figures not wish to see a rupture with the current research practices that seem tilted toward the PU Pole? This question is clearly complicated and beyond the scope of our current data set. However, in light of the knowledge developed in sociology on social processes and culture, we might speculate two reasons. First, Bourdieu’s studies (1991, 1993a, b) suggest that it is usually the new players entering a field that act as catalysts of change. Our participants, by virtue of having been selected for their influence on the field to date, are all senior members of the community. One might imagine that these individuals like all players who have evolved and succeeded in a domain of activity for many years, have internalized the culture peculiar to the field of medical education research. It can thus be difficult for them to implement the transformations that they nevertheless perceive to be potentially advantageous. Second, it is possible that the influential players do not feel that they have the sufficient support from the clinician educators and the medical school administrators to implement research practices characterized by a greater academic profile (i.e., a production more destined to the producers). Further studies exploring the economic, political, administrative, and cultural dimensions of the medical education research milieu could, in this regard, contribute to a better understanding of the structural conditions underlying the development of research in this domain.

Do the obstacles to greater autonomy in the field that we have explored constitute inevitable barriers to the improvement of the quality of research in medical education? Very likely not. It is possible to imagine, as did our influential participants themselves, that a better balance between the practices of research associated with each of the two poles and the introduction of new disciplines could improve the quality of research. It may be that the field of medical education research, like other research domains with a strong applied dimension (criminology, social service, engineering, law, etc), could never become a completely autonomous field (functioning primarily according to the rules of the PP Pole). However, according to many influential players, this does not diminish the fact that a diversification of approaches, of methods and of objects of research, could spark a greater number of debates among scholars and, in the end, have a beneficial effect not only on the field, but on the research itself.