- Split View
-
Views
-
Cite
Cite
Mark Dooris, Healthy settings: challenges to generating evidence of effectiveness, Health Promotion International, Volume 21, Issue 1, March 2006, Pages 55–65, https://doi.org/10.1093/heapro/dai030
- Share Icon Share
Abstract
This paper starts by briefly reviewing the history, theory and practice of the settings approach to promoting public health—highlighting its ecological perspective, its understanding of settings as dynamic open systems and its primary focus on whole system organization development and change. It goes on to outline perceived benefits and consider why, almost 20 years after the Ottawa Charter advocated the approach, there remains a relatively poorly developed evidence base of effectiveness. Identifying three key challenges—relating to the construction of the evidence base for health promotion, the diversity of conceptual understandings and real-life practice and the complexity of evaluating ecological whole system approaches—it suggests that these have resulted in an ongoing tendency to evaluate only discrete projects in settings, thus failing to capture the ‘added value’ of whole system working. It concludes by exploring the potential value of theory-based evaluation and identifying key issues that will need to be addressed in moving forward—funding evaluation within and across settings; ensuring links between evidence, policy and practice; and clarifying and articulating the theories that underpin the settings approach generically and inform the approach as applied within particular settings.
INTRODUCTION
With roots in a range of disciplines (St Leger, 1997; Wenzel, 1997; Green et al., 2000), the settings approach to promoting public health has, at an international level, been consistently advocated since the mid 1980s. Building on the Ottawa Charter statement that ‘health is created and lived by people within the settings of their everyday life; where they learn, work, play and love’ [(WHO, 1986] p. 3), the Sundsvall Statement (WHO, 1991) called for the creation of supportive environments with a focus on settings for health, and the Jakarta Declaration (WHO, 1997) emphasized the value of settings for implementing comprehensive strategies and providing an infrastructure for health promotion.
The theory and practice of the settings approach have been described and debated over a number of years [e.g. (Barić, 1993; Kickbusch, 1995; Wenzel, 1997; Dooris et al., 1998; Green et al., 2000; Whitelaw et al., 2001; Dooris, 2004; Paton et al., 2005)]. Whilst there is no clear consensus amongst commentators, commonalities can be identified—suggesting that, at a conceptual level at least, the approach has a number of key characteristics.
Ecological model of health promotion
First, it reflects an ecological model of health promotion, which understands health to be determined by a complex interplay of environmental, organizational and personal factors, largely determined outside of ‘health’ services. It represents a shift of focus from illness towards salutogenesis (Antonovsky, 1987; Antonovsky, 1996), from individuals to populations, and from a mechanistic and reductionist focus on single health problems, risk factors and linear causality—towards a more holistic view, concerned to develop supportive contexts within the places that people live their lives (Kickbusch, 2003).
Systems perspective
Secondly, reflecting this ecological model, it views settings as dynamic complex systems with inputs, throughputs, outputs and impacts—characterized by integration, interconnectedness, interrelationships and interdependencies between different elements (Capra, 1983; French and Bell, 1999; Skyttner, 2001). This systems perspective—illustrated with reference to a university in Figure 1—also acknowledges that each setting is part of a greater whole, functioning as an ‘open system’ in synergistic exchange with the wider environment, and within this, other settings (Green et al., 2000; Paton et al., 2005). Action at different levels is intrinsic to this outlook (Capra, 1997). Within the context of settings, this ensures that the approach addresses rather than detracts from underlying determinants of health; as St Leger comments, there is a need to ‘stay with the big picture’ [(St Leger, 1997) p. 101].
Whole system organization development and change focus
Thirdly, the approach places its primary focus on introducing and managing change within the whole organization (Grossman and Scala, 1993)—applying ‘whole system thinking’ (Pratt et al., 1999). Dooris has proposed a model for conceptualizing the approach, highlighting the need to combine organization development with high visibility projects, to balance top-down commitment with bottom-up stakeholder engagement, and to ensure that initiatives are driven by both public health and ‘core business’ agendas (Dooris, 2004). Emphasizing the need to move beyond rhetoric, Paton et al. have presented their Healthy Living and Working Model, reiterating that the distinctiveness of the approach lies in its prioritization of organization development and systems theory to plan, stimulate and implement appropriate change (Paton et al., 2005).
THE SETTINGS APPROACH: EVALUATION AND EVIDENCE
In terms of ‘effectiveness’, the settings approach is perceived to have a number of benefits derived from the above characteristics (Dooris, 2003; Dooris, 2004). It provides a comprehensive framework within which to work and encourages multi-stakeholder ownership of health; it allows connections between people, environments and behaviours to be explored; it enables interrelationships between different groups of people within a setting to be addressed; it enables interactions between different health issues and initiatives to be recognized and taken account of; it looks outward as well as inward, encouraging corporate citizenship through developing organizational awareness of the wider impacts on health, sustainability and inequalities at local, national and global levels; and it provides an opportunity to maximize the contribution of particular settings to joined-up holistic public health.
However, despite this perceived ‘added value’ and the consistently high international profile afforded to healthy settings work over some 20 years, it would seem that the approach has a relatively poorly developed evidence base. This can be illustrated for a number of specific settings.
In relation to Healthy Cities, de Leeuw and Skovgaard conclude that although there is a fair degree of general evidence that the programme works, this does not translate to a problem-solving perspective that can usefully contribute to informed decision-making (de Leeuw and Skovgaard, 2005).
In relation to workplaces, although it has been suggested that inter-disciplinary, comprehensive approaches are essential for effective workplace health promotion (Breuker and Schröer, 2000), it is also clear that few studies have examined integrated, comprehensive strategies as a whole, focusing instead on the individual components (Dugdill and Springett, 2001).
In relation to health services, evidence of effectiveness of health promotion in general is limited and ‘there is little empirical evidence of a measurable health impact’ of policies focused on creating healthy environments as part of Health Promoting Hospitals [(McKee, 2000) p. 127]. It has, however, been argued that the Health Promoting Hospitals philosophy is ‘based on strong evidence and methods to incorporate health promotion as a core principle in the organization’ [(Groene, 2005) p. 7].
In relation to schools, there appears to be strong consensus concerning the value of a ‘whole school approach’, perhaps because this perspective is well established in educational theory and practice, there is a synergy with the prominence given to children in the ‘life stages’ approach (O'Neill et al., 2000), and the settings approach has been formally developed over more than a decade through Health Promoting Schools and related national initiatives. However, whilst it has been suggested that programmes that are comprehensive in concept and content are most likely to achieve and sustain benefits (National Health and Medical Research Council, 1996; St Leger and Nutbeam, 2000), there remains a relative paucity of studies focusing on such comprehensive programmes—and there are continuing difficulties with both evaluation and implementation (Lister-Sharp et al., 1999; Deschesnes, 2003; Mukhoma and Flisher, 2004).
Thus, although it has been suggested that settings ‘offer opportunities for comprehensive interventions which can be directed at health behaviour change and environmental change to achieve improved health outcomes’ [(Nutbeam, 2000) p. 4] and that the settings concept provides ‘an efficient and effective framework for planning and implementing health promotion initiatives and ultimately assessing their impact’ [(Goodstadt, 2001) p. 209], there remain significant challenges. As St Leger argues:
The settings approach has been legitimated more through an act of faith than through rigorous research and evaluation studies … much more attention needs to be given to building the evidence and learning from it [(St Leger, 1997) p. 100].
Whilst it is beyond the scope of this paper to reiterate debates of recent years [e.g. (Nutbeam, 1999; Eriksson, 2000; Raphael, 2000; International Union for Health Promotion and Education, 2000; Rootman et al., 2001a; McQueen, 2002; Rychetnik et al., 2002; Tang et al., 2003)], it is useful to reflect briefly on the general territory of evaluation and evidence. Evaluation has been defined as ‘the systematic examination and assessment of features of a programme or other intervention in order to produce knowledge that different stakeholders can use for a variety of purposes’ [(Rootman et al., 2001b) p. 26]. It is clearly one important contributor to evidence, which is understood to comprise facts or data that can be used in making a decision or in solving a problem (McQueen and Anderson, 2001) and to involve ‘the interpretation of empirical data derived from formal research or systematic investigations, using any type of science or social science methods’ [(Rychetnik et al., 2002) p. 119].
A number of key issues have been highlighted in the literature on evidence-based health promotion and public health. Not least of these is the tension between the traditional positivist approach to evidence—characterized by a focus on quantitative data, linear causality and ‘scientific’ reliability and validity, based on a hierarchy of evidence in which the randomized controlled trial (RCT) is the ‘gold standard’—and the complex, multidisciplinary, multi-layered nature of health promotion (and, by implication, the ‘new’ public health) (Raphael, 2000; Speller et al., 2005). As Nutbeam has commented:
It is a challenge to assemble ‘evidence’ in ways which are relevant to the complexities of contemporary health promotion, and to avoid the possibility that this may lead action down a narrow, reductionist route [(Nutbeam, 1999) p. 99].
In response to this challenge, there has been increasing advocacy for use of both quantitative and qualitative data, for a breadth of evidence that allows the effectiveness of programmes to be captured without losing their intrinsic richness and diversity, and for an ‘evidence into practice into evidence’ cycle [e.g. (Nutbeam, 1999; Raphael, 2000; McQueen and Anderson, 2001; Aro et al., 2005; Speller et al., 2005)]. There has also been growing recognition that for evidence to be useful, it should demonstrate not only what works, but how and under what conditions it works. In addressing these questions, commentators have emphasized the importance of underpinning theory [e.g. (Birckmayer and Weiss, 2000)]; of context in relation to external validity, generalizability and transferability [e.g. (Banta, 2003; Dobrow et al., 2004)]; and of utility-driven evidence—addressing how evidence will be used in decision-making [e.g. (Dobrow et al., 2004; Petticrew et al., 2004; de Leeuw and Skovgaard, 2005)].
CHALLENGES FACED IN EVALUATING THE SETTINGS APPROACH AND GENERATING EVIDENCE OF EFFECTIVENESS
As discussed above, health promotion and public health have been confronted with a range of general difficulties in responding to the demand for evidence-based policy and practice. However, it can be argued that a number of specific challenges have further mitigated against the generation of credible and convincing evidence for the settings approach, and made it problematic to undertake consistent, rigorous evaluation.
Construction of the evidence base for health promotion: focus on diseases and single risk factors
First, most systematic reviews, meta-analyses and resulting guidance available through recognized bodies are focused on specific diseases and single risk factor interventions rather than a comprehensive settings approach [e.g. (Cochrane Collaboration, 2001); (Evidence for Policy and Practice Information and Co-ordinating Centre); (Centre for Reviews and Dissemination); (Health Evidence, 2003); (National Institute for Health and Clinical Excellence, 2005]. A very limited number of reviews have focused specifically on programmes such as health promoting schools [e.g. (Lister-Sharp et al., 1999; National Health and Medical Research Council, 1996)] and drawn promising conclusions regarding the value of a whole system approach. However, of those reviews that focus wholly or in part on a particular setting (e.g. school, workplace), the vast majority are concerned with interventions designed to impact on one specific risk factor such as smoking or drug use.
Thus, despite Barić's assertion of a ‘paradigm shift’ in health promotion (Barić, 1994), it would appear that the construction of the evidence base has continued to follow a medical model. It is likely that this situation reflects two influences: the continuing prominence given to disease and behaviour based targets in health policy (Ziglio et al., 2000)—resulting in more funding being available for evaluation of issue-based than settings-based initiatives (with a consequent cyclical reinforcing effect); and the fact that much research designed to evaluate holistic, complex multi-issue programmes ‘fails’ to meet the criteria for inclusion within systematic reviews and meta-analyses—although this will hopefully change with the general broadening of approach to allow inclusion of studies beyond RCTs (Nutbeam, 1999; Jackson and Waters, 2005).
Diversity of conceptual understandings and real-life practice
Secondly, there is—alongside the degree of consensus indicated above—a diversity of both conceptual understandings and real-life practice brought together under the banner of healthy settings (Green et al., 2000; Poland et al., 2000; Whitelaw et al., 2001). This presents obvious difficulties in generating a substantive body of research that allows comparability and transferability. A number of issues can be highlighted.
Conceptual variance
Whilst much literature highlights the centrality of systems thinking and organization development, there is a continuing tendency to conflate ‘health promotion in settings’ with the settings approach—a point highlighted by Wenzel (Wenzel, 1997), who argues that the concept of settings has in reality been used to perpetuate individually-focused intervention programmes with defined target groups. Whitelaw et al. propose a representational typology of settings-based health promotion that distinguishes between various forms of practice, reflecting different analyses of the problem and solution in terms of whether the focus should be on the individual or on the setting/system (Whitelaw et al., 2001). Whilst recognizing the dangers of dictating what does and does not constitute a setting approach, it is apparent that this conceptual variance can add confusion to the evidence generation process.
Pragmatic influences
Furthermore, Whitelaw et al. highlight the impact of pragmatic considerations on real-life practice—emphasizing the different degrees of opportunity and constraint within different settings, and the difficulties of translating philosophy into tangible action (Whitelaw et al., 2001). As Dooris has commented, ‘whilst the theoretical framework guiding the work may be rooted in systems thinking and organizational development, the practice is often constrained to smaller-scale project-focused work around particular issues’ [(Dooris, 2004) p. 44]. In terms of evaluation, this is likely to result either in further confusion or in a de facto sense of failure.
Size and type of settings
The literature presents a confusing picture, listing settings as diverse in form and size as homes, schools, hospitals, islands, cities, states and regions. This highlights the need to debate definitions and parameters, and to clarify similarities and differences within and across categories (Poland et al., 2000; Dooris, 2004). In terms of building an evidence base, it is likely that the mechanisms used within ‘total institutions’ such as hospitals and prisons will differ from those used in less formal settings such as homes and communities; and it is arguably easier to demonstrate whole system change within a small clearly defined setting such as a primary school than in a large multi-layered setting such as a university, let alone a city.
Standards and accreditation
A further variation exists between programmes that have been formalized with agreed standards and accreditation criteria (e.g. schools) and those that have no formal national or international programme—and therefore no agreed criteria or benchmarks (e.g. universities). Whilst the introduction of accreditation criteria and award schemes has been subject to criticism (Jones et al., 2002) and must always take social, economic and cultural variations into account, it is evident that it makes evaluation easier.
Complexity of evaluating ecological whole system approaches
Thirdly, leaving aside ‘real-life’ conceptual and operational diversity, and focusing on the settings approach as described above (i.e. characterized by an ecological perspective that draws on systems theory and prioritizes whole system organization development), it is clear that evaluation is extremely complex. This can be elaborated in several ways.
The settings approach and integration
If the approach is understood to be about ‘integrating a commitment to health within the cultures, structures, processes and routine life of organizational and other settings’ [(Dooris, 2004) p. 40], it can be argued that the more successful an initiative is, the more challenging the task of evaluation paradoxically becomes. Integrative approaches allow the language of ‘health’ to recede—and as the work becomes mainstreamed and the effectiveness of organization development becomes more apparent, ‘health promotion’ as an entity becomes more remote. This perspective is echoed in a review of health promotion in the workplace, which comments that many organization-level interventions are ‘performed without any direct link to health and thus have an unspecified effect on ill health and well-being’ [(Breuker and Schröer, 2000) pp. 103–104].
The settings approach, ecology and systems thinking
Ecological perspectives focus on the interactions and interdependence between different elements within ecosystems (van Leeuwen et al., 1999; Grzywacz and Fuqua, 2000; McLaren and Hawe, 2005). Similarly, the application of systems thinking to health promotion demands a focus not only on the different parts of the whole—but on the ‘spaces in between’, on the ‘arrows’ rather than the ‘bubbles’ (Barić and Barić, 1995). To quote Senge:
Systems thinking is a discipline for seeing wholes. It is a framework for seeing interrelationships rather than things, for seeing patterns of change rather than static ‘snapshots’ [(Senge, 1990) p. 68].
This means that for evaluation to capture the ‘added value’ of whole system working and help generate evidence of effectiveness for healthy settings, it must do more than focus separately on each intervention or programme operating within the context of a settings initiative. Instead, it must look at the whole and attempt to map and understand the interrelationships, interactions and synergies within and between settings—with regard to different groups of the population, components of the system and ‘health’ issues (see box 1 and Figures 2–4, which illustrate this in relation to a university). Whilst there are examples of how complexity theory relates to effective community regeneration [e.g. (Stuteley and Cohen, 2004)], it would seem that most healthy settings initiatives have struggled to apply a whole system perspective to evaluation—although the complementary use of Total Quality Management and the Balanced Scorecard approach within Health Promoting Hospitals offers significant promise (Brandt et al., 2005).
A programme aimed at promoting staff well-being through changing ‘unhealthy’ organizational cultures (e.g. working through lunch breaks) is likely to have knock-on impacts among students. This will not only be through quality of teaching, but through a ‘hidden curriculum’ effect—whereby cultural norms and values are informally transmitted and reproduced within the external organizations that students subsequently work within and lead.
A programme aimed at increasing physical activity must not only ‘intervene’ within all the relevant components of the system—both within the university itself and outside (e.g. building and campus design, curriculum, timetabling, transport infrastructure), but also explore the interconnections between the different interventions and possible ‘multiplier effects’. Furthermore, it would be hoped that at least some ‘interventions’ would be designed to encourage parallel practice in other organizations and to influence ‘upwards’—impacting on policy outside as well as within the university. Tracking the impacts of these advocacy, mediation and enablement roles (see Figure 1) should be an integral part of evaluation.
The range of issue-focused programmes operating within a setting do not (and, indeed, within the context of a settings approach, should not!) function in isolation. For instance, transport policy will impact on physical activity and mental well-being; mental health promotion programmes will interact with and impact on sexual behaviour, food-related behaviour and substance use; and wider regulations and action relating to advertizing and sponsorship will influence intra-institutional programmes focused on food and alcohol.
CONCLUSION
It has been argued that those seeking to evaluate and build evidence for the effectiveness of the settings approach face a number of specific challenges, relating to the current construction of the evidence base for health promotion and public health, the diversity of conceptual understandings and real-life practice, and the complexity of evaluating ecological whole system approaches. These have resulted in and reinforced an ongoing tendency to evaluate only discrete projects in settings, and mitigated against the generation of credible and convincing evidence of effectiveness for the settings approach as a whole.
A possible way forward is to apply theory-based evaluation (TBE), which has increasingly been advocated within the fields of health promotion, public health and community change [e.g. (Chen, 1990; Weiss, 1997; Birckmayer and Weiss, 2000; Auspos and Kubisch, 2004)]. One approach is ‘theories of change’ (Connell and Kubisch, 1998), which draws on both logic models and realistic evaluation (Pawson and Tilley, 1997). Serving as both a development and evaluation framework that prioritizes stakeholder participation in theory generation, explores links between activities, outcomes and contexts, and takes account of the relationships between people and their environments, this requires the chain of assumptions and hypotheses on which an initiative is based to be made explicit. It presents a vision and strategic goals, but also sets out context in terms of needs and assets, a rationale for the chosen range of ‘interventions’, expected consequences, and performance indicators. In this way, it explores both process and outcomes, tracking the stages that make up overall programmes, mapping the links between the programmes that comprise a larger initiative, and enabling a more sophisticated and utility-focused understanding not only of whether something works, but also of why and how it works or does not work in particular situations.
‘Theories of change’ has been widely used in evaluating comprehensive community-based initiatives, which by nature function as complex, dynamic, multi-level open systems with diverse interacting factors (Judge and Bauld, 2001; Coote et al., 2004). It follows that TBE could potentially be useful in evaluating healthy settings initiatives, representing a means of understanding and capturing the ‘added value’ of whole system working as well as assessing the effectiveness of individual programmes and projects. In relation to change management in the health service, this is supported by Iles and Sutherland, who suggest that building evidence requires research methods that ‘allow for the process of change to be explored and understood, rather than methods that concentrate on measuring the outcome’ [(Iles and Sutherland, 2001) p. 75].
However, to apply TBE successfully, a number of issues need to be addressed.
Funding evaluation within and across settings
First, dedicated funding is required to enable TBE to be applied within and across a range of healthy settings initiatives in a coherent and co-ordinated way. Many initiatives are currently funded on a shoestring and do not have access to the resources or expertise to develop and implement comprehensive evaluation, yet alone partake in a co-ordinated process that can contribute to the wider knowledge base. Furthermore, the tendency for different settings initiatives to work in isolation from one another makes it challenging to apply a model of evaluation that explicitly tracks beyond and across settings. However, taking account of the fact that people's lives are not neatly bounded by settings, such an approach is crucial if the synergies, impacts and outcomes are to be understood and made explicit. As Dugdill and Springett have argued in relation to workplaces:
… evaluation should attempt to cross the interfaces between work, home life, and the community, to give coherence, continuity and sustainability [(Dugdill and Springett, 2001) p. 304].
Evidence, policy and practice
Secondly, there is a need to engage with decision-makers in planning and carrying out evaluation, so that evidence is being generated to a purpose (de Leeuw and Skovgaard, 2005) and is clearly linked to policy and practice. At an international level, the World Health Organization is an obvious partner in the process, having been the primary initiator and advocate for the settings approach. The International Union for Health Promotion and Education is also a key stakeholder, and it will be important to ensure that evidence concerning healthy settings is generated and transferred through its programmes.
Clarifying and articulating theory
Thirdly, it is necessary to clarify, develop and articulate the theories that underpin the settings approach generically (i.e. across settings) and inform the approach as applied within particular settings. This requires academics and practitioners to move beyond rhetoric and draw upon, grapple with, synthesize and demystify insights from both practice and a range of (to many, mystifying!) fields of enquiry—including ecology, systems theory, organization development, complexity science and network analysis.
This process presents various conceptual and practical challenges. How can we deal with the dilemma that stakeholders might want controversial or radical actions and goals to remain ‘hidden’—and not be exposed as an explicit part of the theory underpinning the settings initiative? How can we work within the world of systematic reviews and respond to Capra's observation (Capra, 1997) that systems thinking requires the traditional scientific focus on substance and structure to be balanced with an increasing focus on form? To do this requires an acceptance that many things cannot be ‘measured’—that the patterns of relationships that make up systems are essentially qualitative and, to be understood, must be mapped. How can we meaningfully articulate theory that embraces a belief in synergy—where the interaction of two or more influences creates an effect greater than the sum of their individual effects? As Curtice et al. have commented in relation to Healthy Cities ‘… the tools are yet to be developed actively to capture the synergistic impact and outcome of a wide range of initiatives implicit in an ecological approach to health promotion’ [(Curtice et al., 2001) p. 310). And how can we acknowledge that settings are not ‘trivial machines’ (Grossman and Scala, 1993), but complex systems that contain elements of unpredictability that must be built into our theory? Echoing Green et al.'s discussion of ‘spillover effects’ between different types of capital within neighbourhood regeneration and the need to move from ‘silo’ to ‘dynamic’ accounting (Green et al., 2001), Sanderson has argued that TBE requires the “capacity to identify and analyse complex ‘synergistic’ effects of multiple, interacting policy measures with potentially irregular non-linear forms” [(Sanderson, 2000) p. 14].
These challenges should not be underestimated. However, it is becoming increasingly clear that 21st century ‘problems’ can only be meaningfully tackled through adopting holistic and comprehensive approaches within the places that people live their lives. In relation to evaluation and evidence, there are grounds for optimism. As McQueen and Anderson contend:
… the complexity of multidiscipinary, compound interventions makes simple, universal rules of evidence untenable … thus, the emerging theoretical perspective on health promotion, which embraces participation, context and dynamism, is being brought into the thinking on evaluation design [(McQueen and Anderson, 2001) pp. 77–78].
The author would like to thank Sharon Doherty, Michelle Baybutt, Claire Drury, Marilyn Dobbs, Deby Gerrard-Brown, Dominic Harrison and Evelyne de Leeuw for their helpful comments, and all in the Healthy Settings Development Unit for their inspiration and support.
REFERENCES
Antonovsky, A. (
Aro, A., van den Broucke, S. and Räty, S. (
Auspos, P. and Kunisch, A. (
Banta, H. (
Barić, L. (
Barić, L. (
Barić, L. and Barić, L. (
Birckmayer, J. and Weiss, C. (
Brandt, E., Schmidt, W., Dziewas, R. and Groene, O. (
Breuker, G. and Schröer, A. (
Centre for Reviews and Dissemination. Http://www.york.ac.uk/inst/crd/ (date last accessed 5 December 2005)
Connell, J. and Kubisch, A. (
Coote, A., Allen, J. and Woodhead, D. (
Cochrane Collaboration. Http://www.vichealth.vic.gov.au/cochrane/welcome/index.htm (date last accessed 5 December 2005)
Curtice, L., Springett, J. and Kennedy, A. (
Deschesnes, M., Martin, C. and Jomphe Hill, A. (
Dobrow, M., Goel, V. and Upshur, R. (
Dooris, M. (
Dooris, M. (
Dooris M., Dowding G., Thompson J. and Wynne C. (
Dugdill, L. and Springett, J. (
Eriksson, C. (
Evidence for Policy and Practice Information and Co-ordinating Centre. Http://eppi.ioe.ac.uk/EPPIWeb/home.aspx?page=/hp/intro.htm (date last accessed 5 December 2005)
French, W. and Bell, C. (
Goodstadt, M. (
Green, G., Grimsley, M. and Stafford, B with Butler, D., Martin, S. and Watkins, R. (
Green, L., Poland, B. and Rootman, I. (
Groene, O. (
Grossman, R. and Scala, K. (
Grzywacz, G. and Fuqua, J. (
Health Evidence. Http://health-evidence.ca (date last accessed 5 December 2005)
Iles, V. and Sutherland, K. (
International Union for Health Promotion and Education (
Jackson, N. and Waters, E. (
Jones, L. and Douglas, J. from first draft by Adams, L. (
Judge, K. and Bauld, L. (
Kickbusch, I. (
Kickbusch, I. (
de Leeuw and Skovgaard (2005) Utility-driven evidence for healthy cities: problems with evidence generation and application.
van Leeuwen, J., Waltner-Toews, D., Abernathy, T. and Smit, B. (
Lister-Sharp, D., Chapman, S., Stewart-Brown, S. and Soden, A. (
McKee, M. (
McLaren, L. and Hawe, P. (
McQueen, D. and Anderson, L. (
Mukhoma, W. and Flisher, A. (
National Institute for Health and Clinical Excellence. Http://www.publichealth.nice.org.uk/ (date last accessed 5 December 2005)
National Health and Medical Research Council (
Nutbeam, D. (
Nutbeam, D. (
O'Neill, M., Pederson, A. and Rootman, I. (
Paton, K., Sengupta, S. and Hassan, L. (
Petticrew, M., Whitehead, M., Macintyre, S., Graham, H. and Egan, M. (
Poland, B., Green, L. and Rootman, I. (
Pratt, J., Gordon, P. and Plamping, D. (
Raphael, D. (
Rootman, I., Goodstadt, M., Hyndman, B., McQueen, D., Potvin, L., Springett, J. and Ziglio, E. (eds) (
Rootman, I., Goodstadt, M., Potvin, L. and Springett (
Rychetnik, L., Frommer, M., Hawe, P. and Shiell, A. (
Sanderson, I. (
Senge, P. (
Speller, V., Wimbush, E. and Morgan, A. (
Stuteley, H. and Cohen, C. (
St Leger, L. (
St Leger, L. and Nutbeam, D. (
Tang, K., Ehsani, J. and McQueen, D. (
Weiss, C. (
Wenzel, E. (
Whitelaw, S., Baxendale, A., Bryce, C., Machardy, L., Young, I. and Witney, E. (
WHO (