research capacity building framework

• Open access
• Published: 27 October 2005

A framework to evaluate research capacity building in health care

• Jo Cooke 1  

BMC Family Practice volume  6 , Article number:  44 ( 2005 ) Cite this article

38k Accesses

201 Citations

12 Altmetric

Metrics details

Building research capacity in health services has been recognised internationally as important in order to produce a sound evidence base for decision-making in policy and practice. Activities to increase research capacity for, within, and by practice include initiatives to support individuals and teams, organisations and networks. Little has been discussed or concluded about how to measure the effectiveness of research capacity building (RCB)

This article attempts to develop the debate on measuring RCB. It highlights that traditional outcomes of publications in peer reviewed journals and successful grant applications may be important outcomes to measure, but they may not address all the relevant issues to highlight progress, especially amongst novice researchers. They do not capture factors that contribute to developing an environment to support capacity development, or on measuring the usefulness or the 'social impact' of research, or on professional outcomes.

The paper suggests a framework for planning change and measuring progress, based on six principles of RCB, which have been generated through the analysis of the literature, policy documents, empirical studies, and the experience of one Research and Development Support Unit in the UK. These principles are that RCB should: develop skills and confidence, support linkages and partnerships, ensure the research is 'close to practice', develop appropriate dissemination, invest in infrastructure, and build elements of sustainability and continuity. It is suggested that each principle operates at individual, team, organisation and supra-organisational levels. Some criteria for measuring progress are also given.

This paper highlights the need to identify ways of measuring RCB. It points out the limitations of current measurements that exist in the literature, and proposes a framework for measuring progress, which may form the basis of comparison of RCB activities. In this way it could contribute to establishing the effectiveness of these interventions, and establishing a knowledge base to inform the science of RCB.

Peer Review reports

The need to develop a sound scientific research base to inform service planning and decision-making in health services is strongly supported in the literature [ 1 ], and policy [ 2 ]. However, the level of research activity and the ability to carry out research is limited in some areas of practice, resulting in a low evidence base in these areas. Primary Care, for example, has been identified as having a poor capacity for undertaking research [ 3 – 5 ], and certain professional groups, for example nursing and allied health professionals, lack research experience and skills [ 5 – 7 ]. Much of the literature and the limited research on research capacity building (RCB) has therefore focused on this area of practice, and these professional groups. Policy initiatives to build research capacity include support in developing research for practice, where research is conducted by academics to inform practice decision making, research within or through practice, which encompasses research being conducted in collaboration with academics and practice, and research by practice, where ideas are initiated and research is conducted by practitioners [ 3 , 8 ].

The interventions to increase research capacity for, within, and by practice incorporates initiatives to support individuals and teams, organisations and networks. Examples include fellowships, training schemes and bursaries, and the development of support infrastructures, for example, research practice networks [ 9 – 13 ]. In the UK, the National Coordinating Centre for Research Capacity Development has supported links with universities and practice through funding a number of Research and Development Support Units (RDSU) [ 14 ]which are based within universities, but whose purpose is to support new and established researchers who are based in the National Health Service (NHS). However, both policy advisers and researchers have highlighted a lack of evaluative frameworks to measure progress and build an understanding of what works[ 15 , 16 ].

This paper argues for a need to establish a framework for planning and measuring progress, and to initiate a debate about identifying what are appropriate outcomes for RCB, not simply to rely on things that are easy to measure. The suggested framework has been generated through analysis of the literature, using policy documents, position statements, a limited amount of empirical studies on evaluating research RCB, and the experience of one large RSDU based in the UK.

The Department of Health within the UK has adopted the definition of RCB as 'a process of individual and institutional development which leads to higher levels of skills and greater ability to perform useful research". (pp1321) [ 17 ]

Albert & Mickan cited the National Information Services in Australia [ 18 ] who define it as

" an approach to the development of sustainable skills, organizational structures, resources and commitment to health improvement in health and other sectors to multiply health gains many times over.'

RCB can therefore be seen as a means to an end, the end being 'useful' research that informs practice and leads to health gain, or an end in itself, emphasising developments in skills and structures enabling research to take place.

A framework for measuring capacity building should therefore be inclusive of both process and outcome measures [ 19 ], to capture changes in both the 'ends' and 'means'; it should measure the ultimate goals, but also measure the steps and mechanisms to achieve them. The notion of measuring RCB by both process and outcome measures is supported within the research networks literature [ 12 , 20 ], and capacity building in health more generally [ 19 , 21 ]. Some argue we should acknowledge 'process as outcome', particularly if capacity building is seen as an end in itself [ 21 ]. In this context process measures are 'surrogate' [ 12 ], or 'proxy' outcome measures[ 16 ]. Carter et al [ 16 ]stress caution in terms of using 'proxy' measures in the context of RCB, as there is currently little evidence to link process with outcome. They do not argue against the notion of collecting process data, but stress that evaluation work should examine the relationship of process to outcome. The proposed framework discussed in this paper suggests areas to consider for both process and outcome measurement.

The most commonly accepted outcomes for RCB cited in the literature includes traditional measures of high quality research including publications, conference presentations, successful grant applications, and qualifications obtained. Many evaluations of RCB have used these as outcomes [ 9 , 10 , 22 , 23 ]. Some argue that publications in peer reviewed journals are a tall order for the low research skills base in some areas of health care practice [ 5 ], and argue for an appropriate time frame to evaluate progress. Process measures in this context could measure progress more sensitively and quickly.

However, using traditional outcomes may not be the whole story in terms of measuring impact. Position statements suggest that the ultimate goal of research capacity building is one of health improvement [ 17 , 18 , 24 ]. In order for capacity building initiatives to address these issues, outcomes should also explore the direct impact on services and clients: what Smith [ 25 ]defines as the social impact of research.

There is a strong emphasis within the primary care literature that capacity building should enhance the ability of practitioners to build their research skills: to support the development of research 'by' and 'with' practice [ 3 , 26 ], and suggests 'added value' to develop such close links to practice. A framework to measure RCB should explore and try to unpack this 'added value', both in terms of professional outcomes,[ 10 ] which include increasing professional enthusiasm, and supporting the application of critical thinking, and the use of evidence in practice. Whilst doing research alongside practice is not the only way these skills and attitudes can be developed, it does seem to be an important impact of RCB that should be examined.

The notion of developing RCB close to practice does not necessarily mean that it is small scale just because it is close to the coal face. Obviously, in order for individuals and teams to build up a track record of experience their initial projects may justifiably be small scale, but as individual's progress, they may gain experience to be able to conduct large scale studies, still based on practice problems, working in partnership with others. Similarly networks can support large scale studies as their capacity and infrastructure is developed to accommodate them.

The framework

The framework is represented by Figure 1 . It has two dimensions

Research Capacity Building: A Framework for Evaluation.

• Four structural levels of development activity . These include individual, team, organisational, and the network or supra- organisational support level (networks and support units). These are represented by the concentric circles within the diagram.

• Six principles of capacity building . This are discussed in more detail below but include: building skills and confidence, developing linkages and partnerships, ensuring the research is 'close to practice', developing appropriate dissemination, investments in infrastructure, and building elements of sustainability and continuity. Each principle is represented by an arrow within the diagram, which indicates activities and processes that contribute towards capacity building. The arrows cut across the structural levels suggesting that activities and interventions may occur within, and across, structural levels. The arrow heads point in both directions suggesting that principles applied to each structural level could have an impact on other levels.

The framework acknowledges that capacity building is conducted within a policy context. Whilst this paper focuses on measurement at different structural levels, it should be acknowledged that progress and impact on RCB can be greatly nurtured or restricted by the prevailing policy. Policy decisions will influence opportunities for developing researchers, can facilitate collaborations in research, support research careers, fund research directed by practice priorities, and can influence the sustainability and the very existence of supportive infrastructures such as research networks.

The paper will explain the rationale for the dimensions of the framework, and then will suggest some examples of measurement criteria for each principle at different structural levels to evaluate RCB. It is hope that as the framework is applied, further criteria will be developed, and then used taking into account time constraints, resources, and the purpose of such evaluations.

Structural levels at which capacity building takes place

The literature strongly supports that RCB should take place at an individual and organisational level [ 8 , 15 , 27 , 28 ]. For example, the conceptual model for RCB in primary care put forward by Farmer & Weston [ 15 ] focuses particularly on individual General Practitioners (GPs) and primary care practitioners who may progress from non participation through participation, to become academic leaders in research. Their model also acknowledges the context and organisational infrastructure to support RCB by reducing barriers and accommodating diversity through providing mentorship, collaborations and networking, and by adopting a whole systems approach based on local need and existing levels of capacity. Others have acknowledged that capacity development can be focussed at a team level [ 11 , 29 ]. Jowett et al [ 30 ] found that GPs were more likely to be research active if they were part of a practice where others were involved with research. Guidance from a number of national bodies highlights the need for multiprofessional and inter-professional involvement in conducting useful research for practice [ 3 , 4 , 6 , 31 ] which implies an appropriate mix of skills and practice experience within research teams to enable this [ 32 ]. Additionally, the organisational literature has identified the importance of teams in the production of knowledge [ 18 , 33 , 34 ].

Developing structures between and outside health organisations, including the development of research networks seems important for capacity building [ 12 , 24 , 34 ]. The Department of Health in the UK [ 14 ] categorizes this supra-organisational support infrastructure to include centres of academic activity, Research & Development Support Units, and research networks.

As interventions for RCB are targeted at different levels, the framework for measuring its effectiveness mirrors this. However, these levels should not be measured in isolation. One level can have an impact on capacity development at another level, and could potentially have a synergistic or detrimental effect on the other.

The six principles of research capacity building

Evaluation involves assessing the success of an intervention against a set of indicators or criteria [ 35 , 36 ], which Meyrick and Sinkler [ 37 ] suggest should be based on underlying principles in relation to the initiative. For this reason the framework includes six principles of capacity building. The rationale for each principle is given below, along with a description of some suggested criteria for each principle. The criteria presented are not an exhaustive list. As the framework is developed and used in practice, a body of criteria will be developed and built on further.

Principle 1. Research capacity is built by developing appropriate skills, and confidence, through training and creating opportunities to apply skills

The need to develop research skills in practitioners is well established [ 3 , 4 , 6 ], and can be supported through training [ 14 , 26 ], and through mentorship and supervision [ 15 , 24 , 28 ]. There is some empirical evidence that research skill development increases research activity [ 23 , 38 ], and enhances positive attitudes towards conducting and collaborating in research [ 39 ]. Other studies cite lack of training and research skills as a barrier to doing research [ 30 , 31 ]. The need to apply and use research skills in practice is highlighted in order to build confidence [ 40 ]and to consolidate learning.

Some needs assessment studies highlight that research skills development should adopt 'outreach' and flexible learning packages and acknowledge the skills, background and epistemologies of the professional groups concerned [ 7 , 15 , 39 , 41 , 42 ]. These include doctors, nurses, a range of allied health professional and social workers. Developing an appropriate mix of professionals to support health services research means that training should be inclusive and appropriate to them, and adopt a range of methodologies and examples to support appropriate learning and experience [ 15 , 31 , 41 ]. How learning and teaching is undertaken, and the content of support programmes to reflect the backgrounds, tasks and skills of participants should therefore be measured. For example, the type of research methods teaching offered by networks and support units should reflect a range and balance of skills needed for health service research, including both qualitative and quantitative research methods.

Skills development also should be set in the context of career development, and further opportunities to apply skills to practice should be examined. Policy and position statements [ 14 , 26 ] support the concept of career progression or 'careers escalator', which also enables the sustainability of skills. Opportunities to apply research skills through applications for funding is also important [ 9 , 10 , 22 , 43 , 44 ].

At team and network level Fenton et al [ 34 ]suggest that capacity can be increased through building intellectual capacity (sharing knowledge), which enhances the ability to do research. Whilst there is no formal measure for this, an audit of the transfer of knowledge would appear to be beneficial. For example teams may share expertise within a project to build skills in novice researchers [ 45 ]which can be tracked, and appropriate divisions of workload through reading research literature and sharing this with the rest of the team/network could be noted.

The notion of stepping outside of a safety zone may also suggest increased confidence and ability to do research. This may be illustrated at an individual level by the practitioner-researcher taking on more of a management role, supervising others, or tackling new methodologies/approaches in research, or in working with other groups of health and research professionals on research projects. This approach is supported by the model of RCB suggested by Farmer and Weston [ 15 ] which supports progress from participation through to academic leadership.

Some examples of criteria for measuring skills and confidence levels are give in table 1 .

Principle 2. Research capacity building should support research 'close to practice' in order for it to be useful

The underlying philosophy for developing research capacity in health is that it should generate research that is useful for practice. The North American Primary Care Group [ 24 ] defined the 'ultimate goal' of research capacity development as the generation and application of new knowledge to improve the health of individuals and families (p679). There is strong support that 'useful' research is that which is conducted 'close' to practice for two reasons. Firstly by generating research knowledge that is relevant to service user and practice concerns. Many argue that the most relevant and useful research questions are those generated by, or in consultation with, practitioners and services [ 3 , 11 , 24 ], policy makers [ 46 ] and service users [ 47 , 48 ]. The level of 'immediate' usefulness [ 49 ] may also mean that messages are more likely to taken up in practice[ 50 ]. Empirical evidence suggests that practitioners and policy makers are more likely to engage in research if they see its relevance to their own decision making [ 31 , 39 , 46 ]. The notion of building research that is 'close to practice' does not necessarily mean that they are small scale, but that the research is highly relevant to practice or policy concerns. A large network of practitioners could facilitate large scale, experimental based projects for example. However, the adoption of certain methodologies is more favoured by practice because of their potential immediate impact on practice [ 47 ] and this framework acknowledges such approaches and their relevance. This includes action research projects, and participatory inquiry [ 31 , 42 ]. An example where this more participatory approach has been developed in capacity building is the WeLREN (West London Research Network) cycle [ 51 ]. Here research projects are developed in cycles of action, reflection, and dissemination, and use of findings is integral to the process. This network reports high levels of practitioner involvement.

Secondly, building research capacity 'close to practice' is useful because of the skills of critical thinking it engenders which can be applied also to practice decision making [ 28 ], and which supports quality improvement approaches in organisations [ 8 ]. Practitioners in a local bursary scheme, for example, said they were more able to take an evidence-based approach into their every day practice [ 9 ].

Developing a 'research culture' within organisations suggests a closeness to practice that impacts on the ability of teams and individuals to do research. Lester et al [ 23 ] touched on measuring this idea through a questionnaire where they explored aspects of a supportive culture within primary care academic departments. This included aspects around exploring opportunities to discuss career progression, supervision, formal appraisal, mentorship, and junior support groups. This may be a fruitful idea to expand further to develop a tool in relation to a health care environment.

Some examples of criteria for measuring the close to practice principle are give in table 2

3. Linkages, partnerships and collaborations enhance research capacity building

The notion of building partnerships and collaborations is integral to capacity building [ 19 , 24 ]. It is the mechanism by which research skills, and practice knowledge is exchanged, developed and enhanced [ 12 ], and research activity conducted to address complex health problems [ 4 ]. The linkages between the practice worlds and that of academia may also enhance research use and impact [ 46 ].

The linkages that enhance RCB can exist between

Universities and practice [ 4 , 14 , 43 ]

Novice and experienced researchers [ 22 , 24 , 51 ].

Different professional groups [ 2 , 4 , 20 , 34 ]

Different health and care provider sectors [ 4 , 31 , 47 , 52 ]

Service users, practitioners and researchers [ 47 , 48 ]

Researchers and policy makers [ 46 ]

Different countries [ 28 , 52 ]

Health and industry [ 53 , 54 ]

It is suggested that it is through networking and building partnerships that intellectual capital (knowledge) and social capital (relationships) can be built, which enhances the ability to do research [ 12 , 31 , 34 ]. In particular, there is the notion that the build up of trust between different groups and individuals can enhance information and knowledge exchange[ 12 ]. This may not only have benefits for the development of appropriate research ideas, but may also have benefits for the whole of the research process including the impact of research findings.

The notion of building links with industry is becoming progressively evident within policy in the UK [ 54 ] which may impact on economic outcomes to health organisations and the society as a whole[ 55 , 56 ].

Some examples of criteria for measuring linkages and collaborations are given in table 3 .

4. Research capacity building should ensure appropriate dissemination to maximize impact

A widely accepted measure to illustrate the impact of RCB is the dissemination of research in peer reviewed publications, and through conference presentations to academic and practice communities [ 5 , 12 , 26 , 57 ]. However this principle extends beyond this more traditional method of dissemination. The litmus test that ultimately determines the success of capacity building is that it should impact on practice, and on the health of patients and comminutes[ 24 ] that is; the social impact of research [ 25 ]. Smith [ 25 ]argues that the strategies of dissemination should include a range of methods that are 'fit for purpose'. This includes traditional dissemination, but also includes other methods, for example, instruments and programmes of care implementation, protocols, lay publications, and publicity through factsheets, the media and the Internet.

Dissemination and tracking use of products and technologies arising from RCB should also be considered, which relate to economic outcomes of capacity building [ 55 ]. In the UK, the notion of building health trusts as innovative organisations which can benefit economically through building intellectual property highlights this as an area for potential measurement [ 56 ].

Some examples of criteria for measuring appropriate dissemination are given in table 4

5. Research capacity building should include elements of continuity and sustainability

Definitions of capacity building suggest that it should contain elements of sustainability which alludes to the maintenance and continuity of newly acquired skills and structures to undertake research [ 18 , 19 ]. However the literature does not explore this concept well [ 19 ]. This in itself may be partly due problems around measuring capacity building. It is difficult to know how well an initiative is progressing, and how well progress is consolidated, if there are no benchmarks or outcomes against which to demonstrate this.

Crisp et al [ 19 ] suggests that capacity can be sustained by applying skills to practice. This gives us some insight about where we might look for measures of sustainability. It could include enabling opportunities to extend skills and experience, and may link into the concept of a career escalator. It also involves utilizing the capacity that has been already built. For example engaging with those who have gained skills in earlier RCB initiatives to help more novice researchers, once they have become 'experts', and in finding an appropriate place to position the person with expertise with the organisation. It could also be measured by the number of opportunities for funding for continued application of skills to research practice.

Some examples of criteria for measuring sustainability and continuity are gibe in table 5

6. Appropriate infrastructures enhance research capacity building

Infrastructure includes structures and processes that are set up to enable the smooth and effective running of research projects. For example, project management skills are essential to enable projects to move forward, and as such should be measured in relation to capacity building. Similarly, projects should be suitably supervised with academic and management support. To make research work 'legitimate' it may be beneficial to make research a part of some job descriptions for certain positions, not only to reinforce research as a core skill and activity, but also to review in annual appraisals, which can be a tool for research capacity evaluation. Information flow about calls for funding and fellowships and conferences is also important. Hurst [ 42 ] found that information flow varied between trusts, and managers were more aware of research information than practitioners.

The importance of protected time and backfill arrangements as well as funding to support this, is an important principle to enable capacity building [9, 15, 24, 58]. Such arrangements may reduce barriers to participation and enable skills and enthusiasm to be developed[ 15 ]. Infrastructure to help direct new practitioners to research support has also been highlighted[ 14 ]. This is particularly true in the light of the new research governance and research ethics framework in the UK [59]. The reality of implementing systems to deal with the complexities of the research governance regulations has proved problematic, particularly in primary care, where the relative lack of research management expertise and infrastructure has resulted in what are perceived as disproportionately bureaucratic systems. Recent discussion in the literature has focused on the detrimental impact of both ethical review, and NHS approval systems, and there is evidence of serious delays in getting research projects started [60]. Administrative and support staff to help researchers through this process is important to enable research to take place [61].

Some examples of criteria for measuring are given in table 6 .

This paper suggests a framework which sets out a tentative structure by which to start measuring the impact of capacity building interventions, and invites debate around the application of this framework to plan and measure progress. It highlights that interventions can focus on individuals, teams, organisations, and through support infrastructures like RDSUs and research networks. However, capacity building may only take place once change has occurred at more than one level: for example, the culture of an organisation in which teams and individuals work may have an influence of their abilities and opportunities to do research work. It is also possible that the interplay between different levels may have an effect on the outcomes at other levels. In measuring progress, it should be possible to determine a greater understanding of the relationship between different levels. The framework proposed in this paper may be the first step to doing this.

The notion of building capacity at any structural level is dependent on funding and support opportunities, which are influenced by policy and funding bodies. The ability to build capacity across the principles developed in the framework will also be dependent of R&D strategy and policy decisions. For example, if policy fluctuates in its emphasis on building capacity 'by', 'for' or 'with' practice, the ability to build capacity close to practice will be affected.

In terms of developing a science of RCB, there is a need to capture further information on issues of measuring process and outcome data to understand what helps develop 'useful' and 'useable' research. The paper suggests principles whereby a number of indicators could be developed. The list is not exhaustive, and it is hoped that through debate and application of the framework further indicators will be developed.

An important first step to building the science of RCB should be debate about identifying appropriate outcomes. This paper supports the use of traditional outcomes of measurement, including publications in peer reviewed journals and conference presentations. This assures quality, and engages critical review and debate. However, the paper also suggests that we might move on from these outcomes in order to capture the social impact of research, and supports the notion of developing outcomes which measure how research has had an impact on the quality of services, and on the lives of patients and communities. This includes adopting and shaping the type of methodologies that capacity building interventions support, which includes incorporating patient centred outcomes in research designs, highlighting issues such as cost effectiveness of interventions, exploring economic impact of research both in terms of product outputs and health gain, and in developing action oriented, and user involvement methodologies that describe and demonstrate impact. It also may mean that we have to track the types of linkages and collaborations that are built through RCB, as linkages that are close to practice, including those with policy makers and practitioners, may enhance research use and therefore 'usefulness'. If we are to measure progress through impact and change in practice, an appropriate time frame would have to be established alongside these measures.

This paper argues that 'professional outcomes' should also be measured, to recognize how critical thinking developed during research impacts on clinical practice more generally.

Finally, the proposed framework provides the basis by which we can build a body of evidence to link process to the outcomes of capacity building. By gathering process data and linking it to appropriate outcomes, we can more clearly unpack the 'black box' of process, and investigate which processes link to desired outcomes. It is through adopting such a framework, and testing out these measurements, that we can systematically build a body of knowledge that will inform the science and the art of capacity building in health care.

• There is currently little evidence on how to plan and measure progress in research capacity building (RCB), or agreement to determining its ultimate outcomes.

• Traditional outcomes of publications in peer reviewed journals, and successful grant applications may be the easy and important outcomes to measure, but do not necessarily address issues to do with the usefulness of research, professional outcomes, the impact of research activity on practice, or on measuring health gain.

• The paper suggests a framework which provides a tentative structure by which measuring the impact of RCB could be achieved, shaped around six principles of research capacity building, and includes four structural levels on which each principle can be applied.

• The framework could be the basis by which RCB interventions could be planned, and progress measured. It could act as a basis of comparison across interventions, and could contribute to establishing a knowledge base on what is effective in RCB in healthcare

Muir Gray JA: Evidence-based Healthcare. How to make health policy and management decisions. 1997, Edinburgh, Churchill Livingstone

Google Scholar  

Department of Health: Research and Development for a First Class Service. 2000, Leeds, DoH

Mant D: National working party on R&D in primary care. Final Report. 1997, London, NHSE South and West.

Department of Health: Strategic review of the NHS R&D Levy (The Clarke Report). 1999, , Central Research Department, Department of Health, 11-

Campbell SM, Roland M, Bentley E, Dowell J, Hassall K, Pooley J, Price H: Research capacity in UK primary care. British Journal of General Practice. 1999, 49: 967-970.

CAS   PubMed   PubMed Central   Google Scholar  

Department of Health: Towards a strategy for nursing research and development. 2000, London, Department of Health

Ross F, Vernon S, Smith E: Mapping research in primary care nursing: Current activity and future priorities. NT Research. 2002, 7: 46-59.

Article   Google Scholar  

Marks L, Godfrey M: Developing Research Capacity within the NHS: A summary of the evidence. 2000, Leeds, Nuffield Portfolio Programme Report.

Lee M, Saunders K: Oak trees from acorns? An evaluation of local bursaries in primary care. Primary Health Care Research and Development. 2004, 5: 93-95. 10.1191/1463423604pc197xx.

Bateman H, Walter F, Elliott J: What happens next? Evaluation of a scheme to support primary care practitioners with a fledgling interest in research. Family Practice. 2004, 21: 83-86. 10.1093/fampra/cmh118.

Article   PubMed   Google Scholar  

Smith LFP: Research general practices: what, who and why?. British Journal of General Practice. 1997, 47: 83-86.

Griffiths F, Wild A, Harvey J, Fenton E: The productivity of primary care research networks. British Journal of General Practice. 2000, 50: 913-915.

Fenton F, Harvey J, Griffiths F, Wild A, Sturt J: Reflections from organization science of primary health care networks. Family Practice. 2001, 18: 540-544. 10.1093/fampra/18.5.540.

Article   CAS   PubMed   Google Scholar  

Department of Health: Research Capacity Development Strategy. 2004, London, Department of Health

Farmer E, Weston K: A conceptual model for capacity building in Australian primary health care research. Australian Family Physician. 2002, 31: 1139-1142.

PubMed   Google Scholar  

Carter YH, Shaw S, Sibbald B: Primary care research networks: an evolving model meriting national evaluation. British Journal of General Practice. 2000, 50: 859-860.

Trostle J: Research Capacity building and international health: Definitions, evaluations and strategies for success. Social Science and Medicine. 1992, 35: 1321-1324. 10.1016/0277-9536(92)90035-O.

Albert E, Mickan S: Closing the gap and widening the scope. New directions for research capacity building in primary health care. Australian Family Physician. 2002, 31: 1038 -10341.

Crisp BR, Swerissen H, Duckett SJ: Four approaches to capacity building in health: consequences for measurement and accountability. Health Promotion International. 2000, 15: 99-107. 10.1093/heapro/15.2.99.

Ryan , Wyke S: The evaluation of primary care research networks in Scotland. British Journal of General Practice. 2001, 154-155.

Gillies P: Effectiveness of alliances and partnerships for health promotion. Health Promotion International. 1998, 13: 99-120. 10.1093/heapro/13.2.99.

Pitkethly M, Sullivan F: Four years of TayRen, a primary care research and development network. Primary Care Research and Development. 2003, 4: 279-283. 10.1191/1463423603pc167oa.

Lester H, Carter YH, Dassu D, Hobbs F: Survey of research activity, training needs. departmental support, and career intentions of junior academic general practitioners. British Journal of General Practice. 1998, 48: 1322-1326.

North American Primary Care Research Group: What does it mean to build research capacity?. Family Medicine. 2002, 34: 678-684.

Smith R: Measuring the social impact of research. BMJ. 2001, 323: 528-10.1136/bmj.323.7312.528.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Sarre G: Capacity and activity in research project (CARP): supporting R&D in primary care trusts. 2002

Del Mar C, Askew D: Building family/general practice research capacity. Annals of Family Medicine. 2004, 2: 535-540.

Carter YH, Shaw S, Macfarlane F: Primary Care research team assessment (PCRTA): development and evaluation. Occasional paper (Royal College of General Practitioners). 2002, 81: 1-72.

Jowett S, Macleod J, Wilson S, Hobbs F: Research in Primary Care: extent of involvement and perceived determinants among practitioners for one English region. British Journal of General Practice. 2000, 50: 387-389.

Cooke J, Owen J, Wilson A: Research and development at the health and social care interface in primary care: a scoping exercise in one National Health Service region. Health and Social Care in the Community. 2002, 10: 435 -4444. 10.1046/j.1365-2524.2002.00395.x.

Raghunath AS, Innes A: The case of multidisciplinary research in primary care. Primary Care Research and Development. 2004, 5: 265-273.

Reagans RZER: Networks, Diversity and Productivity: The Social Capital of Corporate R&D Teams. Organisational Science. 2001, 12: 502-517. 10.1287/orsc.12.4.502.10637.

Ovretveit J: Evaluating Health Interventions. 1998, Buckingham, Open University

Meyrick J, Sinkler P: An evaluation Resource for Healthy Living Centres. 1999, London, Health Education Authority

Hakansson A, Henriksson K, Isacsson A: Research methods courses for GPs: ten years' experience in southern Sweden. British Journal of General Practice. 2000, 50: 811-812.

Bacigalupo B, Cooke J, Hawley M: Research activity, interest and skills in a health and social care setting: a snapshot of a primary care trust in Northern England. Health and Social Care in the Community.

Kernick D: Evaluating primary care research networks - exposing a wider agenda. British Journal of General Practice. 2001, 51: 63-

Owen J, Cooke J: Developing research capacity and collaboration in primary care and social care: is there enough common ground?. Qualitative Social Work. 2004, 3: 398-410. 10.1177/1473325004048022.

Hurst: Building a research conscious workforce. Journal of Health Organization and management. 2003, 17: 373-384.

Gillibrand WP, Burton C, Watkins GG: Clinical networks for nursing research. International Nursing Review. 2002, 49: 188-193. 10.1046/j.1466-7657.2002.00124.x.

Campbell J, Longo D: Building research capacity in family medicine: Evaluation of the Grant Generating Project. Journal of Family Practice. 2002, 51: 593-

Cooke J, Nancarrow S, Hammersley V, Farndon L, Vernon W: The "Designated Research Team" approach to building research capacity in primary care. Primary Health Care Research and Development.

Innvaer S, Vist G, Trommald M, Oxman A: Health policy- makers' perceptions of their use of evidence: a systematic review. Journal of Health Services Research and Policy. 2002, 7: 239-244. 10.1258/135581902320432778.

NHS Service Delivery Organisation : NHS Service Delivery and Against National R&D programme, National listening exercise. 2000, London, NHS SDO

Hanley J, Bradburn S, Gorin M, Barnes M, Evans C, Goodare HB: Involving consumers in research and development in the NHS: briefing notes for researchers. 2000, Winchester, Consumers in NHS Research Support Unit,

Frenk J: Balancing relevance and excellence: organisational responses to link research with decision making. Social Science and Medicine. 1992, 35: 1397-1404. 10.1016/0277-9536(92)90043-P.

National Audit Office.: An international review on Governments' research procurement strategies. A paper in support of Getting the evidence: Using research in policy making. 2003, London, The Stationary Office.

Thomas P, While A: Increasing research capacity and changing the culture of primary care towards reflective inquiring practice: the experience of West London Research Network (WeLReN). Journal of Interprofessional Care. 2001, 15: 133-139. 10.1080/13561820120039865.

Rowlands G, Crilly T, Ashworth M, Mager J, Johns C, Hilton S: Linking research and development in primary care: primary care trusts, primary care research networks and primary care academics. Primary Care Research and Development. 2004, 5: 255-263. 10.1191/1463423604pc201oa.

Davies S: R&D for the NHS- Delivering the research agenda: ; London. 2005, National Coordinating Centre for Research Capacity Development

Department of Health.: Best Research for Best Health: A New National Health Research Strategy. The NHS contribution to health research in England: A consultation. 2005, London, Department of Health

Buxton M, Hanney S, Jones T: Estimating the economic value to societies of the impact of health research: a critical review. Bulletin of the World Health Organisation. 2004, 82: 733-739.

Department of Health.: The NHS as an innovative organisation: A framework and guidance on the management of intellectual property in the NHS. 2002, London, Department of Health

Sarre G: Trent Focus Supporting research and development in primary care organisations: report of the capacity and activity in research project (CARP). 2003

Department of Health: Research Governance Framework for Health and Social Care. 2001, London, Department of Health.

Hill J, Foster N, Hughes R, Hay E: Meeting the challenges of research governance. Rheumatology. 2005, 44: 571-572. 10.1093/rheumatology/keh579.

Shaw S: Developing research management and governance capacity in primary care organizations: transferable learning from a qualitative evaluation of UK pilot sites. Family Practice. 2004, 21: 92-98. 10.1093/fampra/cmh120.

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2296/6/44/prepub

Download references

Acknowledgements

My warm thanks go to my colleagues in the primary care group of the Trent RDSU for reading and commenting on earlier drafts of this paper, and for their continued support in practice.

Author information

Authors and affiliations.

Primary Care and Social Care Lead, Trent Research and Development Unit, formerly, Trent Focus Group, ICOSS Building, The University of Sheffield, 219 Portobello, Sheffield, S1 4DP, UK

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jo Cooke .

Additional information

Competing interests.

The author(s) declare that they have no competing interests.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions.

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Cooke, J. A framework to evaluate research capacity building in health care. BMC Fam Pract 6 , 44 (2005). https://doi.org/10.1186/1471-2296-6-44

Download citation

Received : 12 June 2005

Accepted : 27 October 2005

Published : 27 October 2005

DOI : https://doi.org/10.1186/1471-2296-6-44

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Capacity Building
• Research Capacity
• Research Skill
• Traditional Outcome
• Research Capacity Building

BMC Primary Care

ISSN: 2731-4553

• General enquiries: [email protected]

Building and Evaluating Research Capacity in Healthcare Systems

Case Studies and Innovative Models

Editors Nancy Edwards, Dan Kaseje and Eulalia Kahwa

Contributing Authors Natasha Mortley and Teasdale-Corti Project Teams

Building and Evaluating Research Capacity in Healthcare Systems: Case Studies and Innovative Models

First published 2016 by UCT Press an imprint of Juta and Company (Pty) Ltd

PO Box 14373, Lansdowne, 7779, Cape Town, South Africa

www.uctpress.co.za

This book is a co-publication with the

International Development Research Centre

PO Box 8500, Ottawa, ON, Canada, K1G 3H9

www.idrc.ca / [email protected]

The research presented in this publication was carried out with the aid of a grant from the International Development Research Centre, Ottawa, Canada. The views expressed herein do not necessarily represent those of IDRC or its Board of Governors.

ISBN 978-1-77582-207-3 (print)

ISBN 978-1-77582-221-9 (Web pdf)

ISBN 978-1-55250-588-5 (IDRC ebook)

Project manager: Carlyn Bartlett-Cronje

Editor: Lee-Ann Ashcroft

Proofreader: Daphne Burger

Typesetter: Firelight Studio

Cover designer: Genevieve Simpson

Printed in South Africa by

The authors and the publishers have made every effort to obtain permission for and to acknowledge the use of copyright material. Should any infringement of copyright have occurred, please contact the publisher, and every effort will be made to rectify omissions or errors in the event of a reprint or new edition.

This book has been peer reviewed by experts in the field.

Acknowledgements

Figure 2.1 The integrated framework on page 23. Recreated by authors incorporating elements of Cooke’s Framework. Cooke, J. (2005). A framework to evaluate research capacity building in health care. BMC Family practice, 6:44.

Table of Contents

About the editors, about the contributors.

List of acronymns

Chapter 1    The Case Study Project: An Introduction — Nancy Edwards

Highlights for Funding Agencies and Links to Pertinent Chapters

Chapter 2    Research Capacity Building: A Literature Review and the Theoretical Framework — Eulalia Kahwa, Nancy Edwards & Natasha Mortley

Chapter 3    Improving ‘Research-to-Action’ through Research Triads — Ronald Labonté, Corinne Packer, Nikki Schaay & David Sanders

Chapter 4    Developing Capacity through a Participatory Action Research Approach — Amy Bender, Nancy Edwards, Eulalia Kahwa & Dan Kaseje

Chapter 5    Process Evaluation of a Course Aimed at Improving Research Skills Emerging in Mexican Obesity Researchers — Mariane Héroux, Lucie Lévesque, Juan López Taylor, Edtna Jáuregui Ulloa & Ian Janssen

Chapter 6    Building a Framework to Strengthen Research Capacity within the Caribbean Region — Martin Forde, Karen Morisson, Neela Badrie & Eric Dewailly

Chapter 7    Innovation, Research Capacity Building and Applied Learning: The Canada-China Project for Vulnerable Populations at Risk for HIV and STIs in Shanghai, China — Dan Allman, Liviana Calzavara, Lai-yi Kang, Ted Myers & the Canada–China Project Study Team

Chapter 8    Capacity Building with Research Partners: Experiences from Proyecto Araucaria in Chile — Katherine Lippel, Ximena Díaz & Amalia Mauro

Chapter 9    Research Capacity Building and Dissemination Components of the HIV Prevention for Rural Youth Project in Nigeria — Andrew Onokerhoraye & Eleanor Maticka-Tyndale

Chapter 10 Indicators for Measuring and Evaluating Research Capacity Building — Dan Kaseje, Nancy Edwards & Natasha Mortley

Nancy Edwards, RN, PhD, FCAHS

School of Nursing and Community Medicine, University of Ottawa, Ottawa, Canada

Nancy Edwards is a nurse epidemiologist with nearly 40 years’ experience in the field of public health. Her programme of research examines multi-strategy and multi-level interventions in community health. She has taught and/or led research projects in various international settings, including Sierra Leone, Kenya, Uganda, South Africa, China, Pakistan and Jamaica.

Dan C. Owino Kaseje, MBChB, MMed, MPH, PhD

Great Lakes University of Kisumu, Kisumu, Kenya

Dan Kaseje has over 30 years of professorial experience in providing technical and advisory services to governments, NGOs, intergovernmental, multi-lateral, bilateral and other international agencies in the implementation of health and development programmes in Africa.

Eulalia Kokuangisa Kahwa, RN, PhD

The UWI School of Nursing, Mona, University of the West Indies, Kingston, Jamaica

Eulalia Kahwa is a nurse and nutritionist with over 30 years’ experience in healthcare and nursing education. She has taught and/or led research projects on HIV/AIDS and asthma in Jamaica.

Dan Allman, MA, MSc, PhD

HIV Social, Behavioural and Epidemiological Studies Unit, Dalla Lana School of Public Health, Toronto, Canada

Dan Allman’s work focuses on the social and structural production of risk and well-being, particularly for those considered marginal, vulnerable or peripheral to a society’s core. His interests include the sociology of health and medicine, social equity, global health and new techniques for public health research.

Neela Badrie, BSc, MSc, PhD

Department of Food Production, Faculty of Food and Agriculture, University of the West Indies, St. Augustine, Trinidad and Tobago

Professor Neela Badrie is a member of the Department of Food Production, Faculty of Food and Agriculture at the University of the West Indies, St. Augustine. She is a lecturer and researcher in microbiology, food microbiology, food processing, food safety and quality assurance, risk analysis, international trade and food legislation, public health, epidemiology and food-borne diseases and nutrition.

Amy Bender, RN, PhD

Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Canada

Amy Bender is a nurse educator with 18 years’ clinical and community-based mental health experience. Her scholarly work combines the relational work of nurses, public health, mental health, and best practices in teaching-learning in these areas with the emphasis on critical pedagogies and participatory, qualitative methodologies.

Liviana Calzavara, PhD

Liviana Calzavara has developed expertise in social and behavioural aspects of HIV/AIDS during the course of her career. In the past, her research focused on marginalised individuals. Dr. Calzavara has led multi-disciplinary research teams in international research projects in Russia and China. She is currently the Director of the CIHR Social Research Centre in HIV.

Eric Dewailly, MD, PhD

Laval University, Laval, Canada

Up until his tragic death in 2014, Eric Dewailly served as Professor of Medicine at Laval University, directed the Laval University Medical Centre’s Public Health Research Unit, and was Scientific Director of the World Health Organization’s Collaborative Centre in Environmental Health. He was an eminent professor and Scientific Director at Centre hospitalier universitaire de Québec Research Center, and also served as the director of the CIHR-funded Nasivvik Centre for Inuit Health.

Ximena Díaz, Lic .

Centro de Estudios de la mujer (CEM), Santiago, Chile

Ximena Díaz is a sociologist and a founding member of the Centro de Estudios de la mujer (CEM), a multi-disciplinary research centre in Santiago, Chile, founded in 1984. She has authored many publications on women and work, aimed at policy-makers, academia, and also at workers, unions and organisations promoting women’s rights.

Martin Forde, ScD, P. Eng

Department of Public Health and Preventive Medicine, St. George’s University, St. George’s, Grenada, West Indies

Martin Forde’s research interests are primarily focused in the area of environmental and occupational health. Specific areas of interest include workplace evaluation, toxicology, and zoonotic infectious diseases. He has also been involved in managing several large North–South research collaborations and is interested in the ethical issues that arise in doing global population health research.

Mariane Héroux, PhD

School of Kinesiology and Health Studies, Queen’s University, Kingston, Canada

Mariane Héroux’s research examines the role that nutrition and physical activity play in the development of obesity in both children and adults.

Ian Janssen, PhD

Ian Janssen’s population-based research programme examines the role that physical inactivity and obesity play in the development of poor health in both child and adult populations.

Edtna Jáuregui Ulloa, PhD

Departamento de Ciencias del Movimiento Humano, Educación, Deporte, Recreación y Danza, Universidad de Guadalajara, Jalisco, México

Edtna Jáuregui Ulloa is head of Morphological and Functional Assessment with the Institute of Applied Sciences of Physical Activity and Sport at the University of Guadalajara, Mexico. She is also the coordinator of physical activity initiatives for the Secretary of Health in the state of Jalisco, Mexico.

Laiyi Kang, MD

Shanghai Centre for Disease Control, Shanghai, China

Laiyi Kang is a Professor Emeritus and Chief Medical Doctor at the Shanghai Municipal Center for Disease Control and Prevention (SCDC), in Shanghai, China, and an Adjunct Professor at the University of Toronto, Canada. Dr. Kang has worked in Public Health for nearly 60 years, with over 30 years’ experience in HIV management, epidemiology, clinical evaluation, laboratory and intervention research.

Ronald Labonté, PhD, FCAHS, HonFFPH

School of Epidemiology, Public Health and Community Medicine, University of Ottawa, Ottawa, Canada

Ronald Labonté is a public health sociologist. He has over 25 years’ experience as a community health practitioner, and 20 years as an academic researcher. His current research includes health systems strengthening, globalisation and health equity, impacts of trade and investment agreements on health, health worker migration, medical tourism, and trade and tobacco control. He has led or facilitated research in over 20 countries, from least-developed to high-income.

Lucie Lévesque, PhD

Lucie Lévesque has, over the past 12 years, focused mainly on physical activity intervention effectiveness evaluation from an ecological and holistic perspective. She is especially interested in enhancing the accessibility of physical activity opportunities for youth and has a keen interest in working with indigenous communities to eliminate health disparities related to physical inactivity.

Katherine Lippel, LLL, LLM, FRSC

Civil Law Section, Faculty of Law, University of Ottawa, Ottawa, Canada

Katherine Lippel is a professor of law specialising in occupational health and safety and disability insurance. Her research programme integrates knowledge of regulatory contexts in research examining prevention, compensation and return to work issues for workers in Canada and around the world. She has studied regulatory challenges associated with protection of workers’ mental and physical health and much of her research has been undertaken with community partners.

Juan López-Taylor, PhD

Instituto de Ciencias Aplicadas a la Actividad Física y al Deporte, Universidad de Guadalajara, Jalisco, México

Juan López-Taylor is a sports medicine physician and physical activity researcher at the University of Guadalajara in Guadalajara, Mexico, where he serves as the Director of the Institute of Applied Sciences of Physical Activity and Sport.

Eleanor Maticka-Tyndale, PhD

Faculty of Arts, Humanities and Social Sciences, University of Windsor, Windsor, Canada

Eleanor Maticka-Tyndale’s research focuses on sexual rights and citizenship and HIV prevention. She has led and co-led 31 research projects with partners in Canada, the United States, Europe, Africa, and South and Southeast Asia. Her research has led to over 150 publications, development of HIV-prevention programming deployed over provinces, states, and entire countries, and changes in legislation, as well as changes in the delivery of sexual health services.

Karen Morrison, PhD

Faculty of Environmental Studies, York University, Toronto, Canada

Karen Morrison’s research interests are in ecosystem approaches to human health, including social-ecological resilience, social and environmental determinants of health, health communication, community development and social learning.

Natasha Kay Mortley, PhD

Regional Coordinating Unit of the Institute for Gender and Development Studies, University of the West Indies, Mona, Jamaica

Natasha Kay Mortley engages in a multi-disciplinary approach to the key developmental challenges which are of particular relevance to developing countries. Her work focuses on Caribbean healthcare systems, medical tourism, migration and diaspora studies, diaspora tourism, gender and leadership, gender and climate change, and gender and entrepreneurship.

Amalia Mauro, Lic., MA

Amalia Mauro is a sociologist and the Director of the Centro de Estudios de la mujer (CEM), a multi-disciplinary research centre in Santiago, Chile, founded in 1984. She has authored many publications on women and work aimed at policy-makers, academia and also at workers, unions and organisations promoting women’s rights.

Ted Myers, PhD

Ted Myers has extensive experience in socio-behavioural and behavioural-epidemiological research with marginalised populations, including First Nations people, gay and bisexual men, and drug-using populations, including injection drug users. He has undertaken a number of province-wide and national studies of behaviour and service delivery as well as studies in other countries.

Andrew G. Onokerhoraye, PhD

Centre for Population and Environmental Development (CPED), Benin City, Nigeria

Andrew Onokerhoraye has been involved in grassroots research in Nigeria, focusing on issues of equity in development as they affect the poor and the disadvantaged. He has published over 25 books and more than 100 academic papers. He has also obtained individual grants from national and international agencies to direct over 200 projects which have influenced public policy in Nigeria.

Corinne Packer, PhD

Corinne Packer is a senior researcher at the University of Ottawa and was coordinator of the Revitalizing Health for All research project. She has authored two books on reproductive health and international human rights law, and is co-author and co-editor of publications on diverse subjects, such as the migration of health professionals, the African Union, harmful traditional practices, and comprehensive primary healthcare.

David Sanders, MBChB, DCH, MRCP, DTPH HonDSc

School of Public Health, University of the Western Cape, Cape Town, South Africa

David Sanders is Professor Emeritus and founding Director of the School of Public Health at the University of the Western Cape. A paediatrician and retired professor, he maintains involvement in numerous projects ranging from safe motherhood to community health. He has 30 years’ experience in health policy and programme development in Zimbabwe and South Africa, having advised governments, NGOs and UN agencies in primary healthcare, child health and nutrition, and human health resources.

Nikki Schaay, MPH

Nikki Schaay contributes to the School of Public Health’s teaching activities by convening the core module on comprehensive primary healthcare and facilitating a course on community participation. Her current research initiatives involve contributing to a review of home-and community-based services for the provincial Health Department in the Western Cape, South Africa.

Many contributed to the preparation and writing of this book. We are very grateful to teams that contributed chapters and to those who participated in interviews. We would especially like to thank individuals who have worked behind the scenes on this effort. Renee Nossal and Katie Hoogeveen provided exceptional administrative support and assisted with many facets of the project, including literature reviews, meeting coordination, formatting and editing text, and preparing project reports and ethics submissions. Cody Anderson provided support for some of the final stages of manuscript preparation. Natasha Mortley ably coordinated the overall project, tracked down participants, organised many Skype meetings and kept the editors on track. We thank our interviewers and translators in Jamaica who assisted with some of the Spanish interviews. Thanks are also due to Susan Roelofs who provided some excellent feedback on drafts of some chapters.

We also thank the Global Health Research Initiative (GHRI) for funding this project, allowing us to capture the rich experiences of the Teasdale-Corti teams. The secretariat staff of the initiative, and in particular Ibrahim Daibes, provided helpful guidance and support throughout the preparation of this book. We are grateful for the assistance received. This work was carried out with the aid of a grant (grant#103460-045) from the Canadian government’s GHRI, a partnership between the International Development Research Centre (IDRC), the Canadian Institutes of Health Research (CIHR), and the Canadian International Development Agency (CIDA). The Canadian International Development Agency was folded into the Department of Foreign Affairs and International Trade in 2013, creating the Department of Foreign Affairs, Trade and Development (DFATD) which is now known as Global Affairs Canada. In October 2015, DFATD was renamed Global Affairs Canada.

A CKNOWLEDGEMENTS FOR C HAPTER 7

We would like to acknowledge the contributions of co-investigators Robert Remis, Qi Chao Pan, Zhen Ning, Alan Li, Hui Fang, Yi Le Xue, William Flanagan, Sheng Nian Zhang, Irving Salit, Ming Hua Zhuang, Carol Major and Ping Zhong and others on the Canada-China Project Study Team based at the Shanghai Municipal Center for Disease Control and Prevention; the Regent Park Community Health Centre; the Faculty of Law, Queen’s University; the Toronto General Hospital; the Canadian HIV Initiative Program; and the Ontario HIV Epidemiologic Monitoring Unit; the HIV Social, Behavioural and Epidemiological Studies Unit, and the CIHR Social Research Centre in HIV Prevention, Dalla Lana School of Public Health, University of Toronto.

List of acronyms

Chapter 1 The Case Study Project: An Introduction

Nancy Edwards

Building research capacity is considered an essential component of strengthening healthcare systems in lower- and middle-income countries (LMICs) (ESSENCE on Health Research, 2011). When research capacity is limited, there is a void in the production of contextually relevant research, in the mobilisation of research evidence for societal benefits and in the synthesis of research evidence to inform practice, programmes and policies. Although many long-standing programmes have aimed to improve research capacity in LMICs, a substantial number of these have been primarily oriented towards enhanced graduate or postgraduate training opportunities (eg, WHO-TDR Fellowships), often with a basic biomedical research focus.

While these types of programme have undoubtedly played an important role in the careers of trainees, such approaches are no longer considered adequate. Recognition of the 10/90 gap in research (that only 10% of worldwide resources devoted to healthcare research is put towards those countries where 90% of preventable deaths occur [Global Forum for Health Research 2004]), coupled with limited progress on some of the Millennium Development Goals, has demonstrated the need for new models of research capacity building. In 1998, the Director General of the World Health Organization called for a ‘quantum leap in [research] capacity building’ to improve health and reduce poverty in developing countries (Nchinda, 2002, p. 1701). In response to these concerns, funding agencies have made research capacity building a higher priority and have developed new funding models for capacity building across the spectrum of health research (Gadsby, 2011). These emergent models have attended to issues such as: career trajectories; institutional infrastructure and mentorship support for research; tailoring research capacity efforts to a wide range of disciplines in diverse fields of science and scholarship; and the inclusion of clinical, community and policy partners in the process.

R ESEARCH CAPACITY BUILDING

The concept of capacity building remains somewhat elusive. As described in an initiative led by ESSENCE on Health Research (2011), there is often no clear understanding of what ‘supporting capacity’ implies and how this can be achieved in an efficient and sustainable manner. Some authors have suggested that the terms ‘capacity building’, ‘capacity development’, ‘capacity strengthening’ and ‘capacity enhancement’ are used interchangeably (Blagescu & Young, 2006). In a review of research capacity strengthening by six international donors, Gadsby (2011) concluded that ‘understandings of and approaches to research capacity building are wide-ranging’ (p. 89). While there is no consensus on a definition for the term ‘research capacity building’, there does seem to be agreement that the goals of research capacity building have shifted from a primary focus on generating new knowledge to a more encompassing orientation that addresses the supply and demand side of research production and use (Gadsby, 2011). Furthermore, there has been a realisation that research capacity building must be aimed not only at enhancing skills of individual scientists but also at strengthening institutional systems that enable sustained scientific productivity (Sheikh et al, 2011; ESSENCE on Health Research, 2014; Interacademy Medical Panel, 2013).

M EASURING THE IMPACT OF RESEARCH CAPACITY BUILDING INVESTMENTS

The necessary complexity of using a systems approach to build research capacity leaves donors and others with difficulties in tracking, measuring and accounting for their investments in this area. This measurement challenge has been a long-standing dilemma in both scholarship and practice. Simon (2000) observes that after 20 years of activity to strengthen research capacity and millions of dollars of investments, we still know very little about the impact of these efforts. In his historic review of capacity building among donors from 1960 to 2003, Whyte (2004) concludes that there is a lack of systematic monitoring and evaluation of these initiatives.

Literature examining the impact of research capacity-building initiatives in the health sector is rather sparse and there is limited guidance on the use of indicators to assess these approaches and their sustained impact (Bates et al, 2011; Gadsby, 2011; Whitworth et al, 2008). Consequently, there are very few examples of systematic and integrative reviews documenting what has been learned about research capacity building across settings. The work of Segrott, McIvor and Green (2006), which examines capacity building for nurse researchers, is one of only a handful of reviews that have attempted this kind of synthesis. These authors found only six studies in LMICs that met their criteria. This gap has implications not only for understanding which approaches to building research capacity are effective, but also, in a parallel and closely related area, which approaches to strengthen health systems make a difference. Frenk (2010) observes that ‘the current surge of initiatives on health systems strengthening must be accompanied … by an effort to generate a process of shared learning among countries’ (p. 2).

More recent reports of larger-scale investments in individual and institutional research capacity building (Fogarty International Centre, 2013; Ghaffar et al, 2008; Daar et al, 2008; Ogundahunsi et al, 2015; RAND Europe, 2014) indicate a stronger commitment to evaluating the impact of these programmes using a broader set of indicators. Conventional indicators, such as research funding levels, peer-reviewed publications, physical infrastructure and the number of trainees, are being complemented by indicators that examine institutional and systems strengthening approaches. Examples of emerging indicators in these domains include: research governance, management and administrative capacity; institutional leadership, global networks and research hubs (North–South and South–South collaboration); and sustainability.

The Global Health Research Initiative background

The chapters in this book examine research capacity building and its measurement, using the cases of 13 teams co-led by Canadian and LMIC researchers as the basis for our analysis. The teams were supported by the Canadian GHRI, through a strategic funding model called the Teasdale-Corti Initiative, with the specific target of research capacity building in LMICs. Chapters were contributed by members of these teams. Case study methods, examples and insights from the work of these teams help extend our understanding of critical elements of research capacity building, and in turn, what indicators capture the nuanced impact of initiatives with this focus.

The GHRI was a consortium of federal agencies in Canada which formally joined together in 2002 with the aim of strengthening research capacity in developing countries to tackle global health challenges. In so doing, they promoted collaboration and shared learning across disciplines and sectors and between low- and middle-income countries and Canada (International Development Research Centre (IDRC)/Centre de recherches pour le développement (CDRI)). The consortium originally included two research funding councils (Canadian Institutes of Health Research and IDRC), as well as Health Canada and the Canadian International Development Agency. The Public Health Agency of Canada joined the consortium in 2009.

The aims of the GHRI are in keeping with principles that have been outlined by other organisations such as the Swiss Commission for Research Partnerships with Development Countries (KFPE) (1998), and in international agreements such as the Paris Declaration on Aid Effectiveness and the Accra Agenda for Action (2005/2008), and the Bamako Call to Action on Research for Health (2008).

A number of research initiatives were funded through the GHRI, including the Teasdale-Corti programme, which is the basis for the cases presented in this book. This programme was named after Drs Lucille Teasdale and Pierre Corti, who worked in health and healthcare in rural Uganda for many years until Lucille’s death in 1996 from occupationally acquired AIDS.

The aims of the Teasdale-Corti team grants were to:

•   enable the creation or further development of South–North teams of researchers working with research users to address one or more of the eligible thematic areas

•   enable teams to develop and implement integrated programmes that combine capacity strengthening, knowledge generation and its application, to improve the health of citizens, especially those most vulnerable, and strengthen health systems in low- and middle-income countries

•   enable teams and their host institutions to provide strong training and mentoring environments, build capacity, and develop and implement strategies for knowledge translation

•   establish a foundation from which teams, individuals and institutions can more effectively secure other research and development grants in order to address pressing health challenges of LMICs.

The Teasdale-Corti programme focused on four thematic areas:

1)   health policy and system research

2)   prevention and control of pandemics

3)   prevention and control of chronic diseases

4)   interactions between health and environment and development.

There were three components: team grants, supplementary awards and leadership awards. Team grants were awarded based on a two-stage selection procedure. Teams initially submitted requests for project development awards (maximum of $15 000). These seed funds were used to support a variety of proposal development activities, including joint face-to-face meetings for teams to develop their full-scale application. Final protocols were submitted and reviewed in 2007. The awards initially ran for four years, but most teams requested a no-cost extension, and many of the teams continued their work into 2012. Funded teams were eligible to apply for a supplementary grant focused on knowledge translation and scale-up. Leadership awards, aimed at early and mid-career researchers in LMICs, were also launched. Although these award holders were considered part of the Teasdale-Corti programme, none of them worked directly with the Teasdale-Corti teams and this part of the programme operated separately from the team grants. The experience of the Teasdale-Corti teams is the focus of this book.

The Teasdale-Corti team grant programme had a number of features, including requirements for a programmatic research approach, the engagement of decision-makers, integration of a knowledge translation plan, inclusion of gender analysis and plans for a strong training and mentoring component. A total of $24 million (Canadian) was invested in the Teasdale-Corti programme between 2007 and 2012. Fourteen teams were chosen to receive funding through a highly competitive peer review process. These teams spanned 30 LMICs. Their project names and partner countries are provided in Appendix A . The teams addressed a range of substantive issues and included biomedical, clinical, health services and population health research.

As reflected in the overall aims of the Teasdale-Corti initiative, capacity building was a core element of the programme design. During an initial meeting of the co-principal investigators for each of the projects, capacity building was identified as a priority area for joint learning. A plan to assemble the case studies described in this book followed that meeting. A grant was subsequently awarded to the editors of this book to develop and implement the case study analysis.

We began with a review of the literature to identify frameworks that captured elements of research capacity building and were consistent with a systems approach. Individual teams were invited to write chapters that highlighted core elements of capacity building within their projects and approaches to assess the impact of these initiatives. Seven of the 14 teams contributed chapters, discussing critical issues such as leadership for change; linking capacity-building efforts across systems or structural levels; and creating a culture of mentorship, research support and enabling structures for ongoing capacity building and sustainability.

The overall experience of the teams was captured using a case study approach. Thirteen of the 14 teams agreed to participate in this portion of the project. This involved a review and analysis of the teams’ annual and end-of-project reports, and interviews conducted with Canadian and LMIC team members. Results of this composite case study are presented in the final chapter along with a discussion of strengths, gaps and limitations of indicators used and proposed directions for assessing research capacity building in future programmes of research.

In response to calls for a quantum leap in research capacity building in LMICs, funding agencies have developed new funding models.

Emergent LMIC research funding models address issues such as: career trajectories; institutional infrastructure and mentorship support for research; tailoring research capacity efforts to a wide range of disciplines in diverse fields of science and scholarship; and the inclusion of clinical, community and policy partners in the process.

The goals of research capacity building have shifted from a primary focus on generating new knowledge to a more encompassing orientation that addresses the supply and demand side of research production and use, and recognises the need to strengthen institutional systems that enable sustained scientific productivity (see Chapter 2 for discussion of a framework for research capacity building).

The necessary complexity of using a systems approach to build research capacity makes it difficult for donors and others to track, measure and account for their investments in this area (see Chapter 10 for a discussion of potential indicators).

This book contains individual case examples that illustrate various approaches to capacity building (see Chapters 3 – 9 ).

A composite case study of 13 Canadian-LMIC teams funded through the GHRI in Canada highlights the strengths, gaps and limitations of evaluation indicators. This case study also provides direction for funding agencies to assess what research capacity has been built through programmes of research (see Chapter 10 ).

R EFERENCES

Bamako Call to Action on Research for Health: Strengthening research for health, development, and equity. (2008, November 17–19). Paper presented at The Global Ministerial Forum on Research for Health, Bamako, Mali. Available from: http://www.who.int/rpc/news/BAMAKOCALLTOACTIONFinalNov24.pdf (Accessed 24 May 2016).

Bates, I., Taegtmeyer, M., Squire, S.B., Ansong, D., Nhlema-Simwaka, B., Baba, A., & Theobald, S. (2011). Indicators of sustainable capacity building for health research: Analysis of four African case studies. Health Research Policy and Systems , 9:14.

Blagescu, M., & Young, J. (2006). Capacity development for policy advocacy: Current thinking and approaches among agencies supporting civil society organizations. Working Paper 260: Overseas Development Institute . ODL: London.

Campbell, D., Redman, S., Jorm, L., Cooke, M., Zwi, A., & Rychetnik, E. (2009). Increasing the use of evidence in health policy: Practice and views of policy makers and researchers. Australia and New Zealand Health Policy , 6:12.

Commission for Research Partnerships with Developing Countries (KFPE). (1998). Guidelines for research in partnership with development countries . The Commission: Bern. Available from: http://www.kfpe.ch/download/Guidelines_e.pdf (Accessed 24 May 2016).

Daar, A.S., Whyte, S.R., Mohamed, S.A., Ching-Li, H., Hoffman, S.L., & Berger, M. (2008). TDR thirty years on: Taking stock and envisioning the future for the Special Programme for Research and Training in Tropical Diseases. PLoS Neglected Tropical Diseases , 2 (11), e314.

ESSENCE on Health Research. (2011). Planning, monitoring and evaluation: Framework for capacity strengthening in health research. ESSENCE Good Practice Document Series . Available from: http://whqlibdoc.who.int/hq/2011/tdr_essence_11.1_eng.pdf (Accessed 24 May 2016).

ESSENCE on Health Research. (2014). Seven principles for strengthening research capacity in low- and middle-income countries: Simple ideas in a complex world. ESSENCE Good Practice Document Series . Available from: http://www.who.int/tdr/publications/Essence_report2014_OK.pdf (Accessed 24 May 2016).

Fogarty International Center. (2013). International tobacco and health research and capacitybuilding program: Program review 2002–2012 . Available from: http://www.fic.nih.gov/About/Staff/Policy-Planning-Evaluation/Pages/fogarty-program-evaluation-tobacco.aspx (Accessed 24 May 2016).

Frenk, J. (2010). The global health system: Strengthening national health systems as the next step for global progress. PLoS Medicine , 7 (1), 1–3. doi: 10.1371/journal.pmed.1000089 (Accessed 24 May 2016).

Gadsby, E.W. (2011). Research capacity strengthening: Donor approaches to improving and assessing its impact in low and middle income countries. International Journal of Health Planning and Management , 26:89–106.

Ghaffar, D.A., IJsselmuiden, C., & Zicker, F. (Eds.). (2008). Changing mindsets: Research capacity strengthening in low- and middle-income countries . Council on Health Research Development (COHRED), Global Forum for Health Research, & UNICEF/UNDP/World Bank/ WHO Special Programme for Research and Training in Tropical Diseases (TDR). Available from: http://www.cohred.org/downloads/cohred_publications/Changing_Mindsets.pdf (Accessed 24 May 2016).

Global Forum for Health Research. (2004). The 10/90 report on health research, 2003–2004. Geneva, Switzerland: Global Forum for Health Research. Available from: http://announcementsfiles.cohred.org/gfhr_pub/assoc/s14789e/s14789e.pdf (Accessed 24 May 2016).

IDRC/CRDI. (n.d). Global Health Research Initiative – About. Available from: http://www.idrc.ca/EN/Programs/Global_Health_Policy/Global_Health_Research_Initiative/Pages/About.aspx (Accessed 24 May 2016).

Interacademy Medical Panel. (2013). A call for action to strengthen health research capacity in low and middle income countries. Available from: http://www.iamp-online.org/sites/iamp-online.org/files/IAMP%20Call%20for%20Action%20on%20RSC%20.pdf (Accessed 24 May 2016).

Nchinda, T.C. (2002). Research capacity strengthening in the South. Social Science Medicine , 54:1699–1711.

Ogundahunsi, O., Vahedi, M., Kamau, E., Aslanyan, G., Terry, R., Zicker, F., & Launois, P. (2015). Strengthening research capacity – TDR’s evolving experience in low- and middle-income countries. PLoS Neglected Tropical Diseases , 9 (1), e3380.

Paris Declaration on Aid Effectiveness and the Accra Agenda for Action. (2005/2008). Available from: http://www.oecd.org/development/effectiveness/34428351.pdf (Accessed 24 May 2016).

RAND Europe. (2014). Evaluating the African institutions initiative: Lessons from the first four years . Commissioned by the Wellcome Trust. Available from: http://www.wellcome.ac.uk/stellent/groups/corporatesite/@sf_central_grants_admin/documents/web_document/wtp057114.pdf (Accessed 24 May 2016).

Segrott, J., McIvor, M., & Green, B. (2006). Challenges and strategies in developing nursing research capacity: A review of the literature. International Journal of Nursing Studies , 43:637–651.

Sheikh, K., Gilson, L., Akua Agyepong, I., Hanson, K., Ssengooba, F., & Bennett, S. (2011). Building the field of health policy and systems research: Framing the questions. PLoS Medicine , 8 (8):e1001–1073.

Simon, J. (2000). Commentary: Does strengthening research capacity improve health equity? British Medical Journal , 321(7264):816–817.

Whitworth, J.A.G, Kokwaro, G., Kinyanjui, S., Snewin, V.A., Tanner, M., Walport, M., & Sewankambo, N. (2008). Strengthening capacity for health research in Africa. The Lancet , 372:1590–1593.

Whyte, A. (2004). Landscape analysis of donor trends in international development. In Rockefeller Foundation (Ed.), Human and institutional capacity building (issue 2). New York City, NY: Rockefeller Foundation.

Chapter 2 Research Capacity Building: A Literature Review and the Theoretical Framework

Eulalia Kahwa, Nancy Edwards & Natasha Mortley

There have been recurring international calls to redress the 10:90 gap in research funding, whereby only 10% of global spending on research addresses health challenges that disproportionately affect 90% of the poor, marginalised and disadvantaged (Global Forum for Health Research, 2004). Building capacity for health research is widely acknowledged as one of the most effective ways to close this gap and achieve sustainable development and health equity, particularly in lower- and middle-income countries (LMICs) (Commission on Health Research for Development, 1990; Gadsby, 2011; Global Forum for Health Research, 2004; Nuyens, 2005; United Nations Development Programme, 2010; Wellcome Trust, 2013; White, 2002; Whitworth et al, 2008). To promote greater equity in global research for health, the Bamako Call to Action on Research for Health (2008) by the Global Ministerial Forum on Research for Health strongly recommended that national governments build capacity for health research throughout health systems by ‘identifying national research priorities; … providing a conducive environment for development of a strong research culture; … improving education and training of researchers; integrating research for health within health systems; … and evaluating the impact of research for health’ (p. 2).

In line with these recommendations, progress has been made in research production within LMICs over the last couple of decades. However, significant capacity gaps remain in all fields of health research including biomedical, clinical, health services and population health. Various authors (Bennett et al, 2011; Frenk et al, 2010; Kilama, 2009; Mirzoev et al, 2013; Yamey, 2012) and international panels (Interacademy Medical Panel, 2013) have described these research capacity gaps and related factors in LMICs. For instance, Kilama (2009) highlights the small number of researchers and the lack of competencies among researchers, weak research institutions, weak communication systems, and lack of opportunities for networking and collaboration in some countries and in some fields of research. With respect to clinical and health services research, authors in higher- and lower-middle-income countries describe weaknesses in translating research evidence into policy, guidelines and protocols to guide practice (Kilama, 2009). Yamey (2012) argues that most evidence-based health interventions are not implemented in LMICs mainly due to the ‘know–do gap’ (p. 3). Others (Frenk & Chen, 2011; Frenk et al, 2010; Grimshaw, Eccles, Lavis, Hill, & Squires, 2012; Saxena, Paraje, Sharan, Karam, & Sadana, 2006; WHO Evidence Informed Policy Network (EVIP), 2012; WHO Special Programme for Research and Training in Tropical Disease (WHO-TDR), 2011) have forcefully stated that critical factors underlying this implementation gap concern the relevance of research, which has predominantly been undertaken in higher-income settings and/or been driven by researchers and funding agencies in these settings.

A lack of contextually relevant research to inform health system improvements has been increasingly identified as an important constraint to bridge the know–do gap (Luoto, Shekelle, Maglione, Johnsen, & Perry, 2014; Orton, Lloyd-Williams, Taylor-Robinson, O’Flaherty, & Capewell, 2011; World Health Organization (WHO), 2005). This pertains to the focus of research in a particular setting, and whether applied research has a strong implementation orientation (Chambers, Glasgow, & Stange, 2013; Parry, Carson-Stevens, Luff, McPherson, & Goldmann, 2013; WHO-TDR, 2011). A major related challenge concerns how to build capacity for health research so that less developed countries are not only main beneficiaries of research capacity-building interventions but also drivers of their own health research agenda (ESSENCE on Health Research, 2014). This includes the identification of priorities for utilising research evidence in policy and practice to improve the quality of healthcare and reduce the burden of disease (Bennett et al, 2011; Frenk & Chen, 2011; Yamey, 2012). Finally, the orientation and impact of research capacity-building efforts have also been affected by changes in the burden of disease (eg, HIV/AIDS) and subsequent shifts in research funding. The double burden of infectious and chronic, non-communicable diseases many LMICs are now facing illustrates the need for renewing capacity-building approaches to deal with emerging priorities (EVIP, 2012; WHO-TDR, 2011; Thornicroft, Cooper, Van Bortel, Kakuma, & Lund, 2012; WHO, 2011).

While there is agreement on the need for capacity building, there is no single internationally accepted definition of it (Bates et al, 2006; ESSENCE on Health Research, 2011). A common understanding of what capacity building means would facilitate the design of appropriate research capacity-building interventions and the development of tools to measure subsequent improvements. The lack of consistent terminology across settings, organisations and individuals presents a challenge in planning, implementing, monitoring and evaluating outcomes of capacity-building interventions across projects (Cooke, 2005; Ebbesen, Heath, Naylor, & Anderson, 2004; Goldberg & Bryant, 2012; Miller, Bryant-MacLean, Coward, & Broemeling, 2009). This definitional issue may contribute to differing perspectives regarding the most effective and appropriate strategies, targets and foci for research capacity-building efforts and how to tailor these to national contexts and systems (ESSENCE on Health Research, 2011; Gadsby, 2011; Ghaffar, IJsselmuiden, & Zicker, 2008; Marjanovic, Hanlin, Diepeveen, & Chataway, 2013).

There are no easy answers to this constellation of challenges. However, a framework may provide useful parameters for learning from investments in research capacity-building initiatives. As a basis for developing a common framework to examine capacity building within the projects funded by the Teasdale-Corti programme, we conducted a review of the literature. The objectives of this review were as follows:

•   to explore definitions, concepts, approaches and frameworks for research capacity building

•   to identify frameworks for evaluating research capacity building in healthcare

•   to describe key challenges related to research capacity building in LMICs.

In 2009, we conducted a search of HealthSTAR, CINAHL, PubMed, Global Health and SCOPUS databases to identify literature and theoretical frameworks related to capacity building. We developed a search strategy using keywords to identify relevant articles (see Appendix B ). In 2012, we repeated our search so that we ultimately covered the period from 1999 to 2012.

Articles were deemed potentially relevant if they were identified through a combination of the search terms relating to five concepts (see Table 2 in Appendix B ). Titles and abstracts were reviewed and full articles retrieved for those deemed relevant. Additional articles were identified by searching through reference lists of retrieved articles. We reviewed all the articles to identify definitions of capacity building and potential evaluation indicators. Theoretically oriented articles were read in-depth and details regarding underlying constructs and frameworks were extracted.

D EFINITIONS OF C APACITY B UILDING

There is no shortage of definitions of capacity building (ESSENCE on Health Research, 2011; Goldberg & Bryant, 2012; Hailey & James, 2003; Miller et al, 2009). Some have described capacity building as ‘an elusive concept, [that is] intangible and vague’ (LaFond & Brown, 2003, p. 19). Two classes of definitions of capacity building, namely generic and research-specific, are instructive.

Generic definitions

Generic definitions describe the process of capacity building in different contexts such as development, health and education. The World Bank, for example, views capacity building as the development of people and institutions to enable countries to meet development goals. In a 1998 document, it describes ‘capacity’ as the combination of people, institutions and practices that permits countries to reach their development goals. It sees capacity building as investment in human capital, institutions and practices.

In a 2009 document, the World Bank expands on this definition, integrating elements of leadership, engagement and ownership. In this more recent definition, capacity building is described as ‘a locally driven process of learning by leaders, coalitions and other agents of change that brings about changes in sociopolitical, policy-related, and organizational factors to enhance local ownership for and the effectiveness and efficiency of efforts to achieve a development goal’ (World Bank Institute, p. 3).

The United Nations Educational Scientific and Cultural Organization (UNESCO) – International Institute for Capacity Building in Africa (IICBA) focuses mainly on skill building to facilitate the achievement of development goals through education. It defines capacity building as ‘a long-term process of building technical abilities, values and attitudes that enable countries, organizations, groups and individuals at any level of society to carry out functions and achieve their development objectives over time …’ (Morgan, 1997, p. 2). Similarly, in a framework for building capacity to improve health, New South Wales Health (2001) defines capacity building as ‘… an approach to the development of sustainable skills, structures, resources and commitment to improvement in health and other sectors to prolong and multiply health gains many times over’ (New South Wales Health, p. 3).

Overall, these and other generic definitions of capacity building refer to the multi-level and multidimensional aspects of capacity building, stressing local ownership of capacity-building initiatives. They emphasise its iterative nature; refer to empowerment processes, and suggest that capacity building is aimed at improved performance and efficiency, and better problem-solving to meet goals and facilitate change.

Research-specific definitions

Research-specific definitions of capacity building vary with respect to what capacity is being built. Some definitions exclusively emphasise individual capacity building (Finch, 2003; Miller et al, 2009), albeit with the intention to produce an aggregated improvement in a particular group (eg, ‘the ability within a discipline or professional group to undertake high-quality research’ [Finch, 2003, p. 427]). However, many of the definitions we found incorporate individual, institutional or, even more broadly, systems capacity-building dimensions. Illustrative examples include: ‘enhancing the abilities of individuals, organizations and systems to undertake and disseminate high quality research efficiently and effectively’ (Department for International Development, 2010, p. 3) or improving ‘the ability of individuals, organizations or systems to perform and utilise health research effectively, efficiently and sustainably’ (Bates et al, 2006, p. 1226). Lansang and Dennis (2004) also make reference to this broader set of players in capacity building, noting that it involves individuals, institutions and nations. The Global Forum for Health Research (2004) definition explicitly joins up this multi-level approach with broader development-oriented aims of research capacity strengthening. It emphasises how the process of capacity strengthening yields ‘abilities (individually and collectively) to perform functions effectively, efficiently and in a sustainable manner to define problems, set objectives and priorities, build sustainable institutions and bring solutions to key national problems’ (p. 149).

Notably, many of these definitions describe the research production and dissemination phases, emphasising knowledge translation and uptake (Bates et al, 2006; Department for International Development, 2010; Ghaffar et al, 2008; Lansang & Dennis, 2004). Similarly, Baird and Baron (2009) highlight the importance of strengthening both the supply and demand side of capacity-building efforts. Others (Global Forum for Health Research, 2004; Trostle, 1992; UNDP, 2010) stress research application as a means to contribute to sustainable development, noting for instance that research capacity ‘encompasses skills associated with identifying national health research priorities’ (Ghaffer et al, 2008, p. 19). Similarly, Gadsby (2011) describes research capacity as inclusive of identifying health research priorities, generating and disseminating knowledge from research, and getting research knowledge into policy and practice.

Iterative and concept-specific definitions

Two other aspects of research capacity building are apparent in the definitions. Some authors discuss the iterative nature of capacity building, describing it as ‘an ongoing process of empowering individuals, institutions, organizations and nations to: define and prioritize problems systematically, develop and scientifically evaluate appropriate solutions and share and apply knowledge generated’ (Lansang & Dennis, 2004, pp 764–765). Others highlight the need for context-specific capacity-building processes. Vogel (2011), for instance, defines research capacity strengthening as ‘a context specific, dynamic process that goes beyond a technical or value-neutral transfer of skills’ (p. 12).

Based on these elements, we developed the following working definition of research capacity building and then used this as a basis to identify relevant conceptual frameworks:

Research capacity building: An ongoing and iterative process of empowering individuals, interdisciplinary teams, networks, institutions and societies to identify health and health-related challenges; to develop, conduct and manage scientifically appropriate and rigorous research to address those challenges in a dynamic and sustainable manner; and to share, apply and mobilise research knowledge generated with the active participation of engaged stakeholders and decision-makers.

A systems orientation to research capacity building

Consistent with the working definition presented above, there has been a pronounced shift in descriptions of research capacity building, from an individual orientation to a systems orientation (Interacademy Medical Panel, 2013; ESSENCE, 2014). Traditional models of research capacity building focused on building the research skills of individual researchers (producers of research), mainly through graduate programmes, postgraduate training, mentoring and apprenticeship-style and learning-by-doing approaches. Earlier indications of a systems orientation included the explicit intent of some programmes to build a critical mass of researchers in specific institutions, settings or fields of research (Edwards et al, 2009; Golenko, Pager & Holden, 2012; Higginbotham, 1994; Nuyens, 2005; Wellcome Trust, 2013). In some instances, this was coupled with or followed by the establishment of institutional structures such as research centres or formal networks to mobilise this critical mass. Over the past decade, a systems orientation to capacity building has taken hold, recognising the importance of adequate capacity within not only research production systems but also research utilisation systems. Creating an interface between academics who generate research and front-line clinicians, decision-makers and policy-makers who use research has been a primary focus of discussion and effort. Nuyens (2005), for example, argues that research capacity strengthening is concerned not only with individual researchers and institutions but also with the health system as a whole. While individual capacities are essential, ‘… individual and collective abilities combine in some way into a larger overall systems capacity …’ (Morgan, 2005, p.18). This systems orientation is reflected in evaluation frameworks to examine returns on research investments (Panel on Return on Investment in Health Research, 2009), an extensive literature on knowledge translation, and more recent funding opportunities for capacity building (Sacco et al, 2011; Wellcome Trust, 2013). However, with respect to research capacity building within LMICs, a number of recurring challenges have emerged (ESSENCE on Health Research, 2011; Mahmood et al, 2011):

•   establishing and retaining a critical mass

•   making advances in institutional capacity and support that keep pace with the production of researchers

•   creating the demand side for research in settings where health systems are fragile

•   addressing the challenge of limited within-country funding for research and the subsequent dependence on external agencies.

Morgan (2005), who examined capacity from a systems perspective, argues that ‘capacity is an emergent property that comes about through the interrelations and interactions amongst the various elements of the system of which it is a part’ (p. 18). He highlights the intentionality that underlies collective capacities, noting that ‘collective capacities form within a system and depend to a large degree on the willingness of individuals, groups and organizations to collaborate in pursuit of shared goals’ (p. 18).

Nuyens (2005) suggests that research capacity strengthening could be addressed at different levels, through various functions of the health research system and in multiple phases of the research process. Thus, current models of research capacity building emphasise a systems approach, using a combination of strategies aimed at individual, institutional, organisational, national and international levels, and at research networks (Cooke, 2005; Department for International Development, 2010; ESSENCE on Health Research, 2011; Farmer & Weston, 2002; Ghaffar et al, 2008; Lansang & Dennis, 2004; Nuyens, 2005). Ebbesen et al (2004) also note that capacity building occurs in many inter-connected domains. They argue that for capacity building to take place, there must be change at more than one structural level.

Cooke (2005) observes that the culture of an organisation in which teams and individuals work may have an influence on their abilities and opportunities to do research. The interplay and nested relationships between the different structural levels may have an effect on outcomes at other levels. For this reason, a sound understanding of the relationships and interconnections among system levels is key to measuring progress in capacity development. As Morgan (2005) notes:

In practice, most systems draw information, resources and energy from a whole range of places and span many boundaries and levels. The question is less about the identification of the hierarchical levels and more about the nature, distribution and interconnection of the system actors (p. 15).

C ONCEPTUAL F RAMEWORKS FOR R ESEARCH C APACITY B UILDING

Conceptual frameworks provide a structured approach to designing, coordinating and evaluating the complex processes inherent in research capacity building (Bordage, 2009; Goldberg & Bryant, 2012). Nuyens (2005) argues that organising frameworks for research capacity building is critical in planning capacity-building interventions which are ‘more integrated and less fragmented, more coordinated and less vertical, more long-term and sustainable, less one-shot and unplanned, more client-oriented and less donor-driven’ (p. 35). We sought to identify conceptual frameworks that were consistent with our working definition of research capacity building and which had some demonstrated utility for both implementing and evaluating capacity-building initiatives.

Our literature review yielded a variety of research capacity-building frameworks, ranging from those with a primary focus on individuals (Farmer & Weston, 2002; Whitworth, Haining, & Stringer, 2012) or organisations (Golenko et al, 2012) to those with a stronger systems orientation (Cooke, 2005; Department for International Development, 2010; ESSENCE on Health Research, 2011; Ghaffar et al, 2008; Lansang & Dennis, 2004). Some frameworks (Department for International Development, 2010; Ghaffar et al, 2008; Whitworth et al, 2012) highlighted specific competencies that should be addressed through capacity-building efforts. All the frameworks incorporated strategies and/or incentives that might be used to build research capacity and/or identified potential evaluation indicators. However, the rather sparse empirical evidence supporting best practices for integrated systems-oriented approaches to capacity building was notable.

Frameworks with a systems orientation described the need for strategies directed at many levels simultaneously: individuals, institutional or organisational, national and international, and through networks and collaborative activities (Cooke, 2005; Department for International Development, 2010; ESSENCE on Health Research, 2011; Ghaffar et al, 2008; Lansang & Dennis, 2004). Similarly, these authors recommended the concurrent development of human resources, enhanced research infrastructure and a strengthened research environment as the basis for developing a sustainable national health research system.

A framework to evaluate research capacity building in healthcare

Following a review of existing frameworks, we determined that the widely cited framework developed by Cooke (2005) best captured the systems orientation and articulated critical dimensions of research capacity building more clearly than other frameworks. Given its origins in health system research, and more specifically primary healthcare, Cooke’s framework (2005) reflects the knowledge production–user interface. Others have also presented frameworks reflecting a multi-level approach (ESSENCE on Health Research, 2011; Ghaffar et al, 2008; Lansang & Dennis, 2004), but Cooke’s framework has several additional strengths. It takes into account the reciprocal influences of the research capacity-building process and the implementation context. It recognises the influence of the policy context whereby prevailing sociopolitical conditions may facilitate or restrict the capacity-building process. The framework also explicitly addresses the need to consider sustainability and continuity.

We used this framework as a foundation, but then added a structural level and several dimensions to incorporate the wider set of findings from the literature review. Before discussing these adaptations, we describe the published framework. It was developed to measure progress for initiatives aimed at research capacity building in healthcare systems. The framework is also intended to provide a basis for examining what and how capacity-building initiatives work in the short and long term, providing process and outcome indicators for measuring progress (Cooke, 2005).

Cooke’s framework consists of two elements: structural levels and dimensions. Four structural levels of capacity development activity are identified: individual, team, organisational and the network or supra–organisational support level (networks and support units). Consistent with systems thinking, capacity building occurs within and across structural levels, with capacity changes at one level influencing capacity development at other levels.

The six dimensions of research capacity building in Cooke’s framework are as follows:

1)    Building skills and confidence: this refers to the development of research skills, self-assuredness and a positive attitude towards doing and using research. It is achieved through ongoing training, mentoring and supervision, sharing of skills and expertise as well as providing opportunities to apply acquired skills. The framework emphasises the importance of conducting a needs assessment to identify capacity-building requirements, and planning capacity-building interventions that are appropriate to the participants’ backgrounds and level of skill development and relevant to different professional groups (Cooke, 2005; Farmer & Weston, 2002).

2)    Ensuring research is close to practice: research should be useful, relevant and inform practice. This practice orientation stems from the fact that the framework was originally developed for primary care. The authors emphasise the importance of research questions that are generated in consultation with users (practitioners and other service providers, and policy-makers), in order to produce research evidence that is relevant to prevailing health issues and concerns. The framework encourages the use of research approaches and methodologies to facilitate the engagement of, and ultimately the uptake of research findings by, practitioners, research users and policy-makers. This approach is described as fostering development of a research culture.

3)    Developing partnerships and linkages: this is identified as key to expanding intellectual and social capital, which can facilitate the exchange of research skills, knowledge and expertise. Establishing relationships among diverse groups (professional associations, practitioners and academics, novice and expert researchers, service users, researchers and policy-makers) is the basis for information exchange and collaboration that ultimately provides the required evidence to tackle complex health problems.

4)    Supporting appropriate dissemination: Cooke (2005) recognises dissemination as critical for enhancing the social impact of research and ensuring that it effectively influences practice. She suggests a range of dissemination techniques including peer-reviewed publications and conference presentations; the development of protocols of care, lay publications, and fact sheets; and using the media.

5)    Including elements of continuity and sustainability: Cooke suggests that the capacity gains of individuals can be sustained by establishing essential research structures and by providing opportunities to apply and extend knowledge and skills gained into practice. Cooke also indicates that establishing mentoring relationships in which experienced researchers work with novice researchers is key to sustaining capacity.

6)    Making investments in infrastructure to enhance research capacity building: the final dimension is an appropriate infrastructure to support participation in research and related capacity-building initiatives. Illustrative examples include involving both academic and management staff to supervise and manage projects, providing protected time for research, creating research positions, and enhancing knowledge about and access to research funding opportunities.

These six dimensions of Cooke’s framework are similar to those described in a number of other frameworks (Department for International Development, 2010; ESSENCE on Health Research, 2011; Ghaffar et al, 2008; Lansang & Dennis, 2004; Panel on Return on Investment in Health Research, 2009). However, our review of the literature suggested the need for some revisions to and expansion of Cooke’s framework. We added a fifth structural layer ( national research infrastructure/bodies ), modified two dimensions ( close to practice and dissemination to maximise impact ) and integrated two other dimensions ( leadership and empowerment ). Each of these is discussed below. Figure 2.1 presents the modified framework that we then used to guide the composite case study outlined in the final chapter of this book.

Figure 2.1 The integrated framework. This figure illustrates the elements/dimensions we added to Cooke’s (2005) framework as a result of our literature search.

Source: Adapted from Cooke (2005, p. 44)

The structural layer, national research infrastructure/bodies , was added to Cooke’s framework as there is growing international consensus that this is an essential system element needed to sustain research capacity (ESSENCE on Health Research, 2011; Ghaffar et al, 2008; Lansang & Dennis, 2004).

Within Cooke’s framework, the dimension close to practice is specific to clinical care. We broadened this dimension to cover other domains of health research, specifically health services, policy, and biomedical, clinical and population health. We have therefore expanded this dimension and called it research applicability , referring to the importance of making research relevant to the priorities and challenges of countries, organisations, institutions and key stakeholders.

Appropriate dissemination to maximise impact is another dimension in Cooke’s original framework, but this implies a one-way flow of information. Extensive literature in the field of knowledge translation indicates that a more complex set of push-and-pull strategies is necessary to maximise the uptake and impact of research in practice, programmes, and policies (Ginsburg, Lewis, Zackheim, & Casebeer, 2007; Lavis et al, 2012; Lomas, 2007). We therefore modified this dimension to appropriate dissemination and knowledge translation to maximise impact .

We added the dimension of leadership to the framework since it has been identified as ‘one of the key factors that facilitates research capacity building’ (Ghaffar et al, 2008, p. 55) and authors have described its pertinence at all system levels (Frenk et al, 2010). Leadership skills, such as servant leadership approaches (Jackson, 2008), are necessary at the individual and team level to enable senior researchers to support junior researchers and to champion the development of institutional supports for research (including protected time for research) and essential technical and human infrastructure, such as research space and information technology, financial and administrative policies and procedures, ethics review processes, and experienced research staff. At the organisational level, successful capacity building is dependent on dynamic leadership with a long-term vision. Competent and committed scientific leadership is needed for various aspects of the work, including fundraising; managing teams; developing institutional strategic research priorities that are mirrored in the recruitment of faculty, graduate students and postdoctoral fellows; and engaging with national and international research networks and decision-makers (Ghaffar et al, 2008; Nchinda, 2002). Leadership and management competence are also necessary for effective national health research systems (Lansang & Dennis, 2004). Political will and vision are required at the highest level of government (Bamako Call to Action on Research for Health, 2008), and competent leaders need to be stewards of the health research system.

Finally, we added the dimension of empowerment to the framework. Empowerment is the process by which all stakeholders are given the necessary tools to engage in and undertake effective and relevant research (Farmer & Weston, 2002; Lansang & Dennis, 2004; Rifkin, 2003). It is particularly relevant to research capacity building within LMICs because it can help shift the power dynamics and subsequently the control over decisions about resources when research funds are externally driven (Goldberg & Bryant, 2012; Rifkin, 2003). Thus, the addition of empowerment to the framework emphasises the need to give LMIC researchers and research users the power to prioritise context-relevant approaches to capacity building, and to use and mobilise fully their skills, influence, networks and resources to get things done. This is an essential dimension to influence both shorter and longer term change.

Recommended approaches for effective research capacity building

In line with the modified version of Cooke’s framework, a number of approaches that are critical for effective research capacity building are described in the literature. These include assessing capacity-building needs and gaps, involving stakeholders at the outset and throughout the entire process, and using systems-oriented and participatory approaches to build partnerships. In order to be effective, research capacity-building initiatives have to start with assessing existing strengths, gaps and needs at individual and organisational levels (Cooke, 2005; Department for International Development (DFID), 2010; UNDP, 2010). Interventions can then appropriately build on those strengths and local expertise to tailor capacity-building strategies to particular contexts (Baillie, Bjarnholt, Gruber, & Hughes, 2009; Department for International Development, 2010; ESSENCE on Health Research, 2011).

The United Nations Development Programme (UNDP) (2010) describes capacity assessment as ‘an analysis of desired capacities against existing capacities, which generates an understanding of capacity assets and needs that informs the formulation of a capacity development response’ (p. 2). DFID (2010) proposes a four-step approach to research capacity building which entails capacity assessment, strategising and planning, implementation and monitoring and evaluation. The assessment process involves an analysis and identification of strengths and weaknesses at the individual, institutional and organisational levels. DFID suggests conducting a skills audit to identify knowledge gaps and the quality of research produced in order to identify training needs.

Capacity assessment at the organisational level should focus on systems, resources and core capabilities, and should be undertaken collaboratively by engaging all key partners (DFID, 2010). Engaging stakeholders in consultative processes about capacity needs has been identified as key to the success of interventions (Cooke, 2005; Ghaffar et al, 2008). However, conducting a detailed assessment, particularly in resource-poor settings, may prove a big challenge depending on the context, the diversity of stakeholders, their interests and available resources (Goldberg & Bryant, 2012).

Both Cooke’s framework (2005) and the ESSENCE document on health research (2011) highlight the importance of analysing contextual factors such as socio-cultural norms and practices within organisations, as well as the broader socio-political and policy contexts, in order to understand the opportunities for and barriers to capacity building. The authors suggest that identifying critical gaps in research capacity at the design phase of a project or activity will enable those gaps to be addressed. According to the Department for International Development (2010), an understanding of existing capacities and the contextual factors that might influence the capacity-building process will facilitate needs-based priority-setting for research, build on existing skills and experiences, and form a basis for monitoring and evaluation. Morgan (2005) proposes a situation-specific contextual analysis and emphasises the importance of recognising the capacity to adapt to change and innovation.

Consistent with the principles articulated in the Paris Declaration on Aid Effectiveness in 2005 and the Accra Agenda for Action in 2008, Marjanovic et al (2013) suggest the alignment of capacity-building initiatives with the researchers’ experiences, interests and capacity-building needs, and with institutional research agendas and regional health priorities. They also note that identifying capacity-building priorities with all partners at the onset, through a consultative process, contributes to early stakeholder buy-in. Bates et al (2011) suggest a phased approach to capacity building and emphasise engaging stakeholders from the start. Similarly, a report by ESSENCE on Health Research (2011) concludes that effective capacity building requires approaches which align with local, national and institutional research priorities.

Effective capacity building also requires enabling approaches and processes which support systems learning (Morgan, 2005; ESSENCE on Health Research, 2011). Lansang and Dennis (2004) suggest that a combination of short- and long-term, multi-level strategies will contribute to a sustainable system of health research in LMICs. Additionally, adequate human resources in number, type, skill levels and mix are key components of research capacity-building processes. They propose the use of joint approaches such as co-supervising and co-leading graduate/postgraduate training, and establishing institutional partnerships between developing and developed countries or linking centres of excellence in different LMIC countries.

According to Lansang and Dennis (2004), an effective health systems research entity in developing countries requires ‘competent indigenous scientists and a strongly supportive and enabling environment that will allow research communities to grow’ (p. 764). Therefore, training programmes for research users, such as policy-makers, programme managers, healthcare practitioners and civil society, are necessary. This kind of activity would foster knowledge sharing and contribute to developing a critical mass of researchers and stakeholders, creating an environment in which the producers and the users of research can learn together and contribute to the development of a health research agenda (Hoffman et al, 2012).

Capacity building occurs through inter-relationships and interactions (Morgan, 2005). Processes involving feedback and mentoring, information sharing and knowledge management are critical. Lansang and Dennis (2004) highlight the development of partnerships between developing and developed countries as key to reducing global inequities and facilitating mutual access to new ideas, technical expertise, best practices and resources. They recommend learning-by-doing approaches, supported by developmental or seed grants and hands-on training provided through ongoing mentorship programmes that complement individual training. Participatory approaches can facilitate the development of partnerships and networks, by engendering a commitment to learning and ultimately the ownership of newly acquired knowledge, skills and experiences. Furthermore, participatory processes enable participants to learn experientially, share knowledge and ideas, and work together toward common learning goals. These approaches may also empower participants to analyse their own situation critically, identify factors that hinder or facilitate their capacity and take steps to improve it (Neilson & Lusthaus, 2007). Flexibility is necessary so that differences in learning styles and pace can be acknowledged and taken into account to build on the group’s existing knowledge and experiences.

K EY C HALLENGES

Monitoring, measuring and evaluating research capacity building is necessary to determine progress, examine intervention effectiveness and assess whether the investments in capacity building are producing the desired outcomes (Bates et al, 2011; Cooke, 2005; LaFond & Brown, 2003; Wellcome Trust, 2013). But there are some critical challenges in this field.

Lack of consensus on the concept(s) and definition of research capacity has adversely affected the development of meaningful assessment, monitoring and evaluation measures (LaFond & Brown, 2003). In part, this may account for the lack of valid and reliable measures to assess capacity development. In addition to these concerns and constraints, Hughes, Black and Kennedy (2008) identified two factors which have affected research in this field:

1)   the context-specific nature of tailored research capacity-building interventions and the limitations this poses on the external generalisability of findings

2)   the inherent complexity and multidimensional nature of capacity-building strategies.

Both these factors make it difficult to attribute successful elements to specific interventions.

Lack of research funding, particularly in LMICs, is one of the main issues adversely affecting the development of strong and active national research systems (Lansang & Dennis, 2004). Many researchers and organisations based in LMICs seek funding from international agencies based in higher-income countries because of the lack of local funding. However, the dependency of LMICs on external funding from donors may result in short-term, intermittent and fragmented capacity-building initiatives. For highly trained researchers, the lack of local funding may limit their career options and encourage them to seek academic posts outside their country, contributing to a brain drain (Lansang & Dennis, 2004). This in turn jeopardises institutional capacity and local mentorship and supervisory support for trainees (ESSENCE, 2014). LMIC’s dependence on external donors also means that donors have an undue influence on setting agendas, which may not reflect the health research needs and priorities of the recipient countries. The research findings may neither respond to national priority issues nor be valued, leading to low demand for research generation (Ghaffar et al, 2008). South—South partnerships among LMICs have been described as one means to address some of these issues (ESSENCE, 2011).

A third challenge concerns the lack of recognition and tenuous career paths for researchers in developing countries (Cooke, 2005; Ghaffar et al, 2008; Segrott, McIvor & Green, 2006). Uneven opportunities for graduate training in certain fields and the lack of funding opportunities for postgraduate training have also been cited as research career barriers (Edwards et al, 2009; Segrott et al, 2006). For those who have completed graduate preparation, low salaries, lack of status and limited opportunities for career growth have been described as key factors that undermine research capacity building. Proactive retention strategies for experienced researchers in LMICs are badly needed (Ghaffar et al, 2008).

Research infrastructure constraints have been described as another important challenge in many LMIC settings. Regardless of the discipline, the growing digital divide is of particular concern. While fundamental global changes to the dissemination of knowledge, including publishing in open access journals, are helping to make research more accessible, access is still dependent on critical intermediate technology and related supports such as computers, regular internet providers and technical support, and electricity. This digital divide has many direct and indirect consequences. It may adversely affect the quality of research applications submitted by those in LMICs and it influences the scope and scale of research networks. Other infrastructure constraints in LMICs are the lack of laboratories, modern research equipment and office space in which to accommodate researchers. These deficiencies may hinder research productivity and influence the quality of research produced in LMICs.

C ONCLUSION

The importance of a systems approach to research capacity building for health research in LMICs has been well articulated. Numerous challenges must be overcome to make significant gains in this arena. Several multi-level and systems-oriented frameworks for research capacity building exist. We have proposed modifications to Cooke’s existing framework, integrating many of the research capacity-building concepts we have identified in the literature. The following chapters illustrate how the Teasdale-Corti teams operationalised and assessed various dimensions of capacity building and the challenges they faced. The final chapter uses our amended framework as a basis for identifying the indicators to assess capacity building across a composite case study by the Teasdale-Corti teams.

A scoping review explores definitions, approaches and frameworks for research capacity building that are pertinent to LMICs. Detailed methods for the review are provided in Appendix A .

The definitions of research capacity building make reference to its multi-level and multidimensional structure, stressing the importance of local ownership of capacity-building initiatives. Capacity building is described as iterative; context-specific, involving empowerment processes for individuals, organisations and nations; and encompassing knowledge production and use. It is aimed at improved performance and efficiency, and better problem solving to meet goals and facilitate change.

There has been a pronounced shift in concepts of capacity building from those targeting individuals to those with a systems orientation.

Cooke’s framework for research capacity building is introduced. It contains six dimensions which are featured in case studies in later chapters:

•   building skills and confidence ( Chapters 3 , 5 , and 7 )

•   ensuring research is close to practice ( Chapters 4 and 7 )

•   developing partnerships and linkages ( Chapters 3 , 6 , 7 and 8 )

•   supporting appropriate dissemination ( Chapters 7 , 8 and 9 )

•   including elements of continuity and sustainability ( Chapters 6 and 9 )

•   making investments in infrastructure to enhance research capacity building ( Chapter 6 ).

Our changes to Cooke’s framework include adding a fifth structural layer (‘national research infrastructure/bodies’), and modifying two dimensions (‘close to practice’ and ‘dissemination to maximise impact’). We also integrate two new dimensions (leadership and empowerment). Examples of these dimensions appear in most of the case studies.

The following key challenges to research capacity building in LMICs are identified:

•   few valid and reliable measures to assess the complex entity of capacity development and the context-specific nature of tailored research capacity-building interventions (see Chapter 10 for a discussion of indicators)

•   lack of local research funding, coupled with a dependence on external donors (see Chapters 3 , 4 and 6 for examples of innovations in funding)

•   lack of recognition and tenuous career paths for researchers and the need for proactive retention strategies for experienced researchers in LMICs (see Chapters 5 and 6 for a discussion of approaches used)

•   research infrastructure constraints and the digital divide (see chapters 6 and 8 ).

Baillie, E., Bjarnholt, C., Gruber, M., & Hughes, R. (2009). A capacity building concept framework for public health nutrition practice. Public Health Nutrition , 12 (8):1031–1038.

Baird, A., & Baron, S. (2009, April). Research capacity building: A Scottish model . Paper presented at the American Educational Research Association, San Diego, CA.

Bamako Call to Action on Research for Health: Strengthening research for health, development, and equity. (2008, November 17–19). Paper presented at the Global Ministerial Forum on Research for Health, Bamako, Mali. Available from: http://www.who.int/rpc/news/BAMAKOCALLTOACTIONFinalNov24.pdf (Accessed 24 May 2016).

Bates, I., Akoto, A.Y.O., Ansong, D., Karikari, P., Bedu-Addo, G., Critchley, J., & Nsiah Asare, A. (2006). Evaluating health research capacity building: An evidence-based tool. PLoS Medicine , 3 (8):e299. doi: 10.1371/journal.pmed.0030299.

Bates, I., Taegtmeyer, M., Squire, B., Ansong, D., Nhlema-Simwaka, B., Baba, A., & Theobald, S. (2011). Indicators of sustainable capacity building for health research: Analysis of four African case studies. Health Research Policy and Systems , 9 (14).

Bennett, S., Agyepong, I., Sheikh, K., Hanson, K., Ssengoodba, F., & Gilson, L. (2011). Building the field of health policy and systems research: An agenda for action. PLoS Medicine , 8 (8).

Bordage, G. (2009) Conceptual frameworks to illuminate and magnify. Medical Education , 43 (4):312–319.

Chambers, D., Glasgow, R.E., & Stange, K.C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science , 8 (1):117.

Commission on Health Research for Development. (1990). Health research: Essential link to equity in development . New York, NY: Oxford University Press. Available from: http://www.cohred.org/downloads/open_archive/ComReports_0.pdf (Accessed 24 May 2016).

Cooke, J. (2005). A framework to evaluate research capacity building in health care. BMC Family Practice , 6:44.

Department for International Development. (2010, June). How to note: Research capacity building . A DFID practice paper. Available from: http://www.gov.uk/government/uploads/system/uploads/attachment-data/file/187568/HTN_Capacity_Building_Fianl_21_06_10.pdf (Accessed 24 May 2016).

Ebbesen, L.S., Heath, S., Naylor, P., & Anderson, D. (2004). Issues in measuring health promotion capacity in Canada: A multi-province perspective. Health Promotion International , 19 (1):85–94.

Edwards, N., Webber, J., Mill, J., Kahwa, E., & Roelofs, S. (2009). Building capacity for nurse-led research. International Nursing Review , 56: 88–94.

ESSENCE on Health Research. (2011). Planning, monitoring and evaluation: Framework for capacity strengthening in health research . ESSENCE Good Practice document series. Available from: http://www.who.int/tdr/publications/non-tdr-publications-essence-frameworks-on/ (Accessed 24 May 2016)

ESSENCE on Health Research. (2014). Seven principles for strengthening research capacity in low- and middle-income countries: Simple ideas in a complex world . ESSENCE Good Practice document series. Available from: http://www.who.int/tdr/publications/Essence_report2014_OK.pdf (Accessed 24 May 2016).

Farmer, E., & Weston, K. (2002). A conceptual model for capacity building in Australian primary health care research. Australian Family Physician , 21 (12):1139–1142.

Finch, J. (2003). Commentary: A look at the bigger picture in building research capacity. Nursing Times Research , 8 (6):427–428.

Frenk, J., & Chen, L. (2011). Overcoming gaps to advance global health equity: A symposium on new directions for research. Health Research Policy & Systems , 9:11.

Frenk, J., Chen, L., Bhutta, Z.A., Cohen, J., Crisp, N., Evans, T., & Zurayk, H. (2010). Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. The Lancet , 376:1923–1958.

Gadsby, E.W. (2011). Research capacity strengthening: donor approaches to improving and assessing its impact in low- and middle-income countries. International Journal of Planning and Management , 26 (1):89–106.

Ghaffar, A., IJsselmuiden, C., & Zicker F. (Eds.). (2008). Changing mindsets: Research capacity strengthening in low and middle income countries . COHRED, Global Forum for Health Research, and UNICEF/UNDP/World Bank/WHO Special Program for Research and Training in Tropical Diseases (TDR) Report. Geneva: WHO Press. Available from: http://www.cohred.org/downloads/cohred_publications/Changing_Mindsets.pdf (Accessed 24 May 2016).

Ginsburg, L.R., Lewis, S., Zackheim, L., & Casebeer, A. (2007). Revisiting interaction in knowledge translation. Implementation Science , 2 (34).

Global Forum for Health Research. (2004). The 10/90 report on health research, 2003–2004 . Geneva, Switzerland: Global Forum for Health Research. Available from: http://announcementsfiles.cohred.org/gfhr_pub/assoc/s14789e/s14789e.pdf (Accessed 24 May 2016).

Goldberg, J., & Bryant M. (2012). Country ownership and capacity building: The next buzzwords in health systems strengthening or a truly new approach to development? BMC Public Health , 12:531.

Golenko, X., Pager, S., & Holden, L. (2012). A thematic analysis of the role of the organization in building allied health research capacity: A senior managers’ perspective. BMC Health Services Research , 12:276.

Grimshaw, J.M., Eccles, M.P., Lavis, J.N., Hill, S.J., & Squires, J.E. (2012). Knowledge translation of research findings. Implementation Science , 7:50.

Hailey, J., & James, R. (2003, November). NGO capacity building: The challenge of impact assessment . Paper presented at the New Directions in Impact Assessment for Development Methods & Practice Conference. Available from: http://portals.wi.wur.nl/files/docs/ppme/Hailey.pdf (Accessed 24 May 2016).

Higginbotham, N. (1994). Capacity building for health social science: The International Clinical Network (INCLEN) social science program and the International Forum for Social Science in Health (IFSSH). Acta Tropica , 57 (2–3):123–137.

Hoffman, S.J., Røttingen, J.A., Bennett, S., Lavis, J.N., Edge A.S., & Frenk, J. (2012). Background paper on conceptual issues related to health systems research to inform a WHO global strategy on health systems research . Available from: http://www.who.int/alliance-hpsr/alliancehpsr_backgroundpaperhsrstrat1.pdf (Accessed 24 May 2016).

Hughes, R., Black, C., & Kennedy, N.P. (2008). Public health nutrition intervention management: Capacity assessment . JobNut Project, Trinity College Dublin. Available from: https://www.researchgate.net/profile/Roger_Hughes/publication/8947020_Public_health_nutrition_workforce_composition_core_functions_competencies_and_capacity_perspectives_of_advanced-level_practitioners_in_Australia/links/53d787c10cf29265323ccbf2.pdf (Accessed 24 May 2016).

Interacademy Medical Panel. (2013). A call for action to strengthen health research capacity in low and middle income countries . Available from: http://www.iamp-online.org/sites/iamp-online.org/files/IAMP%20Call%20for%20Action%20on%20RSC%20.pdf (Accessed 24 May 2016).

Jackson, D. (2008). Servant leadership in nursing: A framework for developing sustainable research capacity in nursing. Collegian (Royal College of Nursing, Australia) , 15 (1):27–33.

Kilama, W.L. (2009). The 10/90 gap in sub-Saharan Africa: Resolving inequities in health research. Acta Trop , 112 (S1):S8–S15.

LaFond, A., & Brown, L. (2003). A guide to monitoring and evaluation of capacity building interventions in the health sector in developing countries. MEASURE Evaluation Manual Series , ( No. 7 ). Available from: http://gametlibrary.worldbank.org/FILES/610_M&E%20of%20Capacity%20Building%20Interventions.pdf (Accessed 24 May 2016).

Lansang, M.A., & Dennis, R. (2004). Building capacity in health research in the developing world. Bulletin of World Health Organization , 82 (10):764–770.

Lavis, J.N., Røttingen, J.A., Bosch-Capblanch, X., Atun, R., El-Jardali, F., Gilson, L., & Haines, A. (2012). Guidance for evidence-informed policies about health systems: 2) Linking guidance development to policy development. PLoS Medicine , 9 (3): e1001186.

Lomas, J. (2007). The in-between world of knowledge brokering. British Medical Journal , 334 (7585):129–132.

Luoto, J., Shekelle, P.G., Maglione, M., Johnsen, B., & Perry, T. (2014). Reporting of context and implementation in studies of global health interventions: A pilot study. Implementation Science , 9 (1):57.

Mahmood, S., Holt, K., Ahmed, S., Salam, M., & Cravioto, A. (2011). Strategies for capacity building for health research in Bangladesh : Role of core funding and a common monitoring and evaluation framework. Health Research Policy and Systems , 9:31.

Marjanovic, S., Hanlin, R., Diepeveen, S., & Chataway, J. (2013). Research capacity building in Africa: Networks, institutions and local ownership. Journal of International Development , 25 (7):936–946.

Miller, J., Bryant-MacLean, L., Coward, P., & Broemeling, A-M. (2009). Developing strategies to enhance health services research capacity in a predominantly rural Canadian health authority. The International Electronic Journal of Rural and Remote Health, Research, Education: Practice and Policy . 9:1266.

Mirzoev, T., Le, G., Green, A., Orgill, M., Komba, A., Esena, R., & Gilson, L. (2013). Assessment of capacity for health policy and systems research and analysis in seven African universities: Results from the CHEPSSA project. Health Policy and Planning , 1–11.

Morgan, P. (1997). The design and use of capacity development indicators . Paper prepared for the policy branch of CIDA, December 1997. Available from: http://www.oecd.org/development/governance-development/1919953.pdf (Not available).

Morgan, P. (2005). The idea of practice of systems thinking and their relevance for capacity development. European Centre for Development Policy Management. Available from: http://portals.wi.wur.nl/files/docs/SPICAD/13.%20Idea%20and%20practice%20of%20systems%20thinking%20(ECDPM).pdf (Accessed 24 May 2016).

Nchinda, T.C. (2002). Research capacity strengthening in the South. Social Science & Medicine , 54 (11):1699–1711.

Neilson, S., & Lusthaus, C. (2007, February). IDRC-supported capacity building: Developing a framework for capturing capacity changes. Universalia . Available from: http://https://idl-bnc.idrc.ca/dspace/bitstream/10625/29146/1/125252.pdf (Accessed 24 May 2016).

New South Wales Health. (2001). A framework for building capacity to improve health . North Sydney, Australia: New South Wales Health Department. Available from: http://www.redaware.org.au/wp-content/uploads/2014/07/A-Framework-for-Building-Capacity-to-Improve-Health.pdf (Accessed 24 May 2016).

Nuyens, Y. (2005). No development without research: A challenge for research capacity strengthening . Geneva, Switzerland: Global Forum for Health Research. Available from: http://announcementsfiles.cohred.org/gfhr_pub/assoc/s14828e/s14828e.pdf (Accessed 24 May 2016).

Orton, L., Lloyd-Williams, F., Taylor-Robinson, D., O`Flaherty, M., & Capewell, S. (2011). The use of research evidence in public health decision making processes: Systematic review. PLoS ONE , 6 (7):e21704.

Panel on Return on Investment in Health Research. (2009). Making an impact: A preferred framework and indicators to measure returns on investment in health research . Ottawa, ON: Canadian Academy of Health Sciences. Available from: http://www.cahs-acss.ca/wp-content/uploads/2011/09/ROI_FullReport.pdf (Accessed 24 May 2016).

Paris Declaration on Aid Effectiveness & the Accra Agenda for Action. (2005/2008). Available from: http://www.oecd.org/development/effectiveness/34428351.pdf (Accessed 24 May 2016).

Parry, G.J., Carson-Stevens, A., Luff, D.F., McPherson, M.E., & Goldmann, D.A. (2013). Recommendations for evaluation of health care improvement initiatives. Academic Pediatrics , 13 (6):S23–S30.

Rifkin, S.B. (2003). A framework linking community empowerment and health equity: It is a matter of CHOICE. Journal of Health, Population & Nutrition , 21(3):168–180.

Sacco, R.L., Smith, S.C., Holmes, D., Shurin, S., Brawlet, O., Cazap, E., & Varmus, H. (2011). Accelerating process on non-communicable disease. The Lancet , 382 (9895):e4–5.

Saxena, S., Paraje, G., Sharan, P., Karam, G., & Sadana, R. (2006). The 10/90 divide in mental health research: Trends over a 10-year period. The British Journal of Psychiatry , 188:81–82.

Segrott, J., McIvor, M., & Green, B. (2006). Challenges and strategies in developing nursing research capacity: A review of the literature. International Journal of Nursing Studies , 43 (5):637–651.

Thornicroft, G., Cooper, S., Van Bortel, T., Kakuma, R., & Lund, C. (2012). Capacity building in global mental health research. Harvard Review of Psychiatry , 20:13–24.

Trostle, J. (1992). Research capacity building and international health: Definitions, evaluations and strategies for success. Social Science and Medicine , 35:1321–1324.

United Nations Development Programme (UNDP). (1998). Capacity assessment and development in a systems and strategic management context . Management Development and Governance Division Bureau for Development Policy. Technical Advisory Paper (No.3). Available from: https://www.cbd.int/doc/pa/tools/Capacity%20assessment%20and%20development.pdf (Accessed 24 May 2016).

United Nations Development Programme (UNDP). (2010). Capacity development: Measuring capacity . Available from: file: www.undp.org/content/undp/en/home/librarypage/capacity-building/undp-paper-on-measuring-capacity.html (Accessed 24 May 2016).

Vogel, I. (2011). Research capacity strengthening: Learning from experience . Report from the UK Collaborative on Development series. Available from: http://www.ukcds.org.uk/sites/default/files/content/resources/UKCDS_Capacity_Building_Report_July_2012.pdf (Accessed 24 May 2016).

Wellcome Trust. (2013). Population and public health, 1990–2011: Portfolio review . Available from: http://www.wellcome.ac.uk/stellent/groups/corporatesite/@policy_communications/documents/web_document/WTP052282.pdf (Accessed 24 May 2016).

White, F. (2002). Capacity building for health research in developing countries: A manager’s approach. Rec Panam Salud Publica/Pan American Journal of Public Health , 12 (3):165–172.

Whitworth, A., Haining, S., & Stringer, H. (2012). Enhancing research capacity across health care and higher education sectors: Development and evaluation of an integrated model. BMC Health Services Research , 12:287.

Whitworth, J.A.G., Kokwaro, G., Kinyanjui S., Snewin, V.A., Tanner, M., Walport, M., & Sewankambo, N. (2008). Strengthening capacity for health research in Africa. The Lancet , 372 (9649):1590–1593.

WHO Evidence Informed Policy Network (EVIP). (2012). 2012–2015 strategic plan . Geneva: WHO Press. Available from: http://www.who.int/evidence/OMSEVIPNetStratPlan.pdf?ua=1 (Accessed 24 May 2016).

WHO-TDR (Special Programme for Research and Training in Tropical Disease TDR)). (2011). Implementation research for the control of infectious diseases of poverty: Strengthening the evidence base for the access and delivery of new and improved tools, strategies, and interventions . Geneva: WHO Press. Available from: http://www.who.int/tdr/publications/documents/access_report.pdf?ua=1 (Accessed 24 May 2016).

World Bank. (1998). Assessing aid: What works, what doesn’t, and why . Oxford: Oxford University Press.

World Bank Institute. (2009). The capacity development results framework: A strategic and results-oriented approach to learning for capacity development . Washington, DC: World Bank.

World Health Organization (WHO). (2005). WHO Global strategy for containment of antimicrobial resistance . Geneva: WHO Press. Available from: http://www.who.int/drugresistance/WHO_Global_Strategy_English.pdf?ua=1 (Accessed 24 May 2016).

World Health Organization (WHO). (2011). Global status report on noncommunicable diseases 2010 . Geneva: WHO Press. Available from: http://www.who.int/chp/ncd_global_status_report/en/ (Accessed 24 May 2016).

Yamey, G. (2012). What are the barriers to scaling up health interventions in low and middle income countries? A qualitative study of academic leaders in implementation science. Globalization and Health , 8 (11). Available from: http://www.globalizationandhealth.com/content/8/1/11 (Accessed 24 May 2016).

Chapter 3 Improving ‘Research-to-Action’ through Research Triads

Ronald Labonté, Corinne Packer, Nikki Schaay & David Sanders

The Revitalizing Health for All: Towards Comprehensive Primary Healthcare (RHFA) project began in 2007 as an innovative health study formulated and implemented by approximately 50 collaborators in some 20 countries throughout Africa, Asia, Latin America, Europe, North America and Australia. The project was concluded in July 2011. The principal goal of the RHFA project was to enhance the capacities of researchers to generate and use new comprehensive primary healthcare (CPHC) research knowledge for policy and programme change in specific country contexts. As outlined in the Alma Ata Declaration (World Health Organization & UNICEF, 1978), comprehensive primary healthcare (CPHC) not only combines first-line medical and allied healthcare workers offering a range of care from prevention to treatment but also includes other elements such as equity of access, collaboration across sectors beyond health, and community empowerment and participation in the services made available. The importance of primary healthcare was reaffirmed on the 30th anniversary of the Declaration in the WHO World Health Report 2008 (World Health Organization, 2008) and a special issue of The Lancet was devoted to the theme ( The Lancet , 2008).

From the outset, the RHFA project aimed to improve ‘research-to-action’. To do so, 20 research teams, from 16 countries and four different regions, were accepted into the RHFA project to conduct new studies on CPHC and to participate in annual capacity-enhancing workshops in all aspects of such studies facilitated by RHFA project faculty. Uniquely, all research projects had to be comprised of triads: early career researchers, research users and senior research mentors. By establishing such mixed research teams, it was anticipated that the research undertaken would better suit the needs and interests of the research user, would find stakeholder support throughout the data collection process, and would increase the likelihood of uptake or implementation of the research findings. No explicit theory for the use of triads was used in this research study. Rather, the concept of triads was developed from the two co-principal applicants’ experiences in enhancing research capacities in low-resource settings (such as many low-income countries), where the role of research mentors is important. In the several community/university research partnerships undertaken by Labonté in the 1990s and early 2000s, the engagement of research users throughout all stages of research activity (including design of questions, methods, analyses, and final write-up and publication) proved successful in developing policy responsiveness to the study findings.

The researcher in each team was an individual at the early stage of his or her career and was assigned to manage, facilitate and implement the research project. The research user was a health practitioner, manager or policy-maker in a mid- or senior-level position. Finally, the mentor was an experienced researcher who was required to help develop and nurture the collaboration between the early career researchers and research users. All three members of the team had to submit a letter signed by their institutions stating that they would be given the time to commit to this project and participate in the capacity-building and research process over a three-year period. Together, they would develop the research protocol, implement the study and disseminate the findings.

Our funded mixed teams conducted regular self-assessments of the effectiveness of their partnerships in order to advance their projects’ potential to transform research into practice. This process began with a structured exercise during the second round of regional training sessions with the focus on some of the key elements of a ‘knowledge translation and exchange’ (KTE) process:

•    Context evaluation: what problems and conditions are most appropriate to research/research-user engagement? Context evaluation assumes that some research may not be adopted by research users, either for political or potential bias reasons, or due to time and financial constraints. Although this was discussed among the groups, we were confident that the process by which we had elicited expressions of interest for research triads, and the time spent refining research questions and approaches during the first regional meeting, had already ensured that the problems they were studying were a good fit for such engagement.

•    Process evaluation: how can the researcher/research-user relationship be best developed and conducted? The process evaluation addressed such issues as trust, reciprocity/benefit sharing, confidentiality, day-to-day research task management, data analysis and interpretation, ‘ownership’ and use, publication and the means by which these relationship issues could be best managed by the teams. It also explored barriers and the organisational conditions that enable effective KTE experiences.

•    Outcome evaluation: what new or different changes arise among researchers/research users (and their institutions) as a result of the team/KTE experience? Outcome evaluation examined the perceived benefits to the parties of a KTE experience. These benefits could be in terms of new content knowledge gained; new appreciation for the demands faced by researchers, and by policy and programme decision-makers; new opportunities; an assessment of whether benefits offset costs; and other outcomes that directly affect the individuals and their organisations participating in the KTE experience.

•    Impact evaluation: how did the researcher/research-user partnership and overall KTE experience improve the potential to achieve greater health equity within and between population groups? Impact evaluation in our study consisted of a final reflection by research teams on what they anticipated in new policy or programme changes based upon their strategies to: 1) locate their own study within a broader research literature, and 2) share the findings across a wide range of individuals, groups and organisations they had ‘mapped’ (described later) as being important to achieve the research-informed changes.

S ELECTING P ARTICIPATING T EAMS: T HE F IRST A SSESSMENT OF P OTENTIAL FOR C APACITY D EVELOPMENT

A ‘call for expressions of interest to participate in new research and research training in CPHC’ was broadly circulated in English and Spanish through the Teasdale-Corti CPHC network and relevant listservs, notably that of the People’s Health Movement. Among the criteria for submitting an expression of interest (EOI) was that each team had to be made up of three individuals:

1)   an early career researcher

2)   someone working in health systems development or implementing primary healthcare policies or programmes (ie, a research user)

3)   a mentor or more senior researcher with experience in research on CPHC, health systems, health and development or other related social development areas.

Eighty-five applications were received. These were separated by region and given to regional teams of evaluators along with a protocol for reviewing the EOIs, which included a scoring chart. The protocol enabled evaluators to score the EOIs on 18 different criteria, which included the following:

•   whether the research-user team member had a mid- or senior-level position capable of ensuring relevance and uptake of the research findings

•   the effectiveness of the proposed strategy for working with a broader network of research users to ensure relevance and uptake of the research findings

•   the potential for research activity to build the research capacity of early career researchers

•   the degree of engagement of research triad members with civil society groups active in promoting CPHC reforms/programmes within their community or country context.

These criteria ensured that teams had the fundamental capacities to engage critically with the research, and to enhance the research/research-user relationship as the study progressed. In the end, 20 teams were selected for participation in the FH (RHFA) project. Two extra teams based in Ecuador and Nicaragua participated in the regional training sessions in Latin America at their own cost throughout the life of the project. Another team in Asia (Gonoshasthaya Kendra in Bangladesh) also participated in the regional training sessions in Asia, at their own cost, although extra funding was sought and received from IDRC to cover the costs of carrying out their research.

M ENTORING G UIDELINES

Given the important role of the mentors in the research partnerships, the RHFA project staff researched existing guidelines on mentoring and developed its own set of mentoring guidelines. These guidelines were delivered and discussed with the mentors at the end of the first round of regional training sessions (see Box 3.1 ).

Box 3.1 Guidelines for Mentors

As mentors to teams of researchers and research users formulating and implementing research projects on CPHC, you hold knowledge and experience which will accelerate their process of learning. The main features of mentoring are that you help the researcher and research user (hereafter referred to as the ‘team’):

•   think critically

•   think realistically (ie, what can realistically be done in the parameters of time, funds and resources at hand)

•   think ethically (keep the team thinking about the community intended to benefit from the research; ensure the community is involved)

•   think strategically (eg, how will the ideals, questions and findings studied fit into the policy agenda of decision-makers?).

Over the course of the research, mentors should plan to have regular meetings with their teams either in person (if the stipend budget enables such meetings) or virtually.

The mentor’s principal role is to provide guidance and support to the team and ensure that it is undertaking the research according to schedule and budget. Sometimes, unexpected challenges arise threatening the development and implementation of the research. It is at this time that teams need the extra support which comes from the experience and wisdom of mentors.

Development of the research protocol phase

During this phase of the research, mentors can assist by

•   guiding the team to apply what they learned in the regional training in developing a successful research protocol

•   ensuring the researcher and research user are working together

•   challenging the team to stick to their schedule

•   ensuring the team is giving appropriate consideration to the community they are researching

•   ensuring the team seeks ethics approval from the relevant body (note that continued funding for research under the RHFA grant is incumbent on obtaining ethics approval by the relevant domestic body)

•   ensuring compliance with the terms of institutional ethics approval for carrying out the research.

Research and data analysis phase

•   checking that the team is sticking to their schedule

•   providing critical input when the team undertakes the literature review (eg, by suggesting literature to review or advising on the review process)

•   checking on the quality of the research process

•   being ‘on call’ to provide advice, particularly when challenges arise

•   providing guidance on preliminary data analysis.

Knowledge translation phase

The mentors will assist the team to

•   focus on how best to undertake knowledge translation

•   present the data in a manner which suits the audience

•   identify where best to publish or present their results

•   develop policy briefs

•   engage with policy-makers.

Other support throughout the research

Mentors can also help the team access other resources if needed to complete the research.

Communication with RHFA project managers and regional trainers

Mentors can contact any of the RHFA project managers or regional training coordinators for feedback or support.

M ETHODOLOGY

All team members participated in three annual training sessions, offered over three years, and representing over four weeks in total, in which aspects of research and policy development were explored. The curriculum at the first training session addressed the following:

•   understanding the history, theory and practice of CPHC

•   an overview of basic research and evaluation skills for proposed studies

•   revision and development of research protocols from original EOIs.

By the end of the training session, teams were expected to have collaboratively developed and finalised robust research protocols which were ready to be submitted for ethics approval by local academic institutions.

At the second training session, teams reviewed and analysed preliminary results from their field research. The facilitators also worked to help them develop preliminary knowledge, translation and exchange strategies. At the third and final training session, teams presented the final analyses of their research findings and discussed their publication strategies. The facilitators addressed how researchers could build up their skills in relation to dissemination of the research results (eg, through the media or in policy reviews).

Given the emphasis on ‘hands on’ learning and capacity enhancement that characterised the project, the triads were asked to assess the effectiveness of their partnership. At each training session, teams were given time to reflect on the ‘triad’ experience. They described the difficulties, expected and unexpected challenges, and successes they had encountered while working together as a team, as well as the strategies they had developed to overcome these problems.

A generic KTE and outcome mapping curriculum was adapted prior to the training and incorporated into each of the regional training sessions. The KTE curriculum drew extensively from the SUPPORT tools for evidence-informed health policy making (STP) resource (Oxman, Lavis, Lewin, & Fretheim, 2009a; 2009b), while the mapping exercise was adapted from the outcome mapping tool developed by IDRC (Earl, Carden, & Smuytlo, 2001).

Most of the work done by the triads in developing their dissemination strategy using these tools began in the second regional training session. This work included the creation of detailed charts and visual maps of those with whom they worked (‘boundary partners’) who would be in positions to change policies or programmes based upon their own study findings, and the larger context of the studies and literature reviews undertaken by the RHFA project as a whole. This work was revisited for discussion and updating at the third regional training session.

To help the core faculty and research triads understand how well this novel research team approach was working, teams were sent a list of questions (see Box 3.2 ) prior to the second regional training session and they were asked to prepare responses ahead of time. They were also given a small portion of time to prepare during the training sessions.

Box 3.2 Questions on Research / Research-User Partnerships

1. How did the triad work together during the research period (frequency of meeting, nature of meeting, or any other forms of interaction)?

2. Did the triad experience challenges working together?

3. Did their research/research-user partnership(s) bring forth benefits or barriers in preparing research protocols? If so, what were these?

4. How were both parties involved in the development of the research protocols and data collection?

5. What suggestions do you have for how the research/research-user partnership can be improved?

6. What were the major barriers or benefits arising from the researcher/research-user partnership(s) up to now?

7. What obstacles or opportunities from this researcher/research-user partnership might exist in the next phase of your research (ie, the data analysis and drafting of findings)?

8. How do you see such research partnerships contributing to the uptake of your findings by ‘boundary partners’ — and specifically in advocating for the implementation of more comprehensive approaches to primary healthcare?

9. Given this Teasdale-Corti experience, to what extent would you as a researcher or a mentor and a research user insist on an identical or similar type of arrangement in other/future research projects? Would there be anything that you would modify or change in terms of the arrangement that has existed within this project?

At the third (and final) regional training session, teams were asked to reflect on a similar set of questions that focused more closely on their experience in their triads. Answers to both sets of questions and the discussions that followed are summarised in the following section.

F EEDBACK FROM T EAMS

On the triad experience.

Overall, triad groups met quite frequently (monthly) and informally to discuss the scope of the research. Meetings were held in person, via e-mail, by teleconference, or by Skype or Vonage. Regular communication was upheld between the researcher and the research user. The frequency of the meetings was dependent on the need for discussions about specific issues at that time.

Working in a triad was phenomenal. I really feel that the arrangement is a revolutionary innovation and hope that it will be replicated in many research initiatives in the future. Working with a mentor and a research user made for rich and stimulating input while the research user added a practical dimension. We as researchers/academics have now forged a long-term relationship (researcher).

I really appreciate the ‘triad’ concept. This has made it possible to bring out the best – to hand hold the young researcher, utilise the wisdom of mentors and include the research user from the beginning to facilitate ownership of the research findings and recommendations. It was a good learning experience (research user).

It was lovely to work with (researcher) and see her growing confidence in writing and analysis over the course of the project (mentor).

On the challenges experienced

The biggest challenge reported was that of getting everyone together for physical meetings. Multiple teams also reported having difficulties with virtual meetings due to time differences, internet connectivity and speed, and the differing availability of participants. A few teams reported that they experienced no difficulties at all working together.

On benefits or barriers in preparing research protocols as triads

The positive reports included the advantage of research protocols already addressing the various possible approaches and interests. It was also stated that the joint (researcher/research user) preparation of research protocols played a role in quick and successful ethics approval, field-trip preparations and easy collaboration with universities.

As a research user — a total pragmatist and definitely not an academic — this has been a fascinating learning experience for me. I have always been a user of anecdotal information in project proposals and reports; I now see its place in a rigorous framework of a research protocol to bring data to life and communicate the living reality of some pretty graphs and tedious tables that are produced by quantitative methods. I love these mixed methods (research user).

The only barriers addressed were those of differing skill sets and interests.

On how the researcher/research-user partnership can be improved

Many suggestions were made, including the following:

•   Have more time for preliminary discussions to get a sense of research users’ needs.

•   Have regular interaction and updates.

•   Attempt a joint effort in publication (many teams have done this).

•   Have joint participation in international seminars and meetings (some teams have managed to do this).

•   Share the results of the study within both the researcher’s and the research user’s organisations for broader partnerships (all teams have accomplished this task).

•   Establish broader stakeholders within the country (eg, with domestic or international NGOs); this was partly accomplished through one or two day open events that took place during each regional training session.

•   Have joint presentations of the research (this was accomplished by several teams).

•   Make a link between the researcher and government body (policy-makers, local universities, etc) through the research user so that the research findings can be effectively disseminated and research findings can be translated into policy (this was achieved by many of the teams).

•   Better engage the research user in different stages of the study (this varied by the availability of the research user, but was a characteristic of all teams).

On the major benefits or barriers arising from the researcher/research-user partnership

The benefits that were reported include:

•   having an opportunity to work together and translate research findings into action

•   gaining access to relevant actors in government

•   gaining access to rich resources of government primary healthcare policy documents for review

•   gaining access to facilities, clinics and the like

•   assistance in overcoming complicated official bureaucracy.

As a researcher who was early in the field, the triad gave me the space to grow in leaps and bounds. It gave me the space to take a lead in the content direction of the research. It gave me the space to make fundamental decisions regarding the research process and most importantly, the triad showed confidence in me to present our work to the rest of the world (researcher).

Few barriers were reported, in some cases, as a result of differing interests between researchers and the research users.

On obstacles or opportunities arising from this form of partnership

The opportunities that were noted included the informal addition of research users, ie, although only one research user may have been formally indicated as a member of the team, that research user was able to draw in other research users. This allowed for better input and a strengthening of the projects.

Other opportunities were the learning and gaining of experience in performing qualitative research, which would help strengthen linkages within field sites. Partnerships in publication, and exploring further joint research, were other opportunities mentioned.

I would also like to thank you for the opportunity to participate throughout the process, and I would like to say that as a research user, the experience was very enriching. It was much more than just researching the answers to questions, I was able to review the vision of comprehensive primary care, its different concepts and understandings, in different contexts (research user).

Again, few obstacles were noted with the exception of difficult time factors and the complexity of data analysis and interpretation.

On how research partnerships contribute to the uptake of findings by ‘boundary partners’

The mapping exercise was instructive in that many teams began by listing every potential interested group, creating an impossibly large wish list. The requirement to name specific people to approach, how they would do so, and what they expected to achieve by this (from minimal response to idealised outcome) forced an important condensing of the impossible to the feasible. Triads reported that they saw their research partnerships contributing to successfully raising primary health care (PHC) issues with policy-makers/stakeholders in health, generating public debates on PHC, providing information to NGOs for advocacy for change, and to informing/influencing policy-makers so that policies may be reviewed or changed.

On overall lessons learned on research/research-user partnerships

Triads reported many lessons learned through their team experience. The following were recurring themes gathered over the course of the research, notably from the second and third regional training sessions; and then from the final global meeting of all of the research teams held in Ottawa in June 2011:

•   Have flexible research protocols to be able to adapt to field realities.

•   Ensure clear role definitions for members of the triad.

•   Ensure research users engage actively in the study itself, to increase their sense of ownership in the study and its findings.

•   Ensure researchers are engaged in policy making and implementation.

The feedback from the research groups on the triad experience was overwhelmingly positive. Most groups suggested that having individuals with different expertise enhanced the research relevance and usefulness as well as the ability to develop relationships and disseminate the research findings. The major negative factor appeared to be difficulty communicating between group members due to location differences and schedule conflicts.

The involvement of the research user was perceived as significantly increasing the opportunities and effectiveness of the programme for better relationships with both policy developers and local health workers. Many of the teams commented that the triad was a model that other researchers, government organisations, NGOs and communities could use to strengthen their research-to-action, and to bridge effectively the worlds of people engaged in policy and programme design and implementation, and those involved in generating research and evaluation evidence.

One interesting and unexpected finding from the overall experience of the triads was that: the distinction between researcher and research user was often artificial and, in some cases, was resented. ‘Research users’ were often competent researchers themselves and not passive consumers of research knowledge or subordinate members to the methodological expertise of researchers. Researchers, in turn, often had important insights into programme and policy work. In one case, there was actually a full swapping of roles within the research triad as the political context of the study changed: the original research user became the junior researcher, the junior researcher became the research mentor, and the research mentor became the research user. This in no way weakens the conceptual importance of establishing research triads, but cautions that care and sensitivity need to be invoked when assigning the different labels.

Earl, S., Carden, F., & Smuytlo, T. (2001). Outcome mapping: Building learning and reflection into development programs . Ottawa, ON: International Development Research Centre.

Oxman, A.D., Lavis, J.N., Lewin, S., & Fretheim, A. (2009a). SUPPORT tools for evidence informed health policymaking (STP) 1: What is evidence-informed policymaking? Health Research Policy and Systems , 7 (1):S1.

Oxman, A.D., Lavis, J.N., Lewin, S., & Fretheim, A. (2009b). SUPPORT tools for evidence-informed health policymaking (STP) 2: Improving how your organization supports the use of research evidence to inform policymaking. Health Research Policy and Systems , 7 (1):S2.

The Lancet . (2008). A renaissance in primary health care. The Lancet , 372 (9642):863.

World Health Organization. (2008). The World Health Report 2008. Primary Health Care – Now More Than Ever . Geneva: WHO Press.

World Health Organization, & UNICEF. (1978). Report of the International Conference on Primary Health Care, Alma-Ata, USSR, September 6–12, 1978 . Geneva: WHO Press.

Chapter 4 Developing Capacity through a Participatory Action Research Approach

Amy Bender, Nancy Edwards, Eulalia Kahwa & Dan Kaseje

Common sense suggests a natural fit between participatory action research (PAR) as a methodological approach and capacity development as an objective in international health research. This fit is explored more closely in this chapter, drawing from the experiences and outcomes of the Strengthening Nurses’ Capacity in HIV Policy Development in sub-Saharan Africa and the Caribbean programme (hereafter, Strengthening Nurses’ Capacity programme). This research collaboration aimed to promote dialogue and collaboration among health professionals and community members at all levels of the health system and integrate best practices for knowledge translation into nurses’ and midwives’ HIV/AIDS work in Jamaica, Kenya, South Africa and Uganda. To meet capacity-development objectives, nurses and midwives were supported to develop and carry through empirically based strategies for change at the practice and policy level of HIV programming in their respective countries through a PAR approach involving the establishment of ‘leadership hubs’. This chapter presents a critical analysis of PAR and how this approach in particular contributed to the development, functioning and contributions of these hubs in terms of building capacity.

The analysis is guided by Heron and Reason’s (1997) explanation of the participatory worldview, including concepts of critical intersubjectivity and collaboration. After a brief background of PAR and overview of this theoretical perspective, a description of the Strengthening Nurses’ Capacity programme is provided. The themes, generated from the comparative analysis of five cases (four being country-specific: Jamaica, Kenya, South Africa and Uganda; and one international), are then discussed: ‘power and empowerment’, ‘change’, ‘context and time’, and ‘values and assumptions’. The chapter concludes with identification of some key points from across these themes that compliment and extend Heron and Reason’s philosophical standpoint, particularly as one considers the international situation, when research projects span continents and involve economically and socio-politically diverse settings.

B ACKGROUND : PAR M ETHODOLOGIES AND C APACITY B UILDING IN H EALTH R ESEARCH

PAR is an approach to research that has its beginnings in the work of academics committed to using research for the explicit purpose of bringing about social change (Borkman & Schubert, 2009; Brydon-Miller, Greenwood & Maguire, 2003; Young, 2006). Some authors make distinctions between ‘participatory research’ and ‘action research’, but one is implicated in the other. The manifesto of the Action Research Journal (2012) declares that PAR is a reflexive process, oriented towards action and evolving in partnership and participation; that is, through ‘the quality of the relationships we form with primary stakeholders and the extent to which all stakeholders are appropriately involved in … the inquiry and change’ (p. 1). Bradbury-Huang (2010) argues that ‘theory without practice is not theory but speculation’ (p. 93). Researchers claiming a PAR approach are confronted with two challenges: is it truly participatory, and what constitutes action? For the first challenge, authors have raised such issues as structuring and defining the process of participation (Bell, Morse, & Shah, 2012; David, 2002); power relations outside the research team (Dworski-Riggs & Langhout, 2010); and seeing participation as work which requires compensation (Davidson & Page, 2012). The second challenge centres on issues of emancipatory practices and social justice (Heron, 1996; Reason & Bradbury-Huang, 2001); advocacy as action (David, 2002), and sustainability of change (Bell, Morse, & Shah, 2012). In short, the thrust of PAR ought to be toward making lasting change in the interest of a more equitable world. In this sense, PAR is an approach to knowledge production which brings evidence and evidence-based actions together as one research endeavour.

T HEORETICAL P ERSPECTIVE : A P ARTICIPATORY W ORLDVIEW

Our knowing of the world is consummated as our action in the world, and participatory research is thus essentially transformative (Heron & Reason, 1997, p. 288).

Heron and Reason (1997) describe the concepts of critical intersubjectivity and collaboration, and emphasise practical knowing in their articulation of the participatory worldview in research. A participatory worldview presupposes that human beings understand the world in at least four interdependent ways: experientially, presentationally, propositionally and practically (Heron, 1996; Heron & Reason, 1997; Reason, 1999). Experiential understanding refers to direct encounters with some person, place, thing, energy or process through which we come to ‘know’; the kind of knowing that is most difficult to put into language. Presentational knowing involves expressive imagery such as art forms, metaphor and symbols. Propositional knowing means knowing in conceptual terms, through statements and theories. Finally, knowing how to do something (competence and skill) is referred to as practical knowing , which is considered central since the previous three are made most apparent in action.

Critical subjectivity is an awareness of these four ways of knowing, how they intersect in the moment, and how the relations among them change. It means being self-reflexive about our experiences; using presentational and propositional knowing; to clarify or explain experiences in a way that informs practical knowing and leads to new understandings. Yet, this ‘critical consciousness’ of the world is not solitary; it develops partly by sharing experiences and dialoguing with others. In this way, subjectivity extends to critical intersubjectivity (Heron & Reason, 1997).

Research, as a form of knowledge creation, rests on critical intersubjectivity. Heron and Reason critique more traditional research paradigms for stopping short at practical knowing; they emphasise action as this approach’s overt commitment to social change. They suggest that finding an ‘enabling balance within and between people of hierarchy, cooperation, and autonomy’ can be an end in itself (1997, p. 287) and therefore stress the collaborative nature of good research.

Collaboration is a natural extension of the participatory worldview and guided by two key principles:

1)   All knowledge is participatory in nature: any propositional knowledge, including that of researchers and policy-makers, rests on experiential knowledge (ie, researchers are also subjects).

2)   Participation is political: people have a human right to participate in research being done about them, which calls for the democratisation of the research process (ie, subjects are also researchers) (Appadurai, 2006; Heron, 1996; Heron & Reason, 1997; Koch, Selim, & Kralik, 2002).

Collaboration is a form of this democratisation. It involves, as far as socio-structural realities will allow, ‘subjects’ engaging in the range of research issues and decisions where possible, and ‘researchers’ directly participating in the contexts under investigation as much as possible. This serves in meeting the participatory-action aim of conducting research with, rather than on, people and in sharing outcomes beyond the academic research community which reflect ‘creative action of people to address matters that are important to them’ and ‘enabling balance within and between people of hierarchy’ (Reason, 1999, p. 208). Heron and Reason’s emphasis on collaboration as an indicator of good research is found in the Strengthening Nurses’ Capacity programme’s hub membership intentionally designed so that various levels of the health system were represented. That is, hub members were engaging and collaborating in both ‘horizontal’ relationships (those with professional peers) and ‘vertical’ relationships (those that cross levels of authority in the healthcare system).

T HE S TRENGTHENING N URSES ’ C APACITY PROGRAMME

The capacity development objectives of the Strengthening Nurses’ Capacity programme addressed system, organisational, work team and individual capacities. With the focus being on HIV/AIDS, the programme comprised four intersecting research projects (Edwards et al, 2007). It was aimed at the following:

•   promoting ongoing, long-term, interactive dialogue and collaboration among various stakeholders

•   strengthening academic institutions’ and research-user groups’ capacity to engage in nurse-led relevant health research

•   integrating knowledge translation best practices into nurses’ work at senior levels

•   providing mentorship for nurses as they learned to conduct and use policy-related research.

The creation of ‘leadership hubs’ was the foundational strategy for meeting these objectives. These hubs were intended as collaborative work groups to bring about policy and practice change in the area of HIV/AIDS care and involved a mix of front-line and management nurses among other stakeholders. To further strengthen nurses’ links with decision-making higher up in the health system, national advisory committees were established in three of the countries. Specific individual and institutional capacity-focused efforts included policy workshops for hub members (topics determined jointly by senior researchers and hubs), teaching modules to be used by hub members in workplaces and other professional settings, communiques of the key baseline findings, and the hub evaluation projects that were based on members’ identification of context-relevant aspects of these findings. There was ongoing mentoring for hub members, the research assistants (RAs) who coordinated the hubs, and the graduate students affiliated with the participating institutions. Research team meetings included senior researchers, country directors, the programme coordinator and, over time, country RAs.

T HE PAR C ASE S TUDY P ROJECT

A comparative case study of the PAR approach in this programme was undertaken in order to articulate how the participatory and action-oriented aims of the overall initiative translated into strengthening capacity through the development, functioning and outcomes of leadership hubs. The specific guiding questions were as follows:

•   How is the PAR approach understood and enacted in meeting the capacity-building and knowledge translation objectives in and across the four countries involved?

•   What are the similarities/converging factors across countries in how the PAR approach manifested and contributed to meeting capacity-building objectives?

•   What are the differences/diverging factors across countries and how are these differences accounted for?

A case study essentially involves two parts: the creation of cases, and comparisons across cases (Anaf, Drummond, & Sheppard, 2007; Aryes, Kavanaugh, & Knalf, 2003; Yin, 2009). In this project, the leadership hubs were used as the focus for creating five cases: Jamaica, Kenya, South Africa, Uganda, and the international network of hub members. Descriptive outlines of each country context and the coordination role of the international network provided a basis for comparisons across cases. These comparisons fuelled a detailed thematic analysis of PAR as the overall approach.

Data collection

Yin (2009) suggests that good case studies adhere to the following three principles regarding data sources:

1)   Multiple sources are used to enhance the development of converging lines of inquiry and to strengthen triangulation.

2)   A database is created to organise case study data.

3)   A chain of data is maintained which indicates how it all fits together and conclusions are made.

Sources of data in the Strengthening Nurses’ Capacity programme included hub leaders (HLs), RAs, country directors, hub focus group transcripts, programme documents, and field notes of the first author. The documents included the programme proposal and appendices, teams’ meeting minutes, RA progress notes, hub action plans and newsletters. All data (documents and transcripts) was stored, managed and coded using Word and Nvivo 9 software, in folders created for each type of document and in folders for each ‘case’ to facilitate within and cross-case comparisons, thereby creating a ‘chain of data’ (Yin, 2009). All data was encrypted and stored in a password-protected computer.

Semi-structured conversational interviewing was used to allow for open dialogue that would tap into participants’ experiential knowledge of the hubs and yet still followed lines of inquiry which addressed the research question (Yin, 2009). The first author conducted six separate RA interviews via Skype and telephone. The focus of these interviews was to understand PAR from their coordination perspective, particularly the experience of adopting such an approach in terms of facilitating capacity development for respective hubs as well as their own research capacity. This was followed by face-to-face group interviews with HLs (10 participants, grouped by country). The first author interviewed HLs separate from RAs, with the aim of understanding what it was like to be involved in a PAR project and how their relationships with research team members facilitated capacity development for themselves and all hub members in general. These 16 interviews were audio-recorded and transcribed. Additionally, conversations with country programme directors provided important contextual information about the country, which formed the basis of the background for each case.

Recruitment occurred in two phases. First, an introductory email was sent to all RAs (past and current) explaining the study, with a consent form attached. Those willing to participate returned the signed consent form and interviews were scheduled. Next, RAs were asked to seek permission from all HLs to be contacted regarding their participation in the case study project. The contact information for those who were willing was sent to the first author, who sent an introductory email explaining the study, obtained written consent and arranged interviews. One notable exception occurred in Jamaica, where the RA interviewed individual HLs. These interviews were followed up by the first author with the group of HLs to ensure that they had the opportunity to comment freely about their relationship with the RA. All email communications for the purposes of recruitment and clarification of interview data was saved with the consent of participants.

Data analysis

Data analysis was structured through the use of matrice, which helped to organise the converging, diverging and complementary points in and across cases according to research questions and to guide interpretations (Averill, 2002). Along one axis were the following aspects of the Strengthening Nurses’ Capacity programme:

•   structural conditions/elements (funder, participating universities, workplaces of hub members and the broader social/geographical context)

•   research process elements (the research proposal and supporting documentation, data collection tools, timelines, personnel issues and communications)

•   capacity-building outcomes (intended and unintended)

•   knowledge translation outcomes (intended and unintended).

Two aspects of PAR comprised the second axis: the ‘participatory’ (experiential knowing) and the ‘action’ (practical knowing) elements. Each of these were subdivided into ‘facilitating’ and ‘constraining’ as a way to ensure that positive and negative aspects of PAR would be considered. Analysis generally proceeded through multiple readings of documents, interview and focus group transcripts, conversations with principal investigators regarding evolving thematic categories, and detailed researcher notes for each case. While the programme documents were a vital component, emphasis throughout analysis was given to the perspectives of hub members, hub leaders and the RAs. This was done with the intent of staying close to participants’ experiences of PAR and the related perceived changes in capacity stemming from this research approach.

Case descriptions

Two of the participating countries are classified as low income (Kenya and Uganda) and two as upper middle income (South Africa and Jamaica) (World Bank, 2012). The histories of all four are marked by the effects of colonisation and corresponding struggles for independence. These countries have produced political and social leaders who have made significant contributions to the international understanding of human rights, decolonisation and economic cooperation.

All of the country directors for the programme were affiliated with academic institutions. Three had primary responsibilities within the University of West Indies (UWI) in Jamaica, the Great Lakes University of Kisumu (GLUK) in Kenya and Northwest University (NWU) in South Africa respectively, and the fourth (affiliated with Makerere University in Uganda) had primary responsibilities as an administrative leader for large teaching hospitals. The cases are presented below.

Jamaica is the largest English-speaking island in the Caribbean Sea, with its closest neighbours being Haiti, Cuba and the Cayman Islands. In 2012, Jamaica celebrated its 50th anniversary of independence and still remains a Commonwealth nation, made up of 14 parishes. As of 2010, the population was 2.7 million and levels of international migration were high, notably among graduates of the health professions. In 2007, approximately 27 000 people were living with HIV and as of 2010, life expectancy at birth for men/women was 69/74 years (WHO, 2013a). Though recently reclassified as an upper-middle-income country, poverty remains a concern, particularly in rural areas where the economy is mostly based on agriculture, mining and natural resource-based tourism. Its geographical location puts Jamaica in the path of tropical storms and hurricanes, which means that the country has needed to rely on international development partners to assist in post-hurricane recovery and rehabilitation (WHO, 2010). Public sector health service delivery is managed by the country’s four regional health authorities, via two avenues: 1) a network of secondary and tertiary care organisations consisting of 24 hospitals, five of which are specialised facilities, and 2) primary care organisations consisting of 348 health centres.

Additionally, there are two national hospitals. However, the continuing shortage of healthcare providers to staff these facilities has a significant impact on key areas of service delivery (WHO, 2013a).

Parishes where leadership hubs were established in Jamaica were chosen based on proximity to the UWI Mona campus, located in the capital city of Kingston and falling under the Southeast Regional Health Authority. Three hubs were formed in the parishes of Kingston and St Andrew, St Catherine and St Thomas, with the St Thomas hub being the farthest distance (90 minutes by car) from UWI. A co-investigator of the Strengthening Nurses’ Capacity programme provided mentorship to each hub.

Key contextual elements in Jamaica

Geography and deliberate choices based on proximity.

All hub trainings and meetings were held at UWI. A room and food were provided, which helped hub members to focus on the work at hand without distraction. The close proximity also meant that the researchers could easily travel to the participating institutions to give information sessions as necessary. This was vital for relationship building in these settings, to familiarise managers and other staff with the project. Part of this deliberate choice of closer proximity was the researchers’ intention to change the way researchers and practitioners typically worked together. It was noted by the Jamaican team that front-line nurses and nurse managers were generally intimidated by the research process, not perceiving it as something within their role. Therefore, the short distance between the researchers and the hubs helped create a more direct and collegial working relationship by allowing time for the researchers to be more ‘hands-on’ with the hub members and to be present to address their learning needs as projects progressed. For example, heavy writing work such as proposals and reports was cited as an activity that was better done at UWI where the hub members could focus on the work and have immediate help and feedback as they were writing. Conversely, as particular hubs carried out activities on site in their respective settings, researchers from UWI could attend more easily. This all served in cultivating strong working relationships.

Severe nursing shortage and professional politics

A dominant issue affecting the hubs that was identified by the Jamaican researchers and the participants was the severe shortage of nurses in all institutions in the three parishes. While the nurses were interested, the challenge came in their being able to participate regularly in meetings. Despite initial buy-in from the institutions, when nurses left to attend meetings at UWI they encountered problems with managers and colleagues in their work settings because it usually meant leaving an already under-staffed unit even more short of nurses. A further complication was that although broad representation from parish hospitals and health centres was the aim, in the Kingston-St Andrew hub, for example, the majority of nurses were from one large general hospital and a maternity hospital, given their large size and staff complement.

Professional politics in terms of interpersonal stressors and power struggles was at times an issue within the hubs. Some of this had to do with differences in interpretations of nursing between those with longer histories as practitioners and those who were newer to the profession. In one instance, an older nurse’s perspective of ‘knowing best’ produced a dominating kind of facilitation that felt argumentative for younger hub members and inhibited their participation. This situation was resolved through one-to-one supportive dialogue with the individuals involved, and is an example of what Jamaican participants felt was partly a reflection of the hierarchical culture of nursing and healthcare at a broader level in Jamaica.

With a population of 40 million, Kenya is ranked 143rd of 177 countries on the Human Development Index. As of 2009, approximately 1.5 million people were living with HIV in this eastern African country (World Bank, 2009), and life expectancy at birth for men/women was 58/62 years (WHO, 2012b; CIA, 2012). At the time of this study, the health sector was governed through two ministries: the Ministry of Medical Services and the Ministry of Public Health and Sanitation. Government is the main provider of health services. However, it faces challenges in terms of inadequate human resources, infrastructure and financial resources. Therefore the private sector provides care to a significant proportion of the population (WHO, 2009). Nyanza province, the site of the Strengthening Nurses’ Capacity programme, is located in the western part of the country on Lake Victoria. Politically, this area has been the seat of opposition for more than 40 years. At the time of independence in 1963, it enjoyed the benefits of many universities, lawyers and health professionals. It was historically at the top of national health indicators and is now at the bottom.

The Kenyan leadership hubs represented three levels of the health system and were located in three districts: Kisumu, Nyando and Suba. Kisumu city is where GLUK is located, so these members had the easiest access to university-based hub activities. Nyando is the next closest (30 minutes by car), and the Suba hub represented a district that is geographically comprised of a number of islands, meaning hub members here provided services for very hard-to-reach communities of 10 000–15 000 people. These locations were selected according to the government’s restructuring of districts into smaller ones after 2002. Although the newly formed districts remained similar in terms of allocation of health resources, there were other changes that affected the functioning of the hubs after the 2008 post-election violence. Kisumu retained the older health services infrastructure. Both the old district hospital and the provincial general hospitals are located in Kisumu district, since Kisumu city is the headquarters for both Kisumu district and Nyanza province. Suba and Nyando, having been carved out of other districts, had at the time newer and less developed health infrastructures. Therefore, Kisumu had well-established, complex systems in place while new districts, like Nyando and Suba, needed time to reorganise in the years following the restructuring. It was agreed that the programme would continue according to former administrative units, partly due to the lack of sufficient system structures in the newly formed provinces (RA progress report, 2009).

Key contextual elements in Kenya

National election of december 2007.

The 2007 pre- and post-election context for Kenyan hubs was one of relative deprivation of the region of chosen districts and political upheaval for the country as a whole. Nyanza was the location of comparatively high rates of infectious diseases and poverty. This meant generally higher workloads for nurses and other health professionals than in other parts of the country. It also meant that the establishment of hubs was affected by pre-election politics. Practically, hub activities needed to be altered at times due to campaigning and related activities. Post-election, when reportedly fraudulent results were announced and the ruling president was sworn in at night, opposition protests began and erupted into violence. People were expelled from areas based on ethnic community and political party divisions. This affected hub members, some of whom had to leave the region because it was not safe for them, meaning a depletion of members. In an estimated period of seven weeks following the election, roads were blocked and daily life came to a standstill. The hubs, just forming, were greatly affected. No hub activities took place until after successful negotiations between parties resulted in a new coalition government and the opposition leader being placed in the newly formed position of prime minister several months later. The tangible effects of this on the hubs were that because many members had to leave, hubs became more ethnically homogenous, and a sense of solidarity emerged within and across them in terms of the history and the hope that people had for change.

GLUK as the setting

The influence of GLUK, as the university setting for the Strengthening Nurses’ Capacity programme, cannot be overemphasised as a key element in the formation and outcomes of the hubs. As a private university set up explicitly to offer an alternative to the metaphorical ‘ivory tower’ of academia, GLUK’s philosophy is one of shaping students for the context and kind of work the country requires. Embedded in this is an assumption that learners ought to be making a contribution as they learn. Engaging students in this way is thought to enhance their capacity so that they are equipped to manage the environment and their resources. Hub members have been deliberately included as learners in this environment with the intention of bringing about changes that are in line with principles of primary healthcare. As RA2 stated, ‘we come back to the people, we work with the people, … and hub members know the reputation of GLUK’. In short, PAR is already a way of working at GLUK.

South Africa

South Africa occupies the southern tip of the African continent. It is a multicultural country with 11 official languages, but politically it is still a relatively young democracy (WHO, 2011). The country has one of the strongest economies in the African region, and has high rates in some health-related indicators, such as adult literacy (82.5%), household access to piped water (92%) and waste removal services (61.6%) (WHO, 2011). However, average life expectancy at birth in 2010 was still low for men and women (55.3 and 60.4 years, respectively) (WHO, 2011). This is partly attributable to an 11% HIV prevalence rate in 2009 (WHO, 2011). Poverty remains an important concern despite its ranking as an upper-middle-income country. Contextual factors that support health in the country have been identified as a favourable political environment for investment and social development, well-developed health policies with budgetary support and a national health act along with a functioning decentralised government structure (WHO, 2011).

The initial plan for leadership hubs in South Africa was to include two provinces, namely North-West and Gauteng, given the close proximity of NWU in Potchefstroom to the border between them and two major cities, Johannesburg and Pretoria. However, the team recognised the challenges of developing hubs in both, and decided to restrict the programme to the North-West province, with hubs in Potchefstroom, Dr Kenneth Kaunda district; Rustenburg, Bojanala district; and Lichtenburg, Dr. Ngaka Modiri Molema district. Early on, two RAs divided the work along geographical lines; that is, one took responsibility for the Rustenburg and Lichtenburg hubs, while the other took responsibility for the hub in Potchefstroom along with administrative duties for the programme. This division of responsibility worked until the former RA withdrew from the programme, leaving the latter to establish working relationships with these hubs. Maintaining the Bojanala hub became a challenge in terms of members’ participation in hub activities. The other two hubs sought ways to connect with Bojanala, to involve their members and to ‘revitalise’ that hub. In the end, in spite of several attempts and various means to engage, the best attendance of Bojanala at joint hub meetings was only ever one or two people.

Key contextual elements in South Africa

Elections and the world cup 2010.

Two important events took place during the span of the programme that were seen to have a small effect on the hubs. First, the elections in early 2010 created some tension in how hub members related to one another during that time. As explained by South African participants, there was no doubt of the African National Congress winning again nationally, but other parties had gained momentum in the years leading up to the election and in fact won elections in several subdistricts. During the election period, there were some subtle shifts in how hub members communicated and worked with one another, depending on their political views. While violence like that seen in Kenya in 2007 did not happen, there was a palpable tension in different institutions. However, this passed post-election. Also, only minor changes in local structures post-election meant that hub activities carried on as planned within a month afterward.

South Africa also hosted the World Cup in 2010. Described as a time when Africans came together, regardless of race or culture, it generally lightened the air in hub dynamics. It was a big event, which added work for hub members, and participation in meetings was affected for some because they were managing extra commitments to public health campaigns during World Cup events. However it did not significantly sidetrack members from hub activities.

Distances within provincial boundaries

The selection of hub locations within one province presented a unique challenge and opportunity. Distances between the hubs and NWU within North-West province were far greater than the original choices of districts in Gauteng and Free State. Since the hub members came from a single province, they formed a provincial ‘identity’ or ‘loyalty’. As the members worked together, they began thinking at a provincial level rather than an individual institutional level. Their commitment to making changes at the provincial level also grew. They reportedly wanted their work to have an impact further up the healthcare system. Hub activities brought people together who might otherwise never have met or worked as a team. But this came with a significant challenge in terms of the practicalities of travel between locations and the struggles of relationship building at a distance. Despite the distances, the RA described a sense of reward in contributing to change in rural/remote areas that tended to get overlooked for research projects.

Crossing institutional cultures and politics

Many administrative activities are necessary in all research and particularly in a programme such as this. In South Africa, the time-intensive practicalities of managing these activities stood out. For example, the financial reimbursement of hub members for their travel expenses related to joint hub meetings was a two-day process for the RA: one day was needed to do the paperwork and obtain a cheque, and the next day to go to the bank, stand in the queue and cash the cheque. This was necessary so that on the day of the meeting, there would be cash in hand to give to hub members. Additionally, there were different requirements for financial and staffing record-keeping across institutions (University of Ottawa versus NWU as well as study hospitals); this also increased the administrative workload.

More subtly, NWU has traditionally been a conservative Afrikaans university and has three campuses in the province, including one considered to be a black university with strong partnerships with the Rustenburg hospital. These historical cultural differences have continued to play out in relations among the campuses, which may also have affected relationship building with the Bojanala hub members. When attempting to engage some of the Bojanala hub members, it became evident that their perception of NWU was not positive. This might have contributed to administrators and nurse managers in Rustenburg restricting staff from attending the Bojanala hub meetings. Despite the sensitive nature of these communication challenges, the leadership hubs created rich opportunities for cross-cultural learning.

Uganda is located in Eastern Africa and is a fast growing country with an estimated population of 35.5 million, 78% of whom live in rural areas. It is ranked 143rd of 177 countries on the Human Development Index (World Bank, 2012) and receives high levels of development assistance. In 2007, there was a 5.4% prevalence rate for HIV and almost one million people were living with the disease (WHO, 2013b). Peace talks in the long-standing armed conflict in Northern Uganda have brought about the return of 95% of the approximate 1.6 million internally displaced people to their homes (UNHCR, 2013). In 2011, national elections resulted in the re-election of the president. In terms of health service delivery, the key challenge, as of 2009, was identified as inadequate capacity in planning, management and human resource development (WHO, 2013b).

The Ugandan leadership hub locations selected were the districts of Luwero, Jinja and Kampala. The members were individually invited to join in each district based on their known interest in international research and word-of-mouth recommendations. Kampala, as the capital of the country and largest city, represented an urban location. Luwero is a rural area, and Jinja is considered semi-urban. All districts are within relatively easy travel distance for meetings in Kampala. The composition of Ugandan hubs was considered in terms of having a mix of members with research knowledge and with the intention of providing good learning opportunities.

Key contextual elements in Uganda

Working for the government.

Changes in the research team and the hubs came about as a result of several members being government employees. After the election, the Ministry of Health was reshuffled, affecting everyone in leadership positions including the country director, who moved from Kampala to Jinja. This disorganised the work of the hubs, which contributed to a decline in momentum. For example, changes in leadership at Jinja hospital (where most of the hub members worked) contributed to the slow start of their hub. According to the country director, these hub members struggled on their own for a while but ultimately the work of the hubs was completed.

Social dimensions of nursing

There was a sense of strong nursing leadership and advocacy in the identity of the Ugandan hubs. Despite workplace leaders’ interest in the Strengthening Nurses’ Capacity programme, their support for staff nurses to participate in the hubs was lacking. This gave rise to the need for extra support and encouragement from HLs and researchers in terms of flexibility in meeting schedules for members, periodically assuming heavier portions of the workload and coaching members on new learning, such as computer skills. Additionally, the nursing profession was framed within an understanding of collegial relationships. Nurses identify with each other socially, which aided in recruitment for the hubs as well as a kind of peer support to continue the work of the hubs. In fact, some individuals were approached directly to participate because they were known to have particular skills, including interpersonal ones. There seemed to be an understanding of colleagues’ personal lives and circumstances that was assimilated as part of sorting out challenges in completing hub work.

Elections, travel and safety

The social and political unrest around election time was an important contextual point. That is, as the country director explained, such instabilities created a need to strategise and to have a back-up plan. For instance, there was a situation during data collection when members of the research team felt anxious, recalling the post-election violence in Kenya. They discussed a safety plan for completing data collection.

International network

The Strengthening Nurses’ Capacity programme was comprised of a network of researchers, programme staff and hub members, as well as experts in healthcare in and across countries who provided advisory guidance. Central coordination took place from Canada, which was the fifth country location. Canada’s research team had representatives from Ottawa, Edmonton, Halifax, Lethbridge and Toronto. There was no leadership hub in Canada. Notably, the global composition of the senior research team was based on pre-existing relationships across Canada and across countries.

Key contextual elements internationally

The detailed guidelines/structure of the programme.

Handbooks created by the programme coordinator provided clear guidelines for each research activity and an overall reference point for the research team as the programme proceeded over five years. The importance of these guidelines is reflected in country directors’ and RAs’ reference to them as ‘the bible’. Correspondingly, RA training sessions were offered that laid out, step by step, the requirements of every data collection tool. The training sessions were structured with short didactic presentations of information, group discussion and related exercises and activities. Several research team members were involved in the development and facilitation of these sessions, which were held on-site in the countries and via Skype.

Communications across distance

The second element, communications across distance, is an aspect of working within an international network that is often taken for granted. It stands out in this programme as a pragmatic consideration that was given careful attention in ensuring completion of all projects in all four countries, and in the case of PAR, for its relational effects. There were trips by team members, RAs and some hub members across the participating countries at various points in time allowing for focused face-to-face work, much like hubs were brought to the university within countries to work without distraction. An RA described how much could be accomplished by being physically together as a team: ‘ now we have to sit together and agree on how to do this … we worked as a team as never before; for the first few hours we took to argument and discussion … [then] we settled to working and doing the writing ’ (South Africa training notes). The telephone and creative uses of the internet also served not only as a means of information sharing but also as a means of support for participants in-between face-to-face meetings. Senior researchers were accessible and approachable across distances, and the programme coordinator was consistently responsive to the needs of the programme team and participants. The importance of tending to communications across distance is well described by another RA:

I wouldn’t have learned so much as I have learned if it was only a national study. Being an international study you have this intangible thing inside you that actually drives you because [other countries are] doing it, so [my country] also has to do it. And if you run into a roadblock then you have the international network … And through that process, even if it was virtual … you can bring that back to the environment you function in … (RA4).

Quality relationships and relationship-building

The last key element of the international network is inseparable from the first two points. The quality of relationships between researchers and participants in a PAR programme relies to some extent on preexisting relationships. Part of the foundation for working well together was the fact that researchers knew and respected one another before this programme formed. Furthermore, pre-existing relationships between country directors, RAs and even some hub members created a strong relational foundation for the programme. The research team’s own interpersonal and group dynamics were consistent with the in-country team dynamics, which in turn were consistent with the researcher–hub member relationships. This modelled for hub members a way of behaving in vertical relationships in their own work contexts.

In a slightly different vein, the relationship of the funder to the programme bears noting. The ‘representative of research funding agencies’ is an administrative role and often the actual person in this role is only known by the researchers. In this programme, the representative was known to participants as well, having made visits to the countries during which he heard directly from hub members about their work. From the participants’ perspective, this served in building presentation skills and allowed a ‘rich exchange’ between the funder and the local participants who may not otherwise ever be engaged in such discussions (GHRI visits to country sites). Hub members’ learning happened in preparing for and attending these meetings. With the help of the research team, they reflected together in advance on what they hoped to convey about themselves and the programme.

Each case presents interesting ideas for considering how PAR works in different international contexts. From the Kenyan experience, we learn that the support of an organisation that embraces tenets of PAR in its own mandate is invaluable grounding for successful programme outcomes. In light of the election situation in 2007, it seems remarkable that the leadership hubs survived at all let alone completed evaluation projects and developed momentum toward sustainability. There is something in this example that also reflects PAR’s commitment to change, not only in terms of bringing it about but also in the perseverance required when sudden change is thrust upon a programme and new strategies are required.

In South Africa, the situation of Bojanala is a poignant reminder of how subtle cultural differences and historical context are always in play. It highlights that awareness of cultural difference grows through careful and critical reflection about the programme, the participants and how to cultivate trust in working relationships. It was suggested that had the approach not been PAR, these cultural aspects of the South African experience may have been lost as a part of the learning and the outcomes. PAR gave ‘the freedom’ for cross-cultural learning to happen precisely because the expectation was set that hub members actively participate with one another and with the researchers, ethnicity or race notwithstanding.

In Uganda, the biggest factor in managing the hubs was the nurses’ lack of familiarity with research and their nervousness about this knowledge gap, yet this was countered with supportive social relationships within nursing networks across hospital and academic settings. We are reminded by this case of the necessity to appreciate fully participants’ starting standpoint; that is, the capacities they bring to the research programme and the kind of persistent support that is required – both social and research focused.

The Jamaican case highlights interesting questions regarding how strengthening the capacity of individual nurses strengthens the profession as a whole. Conversely, the question of how the culture and internal politics of a profession strengthen or inhibit the capacity of its members seems equally important to consider.

The case of the international network highlights a need to reflect on the labour-intensive nature of PAR and the importance of having a clear plan laid out initially by those with the most research knowledge, adapted and adjusted over time by those with the most context knowledge. The plan must be carried out through respectful and consistent communication, which leads to trusting relationships between researchers and participants, and develops into successful outcomes.

Overall, the cases offer three important converging points. The first is that institutional and cultural factors bear consideration in planning an international PAR programme. Specifically, how carefully programmes adapt to differences in bureaucratic processes across countries has a direct impact on how and when the work gets done. Next, the hurdles of distance, geography, safety, infrastructure and political divides are also representations of difference that researchers and participants must negotiate together if the programme is to be successful. The third point arises from hub members identifying themselves not only as part of a local or even national initiative but also as part of a global movement. This can be attributed in part to the careful attention to communication at all levels of participation across international borders. This sense of being part of something ‘bigger’ facilitated new learning, new connections to others doing similar work, and new educational goals and possibilities.

T HE PAR T HEMES

This section focuses on the results of the comparative analysis of the cases, namely, four themes that represent the complementary points about the experience of PAR across countries – power and empowerment, change, time and context, and values and assumptions. All participants were asked how they understood PAR. Various definitions were offered that reflect both participation and action, as seen in the following examples:

different kinds of people, different groups of people in the whole research process that includes right from the design of the research project and implementation … putting all that together and then sharing so that everybody works hard (RA2).

where somebody is asking you to come and participate in the research … not just to participate but you are supposed to be an active member of the research approach whereby there will be different categories of people so that there should be a clear understanding of the project itself. … you don’t just participate and keep quiet … you are hands-on, you are the one that is doing that so it becomes more visible…you are the one that is expected to give feedback … (HL6).

HLs and RAs distinguished PAR from ‘the typical’ or ‘usual’ way of doing research. As one RA explained, ‘ not the other way of doing research, action comes later, but action or responses or participation comes along as you do your research ’ (RA6). It was seen as something ‘ongoing’, ‘cyclical’, ‘continuous’ – a process whereby information was the basis for action and reflective group discussion about actions and this, in turn, provided new information (RA6, HL4, HL6). The importance of relationship-building between researchers, HLs, members and their respective communities was also stressed. Certainly there were time and expense issues in determining the setting up of hubs, as one RA pointed out, but

when you have the relationship, really it’s much easier and it’s context specific – you’d have individuals in the same [work] context and the impact of that relationship boils over to other aspects … which is a good thing. If you have the typical approach of just entering a country, doing your research, and then turning around and heading out, that will actually have a bad impact. But with the PAR approach, there’s a prolonged positive impact on the context where the research happened (RA4).

One such positive impact was described by an HL as she explained her understanding of PAR: ‘ in cases where you involve the communities, you get information in much simpler terms that [they] are able to understand … from maybe bottom up and stuff like this ’ (HL7).

But interviewees also articulated drawbacks of such an approach, the first being that as participants become engaged, particularly as they recognise new areas of need, there is a risk of losing focus on achievable objectives for the project. One interviewee put it like this: ‘ When you go to the field something else is happening so … you can end up totally digressing from your objective ’ (HL10). An example from Uganda highlights this as they struggled to identify an evaluation project topic:

Initially we had a whole big five-year project. So we’re looking at HIV as something wide. We wanted to work on stigma, we wanted to work on all the dimensions of HIV. So because of the direction we were given, because of the guidance … we ended up with a specific focus. But otherwise we had about five objectives … [country director] was always on us, ‘you prioritise those objectives, you identify one or two that you can work on in this small period with those resources’. But for us,… we want to do stigma, we want to go to the caretaker … to the nurse, and the community (laughing)… so that was very important for us that we had a sense of focus, a sense of direction … and specificity (HL9).

The participatory nature of the PAR approach also added to confusion at times. For example, ‘ you may wonder, am I evaluating a project that was already implemented or … what am I evaluating? ’ (HL10). So PAR, in this programme, is a cyclical process that engages different people from different areas throughout its duration. It means translating information into understandable terms for all involved, taking action in a focused, deliberate way, and expecting change to come from both these actions and the relationships that formed along the way. This understanding of PAR is further refined through the following themes.

Power and empowerment

This theme highlights assumptions about the power of all individuals (ie that capacities for different kinds of work/engagement already exist), reflected in the emphasis on personal and collective responsibility. The theme of power and empowerment is foundational in understandings of PAR, expressed in several ways across all interviews. First, the intentional arrangements of roles within the research team, the leadership hubs’ membership, and the relationships between these groups were established with explicit attention to power.

On the premise of relational capital, membership in hubs and delineation of roles in the research programme were given careful consideration, as reflected, for instance, in the terms of reference for membership of leadership hubs and national advisory committees. This was all set within parameters laid out by the senior team and in how the team communicated with each other and with RAs and hubs. Transparency and accountability were central, which is thought to have been a contributing factor in the success of the programme.

Hubs were purposefully created, bringing together professionals and lay people from different sectors in the health system with different levels of power and authority. This established structure served to acknowledge power relationships and opened a way for the process and content of the work to become empowering, notably in regard to individual knowledge, confidence and skills. One HL explained it as members developing ‘ confidence and certainty in the ongoing activities, which contributes to self-worth ’ (HL4). Another shared that ‘the experience leads you to also develop your own assessment at your workplace to say, oh there’s a problem, how can I approach it? … It’s not like [it was previously] to say there’s nothing that I can do ’ (HL7). RAs also observed this empowerment:

… [hub members] were able to write the [evaluation] proposals themselves … It is one thing to identify the problem and another thing to put the problem into scientific writing. They were able to write … with guidance from us and the team in Ottawa of course (RA6).

This was a nurse who from the onset said no to the hub activity, but the first day she came in… she contributed to the writing of the protocol … She also said she had learned a lot from the research project … I would say apart from improving their participation in pushing forward the agenda for quality health service, I saw them push the agenda of research … (RA3).

Another stated that empowerment was evident in ‘ the way they express themselves, they become more confident ’ (RA2), adding that rather than typical didactic teaching about research design, this programme involved nurses throughout a whole process of proposal development, which built writing and presentation skills as well as communication skills for speaking with leaders in the health system. The RA pointed out that these things are often taken-for-granted skills that are therefore often passed over in research courses, and she continued, shifting to her own new sense of empowerment:

In many research programmes, as a fact, like all you do is push paper but in [Strengthening Nurses’ Capacity] the research assistant title is a lot, it means you’re involved in every aspect of the programme. It means that … your capacity is built … Before I was on the team, I had absolutely no experience … it was only when I joined that I learned how to connect, to transcribe, to do preliminary analysis and that is something that I could take away … when the project is coming to an end I have more resources, I have more confidence and experience as a researcher (RA2).

Other RAs echoed this. One explained that through his role in this programme, he learned to critique and write grant proposals, having ‘ full samples to fully understand ’ (RA3). Another named empowerment directly: ‘ the empowerment of myself as a novice researcher was, I cannot describe it in words. It was marvelous ’. (RA5).

This experience of empowerment has two dimensions: responsibility and support. One RA pointed out, ‘ interest doesn’t equal change ’ (RA6), referring to the responsibility required of hub members. There seemed to be an emphasis on the difference between desire to be empowered and actually exercising one’s power. In this sense, participants identified that with power and increased capacity to take action came responsibility. ‘ Those who have taken the responsibility and leadership … we have seen that they have come out as very different people ’ (RA2). One example is a Kenyan hub member who was not active early on, but then began volunteering to do things to get hub activities moving. The RA witnessed that as he took on more responsibility, his confidence grew. One HL also voiced a new awareness of the responsibility that comes with a leadership role:

… we always feel, if I don’t do this I’m going to fail the whole project and it’s not what I want to happen. How are other people going to see me? When everybody’s busy trying to make a difference then I come with failures …? (HL6).

Clarifying levels of responsibility was also highlighted. As explained by the programme coordinator, each country had an identified programme director and researcher who assumed responsibility for ensuring follow-through on all projects. They also assumed responsibility within their respective institutions for administering programme funding, which meant that other levels of authority required to manage funds needed to be clearly established. Similar kinds of administrative clarifications were learning opportunities for some hubs. The value of documentation, for example, was highlighted by one HL. The senior researchers’ insistence on having signed administrative approval for evaluation projects was new to these members and initially even seemed excessive:

there was a lot of emphasis in ethical consideration so we as a team in a district are not seeing why it is important to ask for permission to carry out an evaluation in our own district … because they insisted we had to follow the right process and this really helped us a lot because at one given time, the politicians wanted to question our evaluation … So we produced the initial letter that was signed and stamped by the district health officer … otherwise if we didn’t have that letter we would have had many questions. And of course when politicians intervene in the project so much it may not succeed as it is intended. So that ethical consideration became a reality to us … the fact that we had to practically do it helped us to learn more about its implications (HL9).

The first author probed with another HL from the same country when she, too, raised this example: ‘ So what’s the benefit of doing that – insisting on permissions, having letters, having ethics consent forms … over the way that it’s traditionally done? ’ HL8 then named it: ‘ It gives you a sense of responsibility in terms that whenever you’re doing something it should be evidenced, it should be on paper, it should be signed making an agreement. That shows that really this person is doing something ’.

Appropriate support is another important dimension of empowerment. An empowering process is not simply about giving information and assigning responsibility; it must involve engaging people in new ways of learning, providing guidance, and clearly identified milestones. This was most pertinent to the development of hub evaluation projects. One HL expressed that ‘ if it were not for the involvement of research team members ’, hub members would not have been able to identify a gap to focus on; they were present ‘ physically and through email ’ throughout the proposal-writing and implementation phases, and followed up regularly with encouraging questions like, ‘ How far have you gone? And how are you ?’ (HL9).

HLs and RAs also talked about the importance of realising they were not alone in the process, either individually or collectively as a hub, or network of hubs. This is reflected in the following quotes:

The country directors used to organise joint hub meetings for the capacity building in our countries and … we had mentors, we had collaborators, and also we had the advisory team from Teasdale-Corti who are reviewing each particular proposal being made’ (HL8).

The PAR provided hub members with quality time to adequately gain from facilitators who form part of the team in all stages of the project’ (HL4).

[The research team] made time, always ready to assist. And I think that is part of the capacity building that we are capable of facing decision-makers … we are one of the decision-makers in my mind… because you think, am I going to be able to communicate with these people? But we can because they’re very approachable (HL7).

We all had our way in which we were supporting each other’s progression … It was recognised that it was necessary for the coordinators of the various programmes to have their own meetings to discuss their issues … It gave the programme the ability to compare and contrast as they moved through each and every challenge (RA3).

The supportive nature of the RA role for hub members was further elaborated by RA6:

I was having issues with the hub members. Their skills of proposal writing were not the best and they needed more efforts so to exchange … what is happening in the rest of the countries, and I [got] really very constructive feedback … to make sure that they all move at the same speed without leaving some behind … I thought it was unique to [my country] but I realised even other hubs in other countries were having troubles related to skills in proposal writing.

This theme accentuates developing capacity as a kind of transformational learning. The PAR approach served to create a culture of learning throughout all activities of the programme, which in itself was empowering. This connection between feeling empowered and learning was expressed by HL8: ‘ You learn a lot. You learn from the research participants and … from the fellow hub members and the team leaders or the director and the technical team when they come to [the country] ’. Moreover, developing capacity meant the researchers supported the hubs and the RAs so that they did not feel alone in more stressful moments of learning (working through something new; adapting new knowledge for the context, etc). The participants identified the critical role of this support not just in terms of providing access to information, but more in terms of relational aspects like active listening and consistency in communications.

PAR insists on intentional action which brings about change through the research process itself. This begs questions regarding what constitutes ‘change’ and how change happens. Capacity development as interpreted in the Strengthening Nurses’ Capacity programme manifested in action-reflection. This emphasis on nurses’ capacity was evident in various opportunities for hubs to reflect critically on changes in their own practice/careers and relationships within their respective healthcare systems. Capacity development as a form of action was stated at the outset of the programme:

Hubs of nurse researchers, front-line nurses and managers, decision-makers and community representatives will be assembled to begin work on building strategies for change processes. We will use existing structures and national advisory committees within each country to identify hub participants from each of the stakeholder groups, based upon local context, availability, potential for buy-in and investment.

The capacity development intention was discussed during hub project meetings: ‘to have a fairly strong and dynamic impact on people’s thinking based on the evidence we are able to accumulate over the course of the project ’.

RAs saw the change in capacity as a starting point for action and a valuable end in itself, even if other actions were not immediately observed. They emphasised the importance of the initial training workshops as the starting point of an iterative process of change, to ‘ help hubs to begin thinking … to consider actions ’ (RA1 debriefing interview). Another RA stated that ‘ process outcomes do count ’ because ‘ several learning elements ’ are happening simultaneously in the hubs; ‘ even if hubs fail with implementing their projects they will have still succeeded in many ways ’ (RA3 debriefing interview). An example from a Kenyan hub supports this RA’s observation that even the ability to identify relevant policy documents was a positive change. In the process of deciding which issue the hub wanted to address, one nurse identified a reproductive health policy document that encompassed motherhood. The RA noticed that the member was applying what she had learned in the hub training to examine real-life policies in her context, and in fact this discussion contributed to the hub’s decision about the next steps for action (Kenya debriefing interview).

In Uganda, change was similarly observed in the development of hub members’ interests in research, which led the majority of them to attend monthly journal clubs at their institutions (2010 RA training notes). The presentation of earlier projects’ findings in a Jamaican all-hubs meeting generated rich discussion about nurses’ knowledge of and access to policies. This discussion stirred members to locate their relevant institutional policies. In carrying through with this small action, one hub discovered a gap between the policy and practices of voluntary consent and testing (VCT) consent procedures, and addressing it became the focus of their evaluation project (2010 RA training notes).

These examples emphasise how the participatory dimension of PAR brought about a change in individuals, perhaps not consciously identified in the immediate moment, but rather a shift that manifested at some later point. This was articulated by one RA as the ‘ripple effect’:

It’s impossible to quantify or measure the capacity that has happened and the ripple effect is definitely big. I think some of the informal feedback that I get when I’m in the field – it’s really good feedback. There’s also the flip side to that where nothing has happened, which is the bad side of it but again there’s a ripple effect on places that you didn’t expect. Maybe to give an example… I had an interview with one of the leadership hub members and afterwards I talked about … the action plans, and she mentioned that she did a total turnaround at this one institution, which was a clinic in a rural area, whereby they took down all the policies they had in the institution, physically putting everything on the table and they went through all the policies and they made sure … they did a policy check … and then they also started to do training with the staff on those policies and that’s still happening. And that to me was wonderful. Unfortunately it wasn’t an institution that’s part of the study but that’s the ripple effect (RA4).

The leadership hubs set context-specific goals for practice and policy change, and involvement in the hubs led to individual change for the members. RAs observed these individual changes. In reflecting on where the Kenyan hubs started versus where they ended up, one assistant described a hub that began with only four members and became ‘ a team of eight or nine ’. This along with seeing ‘ great growth in the nurses ’ was considered a ‘ great success ’ of PAR (RA3). In Jamaica, there was one hub member ‘ who at the beginning was very shy, very quiet, didn’t really participate ’, perhaps due to a lack of experience with research, but over time ‘ blossomed ’ and became ‘ very vocal ’ in meetings (RA1).

Individual changes fuelled collective changes. For example, in a final focus group, Kenyan hub members articulated changes that included members agreeing to ensure policy utilisation in their workplaces, and to use research findings ‘ for the improvement of the service delivery ’. Another HL explained that participation in the programme led to ‘ ideas of what we could do looking forward ’, for instance, HIV home care in Jamaica, ‘ to see how we could improve patients who … don’t come to primary care and they’re not in secondary care ’ (HL1). Another Jamaican hub wanted to change the method of voluntary counselling and testing in their institutions by drafting a policy (RA1). Kenyan hubs identified the unmet needs of nurses and other staff caring for HIV-positive patients as a problem, and planned to develop ‘ something known as awareness centre ’ (Kenya focus group 1).

The change depended on levels of interest, motivation and involvement. Recognising that the programme would not ‘ change everyone ’ and needed ‘ the leaders to get it moving ’ (RA2), RAs focused on identifying and supporting leaders whose motivation was high, who were ‘ keen no matter what, [to] take initiative ’ (RA debriefing interview). For example, a South African hub was led by a nurse ‘ with minimal research skills but very motivated ’ (RA debriefing interview). Other quotes demonstrate similar observations in Kenya, Uganda and Jamaica:

Members seem to have huge expectations from this project. When I thought about this later, I felt that I should have told them that the hub will become what they want it to be. If they want it to be a wheel of change, it will. The opposite is equally true (RA2).

They are very enthusiastic and they have big plans because our hubs are mixed with many different kinds of people … They are really focused now and they know what they want. Especially when we had the last workshop and they were able to identify their own projects (RA6).

… the willingness to participate never left at any point … At times it was challenging and difficult, however, we found ways to work around that … at the end of the day everybody had an input of some sort and was able to participate in some way (HL1).

This motivation was fuelled by new learning, particularly in regard to understanding the findings from specific projects and interpreting their meaning for specific workplaces. In this sense, change happened by connecting data with experience. One HL described when PAR fully made sense for her:

when the results of the initial findings were disseminated to us. That’s when I said, yes, this is really a problem and it’s a problem affecting nurses so it’s nurses who must do something about this … Then we went ahead to say what can we do? So through that process of asking ourselves and sharing we came up with a gap and an action goal (HL9).

RAs observed this change across all hubs. One stated, ‘ it is during the application that it becomes exciting for the hubs ’, adding that ‘ huddling ’ around laptops to review findings and draft evaluation project proposals, ‘ even if their computer skills weren’t so good ’ could not be ‘ underestimated [as] capacity building ’ (RA4 and 5 debriefing interview). One RA explained that as hub members were encouraged to discuss findings, stories from practice came out and they began to identify issues:

They started visualising, saying that these were the everyday situations that we’re living within [our] districts. For example, there was a hub that designed an intervention to look at HIV stigma among nurses. We were just presenting figures and percentages, but when they started talking, actually they were citing examples of some of the nurses that have been stigmatised. It was hard to seek care [because] they were the same services being offered at their [work]places … And they moved forward to develop the full proposal which was later funded (RA6).

As the same RA explained to the funder representative during a country visit,

this being a multi-country programme of research, different aspects of research apply differently in the different countries. Even within this data that is collected, some things may not be clearly understood by just mentioning them. So what happens is that after the data has been collected and analysed, it’s the hubs, because these are nurses that are facing these challenges every day,… that provide country specific interpretations to these findings (meeting notes, country visit, 2011).

Participants were careful to point out that interpreting research findings is not an easy or simple process, that is comprised of one activity like a presentation of findings to hubs and that is enough: ‘ it does take time and a lot of patience to share findings ’; hub members participated more when they had detailed summaries of findings explained to them, when they felt there was a trusting working relationship with the research team, including RAs who were open about what they did not know and when they needed to ‘ follow up and provide missing information later ’ (RA3, South Africa 2010 training notes). And, as one HL explained, hubs needed to recall repeatedly what they were supposed to be doing with the data:

This happened to all of us, even at the latest stage of implementation we’re still asking ourselves, what are we evaluating? So for us until we remembered that oh there is a baseline which was done. There was baseline information that we have about nurses’ knowledge, attitude and practice about. Then we are looking at it’ (HL9).

Another key dimension of understanding change is that it happens in and through vertical relationships. In envisioning the leadership hubs, the research team articulated ‘purposeful connectivity’ as a key concept. This requires learning to network, specifically to cultivate vertical relationships in the healthcare system, in an evidence-based and purposeful way that also draws from experiential knowing. This was translated through the participatory nature of the work, and hub members and RAs developed a critical consciousness of their power to affect change in vertical relationships and particularly through the vertical relationships within the Strengthening Nurses’ Capacity programme.

To get ‘political’ blessing we involve the District Medical Officer of Health (DMOH). The DMOH is in charge of all health services in a district and the secretary in the District Health Management Team (DHMT). We have had a strong partnership with the DMHT and this could make a positive contribution (Kenya field notes, 2010).

Another Kenyan example reflects the growing awareness and comfort of communicating in vertical relationships:

the decision that we made after the research [was that] we gave feedback to our … top decision-makers. And also since the project was coming to an end, we were wondering what next, [so] we gave feedback to the top decision-makers and they really supported us and I think they are going to follow up (Kenya FG1).

This comfort in vertical relationships worked both ways. Hub members who held management positions in the healthcare hierarchy also experienced new realisations about their place in effecting change. One focus group participant explained that she became aware of many gaps in her manager role: ‘ I find the hub to be very resourceful such that you realise that policy issues are just one aspect of leadership… so I am really yearning to get other aspects … ’ (FG Kenya).

The PAR approach as it manifested in the research team’s conduct, rather than the training content per se, is what participants identified as a unique feature of the changes they experienced. Specifically through these vertical relationships within the programme, hubs learned accountability in ‘ sharing with the people you’re meant to share it with ’ (RA6). The empowering effect of this realisation was expressed by several HLs as a kind of role modeling of new ways of doing things. One such example was the research team’s insistence on having signed institutional approvals for hub evaluation projects as well as time away from work. When asked to clarify, one HL explained,

… it paved the way for us. Because we had to think always about ethical considerations and the responsibility … now even if [a nurse is] going for two days [she must] seek for permission from immediate supervisor and they approve. Like the time we went to Kenya … it was very easy for us, we got their permission, written. So it a bit protected us because we are sure our supervisors are aware that we are in this place (HL9).

This role modelling was also evident in hub members’ descriptions of how the researchers and RAs communicated with them. Specifically, there were consistently firm messages about the obligations and expectations associated with the work of the hubs coupled with encouragement and implicit positive expectations of hubs’ capacity. This was important for the hubs, who at times felt that change would be nearly impossible. For example, the first author asked one HL to clarify a comment about feeling hesitant about her ability to effect change. She answered: ‘ Yes, we were seeing this as something that cannot be done. So we are seeing this as something that is big. We did not have the confidence … it is very difficult to influence policy ’ (HL9).

Role modelling was identified as part of the PAR approach. It was described by one HL as ‘ they are there, they are in the session, … they communicate at the level [where] we understand each other …’ (HL6). Regarding the country’s RA specifically:

If you come in with an idea he won’t say, no, no, no, it’s out of line or whatever. He’ll always go into … let me understand what you are trying to say so that you should reach consensus. There are people that say, no, that’s out. He’s not that kind of a person. He went to the bottom of an issue to really understand…

This HL credited the PAR approach, ‘ where you involve people, try to get their ideas, don’t just impose, get to the root of problems ’, as the basis for learning a different approach to communicating with people at different levels of power.

Finally, conceptualising change as the core of action-oriented aims in capacity development revealed two important considerations about PAR that derived from this programme. First, a key challenge of PAR is setting and meeting achievable objectives (making intentional changes), while remaining open to the mystery of unintended change that is particularly the case when capacity is the target. Developing capacity is a kind of change that is not necessarily seen in measurable or immediate ways. One RA articulated it this way:

It’s very difficult to pinpoint those actions … We can say for instance this capacity development workshop we had on this day, we can get immediate feedback from the participants involved in that workshop, but we can’t necessarily pinpoint anything happening afterwards. We can get feedback from the members in terms of maybe what happened at the institutions or things like that but you can’t really… say, here is my report in terms of exactly what happened with this. So I think the [participatory] action plays a big part but I think it’s very difficult … to flag them the whole time or to say for all the inputs these were the outcomes (RA4).

This links to a second consideration, that developing capacity rests on a good understanding of the capacities that already exist. In the case of the Strengthening Nurses’ Capacity programme, this was not neatly determined at the beginning, but became apparent as the hubs formed and specific strategies were carried out. In some cases, as RA6 identified, minimal or non-existent capacity in research meant that hub members needed first to learn about the research endeavour itself. RA2 explained it this way:

The main challenge is that people are at different levels in terms of capacity, in terms of research, in terms of involvement and so, because in PAR people are supposed to work together, sometimes it takes much longer for some people to get in cue.

Context and time

PAR, as manifested here, emphasises the need for openness and adaptability in terms of time and shifting social circumstances in different contexts, especially in an international collaboration. The context and time challenges influenced both hub membership and hub functioning. For example,

the initial members … in South Africa and the ones in, for instance, Uganda were a bit different. It did change over time because I did realise that it’s not necessarily how you present yourself but it’s what you know of your context and what you know of what you’re busy with … and even in Kenya it seems that … there’re people doing their Master’s degrees, there’re people doing their PhD degrees and … I think they have more support when it comes to research because they have this person they can talk to, they don’t necessarily have to approach the RA or somebody at a tertiary institution …. in Jamaica, well I think the picture there is very different because it’s a very small community, Jamaica is an island, it’s small … The people know of each other… (RA4).

Geographical proximities played a vital role in the establishment and activities of the hubs in all countries. Proximity was considered at early and later stages of planning the leadership hubs. Participants described it as the single most challenging feature of the PAR approach. The physical distance between hubs and countries revealed a kind of paradox of the PAR approach in this international programme; that is, capacity developed both because of and in spite of communication channels. Telephone and internet connectivity within countries was often poor or inaccessible and the associated costs with either of these means of communication were high. Kenya, South Africa and Uganda shared similar challenges in this vein, while in Jamaica these issues were perhaps less significant. For example, it was estimated that of all the hub members in South Africa, only three could reliably communicate by email, and some even had no access to fax machines, in which case the document would be faxed to a colleague who could pass it on. In short, material communication channels were often poor and this is where the relational channels of communication were critical.

All participants agreed that meetings in person were most valuable because, as RA4 explained, ‘ if you [only] talk to a person over the telephone … it’s not [as likely that] something [is] going to happen after the conversation ’. However, in-person meetings involved travel issues of scheduling and cost. For example, RA3 described a meeting that started almost three hours late with members still ‘trickling in’ later, which he knew was due to the distance and work assignments. The reverse situation also occurred for RAs visiting the hubs:

If I drove from here two and a half hours to the venue and there’s only three or four members, that was very difficult …. I remember one morning I was driving back from [hub location]… at three in the morning, because I included the night shift staff to also become aware of the research program… And another time it was eleven’ (RA5).

The value of working in close proximity to one another created opportunities for more frequent meetings, more time just to check in, which overall facilitated more familiarity and trust. One RA explained how much he came to value the ‘drop-in’ aspect of the work in conveying to hub members an inclusive ‘team’ attitude:

… those individuals that you said ‘hi’ to are the ones who attended or really tried to attend meetings. And when it came to doing the action plans, or for them to go to their supervisors, it was easier for those individuals because they had that relationship with the project, which made them feel good and … gave them the willingness to do more in their context (RA4).

There were also weather considerations, a point not often identified as a factor in conducting traditional health research, but an important one for PAR projects in places where access and infrastructure are key issues. In Jamaica and South Africa, for example, storms led to bridges being washed away at times when hub meetings were planned.

Workplace arrangements and conditions also played a significant contextual role throughout the programme. PAR assumes expectations about people participating and taking action to bring about change. The hub members’ ability to participate and take action in this programme was influenced by workload, schedules and other unforeseen demands that are part of the everyday nature of healthcare, especially in resource-constrained conditions. Yet, the hub members’ participation led to new learning, particularly regarding policies, and dissemination of their new learning with colleagues. In short, the workplace (and by association, the nature of the work) both dictated the level of involvement in the leadership hubs and reaped the beneficial outcomes of hub activities. This was an ever-present understanding throughout the programme. For example, ‘ I know in Jamaica we were able to pull ourselves away … not all the hub members every time, [but] a representative or two from each hub to be at a central meeting ’ (HL1).

Challenges experienced included movement of members and their replacement occasionally pulling the team back, since it would require bringing new members on board to enable the whole team to move at the same pace. Convening meetings became challenging with time since all members moved from their original stations and with added responsibilities, meetings were rescheduled many times (HL5).

Timing also factored in the Strengthening Nurses’ Capacity programme, not unlike other PAR projects. This theme is captured in the phrase, ‘ the importance of understanding the pace ’ (RA6), and the nuances of this are summarised in the following quote:

The two biggest things come to mind, one is time and the other one is expectations … I think there was so much happening … with the data collection and it was really hard to actually do the initial relationship building with the individual members of the hubs and getting them involved and making sure they attended all the different meetings because you can only really progress if there is continuous attendance by the members because … what we’ve realised in our context is they start to carry and learn and teach each other on what’s happening. So I think the time … the time and the amount of work that had to be done within the small timeframe – I think that was a big challenge. Because of the distances that the hub members travelled we were very limited … and finances also limited the context that we can have with the members. So we could have been together for one day and then we try to do so much in that one day that I kind of felt that we … the leadership hub members might have experienced a type of information overload, that they might have just taken little bits and snipits of what’s important to them or what got them hooked and they didn’t always walk away with the big picture (RA4).

The five-year timeframe of the programme came to be understood by RAs and HLs as a relatively short span for the full benefits of PAR to be seen. RAs were in agreement that only in the last year and a half of the project could substantial hub activity be seen, recognising that it takes time ‘ to get buy-in from everyone and get them all involved at the level of participatory ’ (RA2). One RA wondered if, because of the time pressure connected with project expectations, they rushed the evaluation proposal process with the persistent reminders of the deadlines.

Pacing therefore became important in planning and carrying out hub meetings. In Kenya, for example, recognising that the original timeframe of quarterly meetings proved to be insufficient for the learning needs and work of the hubs, the team increased the frequency of meetings to allow more time together. This also led to a decision to hold joint meetings of all hubs in Kenya, which not only addressed time issues but also generated richer discussion in the meetings. By contrast, Ugandans kept to the quarterly timeline, but invested more energy in giving lots of advance notice so that hub members could schedule their lives accordingly. The structure of meetings was also continually negotiated and re-adjusted according to members’ needs, and although noted as challenging, moving the agenda at the members’ pace of learning proved most valuable (RA3, RA4). Pacing at the international level also became apparent and the temporal dimension of PAR was expressed in several ways:

At one point the hubs were moving at different speeds so … what I learned [about] the action research was basically understanding the pace … the speed at which we should move with the different hubs and … in the interest of making sure that all do follow … So if it’s going to be participatory action research really it requires time, patience and perhaps mentoring based on those two factors … For example, if people in [one country] are ahead they can go but also give time for people in [another country] to move at that pace that will enable them to absorb things and make meaningful actions (RA6).

The message from me about PAR is that [researchers] need to be aware of the fact that people will come in at different levels of understanding, of interest … and if you want people to be able to understand the research it takes time (RA2).

Hub members identified the long-term commitment required of PAR: ‘ Other approaches do not require time and funds but comparatively, action research needs the same team members from the beginning to the end if reliable information is to be obtained ’ (HL4).

Embedded in the importance of pacing and context is the notion of commitment. RAs and HLs referenced commitment many times in describing their experiences of PAR, alluding to it as a character trait, a kind of tenacity or determination that is required in light of the contextual and timing hurdles. Participants seemed to describe their commitment to hub activities in a way that raises the question of volunteerism as a necessary component of successful PAR projects. For example: ‘ there’s been a lot of them for whom [the work of the hub] is really in their extra time ’ (RA2); and

there was sacrifice, definitely I could sacrifice time before the [evaluation] project was funded. I could sacrifice time and they fund the money for airtime, and I call these people … I had the feeling that if we had started something we really need to see results (HL9).

Closely related was an assumption that the action nature of PAR implied something ‘ongoing’ must come from the work along with a corresponding commitment to ensuring the work continues (RA6; HL6). This raises the need for critical reflection on how interest and motivation of the participants are taken into account in light of careful contextual understanding; that is, there are sound reasons why motivated people are unable to follow through on initiatives.

Values and assumptions

The articulation of values, and the assumptions that derive from them, is a hallmark of the PAR approach. Values were therefore expressed as the premises of the Strengthening Nurses’ Capacity programme and reflected in the proposal submitted for funding and in corresponding documents such as role definitions, guidelines for authorship, and through team practices such as the rotating chair position in team meetings. Examples included the following:

•   Through the use of PAR and authentic engagement, stakeholders will be able to address the root determinants of health and will enhance sustainability of health systems strengthening.

•   PAR enables researchers to be effective brokers for knowledge transfer between front-line nurses and policy-makers.

•   The potential for evidence uptake is enhanced by the development of relational capital and dynamic collaboration among front line nurses, decision-makers and researchers, which promote vertical rather than horizontal relationships.

A declaration of researcher values and assumptions at the beginning of a PAR programme is important as a kind of propositional knowing (Heron & Reason, 1997). However, the experiential knowing that emerged from being engaged in PAR led participants to question the status quo and re-examine their assumptions about health systems and the nursing profession, academia and research, and collaboration and equity.

With respect to the health system and the nursing profession, participants critiqued hierarchical models and reconciled misunderstandings of nursing. For example:

I think [PAR] is actually exactly what’s needed in our context because there’s such a sharp divide in terms of hierarchy on so many levels, you know, that there are … and I mean I’m not only speaking about the healthcare system but if we want to go there and speak only about the healthcare system the hierarchy is … it’s real and the persons at the front line don’t feel valued (RA1).

It has been known far and wide that nurses in general are difficult people to deal with, including myself … when I got into this project everybody asked will you manage to work with the nurses? But one thing that I said from the onset I said I think it’s an opportunity for me to try … I realised I had a very wrong perception of who nurses are … in the process of interacting with them I clearly realised … given their kind of work environment they would be happy to have somebody who listens and understands them (RA3).

There were also assumptions about academia and research, specifically regarding the differences between universities and healthcare institutions that they reflected on or questioned. As described in the Kenyan case, GLUK actively engages students and community members in projects, breaking down traditional ivory-tower stereotypes. An RA in another setting articulated how the academia practice setting relationship was beginning to shift as a result of the PAR approach:

We do place students at the different clinics and at the hospital, but that’s very formal, just over email … with the research and continuous conversations we’ve actually built a relationship whereby they phone us up … and say, we’ve got a community dialogue coming up next week and we would like to talk to them about alcohol abuse and HIV and AIDS. Is it possible that you can give us a brief summary of some of the literature out there that we could use to talk to the youth?… that’s something that happens while you’re just going on and you don’t realise what’s happening … because you would say, yes, we can do that and we quickly get someone to do the literature review and you get the one or two pages out to them. And after you realise, but wow this wouldn’t have happened two years back, … you can leave the idea with them that there is an institution that you can receive academic support from and they can in turn, in their institutions, leave a type of culture behind from the youngsters coming up in that institution that, you know what, there is that place called the university and there are those individuals and they can make research accessible to you and they can make it understandable to you and they can put it in a way that you can actually learn from that. You don’t have to … think because it’s an academic institution and they’re doing research it’s something only for academics (RA4).

PAR assumptions regarding how research is conducted also shifted RAs perceptions of the research endeavour more broadly. For example, RA3 stated:

the world has many problems and already there are many researchers with many research questions … pulling this kind of information and writing it as part of a report for the project … to me is about contribution to solving a problem in society, and it’s more effective than writing a thesis dissertation that is going to sit on a shelf because no one wants to look at it (RA3).

Other RAs highlighted assumptions regarding capacity of the participants in a PAR programme such as this:

I think people’s skills need to be put into consideration really, if there are specific results that you are looking at accomplishing … For example, if you want them to develop proposals, then the people you recruit should be the kind that can really write up. Sometimes beginning at zero you get frustrations from here and there, yet it’s not your problem and it’s not their problem … So for short-term outcomes, I think it needs to consider the skills. Like existing skills within the people and the skills that you may have to give them. Like if it’s participatory action research and along the way you have milestones that you want to accomplish, you can use the milestones to design the kind of mentoring that will make people accomplish those milestones, but also looking at the time in which the slower person can be able to accomplish something within that strict time (RA6).

It does take time and a lot of patience to share findings … [we] need to understand each of the hub members and their strengths and weaknesses. Sometimes they will sharply differ in what they want to discuss, [and] we need to know how to handle this kind of situation (RA3, SA training notes).

There were also assumptions expressed about collaboration and equity, notions of what these terms mean and how they are linked in PAR. Specifically, the understanding was that there cannot be one without the other if the approach is to be truly participatory. This is exemplified in RA1’s comment:

as a dynamic collaboration, we have been able to achieve the whole formation of the hub … working together and recognising their role and shared vision and goals and so on … that has been completely concretised here in our setting … to bring a group of persons together from different backgrounds, different settings etcetera and have them try to work together… to get them to understand and take on their role as a leadership hub member, I think that’s a major success that we have been able to achieve here.

Furthermore, one dimension of collaboration is the responsibility that comes with empowerment. Collaboration, according to RA2, implies ‘ putting all together and sharing [so] that everybody works hard ’. Yet, in the reality of the work itself, this assumption may not be shared by all participants; ‘ in the end you might find that you have participatory action research approaches but maybe not everybody was really involved in the whole research as I think they should have been ’ (RA2).

Finally, one RA talked at length about expectations in the Strengthening Nurses’ Capacity programme which seemed also to express the importance of explicating assumptions in any PAR project:

Individual expectations, when you have a joint leadership hub meeting and you have those individuals in front of you and you say, today we’re going to talk about this practice guideline or about policy development, then there is a certain expectation that you put on the table. And if you put it on the table you take responsibility for that and you would really like them to step away from that session being confident in what they’ve learned. So there’s expectations that you have on yourself, there’s expectations you have for the experts … and there’s also the expectations that you have in terms of capacitating these individuals so that they will have an impact that you will be able to measure at the end of the day … Speaking to the whole PAR approach at different stages there’s different expectations from different individuals and just keeping tabs on what those expectations are and how you would reach them is very difficult to do (RA4).

These themes highlight that assumptions about capacity and how best to develop it are important. Notably, in articulating values, the notions of capacity and power are closely aligned. As stated, PAR makes power explicit, and this programme’s emphasis on transparency in team processes and careful attention to support, rather than to more hierarchical ‘instruction’ or ‘directives’, are examples of doing so. Furthermore, respectful identification and acknowledgement of existing and required capacities of the leadership hubs were critical for effectively developing capacity.

C APACITY D EVELOPMENT AND THE P ARTICIPATORY W ORLDVIEW

The Strengthening Nurses’ Capacity programme serves as an informative case study of the intersection of capacity development as an objective and PAR as an approach to meeting such objectives. Notably, there are three distinctions in the programme’s conceptualisation of capacity development. It was understood as an ongoing process of empowering individuals, interdisciplinary teams and networks, and nations, so they are able to:

1)   identify health and health-related challenges

2)   conduct, develop and manage research and evaluation processes that answer these dynamic challenges as they arise

3)   contextualise, share and apply knowledge generated from research and evaluations.

The achievement of ‘strengthened’ capacity was seen to rest on the active participation of engaged stakeholders and decision-makers. Notable, however, is that the interviewees of this case study used the actual word ‘capacity’ far less than they used variations of the word ‘learn’ as they discussed the leadership hubs. The participants talked about learning opportunities, what they had learned, how learning happened, and the tangible results of their learning in terms of experiencing changes in themselves, their work and workplaces, and feeling empowered to ignite change in their colleagues.

The expression of capacity development as learning resonates with Heron and Reason’s (1997) ‘critical consciousness’ that is part of the intersubjectivity of the research process. As described in the related theme, the hub members encountered change through various learning opportunities – writing, presentations and linking data with their experiences to bring research findings into useable forms. They learned how to dialogue and in dialogue with stakeholders and decision-makers, found balance, as Heron and Reason describe, ‘within and between people of hierarchy’ (p. 257).

Heron and Reason (1997) also stress the collaborative nature of good research. The Strengthening Nurses’ Capacity programme reflected such collaboration, not only between the research team and leadership hubs, but more to the point, within the hubs, that were comprised of staff nurses, managers and leaders in regional health organisations. This deliberate crafting of hub membership created a space, in otherwise hierarchical systems, where vertical relationships could flourish, thereby reflecting the assumption that all knowledge is experientially based, including that of the researchers and decision-makers. Similarly, the call for the democratisation of the research process (Heron & Reason, 1997) was also heeded in this programme. Yet, a subtle clarification of this aspect of what it means to ‘collaborate’ in PAR is noteworthy.

Democratisation, or the meaningful involvement of participants throughout the research process, does not eliminate the need for leadership — from those with more expertise, for those with less. For example, the ‘bible’ of project guidelines was written by the senior researchers and programme coordinator primarily to set direction and to guide data collection across all the projects. This was taken up gratefully by RAs and country directors, interpreted not as a top–down authoritarian ‘rule’ but rather as a useful structure that, while serving the purpose of rigorous consistent data collection across countries, also offered a model, more generally, for conducting quality research studies, thereby enhancing the in-country research team members’ learning (capacity). Also, as RAs stressed, leadership hubs needed to be well understood in terms of how capacity both existed and was lacking, and researcher-directed strategies were necessary to address the latter.

Leadership from the senior researchers, while at times experienced as demanding on workload and timelines for each country, was also valued, welcomed and even requested. ‘Understanding the pace’ in completing the projects meant understanding individuals’ and organisations’ readiness for change. In short, in PAR, changes are enacted only when those involved feel ready, equipped, informed and confident. The success of collaborative efforts in PAR, then, must involve discerning such readiness and ensuring that those who feel less ready are supported by leaders.

The workplace challenges that members faced by participating in the hubs, as well as the unexpected positive linkages between academia and clinical care settings, call attention to questioning the readiness for change at the system level. For example, do individual changes in capacity necessarily translate into institutional capacity? Should this be the expectation of PAR programmes? Specific to the case of the Strengthening Nurses’ Capacity programme, how did the very existence of the leadership hubs affect institutional capacity for the organisations involved?

Heron and Reason (1997, p. 287) state that participation in itself is a form of practical knowing, ‘knowing how to choose to act — hierarchically, cooperatively, autonomously’. As we have seen in this case study, collaboration as the democratisation of research becomes more complicated in international research situations. The strengthening of capacity through PAR does not always fit neatly into measurable outcome definitions or funding timelines. This is particularly the case in international collaborations such as the one presented here. The anticipated as well as the ‘ripple effect’ positive changes in individuals, hubs and institutions were largely attributed, by participants of this study, to the PAR approach taken in the programme. The Strengthening Nurses’ Capacity programme also demonstrates, however, that the significant contextual challenges to relationship building and communication across different social, political, historical and geographical arrangements must be carefully considered and consistently accommodated through an interpretation of PAR that balances ideology with pragmatism.

Action Research Journal. (2012). Manifesto: Action research: Transforming the generation and application of knowledge. Available from: http://arj.sagepub.com/ (Accessed 24 May 2016).

Anaf, S., Drummond, C., & Sheppard, L.A. (2007). Combining case study research and systems theory as a heuristic model. Qualitative Health Research , 17 (10), 1309–1315.

Appadurai, A. (2006). The right to research. Globalisation, Societies and Education , 4 (2), 167–177.

Averill, J.B. (2002). Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qualitative Health Research , 12 (6):855–866.

Ayres, L., Kavanaugh, K., & Knalf, K.A. (2003). Within-case and across-case approaches to qualitative data analysis. Qualitative Health Research , 13 (6):871–883.

Bell, S., Morse, S., & Shah, R.A. (2012). Understanding stakeholder participation in research as part of sustainable development. Journal of Environmental Management , 101:13–22.

Borkman, T., & Schubert, M. (2009). Chapter 2 . Participatory action research as a strategy for studying self-help groups internationally. Prevention in Human Services , 11 (1):45–68.

Bradbury-Huang, H. (2010). What is good action research? Why the resurgent interest? Action Research , 8 (1):93–109.

Brydon-Miller, M., Greenwood, D., & Maguire, P. (2003). Why action research? Action Research , 1 (1):9–28.

CIA. (2012). Factbook 2012 . Available from: URL: https://www.cia.gov/library/publications/the-world-factbook/geos/ke.html (Accessed 24 May 2016).

David, M. (2002). Problems of participation: The limits of action research. International Journal of Social Research Methodology , 5 (1):11–17.

Davidson, P., & Page, K. (2012). Research participation as work: comparing the perspectives of researchers and economically marginalized populations. American Journal of Public Health , 102 (7):1254–1259.

Dworski-Riggs, D., & Langhout, R.D. (2010). Elucidating the power in empowerment and the participation in participatory action research: A story about research team and elementary school change. American Journal of Community Psychology , 45 (3–4):215–230.

Edwards, N., Kahwa, E., Kaseje, D., Mill, J., Webber, J., & Roelofs, S. (2007). Strengthening health care systems for HIV and AIDS in sub-Saharan Africa and the Caribbean: A program of research. Caribbean Journal of Nursing and Midwifery , 2:29–36.

Heron, J. (1996). Cooperative inquiry. Research into the human condition . London, UK: Sage.

Heron, J., & Reason, P. (1997). A participatory inquiry paradigm. Qualitative Inquiry , 3 (3):274–294.

Koch, T., Selim, P., & Kralik, D. (2002). Enhancing lives through the development of a community-based participatory action research programme. Journal of Clinical Nursing , 11 (1):109–117.

Reason, P. (1999). Integrating action and reflection through co-operative inquiry. Management Learning , 30 (2):207–225.

Reason, P., & Bradbury-Huang, H. (Eds.). (2001). Handbook of action research: Participative inquiry and practice . Thousand Oaks, CA: Sage.

UNHCR. (2013). 2013 UNHCR country operations profile – Uganda . Available from: http://www.unhcr.org/pages/49e483c06.html (Accessed 24 May 2016).

World Bank. (2012). Data & statistics, country group, 2012 . Available from: http://www.econ.worldbank.org/WBSITE/EXTERNAL/DATA-STATISTICS/0,,contentMDK:20421402~menuPK:64133156~pagePK:64133150~piPK:64133175~theSitePK:239419,00.html#Low_income (Accessed 24 May 2016).

WHO. (2009). Country cooperation strategy at a glance. Kenya . Available from: http://www.who.int/countryfocus/cooperation_strategy/ccsbrief_ken_en.pdf (Accessed 24 May 2016).

WHO. (2010). Country cooperation strategy at a glance. Jamaica . Available from: (Accessed 24 May 2016).

WHO. (2011). Country cooperation strategy at a glance. South Africa . Available from: http://www.who.int/countryfocus/cooperation_strategy/ccsbrief_zaf_en.pdf (Accessed 24 May 2016).

WHO. (2013a). Jamaica . Available from: http://www.who.int/countries/jam/en/ (Accessed 24 May 2016).

WHO. (2013b). Uganda . Available from: http://www.who.int/countries/uga/en/ (Accessed 24 May 2016).

Yin, R. (2009). Case study research. Design and methods (4th ed.). Thousand Oaks, CA: Sage.

Young, L. (2006). Participatory action research (PAR): A research strategy for nursing? Western Journal of Nursing Research , 28 (5):499–504.

Chapter 5 Process Evaluation of a Course Aimed at Improving Research Skills Emerging in Mexican Obesity Researchers

Mariane Héroux, Lucie Lévesque, Juan López Taylor, Edtna Jáuregui Ulloa & Ian Janssen

Childhood obesity is a global problem that calls for immediate action. While the rise in childhood obesity was first observed in Western countries and progressed at a moderate pace, developing countries are now witnessing similar increases, but at an accelerated rate (Popkin, Adair, & Ng, 2012). In fact, in the last decade, obesity rates have tripled in many low- and middle-income countries (Popkin et al, 2012). As a result, in addition to malnutrition and infectious diseases, obesity and its related chronic diseases are now recognised as being major health problems and research priorities in low- and middle-income countries (Lim et al, 2012; Lozano et al, 2012). For example, in 2010, the Ministry of Health in Mexico, in collaboration with the President of the country, launched a comprehensive, multisectoral national agreement and strategy for preventing and controlling obesity (Barquera Cervera, Campos-Nonato, Rojas, & Rivera, 2010). This call to action is not surprising seeing that the rate of obesity in Mexico has surpassed epidemic proportions and is now comparable to that of Canadian and American children (Ogden et al, 2011).

A substantial amount of research on childhood obesity has been completed in developed countries in an effort to gain a better understanding of the determinants of obesity and strategies for preventing and treating it (Sokar-Todd & Sharma, 2004). Far less research has been produced in developing nations such as Mexico (Janssen et al, 2010). This may be due to limited resources and a lesser priority given to research in the developing world (Popkin et al, 2012). For example, a study by Janssen and colleagues (2010) found that about 0.5% of global obesity research conducted in recent years (about 28 published articles per year) was completed by Mexican researchers, despite the fact that Mexico accounts for 1.6% of the global population (World Bank, 2010). In contrast, Canada accounts for 3.8% of global obesity research (Sokar-Todd & Sharma, 2004) despite the fact that it only accounts for 0.5% of the global population (World Bank, 2010). Clearly, high-income countries such as Canada have more resources to support research and research training in obesity and other disciplines than middle-income countries such as Mexico. Building or enhancing research capacity may be an effective and pertinent way in which high-income countries can provide middle-and low-income countries with guidance to address childhood obesity.

In order to gauge the impact of research capacity building, it is important to document and analyse the processes and outcomes of such initiatives. Others wanting to replicate a successful initiative will require details on how the initiative was designed and implemented, what problems were encountered along the way, and what changes are needed for improvement. A good programme implementation evaluation should provide a clear description of what the capacity-building (or other health promotion initiative) programme staff and participants did and what they experienced as they engaged in those activities (McGraw et al, 1994). This is often measured as programme reach, dose delivered and dose received. While definitions vary slightly among authors, there is a general consensus that reach refers to the proportion of the intended audience that participates in the programme, dose delivered refers to the number or amount of intended units or components provided, and dose received refers to the extent to which participants actively engaged with, interacted with, were receptive to, and/or used materials or recommended resources (Linnan & Steckler, 2002).

Unfortunately, to our knowledge few capacity-building efforts have been implemented in the field of obesity in low- to middle-income countries and even fewer have been evaluated. The Guide for Useful Interventions for Activity in Brazil and Latin America (GUIA) is an example of an initiative between the US Centers for Disease Control and Prevention (CDC) and Brazil that takes a knowledge translation approach (with aspects of research capacity building) to address physical inactivity, which indirectly addresses obesity. Among other things, GUIA strives to establish and build cross-national collaborative relationships with researchers, practitioners and institutions in Brazil, conduct formative research and evaluate ongoing community interventions (Pratt et al, 2010).

Another initiative, CAMBIO, was developed in response to the alarming rise in obesity rates in Mexico (Bonvecchio et al, 2009) coupled with the lack of research on the issue (Janssen et al, 2010). The CAMBIO programme, as a whole, is a knowledge translation initiative. The four main components of the CAMBIO programme are as follows:

1)   to facilitate and fund faculty and student exchanges and researcher visits

2)   to develop and fund a collaborative programme of obesity research between obesity researchers in Canada, Mexico and the US

3)   to build partnerships and promote networking, including the development of a national obesity network within Mexico

4)   to foster research training by developing and delivering an annual obesity research short course for Mexicans.

This chapter and study, focuses on CAMBIO’s fourth component. The specific objective was to conduct a process evaluation of the 2009/2010 CAMBIO short course in an effort to add to the literature pertaining to the evaluation of obesity-related research capacity initiatives. Although the short course was only one aspect of CAMBIO’s research capacity-building efforts, an evaluation was conducted on this aspect only. Essentially, the course was conducted in an effort to improve research skills among emerging Mexican researchers in the field of obesity and was developed to provide these researchers with the tools and knowledge needed to conduct such research with the ultimate goal of empowering participants and promoting confidence to take on the obesity problem in Mexico. The course utilised a ‘learn by doing’ approach (Lansang & Dennis, 2004) and sought to mentor students as well as to promote skill development.

Short course participants

Sixteen Mexican graduate students and new professionals (10 women) between the ages of 22 and 54 (median = 33 years) participated in the 10-day course taught by Canadian, American and Mexican faculty. Participants were selected through an application process that involved nomination by a faculty member working at a Mexican university or research agency and after nomination, submission of their curriculum vitae and a short essay on why they wanted to take the course. Sixteen applicants were selected out of 24 nominations.

Course description

The objective of the short course was to provide an intensive research training experience and to increase research skills within a small, select group of graduate students and new professionals with an interest in research in the area of obesity. The short course was designed to guide participants through the research process — from developing research questions to writing manuscripts — and included theoretical (ie, lectures, web tutorials) and practical (ie, labs, fieldwork) teaching approaches. The course was developed by CAMBIO’s primary investigators (Drs Ian Janssen and Juan López Taylor), other CAMBIO team members (Drs Edtna Jáuregui Ulloa, Robert Malina, Mark Tremblay, Lucie Lévesque and Louis Ortiz Hernandez) and CAMBIO’s management team (Donna Ivimey and Gabriela Ibarguchi). The course components were designed to engage students to learn and undertake the steps involved in conducting research in the field of obesity, using a population and public health disciplinary approach. The course curriculum was based on theoretical and empirical literature and the personal experiences and knowledge of the course instructors. The course instructors were selected based on their methodological and content expertise.

The main part of the course was divided into two in-person sessions that were six and four days long (in November 2009 and February 2010, respectively). The in-person sessions covered eight main domains:

1)   overview of the research process

2)   how to conduct literature searches

3)   how to develop a research plan

4)   how to evaluate and critique research papers

5)   how to obtain measures relevant for the fieldwork component of the short course research project

6)   how to create a database and accurately enter information

7)   statistics and use of the SPSS statistical software program

8)   scientific writing.

Table 5.1 displays the itinerary from the short course outlining the sub-topics covered within each of the eight domains, the amount of time spent on each sub-topic, and when pre- and post-course questionnaires were administered. Specific session objectives were also identified and are also displayed in Table 5.1 .

Table 5.1 Schedule of events for CAMBIO’s obesity short course and associated domains

Source: Authors

A common objective among all sessions was for students to gain proficiency in the topic area. Lectures were presented in Spanish and English, depending on the speaker’s proficiency in the Spanish language. A live translation was provided by a certified translator for all English lectures.

Students were also required to complete an internet-based human subjects ethics training module and test prior to attending the first in-person session, unless they had previously completed equivalent training. The ethics portion of the course was integral as ethics training, while a key aspect of conducting research and a requirement of the ethics boards that approved the short course research activities, was lacking at many Mexican universities. The course was offered in Spanish and included 12 modules. Further details about the ethics portion of the course can be found online ( https://www.citiprogram.org/aboutus.asp?language=english ).

Students also completed two assignments between the in-person sessions. For the first of these assignments, students were asked to write a brief two- or three-page report on one of the research measures used during the fieldwork component of the short course (see Table 5.1 ). Reports were to:

•   provide an explanation of what was being measured (psychological basis of the test)

•   provide an explanation of the relevance of the measure (eg, what does the measure tell us about the current or future health of the participants?)

•   discuss the objectivity, reliability, validity and limitations of the measure within child populations.

This assignment was to be submitted via e-mail within two months of the completion of the first in-person session. For the second assignment, the students were asked to develop a research question based on the data collected during the fieldwork component of the first in-person session, and to prepare the introduction and methods sections for a research paper that they would continue to work on (with guidance and assistance from the course instructors) during the second in-person session. This assignment was to be completed in time for the second in-person session. Students could complete assignments in Spanish, however those proficient in English were encouraged to do so in English so that they could work with the English-speaking faculty.

Finally, the students were asked to complete an assignment after the second in-person session. This assignment consisted of finishing the scientific paper they had started to work on prior to and during the second in-person session. While completing this last assignment, students also had the opportunity to maintain contact with the course instructors, primarily to receive feedback on written work and, if desired, to work towards submitting the paper for publication at a peer-reviewed journal.

Students were asked to bring their personal laptop computers to the in-person course sessions. If students did not have a laptop, they could borrow one from the course organisers. Each student was given a USB flash drive, which contained the course itinerary, assignment instructions and all PowerPoint lecture presentations.

Short course process evaluation components – process measures

Quantitative process evaluation data were collected prior to, during and after the course, and were used to conduct the process evaluation. The questionnaires that the participants were asked to complete were modified for the short course by CAMBIO team members (Donna Ivimey, Mariane Héroux and Gabriela Ibarguchi) and were based in large on evaluative instruments developed by Ottoson and Patterson (2000) and Woodward (2004). The short-course evaluator (Mariane Héroux) administered all of the questionnaires and observed the participants throughout the short course to measure the three process evaluation components — reach, dose delivered and dose received — as defined by Linnan and Steckler (2002). Each questionnaire is described in more detail below in relation to its use in measuring the above-mentioned process evaluation components. Sample questions from each questionnaire (translated into English) are provided in Table 5.2 .

Table 5.2 Sample questionnaire questions

Reach assessed the proportion of the intended target audience that participated in the programme. During the recruitment period, the CAMBIO programme managers attempted to recruit a large group of students from the Guadalajara, Jalisco region (a region of focus for CAMBIO’s research capacity building) and several students from other regions and states. One Kenyan student was invited as he was part of a similar research initiative that was being led by CAMBIO team members. Basic demographics obtained at baseline were used by the CAMBIO programme managers to determine if the participants attending the course fit the characteristics of the target population outlined prior to the start of the study (this being students or new professionals from Mexican institutions interested in obesity research). Attendance at each in-class session was also taken to calculate the proportion of students attending sessions.

Dose delivered recorded the number of intended units actually used. This was measured for the in-class components of the short course by comparing the course itinerary and prepared materials to what was actually taught and what materials were provided during the course. These observations were recorded by the CAMBIO programme evaluator.

Dose received measured the extent to which participants understood what they learned from the lessons that were taught, the degree of perceived knowledge uptake, whether or not participants reported that they continued to use what they had learned after the course was over, and if they were able to successfully apply the skills learned from the course.

Participant understanding of individual in-class lessons was assessed using the session-specific questionnaire. The questionnaire asked participants to rate how much new information was learned. The same five questions (amount of information learned during the session, likelihood of using the information in their current position, likelihood of using the information in the next six months, degree to which the presented information was perceived to be credible, and level of interest in learning more about the topic) were repeated at the end of each course section taught, as outlined in Table 5.1 . The participants’ answers were based on a 7-point rating scale, with 1 representing a negative response and 7 indicating a positive response. An overall mean score was created for each of the five questions by adding the participants’ numeric responses across all 21 session-specific questionnaires and dividing the total by the overall number of questionnaires (ie, answers for Question 1 across all 21 questionnaires were added together and then divided by 21 to obtain a mean score for Question 1 for each participant). The session-specific questionnaires also provided a place for comments where participants were able to provide feedback they felt was relevant to the lesson.

The degree of perceived knowledge uptake was obtained from the pre- and post-course repeated questionnaire. Before the course started, students were asked to complete a pre-course questionnaire, which was repeated at the end of the second in-person session (post-course repeated questionnaire). These questionnaires contained several demographic questions and questions about current knowledge and experience in the eight in-class course domains (research process, literature searches, research plans, evaluating papers, measures, database creation, statistics and writing). Each course domain consisted of nine questions rated on a 5-point scale, with 1 being ‘strongly disagree’ and 5 being ‘strongly agree’. An overall summary score (out of 45) was created for each question by summing the responses for each course domain.

The one-year follow-up questionnaire asked students how often they have applied the skills that they learned during the short course. The questionnaire consisted of eight items (each on a 5-point frequency scale ranging from ‘never’ to ‘daily’) related to the eight previously mentioned course domains. The questionnaire also asked students to list what they perceived as barriers to conducting research. Finally, space was provided at the bottom of the questionnaire for students to write whatever comments and feedback they felt was relevant. A one-year follow-up period was chosen as this allowed adequate time for students to apply the skills acquired during the course. The one-year follow-up also coincided with the funding provided to the CAMBIO programme to complete its research activities.

In addition, a select group of students were contacted 18 months after the course was completed and asked to write a short personal story expressing how they felt the short course helped them become a better skilled and more confident researcher. They were asked to address the following topics in their story:

•   their occupation before attending the course

•   why they were interested in taking the course

•   what they learned from the course

•   how the course helped them with their future career goals.

The personal stories helped illustrate to what degree the course was utilised by the students as well as how it impacted their careers.

Finally, the three assignments given to the students throughout the duration of the course (described in ‘course description’) were developed to give students a chance to apply the knowledge learned throughout the course and to demonstrate whether or not they understood the concepts.

Statistical analysis

All quantitative data was analysed using SAS 9.2 (SAS Institute, Cary, North Carolina). Scales from the session-specific and follow-up questionnaires were assessed using the FREQ procedure. Pre- and post-course repeated questionnaire scores (eg, knowledge and skills in the research process, conducting literature searches, developing a research plan, evaluating papers, obesity measures, creating a database, conducing statistics and writing a paper) were compared within the 16 participants using paired t-tests.

Each of the 16 participants completed a Bachelor’s degree, nine also completed Master’s degrees, two were medical doctors and one completed a PhD. Five participants were still students, while the remaining 10 had just entered the workforce. The students represented four Mexican universities (Universidad de Guadalajara, Universidad de Chihuahua, Universidad Autonoma de Colahuila and La Salle Universidad), one African university (Kenyatta University), one Mexican hospital (Hospital General Instituto Mexicano del Seguro Social) and one Mexican public health institution (Instituto Nacional de Salud Publica). In general, the response rates to the questionnaires were excellent. Each of the 16 participants returned the pre-course questionnaire and 100% of the session-specific questionnaires. Only one student did not complete the post-course questionnaire while four of the 16 students did not complete the one year follow-up questionnaire.

Short-course process evaluation components – process measures Reach

All of the participants met the intended target population identified by CAMBIO managers. There was also a 100% attendance recorded for all course sessions..

Dose delivered

All of the 21 course sections were taught as planned and all four assignments were delivered as planned, with only one not going according to plan. The one exception was for the data dictionary component of the first in-person session, wherein the topic was taught, but the students did not (as originally intended) create a dictionary due to time constraints (data entry and cleaning took longer than expected). All course materials were utilised as planned including PowerPoint presentations, handouts, the use of fitness testing equipment and the use of laptop computers for conducting literature searches, entering data, completing statistical analyses and writing scientific papers.

Dose received

Figure 5.1 shows the average results from the five questions addressed at the end of each of the 21 in-person class sessions. These questions were rated on a 7-point rating scale (1 = most negative response, 7 = most positive response). None of the 16 students had an average score of less than 4 out of 7 for any of the five questions. At least 10 (and up to 15) of the 16 participants had an average score of 6 or 7 out of 7 for all five questions. Thus, high ratings were reported for the amount of new information learned (Question 1), likelihood of using the information taught in current position (Question 2), expectation of using the information in the next six months (Question 3), belief that the information presented was credible (Question 4), and interest in learning more about the topic area (Question 5).

Figure 5.1 Session-specific questionnaire scale responses ( n = 15)

The results from the pre- and post-course repeated questionnaires, which were available for 15 of the 16 students, suggested that perceived knowledge uptake was achieved in different research process areas (see Figure 5.2 ).

Figure 5.2 Comparison of pre- and post-course questionnaire responses regarding sessions (n = 15). Statistically significant results from the t-test (p < 0.05) are denoted by a * Source: Authors

Although statistically significant (p < 0.05) differences from pre- to post-course findings were observed only for one of the eight individual domains (writing), there was a consistent pattern which indicated that the post-course scores for the self-perceived research skills were higher than the pre-course scores. Summary scores across all eight domains were higher post-course (mean = 205, SD = 59) than they were pre-course (mean = 174, SD = 66), but the differences were also not statistically significant (p = 0.20) (data not shown). It must be noted that the curriculum placed significant emphasis on writing activities (in terms of number of hours spent writing and one-on-one interactions with the faculty). Thus significant differences from pre- to post-course for this specific domain may be a reflection of the time dedicated to it.

Figure 5.3 shows how often the students reported that they used the skills learned across the eight course domains in the year following the completion of the in-person short-course sessions.

Figure 5.3 Results from the post-course questionnaire displaying how often students used skills learned in the 12-month period following course completion (n = 12)

This questionnaire was completed by only 12 of the 16 students. Eight of the 12 students prepared an ethics application within the year. Each of the 12 students conducted a literature search within the year, and 5 of the 12 students did so at least weekly. Nine of the 12 students evaluated a scientific paper/report in the year, and four of them did so weekly. All but one or two of the 12 students recorded/entered data, conducted statistical analyses and participated in the writing of a scientific paper/report in the year, but few participated in these activities on a weekly or daily basis.

In terms of the written assignments, 14 out of 16 students handed in the first assignment and 12 of 16 students handed in the second assignment. Although a formal evaluation of these assignments was not conducted, the course instructors who reviewed these assignments, for the most part, felt that they were not of a high quality as defined by Western research standards. Several students maintained contact with course instructors following the second in-person session in an attempt to continue working on their paper until it was ready for submission to a peer-reviewed journal. Five of these students subsequently submitted their papers for publication.

The personal stories submitted by students demonstrated the positive impact that attending the course had on their career paths. Students also spoke about the strong connections that they formed with peers and instructors. Two personal stories are shared in Box 5.1 below.

Box 5.1 Personal Story 1 (English translation)

When the CAMBIO short course started, I was finishing my undergraduate studies and was starting my social service at the Human Nutrition Institute at the University of Guadalajara.

When I found out about the CAMBIO course and that I could be part of it, I was very interested and excited to attend, given that I was starting my research journey and that I had a great interest in learning more about this process. Furthermore, my interest in the CAMBIO course grew as it was focused on childhood obesity which is a very important topic in relation to health and given that this was the topic I wanted to work on for the upcoming stages of my studies. At that time, I was considering starting my graduate studies and I was sure that this course would be the milestone to support my decision.

Once the course began, my expectations were fulfilled and were exceeded given the way we worked and the way the organisers taught. Some learning experiences that marked my development as a researcher are the detailed manner for doing fieldwork, the patience and care of the data entry, the clarity of the statistical classes, the great literature review and the initiation of a scientific article, among others. I also made great friends whom I still have contact with and who have helped me through my professional development. I had the opportunity to meet recognised researchers from different fields who are passionate about health topics and who shared their experiences and comments with all of the course attendees. It is clear to me that the research process is a lot of work, but all that it involves is great and satisfying.

At the end of the CAMBIO course, I continued writing a scientific manuscript which is currently under publication process and from which I have learned a lot. I am currently doing my master’s studies at the National Institute of Public Health and the experience I obtained from the CAMBIO course has been of great help, just as it will be throughout my professional career. I am very grateful to have been allowed the opportunity to be part of the CAMBIO course which I enjoyed and used as much as I could, and I hope I can collaborate with this great team again some day and that I keep in touch for a long time.

Personal Story 2 (English translation)

Before attending the CAMBIO short course, I was working at the Public Health Institute in Mexico, mainly as a researcher assistant. This was a challenging job, because I could not completely understand many aspects of my job. Thus I was only able to collaborate doing small things. I felt frustrated because since I had finished my undergraduate degree I wanted to work in research. My boss told me that the CAMBIO course was a great opportunity to learn more about research, and he helped me to contact the organisation committee and apply to the course.

I was lucky to be selected for this course. When I took it I was preparing, with a researcher team, a proposal to evaluate physical activity in adolescents. It was the perfect time to get some feedback from the course. I learned many useful things for my research project; starting from very simple ones (like making questionnaires), to strategies for cleaning and analysing data. This course also introduced me to many wonderful and generous people willing to teach and help me. Some of them are working at the Mexican National Institute of Public Health. Thanks to this course I had the opportunity to study English at Queen’s University and later to be admitted in their Master of Science programme.

Therefore, this course not only gave me background and tools for my work but it also defined the next steps in my career as a public health researcher interested in physical activity and the prevention of chronic diseases.

I really appreciate the opportunity that CAMBIO gave me. I am in my second year at Queen’s University, completing my course work, and I am enjoying very much my training and the people I have met.

Thanks to the CAMBIO course, I am convinced that research is what I want to do. I also think that junior researchers in Mexico need opportunities such as this. I hope more initiatives like this see the light in the future.

D ISCUSSION

This research training short course was implemented in Mexico in an effort to increase the skills and knowledge of emerging researchers in the field of obesity, with the end goal of instilling confidence in the participants who would be conducting research on obesity in their respective countries. The reported increase in students’ knowledge after completiing the course is encouraging and suggests one of two things:

1)   students’ understanding of and competency in each course domain increased as a direct result of their participation in the course, or

2)   students underestimated their competency prior to the course and came to realise that they had a higher level of knowledge and competency as the course progressed.

Regardless, the increase in perceived knowledge suggests that the short course helped instil confidence in the students to undertake research activities in the future. Furthermore, the increase in perceived knowledge may have contributed to the reported use of several research-based skills in the year following the short course (see Figure 5.3 ), as well as to their participating in research activities and conducting research.

Although very few research capacity-building initiatives have been developed and evaluated in the field of obesity, there is evidence from other research disciplines that such initiatives can impact knowledge and research productivity (ie, grant applications, publications, etc). For example, a nine-day research training workshop offered by the University of Pittsburgh to healthcare professionals in Pakistan resulted in increased local research capacity (Dodani & LaPorte, 2008). The course focused on basic epidemiology, biostatistics, genetic epidemiology and international health, and used face-to-face and video-teleconferencing delivery methods. Student knowledge on research methods was tested before, during and after the course using questionnaires. Results showed that content knowledge across course subjects increased by up to 36% in the face-to-face group and by up to 24% in the video-teleconferencing group. Furthermore, knowledge was retained one year following the course (Dodani & LaPorte, 2008). Unfortunately that study did not measure the participants’ research outputs after course completion. However, other findings suggest that an increase in research knowledge may translate into an increase in research output (White, 2002; 2004).

While CAMBIO’s short course on obesity resulted in a self-perceived increase in knowledge and reported use of skills in the year following course completion, it remains to be seen if this will translate into an increase in research productivity by the short-course participants. However, the networks and connections the students made with the faculty members during the course resulted in the pursuit of additional, more intensive training by some of the students. For instance, a connection made between one of the students and faculty members at the short course led directly to that student completing a Master’s degree in Canada under the supervision of the faculty members in question. It also led to the initiation of several research collaborations between the student’s home institution in Mexico and Queen’s University in Canada. This newly formed partnership has therefore benefited both the student and the faculty members and will likely continue to do so for years to come. The full impact of this training may take years to come to fruition and thus it was not feasible to measure within the current study.

Unfortunately, not all students in CAMBIO’s short course completed the course assignments. This was disappointing to the course organisers and teaching faculty, who were discouraged by the lack of commitment shown by certain students towards their independent work. The organisers and teaching faculty dedicated a significant amount of their time outside of the in-class sessions to prepare for the short course, and would have liked to see the same level of commitment from students. It should be noted that these assignments were to be completed between the two in-class sessions when students had other full-time commitments including school, work and family obligations. Thus, the incomplete assignments are not necessarily a reflection of a lack of motivation or poor time management, but perhaps a reflection of limited time, which was mentioned by most as a barrier to conducting research. It must be noted that many researchers in lower-income countries produce research on their own time (ie, outside of their paid hours), inevitably affecting the amount of time available for research (Costello & Zumla, 2000; Trostle & Simon, 1992). Thus, in order to ensure that all assignments were completed, it may have been helpful to give students more time to work on them during the in-class sessions.

Several other limitations to research in lower-income countries have been identified in the literature and deserve mention. Some of the more prominent of these include limited resources, limited accessibility to the scientific literature, limited funding opportunities, limited time and limited emphasis placed on publishing (Adair, 1995; Costello & Zumla, 2000; Popkin et al, 2012). Although the Western research model is dominant, lower-income countries do not necessarily have the recourses (financial and physical) to adopt this model themselves. Thus, a research short course, such as the one evaluated in the current study, may be very successful at increasing participant research knowledge and competency, but will not necessarily translate into increased research production. Increased research production will be impacted by the various systemic barriers to research encountered by participants after course completion.

Working within the international partnership frame of reference resulted in several challenges and lessons learned by the CAMBIO team and teaching faculty. See Box 5.2 for comments by the CAMBIO management team (Donna Ivimey and Gabriel Ibaraguchi).

Box 5.2 Challenges experienced by the CAMBIO team

Improving research skills in Mexico cannot be understood without considering the context under which researchers in Mexico work. Most of the short course participants were from a university or university-affiliated hospital and the focus of Mexican universities is primarily on teaching with research a marginalised activity in terms of recruitment, resources and systems to acknowledge and reward research productivity. While the short course may have effectively taught important research skills to individuals, without addressing the institutional environment to which these young researchers must return, it is hard to expect the course to have long-term impact.

Enrolment in the short course was limited to junior researchers from different regions in Mexico. Targeting representatives from the regions was necessary to counterbalance the centralisation of research expertise in Mexico City. The majority of participants were from outside of Mexico City and the process for identifying and nominating candidates was transparent. However, future courses should reconsider whether to include senior researchers and university research administrators. Given that higher-level administrators play a significant role in supporting (or impeding) research activities through departmental and institutional processes, it may be beneficial to include them in such a short course or parallel workshop, where they could better understand and appreciate the various steps involved in conducting research.

Streamlined bureaucratic processes and access to research infrastructure is something the North takes for granted. Due to rules at the University of Guadalajara relating to import and acquisition of research equipment, it was easier for the CAMBIO team to order and purchase course-related research equipment and software through Queen’s University, receive it in Canada and then donate and ship it to the University of Guadalajara. Even with these measures, the University of Guadalajara withheld releasing paedometers (a small, inexpensive tool for measuring steps) up until the day before the fieldwork was set to begin.

Access to online journals is another critical research tool that is not widely available to researchers at Mexican institutions. Special permissions allowing access to IDRC databases were granted for the duration of the short course. Devoting a full day to teach and conduct literature searches would be useful for future short courses and would help demonstrate to university administrators the need to purchase online journals to support both research and teaching.

The short course was intended as an individual training and learning-by-doing initiative and was not designed to address institutional capacity directly. The culture of Mexican universities was not fully appreciated by the Northern partners. However, given the significant role institutions play in promoting or impeding research, future courses should look to incorporate parallel sessions for senior research administrators and university leaders.

As with all research, this study has several limitations that must be addressed:

1)   Only 15 Mexican obesity researchers participated in the course. Although, only 8% of the Mexican population holds a Bachelor degree (Santibanez, Vernez, & Razquin, 2005) and, of this 8%, few conduct obesity research, it is certain that eligible students were missed in the nomination process.

2)   Additional time for course delivery and practice would likely have benefited students. Unfortunately this was not feasible due to budgetary constraints and the fact that teaching faculty and students were already taking considerable time away from their institutions to attend and prepare for the course.

3)   Language barriers were an issue as several students spoke little or no English and several instructors spoke little or no Spanish. Although translators were onsite, translation errors are likely to have occurred resulting in miscommunication.

4)   Networking and mentoring were not measured during the evaluation, but both were mentioned as being important aspects and benefits of the course by both the students and teaching faculty.

5)   Institutional infrastructure was not evaluated and is likely to have contributed to the continuation of skill use post-course. While students were provided with the necessary tools to conduct research during the course (ie, access to full-text journals, equipment for data collection, etc), access to such tools post-course depended entirely on their availability within the students’ home institutions.

Despite the above-mentioned limitations, it must be noted that this short course is one of the first research capacity initiatives aimed at increasing obesity research in Mexico. Such an initiative is extremely important as obesity rates are increasing rapidly in Mexico (Bonvecchio et al, 2009) and researchers need the tools and skills to conduct research which can be used as evidence to influence policy and change. Nonetheless, training programmes alone may not be sufficient to support the ongoing use of skills. Without ongoing support and adequate resources, the use of skills learned is difficult to sustain (Lansang & Dennis, 2004).

Courses such as this could potentially help increase the research skills of those interested in doing research in low- to middle-income countries, which may in turn empower countries to address the problem and give researchers the tools and support to conduct research in the field of obesity. Future studies are needed with similar courses being presented and evaluated within different population groups.

Adair, J. (1995). The research environment in developing countries: Contributions to the national development of the discipline. International Journal of Psychology , 30 (6):643–662.

Barquera Cervera, S., Campos-Nonato, I., Rojas, R., & Rivera, J. (2010). Obesity in Mexico: Epidemiology and health policies for its control and prevention. Gaceta Medica de Mexico , 146 (6):397–407.

Bonvecchio, A., Safdie, M., Monterrubio, E.A., Gust, T., Villalpando, S., & Rivera, J.A. (2009). Overweight and obesity trends in Mexican children 2 to 18 years of age from 1988 to 2006. Salud Pública de México , 51 (4):S586–594.

Costello, A., & Zumla, A. (2000). Moving to research partnerships in developing countries. British Medical Journal , 321 (7264):827–829.

Dodani, S., & LaPorte, R.E. (2008). Ways to strengthen research capacity in developing countries: Effectiveness of a research training workshop in Pakistan. Public Health , 122 (6):578–587.

Janssen, I., Jiménez Alverado, J., Gonzalez, C., Ibarguchi, G., Héroux, H., & López Taylor, J. (2010). An overview of obesity research in Mexico. Revista Panamericana de Salud Publica/Pan American Journal of Public Health (submitted).

Lansang, M.A., & Dennis, R. (2004). Building capacity in health research in the developing world. Bulletin of the World Health Organization , 82 (10):764–770.

Lim, S.S., Vos, T., Flaxman, A.D., Danaei, G., Shibuya, K., Adah-Rohani, H., & Ezzati, M. (2012). A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: A systematic analysis for the Global Burden of Disease Study 2010. The Lancet , 380 (9859):2224–2260.

Linnan, L., & Steckler, A. (2002). Process evaluation for public health interventions research: An overview. In L. Linnan & A. Steckler (Eds.), Process evaluation for public health interventions and research (pp. 1–24). San Francisco: Wiley & Sons.

Lozano, R., Naghavi, M., Foreman, K., Lim, S., Shibuya, K., Aboyans, V., & Murray, C.J.L. (2012). Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: A systematic analysis for the Global Burden of Disease Study 2010. The Lancet , 380 (9859):2095–2128.

McGraw, S.A., Stone, E.J., Osganian, S.K., Elder, J.P., Perry, C.L., Johnson, C.C., & Luepker, R.V. (1994). Design of process evaluation within the Child and Adolescent Trial for Cardiovascular Health (CATCH). Health Education Quarterly , 2:5–26.

Ogden, C.L., Gorber, S.C., Rivera, J.A., Carroll, M., Shields, M., & Flegal, K. (2011). The epidemiology of childhood obesity in Canada, Mexico and the United States. In L.A. Moreno, I. Pigeot & W. Ahrens (Eds.), Epidemiology of obesity in children and adolescents (pp. 69–94). New York: Springer.

Ottoson, J.M., & Patterson, I. (2000). Contextual influences on learning application in practice: An extended role for process evaluation. Evaluation and the Health Professions , 23 (2):194–211.

Popkin, B.M., Adair, L.S., & Ng, S.W. (2012). Global nutrition transition and the pandemic of obesity in developing countries. Nutrition Review , 70 (1):3–21.

Pratt, M., Brownson, R.C., Ramos, L.R., Malta, D.C., Hallal, P.C., Reis, R.S., & Simoes, E.J. (2010). Project GUIA: A model for understanding and promoting physical activity in Brazil and Latin America. Journal of Physical Activity & Health , 7 (2):131–134.

Santibanez, L., Vernez, G., & Razquin, P. (2005). Education in Mexico: Challenges and opportunities [documented briefing]. Santa Monica: RAND Corporation.

Sokar-Todd, H. B., & Sharma, A. M. (2004). Obesity research in Canada: Literature overview of the last 3 decades. Obesity Research , 12 (10):1547–1553.

Trostle, J., & Simon, J. (1992). Building applied health research capacity in less-developed countries: Problems encountered by the ADDR Project. Social Science & Medicine , 35 (11):1379–1387.

White, F. (2002). Capacity-building for health research in developing countries: A manager’s approach. Revista Panamericana de Salud Publica/Pan American Journal of Public Health , 12 (3):165–172.

White, N.W. (2004). Capacity building for research: Report on a UNION strategic planning workshop. Report of a workshop held during the International Union Against Tuberculosis and Lung Disease World Conference on Lung Health, Montreal, Canada, 7 October 2002. The International Journal of Tuberculosis and Lung Disease , 8 (7), 910–914.

Woodward, G. (2004). Health promotion capacity checklists: A workbook for individual, organizational and environmental assessment . Saskatoon, Canada: Prairie Region Health Promotion Research Centre. Available from: http://www.usask.ca/healthsci/che/prhprc/programs/finalworkbook.pdf (Accessed 24 May 2016).

World Bank. (2010). World development indicators 2010 . Washington, DC: World Bank. Available from: http://data.worldbank.org/data-catalog/world-development-indicators?cid=GPD_WDL (Accessed 24 May 2016).

Chapter 6 Building a Framework to Strengthen Research Capacity within the Caribbean Region

Martin Forde, Karen Morisson, Neela Badrie & Eric Dewailly

As in most developing regions of the world, there is an ongoing need to develop local research capacity and capabilities in the Caribbean region. In particular, there is a pressing need to strengthen and operationalise the concept of integrative multi-institutional and disciplinary research. It should make allowances for organisational inputs at multiple levels (communities, non-governmental organisations [NGOs], government), and incorporating sectoral perspectives in a way that is practical and sustainable. This is especially challenging to execute in the Caribbean, not only because of the difficulties that arise when trying to change deeply entrenched historical and cultural norms, but also due to the fact that this region is characterised by multiple, diverse small island states, with each island state having unique and divergent economic capacities and political will to support research activities. Indeed, any attempt at creating a homogenous Caribbean-wide research capacity among these heterogeneous island states poses several challenges and issues that need careful attention in order to arrive at initiatives that are workable and sustainable.

Given the context provided above, it is not surprising that research capacity is scattered and unevenly spread across the Caribbean. The key players — academics, researchers, industry professionals, technocrats and research-oriented institutions — that would be needed to respond to ongoing and emerging environmental health challenges in the Caribbean region, already exist. Missing, however, are the knowledge and skills to integrate and network these fundamental ingredients of capacity development in organic, real-time and innovative ways. Thus, the case is made for a considered, deliberate effort to build a framework that integrates all of the above-listed elements, ultimately to improve the research capacity and capability of researchers and health professionals throughout the Caribbean.

The Caribbean EcoHealth Programme (CEHP) is a collection of several thematic research programmes, each of which has human and material capacity and capabilities, and development components incorporated into them. The overall CEHP was structured such that it facilitated and encouraged the bringing together of Caribbean- and non-Caribbean-based academic, governmental, NGO and technical organisations to address key gaps in the Caribbean region as they pertain to the environment and human health. Thus, an overarching goal of the CEHP was to improve the capacity and capability of public and environmental health professionals in the Caribbean Community (CARICOM) to respond in integrative ways to ongoing or emerging epidemiological and environmental health challenges by means of multidisciplinary interventions.

A fundamental CEHP objective was to build a multi-disciplinary and multisectoral team of endogenous professionals that would be capable of pursuing and utilising integrated approaches in solving public and environmental health problems within the Caribbean region. Thus, key parts of the CEHP’s mission were as follows:

1)   To work collaboratively with Caribbean professionals to identify critical regional knowledge and capacity development needs

2)   To build, enhance and strengthen integrated approaches to conducting research on the previously identified research and capacity development needs

3)   To develop, enhance and strengthen the structural and cultural mechanisms that would lead to the creation of a sustainable team of endogenous professionals who would have the confidence and access to resources to come up with solutions to the problems that are experienced in this region of the world.

D ESCRIPTION OF THE CEHP R ESEARCH P ROGRAMME AND O BJECTIVES

In 2006, using a highly collaborative and participatory process, a culminating grant proposal development workshop meeting was held in Trinidad. It was attended by several leading Canadian and Caribbean environmental and public health professionals drawn from both regions’ major technical institutions and knowledge generating centres. From Canada, there were representatives from Laval University, Guelph University, the Public Health Agency of Canada and the Institut national de santé publique du Québec. From the Caribbean, there were representatives from the Caribbean Epidemiology Centre (CAREC) located in Trinidad, the Caribbean Environmental Health Institute (CEHI) in St. Lucia, the University of the West Indies (UWI) in Trinidad, and St. George’s University (SGU) in Grenada. A representative from the regional Pan American Health Organization (PAHO) was also invited to participate in this grant proposal development workshop meeting.

Using an ecosystemic approach, commonly referred to as ecohealth, attempts were made to involve potential end-users in the formulation and design phase of the grant proposal. This approach integrates multiple specialist disciplines along with potential research users (eg, members of the affected community) both at the design phase of the research programme and throughout its implementation (Charron, 2012). Thus, in addition to the above-listed Caribbean, Canadian and para-regional professionals, several government representatives from St. Lucia, Suriname, Guyana and Trinidad and Tobago actively participated in the proposal planning workshop. The CEHP, which was borne out of this collaborative process between researchers and end-users, identified and proposed the following seven key knowledge gaps and capacity development research programmes as important for the region:

1)    Persistent organic pollutants (POPs) study: to determine and evaluate exposures to environmental pollutants such as persistent organic pollutants, heavy metals (mercury and lead) and several commonly used pesticides

2)    Zoonotic infections (ZI) study: to determine and evaluate exposures to zoonotic infections

3)    Foodborne burden of illness (FBOI) study: to provide additional financial support to fund and expand on PAHO/CAREC’s ongoing foodborne burden of illness studies. The CEHP collaborated with these international and regional health institutions to provide additional funding to expand and support the conduct of these studies

4)    Rainwater harvesting (RWH) study: to provide additional financial support to fund and expand the CEHI’s efforts to evaluate and determine the microbial quality of rainwater harvesting systems

5)    Recreational water quality (RWQ) study: to determine and evaluate appropriate microbial water quality indicators for recreational waters in the Caribbean

6)    Eco-toxicological water (ETW) studies: to conduct eco-toxicology water studies to evaluate the quality of drinking water supplies

7)    Food safety training (FST) programme: to develop a degree certification programme in food safety and management.

As previously mentioned, built into each CEHP research programme was a local human capability development component. A programme-wide key goal was to help create and strengthen sustainable communities-of-practice that could successfully come up with solutions to the problems found in each of these thematic areas. Where appropriate and feasible, in addition to the goal of building and enhancing local and regional human resource capacity and capabilities, a secondary goal to develop material resource capacity, such as laboratory equipment and supplies, was also undertaken.

Central to the CEHP’s human and resource development mission was its use of the Atlantis Mobile Laboratory (AML). In 2003, the Canadian Foundation for Innovation and the Québec government funded researchers from Laval University to build the AML, a $3-million mobile laboratory consisting of six 20-foot containers retrofitted to function as a self-contained laboratory. Three of the containers were outfitted to be chemistry, microbiology and eco-toxicology laboratories; the other three served as offices and living quarters, a service unit which housed the standby generator and workshop tools, and a storage container for bulky items and transhipment of a field vehicle and boat.

Given its mobile capabilities, the AML allowed the CEHP to move from one Caribbean island to another and to respond to specific local- and regional-based calls for research and capacity-building opportunities. This, in turn, helped to encourage professional development within the Caribbean region. The AML provided an immediate analytical capacity that was not previously available on the islands to which it was moved. The AML allowed the CEHP to provide local governments with an opportunity to carry out environmental monitoring activities that were demand driven and based on self-identified national priorities. Most importantly, the AML enabled the CEHP to adopt a more collaborative and integrative approach to research.

The AML also provided the CEHP with the unique opportunity to help develop and enhance local laboratory technicians’ capabilities by exposing them to relatively new analytical techniques. Several pieces of lab equipment used in the AML, as well as the analytical techniques employed, were absent in many Caribbean laboratories. Thus, many local lab technicians were given the opportunity to receive training on how to use equipment and techniques. While the AML was the focal point for a wide range of CEHP activities — including research, capacity building, training, promotion and outreach — not all of the CEHP research and capacity-building efforts revolved around the lab. The foodborne burden of illness and the rainwater harvesting studies, for example, worked directly with other national and regional laboratories and helped to upgrade both the human and equipment capacities of these laboratories.

The multi-institutional operational structures and the collaborative frameworks that were put in place to execute CEHP research programmes are shown in Figure 6.1 . The institutions and programme leaders for each of the CEHP research programmes are outlined in Table 6.1 . During the five-year period of the CEHP, the supporting organisations changed and evolved. Each CEHP research programme was significantly expanded and in some cases, the nature and scope of the proposed research programme dramatically changed due to multiple factors. These have been discussed in detail elsewhere (Forde et al, 2011).

Figure 6.1 Caribbean EcoHealth Programme (CEHP)

Table 6.1 List of core CEHPs, their leaders and their leaders’ respective institutions

We will use the modified version of Cooke’s theoretical framework (Cooke, 2005) to provide examples of how the CEHP attempted to meet its research capacity development goals as well as to highlight some of the challenges and issues that arose. Two dimensions, namely structural levels of development activity and guiding principles of capacity building, will be applied to illustrate the capacity development activities of the CEHP.

D ESCRIPTION OF THE CEHP’ S S TRUCTURAL L EVELS OF C APACITY D EVELOPMENT A CTIVITY

Structural level: individual.

Several CEHPs specifically had individual capacity-building and development components built into them. For example, in the persistent organic pollutants research study programme, rather than collect samples and carry out analyses using outside expertise, the CEHP hired and trained local nurses, senior laboratory technicians and other health professionals to do this on its behalf. Furthermore, on each island where an AML was located (Grenada, Dominica and Barbados), local laboratory technicians were hired and trained to operate it.

The number of local people trained on each island by the persistent organic pollutants research study programme is given in Table 6.2 . These locally hired people were trained in research methodology and protocols, research ethics, and in the management of human participants and bio-samples. The goal was to ensure that at the end of the study, the trainees would have the skills and confidence to conduct this type of research within their countries whenever required or desired.

Table 6.2 List of personnel trained in the persistent organic pollutants study

* These include hired persistent organic pollutants study country coordinators, chief epidemiology officers and other government personnel.

# Local laboratory technicians were hired and trained to run the AML on each island.

The AML also allowed the CEHP to facilitate several training programmes and workshops to the benefit of the local countries where the lab was located (see Table 6.3 ). Additionally, at least eight Caribbean and four Canadian graduate students benefited from being directly engaged in AML-based CEHP-related research and training projects.

Table 6.3 Human resource development and enhancement facilitated by the CEHP’s AML

For CAREC’s regional foodborne burden of illness studies, local government representatives, including local graduate students from Caribbean universities, were identified and trained to conduct the studies on their respective islands. A Grenadian student pursuing an MSPH degree was selected for the role of country coordinator to manage and conduct the BOI study in Grenada. Similarly, a Trinidadian MPhil student was chosen to be the Trinidad and Tobago foodborne burden of illness study coordinator. Both students used their country-based foodborne burden of illness studies as the basis of their Master’s thesis, and gained valuable experience working alongside CEHP researchers and ministry of health personnel. Additionally, on each island where a foodborne burden of illness study was conducted, laboratory technicians from different laboratories — including the national clinical, food and university environmental laboratories — were trained to identify and report on an expanded list of foodborne pathogens. Government epidemiologists and surveillance personnel in the field were trained in the collation and statistical analyses of data specific to the foodborne burden of illness study and other similar research projects. Overall, CEHP funding supported CAREC’s foodborne burden of illness programme by building capacity in eight countries in the areas of conducting foodborne burden of illness studies, epidemiological household population surveys, laboratory isolation of foodborne disease pathogens, database development, analysis of surveillance and research data, and the writing of peer-reviewed papers for publication. In total, over 1 000 people, including epidemiologists, laboratory technicians, surveillance nurses and doctors, received some sort of training under the BOI research programme.

The CEHP also mounted other programme-wide training initiatives such as one-week courses on systematic reviews and meta-analysis, and in ocean and human health (see Table 6.4 ).

Table 6.4 Non-AML training and capacity-building activities conducted by the CEHP

Structural level: teams

For each CEHP research programme, a team of academics, practitioners and potential end-users was created. Within each of these teams, the goal was to develop and create an organisational infrastructure that encouraged research capacity development opportunities by reducing barriers, facilitating mentorships, and fostering regional and non-regional collaborations and networking, all anchored in the context of addressing local and regional research capacity needs.

Graduate students from partner CEHP universities received training and research experience that otherwise would not have been possible. This is exemplified in the case of a Caribbean AML technician who was hired to operate the AML in Dominica. Due to the extensive training she received both in Dominica and later in Québec, Canada, she was asked to stay on with the lab when it moved to Barbados. She is now working along with one of our core research partners at UWI on a research project documenting the type of chemical contaminants in Caribbean marine waters. As a result of the network that was established by CEHP, she was also able to secure funding from the Canada Caribbean Scholarship Programme to pursue training at Trent University. She is now using this training to help assess the level of pharmaceutical and endocrine-disrupting chemical pollutants in the Caribbean marine environment.

Structural level: organisational

If one were to view the CEHP as an organisation, then it could be said that from an organisational point of view, both local and external institutional and professional management structures were created to facilitate the best mix of skills and knowledge within each CEHP research programme.

Capacity building was enhanced by the informal organisational structure of the CEHP that drew on the skills of all four principal investigators (PIs) as well as key individuals from the AML and core research programmes. While the larger team met annually at programme team meetings, throughout the year various team members took on responsibilites and initiatives to keep the programme moving forward. Early on in the programme, the team made a conscious decision to meet as individual research project teams, and to prioritise full team meetings to share information, discuss results, leverage opportunities and enhance interdisciplinary collaboration and group learning. While timing issues impeded the realisation of having the different research programmes work together on joint projects as was originally envisioned, this commitment to information exchange and collaboration has provided a strong foundation for future work.

Structural level: supra-organisational (networks and support units)

A core objective of the CEHP was to improve and expand the Caribbean’s community-of-practice network ability of addressing environmental and public health problems. A key feature of these communities-of-practice is that they seek to integrate both regional and external professionals, along with their respective organisations, into sustainable networks that not only do research but also look for opportunities to develop and nurture research capacity in the Caribbean.

The AML has played a particularly important role in creating a unique and effective conduit where the academic community can meaningfully link with the research needs of Caribbean regional institutions, and help create, foster and support a team of engaged professionals interested in researching the link between human and environmental health issues. More generally, the CEHP has made a tangible contribution to improving the chronic shortage of qualified professionals to run local and regional laboratories that oversee the implementation of public health and environmental programmes in this part of the world.

An exciting outcome of the CEHP’s efforts to foster and nurture regional and local multidisciplinary teams is the tremendous amount of interest its research programmes generated throughout the region. For example, the persistent organic pollutants research study programme was expanded to include six additional countries. CAREC’s foodborne burden of illness studies generated so much interest and demand that it was expanded to 10 countries. Due to multiple requests from regional and local governments, the rainwater harvesting research programme was expanded to include a focus on seawater and freshwater sources. Thus several water-focused studies in eco-toxicology were implemented and the recreational water quality indicators study was developed and added to the list of CEHP research programmes. After consultation with regional academics and a review of regional job market needs, the CEHP’s food safety training programme’s original goal of developing a single online certificate course in food safety at the University of the West Indies was expanded into a full diploma/MSc degree in agricultural food safety and quality assurance for their evening university programme.

In most of these cases, the AML was a key factor in helping the CEHP to successfully achieve its mandate of building a collaborative network and community of environment and health researchers and graduate students in the Caribbean. In turn, this network served as the catalyst to several new research studies. For example, a study on marine recreational water quality for the south-western coastline was conducted by the Windward Islands Research and Education Foundation (WINDREF) in collaboration with an SGU MSPH student and the Fisheries division of Grenada. In Dominica, the Ministry of Agriculture and Ross University medical students jointly conducted river pollution profiling outlining spatial changes in the level of water quality by monitoring indicators from water sources all the way to the point of entry into the sea. In Barbados, studies were conducted by postgraduate students of the UWI department of biological and chemical sciences on new and appropriate marine water quality monitoring indicators for the tropics, including testing for resistant microbes and specific pathogens.

Over the five-year operational period of the CEHP, its mission to build and develop regional and local networks of public health researchers resulted in a significant increase in the number of core partner agencies from 8 to over 13. The CEHP’s collaboration with the ongoing PAHO/CAREC foodborne burden of illness studies also demonstrated its commitment to building sustainable capacity and networking.

The CEHP entered into other partnerships that resulted in the expansion and enhancement of several CEHP research study programmes, and helped fulfil human resource development commitments to the Caribbean region. For example, an alliance was formed between the United Nations University in Hamilton, Canada, St. George’s University and Laval University to allow the CEHP to access additional funding for the persistent organic pollutants study from the World Bank Persistent Organic Pollutants Fund. As a result of this partnership, the persistent organic pollutants study was able to expand its research efforts from 4 islands to all 15 CARICOM member states in the Caribbean. This collaboration also led to the creation of a scholarship opportunity for one of the AML laboratory technicians.

CEHP’ S I MPLEMENTATION OF THE C APACITY -B UILDING G UIDING P RINCIPLES

In this section, challenges and successes of the CEHP in building a sustainable research capacity within the Caribbean region are explored using several principles of research capacity building.

Continuity and sustainability

The CEHP spent five years (2008–2012) building and developing a team of endogenous professionals who successfully networked among themselves and efficiently tapped into external resources and knowledge to tackle local environmental and public health problems. As a concrete example of this, it is expected that the food safety and quality MSc and diploma programmes at the University of the West Indies will continue to train food safety professionals who will be capable of handling these important needs of the Caribbean region long after the CEHP has ended.

The laboratory-strengthening training provided by the foodborne burden of illness studies was integrated into national surveillance programmes, which are now able to monitor an expanded set of zoonoses reflecting pathogen variation among countries.

Another indication of the sustainability of the CEHP’s capacity-building programme is illustrated by the interest generated by the AML — a key focal point for local capacity building during the programme. For example, the tenure of the lab in Dominica catalysed interest on the part of the national government to enhance and upgrade their own laboratory capacity as well as enter into partnerships with Ross University School of Medicine professors and other CEHP collaborators. The national government, in addition, requested that the CEHP provide them with a list of equipment that they should invest in, in order to support local environment and health research. This can be seen as a successful capacity-building component of the CEHP’s work in that country, ie building the capacity of decision-makers to value and want to invest in local scientific research capability.

In a similar way, the University of the West Indies entered into formal negotiations with the CEHP to take over the AML when the owners indicated that they were willing to leave it in the region indefinitely. The university is interested in expanding its laboratory capabilities to include the kind of technologies currently available in the AML. Importantly, this signals heightened interest and a demand for more advanced technologies in the region.

Coming from the interest shown by several other regional governments and environmental and public health agencies, efforts are currently underway to expand the analytical capabilities of the AML to better support environmental and public health-monitoring initiatives in the Caribbean. At the time of writing, this mobile laboratory was being set up in Bermuda, where it will contribute to similar local scientific efforts in partnership with a research facility on the island, as well as a private company, in a public–private partnership. Another long-term option that is also being explored is the possibility of mounting the AML on a ship or barge similar to what was done when the AML was sent to work in the Arctic. Having such a mobile analytical resource helps not only in the promotion of new science but also in assisting with post-disaster recovery efforts related to ecological change and human health (eg, water quality testing, active epidemiological surveillance, etc). Given the demand expressed by various institutions in the region, where the laboratory goes next is an interesting and positive problem to have.

There are early indications of an expanding impact of this programme on individuals, institutions and systems. Already, Ross University School of Veterinary Medicine, one of the CEHP’s partner institutions, has credited participation in the CEHP as an important catalyst in the formation of a new graduate studies programme in their institution.

In addition, CEHP ‘alumni’ have banded together to conduct a follow-up study in Bermuda based on novel foodborne burden of illness study findings. This is engaging two CEHP PIs, two CEHP network partners and one CEHP laboratory technician/student, in a project that the student will use to gain a PhD in epidemiology from a Canadian university. Given this student’s connections to the region, and in particular his network with the regional universities, there is a high likelihood that he will join the next generation of Caribbean public health leaders when he finishes his degree. There are, of course, no guarantees that foreign-trained professionals will return to work in their country of origin. Nonetheless, it seems logical that students who are strongly supported by a regional network in their home region are more likely to have access to, and to consider, career opportunities locally when they graduate. This is an example of the lasting mentorship that is possible by programmes such as the CEHP. While this is only one example of a student continuing to build on the opportunities presented by his engagement in the CEHP network, there are other similar successes in the programme, such as a CEHP-supported MSc student currently employed by a national health agency who is pursuing her PhD with a CEHP PI and another CEHP collaborator at the University of the West Indies. Thus, the programme has facilitated on-going South–South and North–South capacity-building opportunities.

In summary, given the magnitude of the CEHP’s multiple research programmes and their local, national, regional and international significance, it has become quite evident that a considerably longer time frame will be needed to evaluate and determine whether the various collaborative organisational teams and networks that have been created will survive and continue to grow and strengthen.

Empowerment

The CEHP’s multi-centred, networked management structure was fundamental to building identity and support for all of the CEHP research programmes in the region. From the outset, well-connected and high-level professional Caribbean leaders were identified through existing connections and the initial programme development meeting in Trinidad. They were all given significant responsibility for the CEHP research and capacity-building programmes. These leaders had extensive networks in the region that were harnessed by the CEHP over its five-year run. In fact, it seemed to their Northern colleagues that the Caribbean partners ‘knew everyone’. This was extremely empowering for the Caribbean partners and the rest of the team, as it enabled the research to move forward in a way that was embedded in the locally driven, regional paradigm. The main sources of empowerment were recognition, technical and managerial support, networking, ‘crisis management’ support and financial resources. The Northern partners were empowered by the recognition provided by the CEHP for their work in the region as well as their access to the Caribbean governance structures and, of course, the research opportunities facilitated by the Caribbean network.

Empowerment was also built and reinforced by having approximately two-thirds of the funds available allocated and administered by Southern partners. This allowed for increased flexibility in deciding how to allocate CEHP funds for maximum regional benefits and how to leverage funds from other agencies strategically. For instance, the Pan-American Health Organization, which managed one-third of the funds, leveraged additional funding from its food safety programme, as well as the Global Foodborne Infections Network. At the institutional level, the CEHP structured its management systems such that research programme leaders were given full responsibility in determining how best to utilise the funds awarded to their respective research programmes.

The CEHP clearly demonstrated the immense utility and benefits that arose from having use of a high-tech mobile laboratory in the Caribbean region. The AML empowered national government officials and community groups to explore questions they had about environmental and health situations in a timely and inexpensive manner (the partners were asked to cover the cost of the supplies used in their studies, not the costs of the laboratory itself or its technicians’ time). This led to greater interest in science-based decision making, as indicated in the section on sustainability above. The empowerment of government officials to ask for, and receive, such support should not be underestimated, particularly in countries where access to trained researchers and equipment is limited. The spin-offs of this engagement will only be seen in time.

For the CEHP research and capacity-building programme, leadership was strongly linked to empowerment. It should be noted that for all CEHP research programmes, the chosen programme leaders were viewed as regional experts on these topics or research domains within the Caribbean. The CEHP empowered these research programme leaders by providing them access to an integrated network of interdisciplinary professionals and other financial and material resources so that they could lead and create new opportunities from a position of strength.

Leadership was fostered by the decentralised management structure, where all CEHP network members were at one time or another given additional responsibilities within the team — whether that was organising and interacting with the Atlantis laboratory, overseeing one of the major research programmes, or facilitating the oceans and human health, and systematic review and meta-analysis field courses. The sheer number of CEHP activities required such an extended leadership model. The CEHP also experienced a notable transition over its five-year history, which was originally dominated by the Northern researchers who had pulled the proposal together, and then was increasingly overseen by the Caribbean partners who took ownership of its major activities. This gradual, organic and mutually supportive leadership transition was facilitated by the professionalism and high level of trust that was built among the core CEHP team members. The tone for this leadership transfer was set early on by the senior Northern CEHP team member, who determined that the funding should be primarily based in the region, and who was also not interested in playing the ‘dominant’ leadership role in the network. He was successful on both fronts, although his leadership was uncontested throughout the process, in no small part because of the role he played in stepping in, at the request of other team members, to help sort out particularly sticky situations as they arose over the course of the five-year programme. In summary, the decentralised and multifaceted nature of the CEHP created numerous opportunities for individual leadership that were fostered and encouraged by the group itself and played a major role in the overall success of the initiative.

The CEHP’s experience provides some insights into the challenges of trying to build a framework to strengthen research capacity in a third world setting. In particular, the CEHP’s experience illustrates that in order to tackle environmental and public health issues successfully, dynamic teamwork and respectful partnerships are often needed. The CEHP’s wide-ranging network of research team members has facilitated the recruitment and engagement of key government and institutional partners, with each party bringing different resources and capabilities to the group. While it is often true that the larger institutions and the Northern partners in the CEHP partnership have access to funds, it is the smaller LMIC partners that are typically more in tune with the critical questions and gaps in research in the region. It is the LMIC partners who have a nuanced understanding of who needs to be engaged in order to get ambitious research projects done.

The CEHP led to the creation of new opportunities in the Caribbean region for graduate students and professionals, providing training and skills development as well as forums where these now highly qualified personnel were encouraged to address regional needs. Multi-year, large-scale research projects such as the CEHP foster interest, excitement and camaraderie for the next generation of global health researchers in the North and in the South. The longer they last, the greater the opportunity to build and foster this critical capacity.

Charron, D.F. (2012). Ecohealth research in practice: Innovative applications of an ecosystem approach to health . Ottawa: International Development Research Centre.

Cooke, J. (2005). A framework to evaluate research capacity-building in health care. BMC Family Practice , 6:44.

Forde, M., Morrison, K., Dewailly, E., Badrie, N., & Robertson, L. (2011). Strengthening integrated research and capacity development within the Caribbean region. BMC International Health and Human Rights , 11 (2):S7.

Chapter 7 Innovation, Research Capacity Building and Applied Learning: The Canada–China Project for Vulnerable Populations at Risk for HIV and STIs in Shanghai, China

Dan Allman, Liviana Calzavara, Lai-yi Kang, Ted Myers & the Canada–China Project Study Team

This chapter describes an approach to research capacity building that is action oriented in its application; it is neither top–down nor bottom–up, but mutual, evolving and flexible. It presents our experiences with capacity building developed across the course of a project to address vulnerable populations at risk for HIV and sexually transmitted infections (STIs) in Shanghai, China, otherwise known as the Canada–China Project. We describe this approach to capacity building as action oriented as it is grounded in hands-on, active and pragmatic skills building and real-time work experience rather than being passive, theoretical and academic.

In this chapter, we argue that the capacity-building efforts of the Canada–China Project were successful in particular, and in great part, because of a national innovation-oriented policy environment known as China’s National Innovation Capacity. This orientation to innovation created a context suitable for the project’s action-oriented capacity building. While many of the mechanisms through which project activities operated may have met with resistance or discomfort given China’s unique historical and political context, the widespread institutional and philosophical orientation towards innovation mitigated such resistance and created a fertile socio-political and policy context within which capacity building could unfold. This chapter reflects on the research capacity-building work of the Canada–China Project in light of these contextual considerations and the project’s achievements, outcomes and novelty as well as lessons learned with regard to advancing our own understanding of capacity building.

B ACKGROUND TO THE C ANADA –C HINA P ROJECT

Structured largely as a partnership between academics in Canada and public health officials in China, the assumption of some team members going into the project was that research capacity building would be top–down; ie expertise, including the tools of research and its analysis, would be translated to the practitioners by the academics. However, in reality, the flow of skills and capacities was far more bi-directional. Differences in experience and approaches were important components of the learning and capacity-building exercises as team members from each of the two countries came to understand how different outlooks and experience could structure approaches to project development and management.

The overarching goal of the Canada–China Project was to create a multi-disciplinary, multi-stakeholder programme of action research by engaging the government, NGOs and vulnerable populations to respond more effectively to the HIV and STI epidemics. This was achieved through an integrated programme of research and interventions which included evaluation, training and capacity building, and knowledge transfer that linked research to policy and practice. The project aimed to do the following:

•   undertake research to better define the distribution of infections and risky sexual behaviour and the experiences of persons living with HIV

•   develop and evaluate preventive interventions for key affected populations to improve their knowledge and enhance skills, and reduce risky sexual behaviours and the transmission of STIs

•   review and analyse laws and policies to identify strengths, weaknesses and gaps, and develop recommendations to address gaps

•   provide training and capacity building for healthcare providers, community stakeholders, and government officials in HIV/AIDS and STI treatment, surveillance, disease prevention, legal and ethical issues, laboratory diagnosis, community outreach and knowledge transfer, knowledge synthesis and policy development.

The programme of work sought to address and build research and institutional capacity for both Chinese and Canadian partners. The Canadian partners benefited as the work increased capacity for global health research and teaching programmes in the Dalla Lana School of Public Health at the University of Toronto and Queen’s University’s Faculty of Law. The programme of work provided the Chinese partners, collaborators and other stakeholders with an opportunity to develop new perspectives and competencies related to professionalism, team membership, research excellence, and knowledge transfer and exchange. Within the partnership, capacity building assisted the Shanghai Centre for Disease Control (SCDC) to further develop the skills necessary to establish a programme of research, intervention and education that moved beyond a behavioural and infection surveillance focus to one of understanding and directly addressing the social determinants of health. The programme provided invaluable information, experience, training, tools, techniques and templates for future work.

Each project component not only produced much-needed information to assist the Chinese in controlling the HIV/STI epidemics and their negative impacts, but also served as a concrete method to acquire and demonstrate practical research skills. These included, but were not limited to, the following:

•   obtaining and synthesising published and unpublished information

•   designing studies to address specific research questions

•   designing and evaluating interventions

•   designing questionnaires and focus group and ethnographic guides

•   interviewing skills

•   data collection and management

•   qualitative and quantitative analytic techniques

•   understanding research ethics

•   proposal writing

•   understanding grant review processes

•   effectively communicating research findings and issues to the public, policy-makers, service providers, researchers and affected populations

•   engaging civil society in planning and implementing programmes.

Specialised training helped ensure that people living with HIV (PLHIV), service providers, policy-makers, clinicians, and laboratory and surveillance staff had access to the most recent information on best practices. The internships and visits to Canada provided direct opportunities for partners from China to learn from the Canadian context and approaches.

F RAMING THE R OLE OF C APACITY B UILDING

The Canada–China Project unfolded in a unique country context. Since the 1980s, China’s rapid economic transition from a centrally planned to a market economy has resulted in mass migration from rural to urban areas. Annual migration is estimated at 220 million, with the approximate migrant population in Shanghai estimated at 6 million (Peng, 2011). A large proportion of this migration is in the service of a booming construction industry (He et al, 2005; Anderson, Qingsi, Hua, & Jianfeng, 2003).

Traditionally, health and social care in China were structured by a fixed household registration system known as hukou . As a social welfare scheme, hukou was not designed to accommodate such vast numbers of mobile peoples, and the result has been the marginalisation of many of them. They often have no access to healthcare and urban employment, or the subsidised food, housing, pension, education and welfare programmes to which registered urban residents are entitled (Chan, 2010; Chan & Buckingham, 2008; Wu & Treiman, 2004, 2007; Wang, 2005; Chan, Liu, & Yang, 1999; Cheng & Selden, 1994).

It is within this socio-political context that the HIV epidemic in China is situated and escalating (UNAIDS, 2004), and it is within this socio-political context that the Canada–China Project evolved.

Historically, the HIV epidemic in China has been concentrated predominantly in rural areas among injecting drug users and plasma donors (Hu, Liu, Li, Stanton, & Chen, 2006; He et al, 2005; UNAIDS, 2004; Gorbach, Ryan, Saphonn, & Detels, 2002; Goodkind & West, 2002). However, the extensive rural-to-urban migration brought about by unprecedented market economy development was accompanied by a large increase in STIs and HIV. Populations most at risk include drug users, sex workers, clients of sex workers, gay men and other men who have sex with men (MSM), and partners of people living with HIV. This population has increased on average by 20–30% since 2000 (Lu et al, 2008).

Within this context and in response to this swelling epidemic, the SCDC was mandated by the Chinese government to develop and implement programmes to reduce the transmission of STIs and HIV (Peng, Zhang, Lu, & Chen, 2003). To fulfil this directive, the SCDC solicited a partnership with the Dalla Lana School of Public Health at the University of Toronto to develop a project of evidence-based interventions, knowledge generation and capacity building focused on preventing the spread of HIV and STIs among at-risk and vulnerable populations, and to improve the quality of life and health of persons living with HIV.

A multidisciplinary, multi-stakeholder five-year programme was developed to engage the government, NGOs and vulnerable populations to respond more effectively to the HIV and STI epidemics. The programme consisted of the following inter-related research projects:

•   An interview with focus group study of people living with HIV, exploring barriers and challenges faced by people living with HIV in accessing testing, treatment and support. Their task was to aid the development of pilot capacity-building activities for people living with HIV and healthcare providers, with the aim of facilitating the process of executing appropriate and effective responses to challenges identified.

•   A longitudinal, event-based web survey of gay men and other MSMs to characterise their knowledge, attitudes, risks and health-seeking behaviours related to HIV and STIs in Shanghai to develop a strategic plan for prevention among this population.

•   A randomised controlled trial of community-based interventions among migrant construction workers, focused on identifying current HIV/STI knowledge, attitudes, risk and health-seeking behaviours; identifying the causes of HIV/STI-related risk factors; and refining, implementing and evaluating prevention interventions.

•   A study to quantify and characterise female entertainment workers in Shanghai; to determine the prevalence and causes of STIs and HIV and sexual behaviours; and to develop, implement and evaluate a strategy to reduce risky sexual behaviours and reduce the incidence of STIs and HIV.

•   A legal and policy review of Chinese laws and policies with a view to identifying those laws, policies and practices that create the largest barriers to an effective response to the HIV and STI epidemics, while engaging intersectoral policy-makers in the process, with the intent of raising awareness and producing policy recommendations to be acted on.

Across all project components, capacity building through training and knowledge transfer and exchange was a core component. These capacity-building activities helped ensure that knowledge generated through research and policy analysis was better able to inform practice, and better able to find its way into practice, particularly in terms of how to understand, frame and address gaps in knowledge, stigma and access to services that increasingly impact public health in Shanghai.

From the inception of the project, it became evident to partners both in Canada and in China that a capacity-building programme based on an applied model of learning needed to be an integral component of the project, and would be most successful if elements of action and sustainability were built in at the very beginning.

The continuity of activities beyond the life of the grant as well as sustainability of these activities were reinforced through linkages, partnerships and collaborations between team members from Canada and China, as well as interactions with other researchers, practitioners, project stakeholders, and staff and management of the SCDC. Linkages and interactions not only helped to ensure sustainability, but also helped to reinforce the applicability and validity of research and policy analysis activities.

To achieve this sustainability, staff of the SCDC and 15 of the 17 local districts it oversees were trained to carry out a variety of research activities to help inform the development and implementation of HIV prevention programming for 19 million people. Specialised training events and skills-building courses were implemented based on needs identified by the SCDC and in the course of the project’s research programme and policy analysis activities.

In order to build on and improve existing capacity in socio-behavioural and epidemiologic research and policy analysis, a training of the trainers approach was employed to build confidence and empower members of the SCDC not only to improve their own capacities but also to take leadership roles in skills building and knowledge dissemination and transfer across the districts and region.

This training and applied learning provided skills-building and applied work experience for staff, interns, students and volunteers in both Canada and China. While traditional capacity-building approaches such as internships, lectures, training and workshops were used, action-oriented learning proved to have the most influence. It is this applied, action-oriented learning, as a component of capacity building, that is a focus of this chapter.

Capacity building as innovation

In the case of the Canada–China Project, the research capacity building occurred in a unique context, that of China in the early years of the 21st century. There is a body of literature which attempts to make sense of the new 21st century China by analysing ‘China’s national innovation capacity’ (Mu & Fan, 2011; Mu, Ren, Song, & Chen, 2010; Hu & Mathews, 2008; 2005; Mathews & Hu, 2007; Liu & Ma, 2001; Liu & White, 2001; Xue, 1997). Essentially, a national innovation capacity (NIC) school of scholarship seeks to trace those elements of a nation’s socio-political and policy context that act to facilitate the promotion and acceptance of innovation as well as any potential barriers or challenges to it. In the case of the Canada–China Project, we argue that the team members from Canada and China found themselves interacting with and engaging in the various research projects and the capacity-building components of those projects in a socio-political and policy context, which is specifically positioned and oriented towards seeking and facilitating innovation. This orientation can help to explain why, rather than being resistant to change and new ideas, project partners from China pushed for them as these were understood to be integral to innovation.

Xue (1997) describes how reforms to China’s innovation system can be traced to the late 1970s, and the period following large-scale economic reform to the agricultural sector. This was a time when efforts to innovate, or reform, were being experimented with on a trial basis at local level. With the 1985 publication of the Central Committee’s resolution on the structural reform of the science and technology system (Central Committee of Chinese Communist Party, 1985) came the cornerstone of China’s ‘innovation system reform’ (Xue, 1997, p. 72). In the 25 years that followed, China’s intent to become an innovation-driven nation was unmistakable (Mu & Fan, 2011). In order to achieve this, NIC is framed as a key means through which China will transform its development pattern. To this end, the Outline of the Medium- and Long-term Plan for National Science and Technology Development (2006–2020) (State Council, 2005) proposed that

China would become an innovation-driven nation by 2020, and that innovation capacity building would be the strategic foundation for science and technology (S&T) development, and would be the pivotal point for industrial restructuring and transforming the pattern of growth (cited in Mu & Fan, 2011, p. 317).

Innovation capacity in this context is defined as ‘the potential of a firm, a region or a nation to generate innovative outputs’ (Neely & Hii, 1998 cited in Mu & Fan, 2011, p. 318). Traditionally, NIC has been applied internationally and in China relative to market enterprises. At the same time, however, our experiences have been that, in general, the nation and its peoples strive to orient themselves and their activities towards innovation across multiple sectors, including health, wellbeing and disease prevention. We contend that orientation towards innovation as reflected in China’s market activities has created the kind of socio-political and policy context that seeks to lend itself to innovation in other areas:

NIC is a complex capacity system that consists of a batch of sub-capacities, such as the capacity of scientific discovery, technological invention, ‘engineerization’, commercialization, production and marketing, and capacity of social diffusion and application. All these sub-capacities synergize to promote national innovative development (Mu & Fan, 2011, p. 319–320).

Scholarship in the area of China’s NIC suggests that throughout the timeframe of the China–Canada Project, the Chinese government took a series of actions in other sectors to strengthen NIC ‘with a view to shifting the main driving force of development from investment to innovation’ (Mu & Fan, 2011, p. 323). Summarising the broad picture, these authors state the following:

Innovation is a complicated social process of value creation, in the fields of science, technology, culture, the economy, and society. It concerns activities ranging from scientific discovery, technological invention, methodological innovation, and their applications as well as social diffusion. The capacity for and the success of innovation are determined by diversified stakeholders of innovation such as scientists, technologists, engineers, entrepreneurs, organizations (firms, research institutes, universities, and governments as well as social entities) (Mu & Fan, 2011, p. 326).

In their research on NIC as reflected by the drivers of manufacturing and patenting within China, Hu & Mathews (2008; 2005) reflect on the ‘important role played by universities in contributing to innovative capacity’ within China (Hu & Mathews, 2008, p. 1476). They indicate this finding is consistent with the limited existing research that explores the contributions of China’s universities to NIC (Chen & Kenney, 2007; Lee, 2005; Xue, 2005). While the focus of those writing about the relationship between NIC and China’s academic activities is on innovation relative to China’s industrial development (see, for example, Liu & White, 2001), we argue that the role of innovation very much helps to inform our understanding of the research capacity-building activities and their successes within the Canada–China Project.

Capacity-building focus and approach

Within a broader social and political environment that welcomes innovation, the capacity-building component of the China–Canada Project set out, primarily, to train local actors who would use the knowledge and skills in their day-to-day work and who in turn could act to further diffuse innovations in policy and practice, and ensure that expertise to address the growing epidemics of HIV and STIs would exist and evolve as required.

To accomplish this, a hybrid instructional approach was utilised, which involved didactic training workshops where lectures, roleplaying exercises, action-based learning and applied planning and undertaking of research projects all contributed toward capacities developed. An aim of the project was to reach a wide variety of stakeholders, for example staff and researchers at SCDC, public health and healthcare providers, laboratory scientists and technicians at local health districts and hospitals as well as people based in the community and policy sectors. Beyond this, the project sought to involve master’s and doctoral students from China and Canada as part of their training, giving them exposure to the international context, the capacity development activities and the multidisciplinary team members from each of the two countries.

Dimensions of capacity building

Within the broader context of a nation receptive to innovation, the capacity-building activities of the Canada–China Project were iterative and collaborative, but also pragmatic. Team members involved in each of the project’s research studies conducted assessments through meetings and other interactions. The aim was to understand what the capacity needs were, what kinds of training might be feasible and useful, as well as the kinds of skill sets to which the training would lead.

Underlying these activities was the key aim to increase capacity for the local district centres of the SCDC to undertake research necessary to develop and evaluate their local programmes. This was accomplished through a number of different initiatives, including targeted workshops and intern opportunities in Canada, the emphasis of which was always applied, hands-on learning.

Throughout the project, partners from each of the two countries prioritised dissemination of activities and of research. Two major national symposia and conferences were held, numerous papers and posters were presented nationally and internationally, and over 40 peer-reviewed papers were published in Chinese journals, with an emphasis on engaging in knowledge transfer and exchange to those outside the Canada–China Project programme of activities.

B UILDING LINKAGES THROUGH PARTNERSHIPS AND COLLABORATIONS

As public health practitioners, many of the SCDC staff had little research training prior to the initiation of this project. The district SCDC focused on designing and implementing HIV and STI prevention programmes, but did not necessarily have the resources or knowledge to conduct the social and behavioural research required to inform such programmes. Even when resources were available, the skills and background required to engage in biomedical and epidemiological research tended to be much greater than the skills and understanding of social research. As such, this was a frequent emphasis of the research capacity-building activities, in part because of the breadth of social science research skills of the team members from Canada, and in part because these elements were not as well understood in the context of China.

The Canada–China team worked in collaboration and partnership with each other as well as key stakeholders to develop, enhance and exchange knowledge and research capacity building. An important example is found in each of the individual concrete research projects. Owing to the structures and aims of these research projects, certain capacities in study design, data collection, data analysis and data interpretation were required. This made it necessary to collaborate to solve problems and make research projects successful. Issue resolution of this type was one form of mutual, bi-directional capacity building. Skills built were applied and pragmatic and each of the projects evolved into a form of research laboratory within which capacity could be built. This, however, was not without its challenges. Symposia, workshops and visiting internships were often developed and experienced as forms of structured learning, similar to those that can be found in a classroom. These were components that lent themselves to planning and fixed timelines, whereas more day-to-day learning within the context of a given research project could be somewhat less structured and more applied. Day-to-day applied learning could prove more challenging in terms of planning and could be more rewarding, owing to the kinds of human interactions that were facilitated and contributions to research output that were produced.

GIPA stands for the ‘greater involvement of people living with HIV/AIDS’, and in many respects, Canada is a world leader in such involvement. This principle was a component of the project’s activities that saw team members from Canada and China work together to involve PLHIV more genuinely than in the past. This included not only the engagement of PLHIV, but also the involvement of sex workers, gay men and MSM as well as other community stakeholders. Each group in their own way found means of greater involvement – through advisory positions, research instrument development and contributions to research capacity building. While in some respects such inclusion was atypical to the SCDC’s previous experiences, this form of community involvement came to be understood by the SCDC as an effective mechanism, with demonstrated utility. To incorporate it into the China context was seen as innovative and arguably in keeping with other broad movements toward innovation throughout other aspects of economic and social reform in China. Further, it was reflective of evolutions in respect to civil society’s engagement in research and prevention programmes. Across the course of the Canada–China Project, international best practices that demonstrated the benefit of engaging NGOs and community representatives to help shape the research and interventions acted to challenge the assumption that only professionals have the knowledge to do so.

S UCCESSES IN THE FIELD

In hindsight, the major successes in building health research capacity at individual and institutional levels were through the engagement of people representing different levels of healthcare and policy across the Shanghai and Chinese landscapes. When the Canada–China Project was initially proposed, the intent was to deal with one institution only, and to contract students to undertake the work. However, as the project entered the field and greater interaction between the project partners occurred, a decision was made to broaden the scope of the project to include not only the centralised SCDC but also the local health districts, which were in need of skills building and knowledge translation. This meant that the capacity-building component of the project, and its emphasis on building better skills and knowledge, was arguably more widespread as well as more applied, because those engaged in capacity-building activities were also those tasked with taking on the day-to-day responsibility for service and healthcare delivery in their respective districts.

An interesting success, and one very close to the work of the project, was the increased capacity on the part of the team members to engage in research with vulnerable populations, and a better understanding of the increased sensitisation required when working with these populations. The approaches to working with vulnerable populations were substantially different between the two country contexts, and it proved very beneficial to be able to share these experiences across team members. Sensitisation of this sort was not a skill or a capacity that necessarily would have been identified by the team members as one for which capacity was required, and yet now that the project has come to an end, it is useful to be able to reflect back and to see that indeed it was.

We believe that of the many project activities, the opportunity for students and trainees from China to engage with the project represents one of the most sustainable impacts. Some of these students had the opportunity to visit Canada for extended periods and to work side by side with peers and colleagues at the University of Toronto. This was particularly successful owing to its applied nature. Being able to see, experience and discuss methods and approaches within a Canadian environment highlighted how various research tools could operate pragmatically within an operational environment and, for the student, how these might be applied in the future in China. By engaging skills and knowledge within an applied context, the potential for innovative application within the China setting became more evident, and lessons learned in turn became more tangible.

H URDLES AND CHALLENGES

The successes of the project did not occur in the absence of challenges. Examples include differences in cultural understandings and in the role and history of centralised political infrastructures. Importantly, challenges of this type were not fundamental obstacles, but rather served to reflect cultural elements and differences between two countries and regions of the world. They showed how such elements and differences can structure practice, and the way an orientation toward innovation can help break down some of these differential structures and in doing so, allow new forms, ideas and approaches to prosper.

Certainly, for many in the project, the limitations in time spent together and the role of cultural and language translation within that time challenged the ability of team members to explore all issues fully when together. This was in contrast to the students and others engaged in the entire programme – both from China and Canada – which, given the nature of longer term and more in-depth involvement, were able to root themselves more fully in project activities, and in so doing experienced an arguably richer and more tangible research capacity-building experience.

In other regards, the limited time that the team leads had to be together, and inherent challenges of working together in languages in which not all team members were fluent, required adaptability in terms of how the project worked to meet its goals. The challenge of language, the need to work to a budget and the limited periods of contact also meant that it was difficult to provide written resources for the research capacity development activities, as few resources in this area exist in multiple languages.

Beyond language, was the reality that participation on the part of most team members was in addition to their regular jobs. As a result, project activities needed to fit within those structures. This required accommodating the real-world limitations that are part of the landscape when working within these kinds of projects. Real-world limitations of this sort need to be aligned with real-world responsibilities, and ultimately with real-world expectations. In our experience, this component of the social context within which the Canada–China Project’s research capacity building unrolled is rarely something that is reflected in the frameworks that structure funding calls and other such opportunities. While the kinds of real-world limitations involved in real-world capacity building, or what Schultz et al (2011, p. 12) describe as ‘realistic capacity building’, are well recognised among practitioner communities and are often informally acknowledged by policy-makers, funders and donors, there has, to our knowledge, never been a wholly successful example of such understandings being translated into the policy or practice of funded research or capacity-building efforts of this type.

An additional challenge came from the different political cultures of Canada and China, evident throughout the course of the project. Despite China’s overarching orientation toward innovation, there remained significant challenges. The centralised nature of the SCDC relative to the district CDCs could function to limit the freedom to act independently. This was compounded by restrictions presented by centralised health infrastructures and political systems, and the limits on freedom and access to information. While such circumstances were normative for Chinese team members, for Canadian team members such limitations could be hard to comprehend and were at times frustrating. While this did not necessarily limit the research capacity building in which the project engaged, it did prove challenging from the perspective of action orientation as skills developed and lessons learned could not always be put to use in the field in as timely a manner as the team might have wished.

Re-allocating resources to meet the challenge

A further key difference in work between Canada and China was the level of access to administrative and management resources. Our HIV-focused work in the Dalla Lana School of Public Health gave us access to numerous research, staff, technological and telecommunication resources. Moreover, as a unit in a high-income country, the school has easy access to telecommunications, computers, software and other related resources that may not be as readily available or affordable in a low- to medium-income country. Although the SCDC had a beautiful, well-equipped laboratory and had recently constructed a number of new buildings and improved its telecommunications, its tools were continuing to evolve.

In order to accommodate the disparity of resources, we created a number of knowledge-sharing and knowledge transfer instruments. For example, the web portal (a secure web interface for uploading and downloading documents and media) allowed all the team members access to the vast literature resources at the university library. The research assistants and professors in Toronto were able to upload relevant articles and documents to the web portal for all team members to see and use. The Shanghai team members also used the web portal to share information and documents related to field research or training sessions with the Canadian team members. Further, a number of methodology texts were purchased and given to the Shanghai Project Office to support knowledge transfer.

Importantly, the Canada–China Project was influenced, challenged and shaped by its unique placement within the larger socio-political, environmental and cultural contexts of China as well as that of the broader bilateral relations between Canada and China. Over its five years, the Canada–China Project experienced and felt the impacts of a number of social, natural and political events. This unique socio-political context had wide-ranging financial, methodological, strategic and practical implications for the project and its stakeholders.

L EARNING TOGETHER

Much of the focus of the Canada–China Project was on the research capacity building of our Chinese partners and institutions. However, the team members from Canada also learned a great deal and built capacity, not only among the core research team members but also among students, faculty and staff, and those from the third sector who were engaged in some element of the team’s projects or who had contact through presentations or other forms of knowledge translation. It was useful to have the opportunity to learn how team members in China work in a context where the link with policy-makers is often much more direct than the link we are accustomed to in Canada. This acts as a reminder that capacity building, even in an innovative context like China, is bi-directional, and that our understanding of capacity building in the context of research benefits from this important lesson. Be it at the individual, team or institutional level, effective capacity building is also a form of capacity exchange, just as knowledge transfer is also a means of knowledge building.

A model of applied learning proved to be an effective way to assess what our team partners knew, what they needed and wanted to learn and subsequently, how well they learned it. Knowledge transfer was an effective way for the team partners from Canada to more fully understand the skills and practices that were essential to the Shanghai context as well as ideas as to how to link future project capacities and projects with policy-makers back home.

Over the course of the Canada–China Project, the team members from Canada and China learned and gained a great deal from each other. They not only improved their technical research and academic skills but also strengthened their skills in cross-cultural communication, negotiation and international project management through a greater overall understanding of the cultural and social complexities of their unique societies and situations. Three specific areas where we were able to foster greater mutual understanding and trust are communication styles, work styles and structures, and resource re-allocation.

Discovering one another through communication

The project was unique in that we were not only separated by an ocean from our partners but also used two different languages representing modes of communication and rules structuring communication that in many respects were binary opposites (Li, 2009). A useful analogy described by colleagues is that a person from the West speaks like an arrow, swiftly moving right to the point, whereas a person from the East speaks like the Yellow River, gently winding to its destination. Although the work of the Canada–China Project was primarily scientific, when managing an international project of this nature we could not ignore such cultural complexities. Since effective collaborative project management is aided by the discussion of plans and initiatives between all partners, it is important to be aware of differences in the understanding and opinion of all project partners, something that only communication can facilitate.

Over the course of the Canada–China Project, we became aware of our differences in communication, and in turn, made modifications to mitigate any misunderstandings or issues that we encountered. We learned that it is necessary to be patient and to take time to understand each other’s communication styles, and how these styles might influence the delivery and receipt of communication and intent. We learned to take time for translation and that this did not merely involve verbatim conversion, but also the transference of any subtleties that might frame what we wished to express and explain. This required more back translations and discussion between the project managers when preparing documents and more face-to-face communications than might be otherwise expected. We took more time to send e-mails and have teleconferences to ensure that we were all in agreement and knew what it was we were agreeing to.

Through our greater understanding of each other, we were able to gain more insight into the problems (and potential solutions) related to HIV/AIDS in China. We were able to truly share our perspective and experience in Canada and to learn how to adapt that to the unique situation faced by our colleagues in China. We built long-standing relationships and friendships that we truly believe will contribute to future work in this area.

The longer the Chinese and Canadian team members were together, the more familiar they became with each other’s environments and needs. In time, with increased trust came more open communication. Through this mutual understanding and cooperation, we were able to work effectively to brainstorm and solve any project problems that arose.

It was clear to all the partners involved that the end result of the Canada–China Project was a novel approach to research capacity building that in many respects was extremely innovative. Despite the challenges and differences, we managed to forge ahead, building capacity to conduct research while conducting that research itself. Further, we managed to do so in a way that engaged community members in the research process in what, in the China context, has traditionally been a top–down endeavour, while at the same time maintaining the engagement of different levels of public health practitioners and policy-makers.

Throughout the project process, we learned that research capacity building and the research to which that capacity building applies, benefits when it is planned in such a way as to be sufficiently responsive to expected and unexpected events. Flexibility, determination and resilience were elements of our own innovation. Such innovation was a vital component for keeping this ambitious, transformative programme of work on track.

Anderson, A.F., Qingsi, Z., Hua, X., & Jianfeng, B. (2003). China’s floating population and the potential for HIV transmission: A social-behavioural perspective. AIDS Care , 15 (2):177–185.

Central Committee of Chinese Communist Party (CCCCP). (1985). The resolution of the Central Committee of the Communist Party of China on the structural reform of the science and technology system.

Chan, K.W. (2010). The household registration system and migrant labor in China: Notes on a debate. Population and Development Review , 36:357–364.

Chan, K.W., & Buckingham, W. (2008). Is China abolishing the hukou system? The China Quarterly , 195:582–606.

Chan, K.W., Liu, T., & Yang, Y. (1999). Hukou and non-hukou migrations in China: Comparisons and contrasts. International Journal of Population Geography , 5 (6):425–448.

Chen, K., & Kenney, M. (2007). Universities/research institutes and regional innovation system systems: The cases of Beijing and Shenzhen. World Development , 35 (6):1056–1074.

Cheng, T., & Selden, M. (1994). The origins and social consequences of China’s hukou system. The China Quarterly , 139:644–668.

Goodkind, D., & West, L. (2002). China’s floating population: Definitions, data and recent findings. Urban Studies , 39 (12):2237–2250.

Gorbach, P.M., Ryan, C., Saphonn, V., & Detels, R. (2002). The impact of social, economic and political forces on emerging HIV epidemics. AIDS , 16 (4):S35–S43.

He, N., Detels, R., Zhu, J., Jiang, Q., Chen, Z., Fang, Y., & Zhao, Q. (2005). Characteristics and sexually transmitted diseases of male rural migrants in a metropolitan area of Eastern China. Sexually Transmitted Diseases , 32 (5):286–292.

Hu, M.C., & Mathews, J.A. (2005). National innovative capacity in East Asia. Research Policy , 34 (9):1322–1349.

Hu, M., & Mathews, J.A. (2008). China’s national innovative capacity. Research Policy , 37 (9):1465–1479.

Hu, Z., Liu, H., Li, X., Stanton, S., & Chen, X. (2006). HIV-related sexual behavior among migrants and nonmigrants in a rural area of China: Role of rural-to-urban migration. Public Health , 120 (4):339–345.

Lee, K. (2005). Making a technological catch-up: Barriers and opportunities. Asian Journal of Technology Innovation , 13 (2):97–131.

Li, W. (2009). Different communication rules between the English and Chinese greetings. Asian Culture and History , 1 (2):72–74.

Liu, S., & Ma, W. (2001). Motif analysis for automatic segmentation of CT surface contours into individual surface features. Computer-Aided Design , 33 (14):1091–1109.

Liu, X., & White, S. (2001). Comparing innovation systems: A framework and application to China’s transitional context. Research Policy , 30 (7):1091–1114.

Lu, L., Jia, M., Ma, Y., Yang, L., Chen, Z., Ho, D.D., & Zhang, L. (2008). The changing face of HIV in China. Nature , 455 (7213):609–611.

Mathews, J.A., & Hu, M.C. (2007). Enhancing the role of universities in building national innovative capacity in Asia: The case of Taiwan. World Development , 35 (6):1005–1020.

Mu, R., & Fan, Y. (2011). Framework for building national innovation capacity in China. Journal of Chinese Economic and Business Studies , 9 (4):317–327.

Mu, R.P., Ren, Z.B., Song, H.F., & Chen, F. (2010). Innovative development and innovation capacity-building in China. International Journal of Technology Management , 51 (2/3/4):4274–4252.

Neely, A., & Hii, J. (1998). Innovation and business performance: A literature review (pp 1–65). Cambridge, MA: The Judge Institute of Management Studies, University of Cambridge.

Peng, J., Zhang, S. N., Lu, W., & Chen, A.T.L. (2003). Public health in China: The Shanghai CDC perspective. American Journal of Public Health , 93 (12):1991–1993.

Peng, X. (2011). China’s demographic history and future challenges. Science , 333 (6042):581–587.

Schultz, J.T., Moodie, M., Mavoa, H., Utter, J., Snowdon, W., McCabe, M.P., & Swinburn, B.A. (2011). Experiences and challenges in implementing complex community-based research project: The pacific obesity prevention in communities project. Obesity Reviews , 12 (2):S12–S19.

State Council. (2005). The outline of medium and long-term plan for national science and technology development (2006–2020) . Guofa (State Council issued) No. 44 (in Chinese).

UNAIDS. (2004). A joint assessment of HIV/AIDS prevention, treatment and care in China . Beijing: UNAIDS China Office and China State Council AIDS Working Committee Office.

Wang, F.L. (2005). Organizing through division and exclusion: China’s hukou system . California: Stanford University Press.

Wu, X., & Treiman, D.J. (2004). The household registration system and social stratification in China: 1955–1996. Demography , 41 (2):363–384.

Wu, X., & Treiman, D.J. (2007). Inequality and equality under Chinese socialism: The hukou system and intergenerational occupational mobility. American Journal of Sociology , 113:415–445.

Xue, L. (1997). A historical perspective of China’s innovation system reform: A case study. Journal of Engineering and Technology Management , 14 (1):67–81.

Xue, L. (2005). University-market linkages in China: The case of university affiliated enterprises . Available from: http://www.iis-db.stanford.edu/evnts/4097/LXue_University-MarketLinkagesinChina.pdf (Not available).

Chapter 8 Capacity Building with Research Partners: Experiences from Proyecto Araucaria in Chile

Katherine Lippel, Ximena Díaz & Amalia Mauro

Proyecto Araucaria (the Araucaria Project) studied work-related mental health problems in Chile and regulatory and policy issues in a variety of jurisdictions, from a gender perspective. It brought together Chilean researchers from the Centro de Estudios de la Mujer (CEM, or Centre for Women’s Studies) and Canadian researchers from the University of Ottawa, Université du Québec à Montréal and Laval University. (The official title of this Teasdale-Corti research programme was ‘Work-related mental health problems in Chile: A gender perspective’; details may be found at http://www.proyectoaraucaria .)

The proposal grew out of an initiative of CEM, which, in 2006, approached researchers from the Centre de recherche interdisciplinaire sur la biologie, la santé et l’environnement (CINBIOSE), a research centre in Québec dedicated to occupational health issues, to develop a research partnership on psychosocial hazards, gender and precarious employment in Chile. For decades, the sociologists and economists working at CEM had been studying the working conditions of women in Chile, focusing on precarious employment, work–life balance and issues related to the quality of employment in Chile. The researchers became aware of an increase in disability claims for mental health problems in Chile, and proposed the partnership idea to CINBIOSE to understand better the contribution of working conditions to the increase in mental health problems among workers. CEM and CINBIOSE focus on a gender-based analysis of working conditions, and have a long history of partnerships with trade unions, community groups and social change agents. While the respective research centres were familiar with each other’s publications, the Araucaria Project was the first formal collaboration between CEM and researchers from CINBIOSE (Lippel, Messing and Bernstein).

When the first contact was made by CEM, Lippel was identified as the CINBIOSE member who was in the best position to meet the needs of the proposed partnership. A specialist in occupational health and safety law and policy, she had worked for decades on the policies governing the prevention of exposure to psychosocial hazards and workers’ compensation for disability related to workplace stressors. She and Bernstein had collaborated for years on the regulation of precarious employment in Québec. She was also involved in a project led by Dr Michel Vézina from the Institut national de santé publique du Québec called the EQCOTESST study (see http://www.inspq.qc.ca/eqcotesst/ ), which sought to measure exposures to various working conditions in Québec and the health measures associated with those exposures. Lippel and Messing have been collaborating since 1993, studying women’s occupational health problems and policy solutions, in partnership with Québec trade unions, in the context of the Invisible qui fait mal research team (see http://www.invisiblequifaitmal.uqam.ca/ ).

Both research centres worked in partnership with community organisations and policy-makers in order to co-construct knowledge about women’s working conditions. Thus, the choice to focus on a gender-based analysis in partnership with community groups, unions, labour inspectors and policy-makers in Chile was a natural extension of previous work undertaken by members of the team. The project was initiated by the Chileans, who identified psychosocial hazards at work as a priority issue because it was under-studied in that country, and the social partners who traditionally collaborated with CEM as well as the CEM researchers themselves, who identified the need to improve their capacity to intervene, as researchers and as social partners, for the protection of workers’ mental health. The fact that Chile was a poster child for the neo-liberal deregulation of working conditions under the dictatorship of Augusto Pinochet made it an interesting country to study, as the labour market was particularly segmented; precarious employment and subcontracting were prevalent, and psychosocial hazards specific to women (like sexual harassment and discrimination) in the workplace were believed to be prevalent.

The project was divided into two axes: axis 1 sought to understand the effect of working conditions on the mental health of Chilean workers and document risk factors and protective factors for their mental health; axis 2 focused on a comparative study of law and policy on mental health, gender and work in order to identify policies and regulations that promote healthier work environments and provide compensation for work disability.

Working conditions are a key determinant of health (Benach, Muntaner, & Santana, 2007) and the significance of their impact on the mental health of working men and women in LMICs led to their acknowledgement as a priority issue by the World Health Organization (Kortum, 2007). The importance of using a gendered approach to conceptualising capacity building with regard to policy changes is well known, notably in relation to an examination of health and safety (European Agency, 2003; Cornish, 2008; Messing & Silverstein, 2009), and more specifically with regard to measures of psychosocial hazards in the workplace (De Smet et al, 2005; Vézina et al, 2011). However, the way to promote such an approach is not without challenges. According to Theobald and colleagues (Theobald, Tolhurst, Elsey, & Standing, 2005) the effectiveness and impacts of capacity-building interventions are known to vary depending on the focus and targets, and they note that bureaucrats, NGOs and academics have very different approaches to gender mainstreaming.

Capacity building formed a central part of the project, which was designed to promote reciprocal exchange of experiences, knowledge and strategies, involving researchers and research users. The Chilean researchers were well skilled in qualitative research methods but less familiar with quantitative measures of psychosocial hazards. For them, Araucaria provided an opportunity to learn, apply and share with research users those skills that provided stronger arguments for prevention and change because they were backed up with data gleaned from a representative sample of the working population. In terms of policy reform, research anchored in a comparative analysis also supported recommendations for change, by contextualising them both regionally and globally.

The programme sought to build not only research capacity but also capacity to perform evidence-based interventions on a local, national and international level to promote better working conditions for workers, while ensuring that the strategies retained were informed by a gender lens. The need for capacity building for institutions responsible for labour inspection is illustrated by recent projects undertaken in the context of trade agreements between Canada and Latin American countries. See, for instance, http://www.hrsdc.gc.ca/eng/labour/labour_globalization/IPPLA/pucp.shtml .

The researchers in the team came from a wide range of disciplines, including sociology, law, medicine, public health and ergonomics. Students in both Canada and Chile were actively involved in the research programme. The research users spanned an equally vast spectrum of fields. In Chile, collaborations were well established with various NGOs, including rural agricultural women workers’ groups, women’s groups, unions and other worker representatives as well as state actors. In Canada, students were involved in the capacity-building process; worker representatives and unions were also involved. Participation in workshops and collective publication projects also enabled capacity building for researchers from other countries who were not necessarily active in the specific field of research focused on by the Araucaria Project.

In this chapter, we will first define terms and then provide illustrations of the various facets of capacity building which have emerged in the context of our project.

W HOSE CAPACITY AND CAPACITY TO DO WHAT ?

When reflecting on our accomplishments with regard to capacity building, it is first necessary to clarify our objectives. From the outset, our team emphasised the importance of improving the capacity of research users, who were chosen to participate as equal partners in our team because of their ability to influence the evolution of civil society and government policy with regard to occupational psychosocial hazards. In that sense, the primary purpose of our capacity-building endeavours was not so much to increase our capacity to do research, although several illustrations allow us to conclude that that too has been accomplished, but rather to mobilise the research process and the research results in a way that was meaningful to those who were in the best position to make changes in light of the research findings. We chose not to focus primarily on training and knowledge transfer, rather adopting Potter and Brough’s (2004) more systemic approach to capacity building instead. Potter and Brough (2004, p. 336) recommend an approach that they call ‘systemic capacity building’ which addresses a ‘four tier hierarchy of capacity building needs’:

1)   structures, systems and roles

2)   staff and facilities

3)   skills

4)   tools.

Araucaria targeted systems, structural capacity and tools, among other aspects. This was done by developing training and tools for the labour inspectorates and NGOs, and by building on the roles of NGOs and state actors in the recognition of psychosocial hazards. It also included promoting and participating in ‘decision-making forums where intersectorial discussion may occur’ (Potter & Brough, 2004, p. 340).

The choice of this approach was grounded in our previous experiences, in Chile and in Canada. From the initial conceptualisation of the project, the researchers from Canada and Chile were well aware of the importance of learning from community partners and providing them with the opportunity to increase their intervention capacity through participation in our research. In the Araucaria Project, the choice of this approach was reinforced by the fact that a major component of our programme sought to influence policy and policy-makers in Chile as well as in other jurisdictions. The most effective mechanism to achieve this objective, in our experience, was by engaging the active participation of decision-makers and community groups, including targeting those actors in the capacity-building process.

In his paper on monitoring policy interventions, Harry Jones described mechanisms to promote a Theory of Change, based on relationships with key actors:

[Theories of Change] for this kind of influencing activity are based on actors, the relationships between them, and the institutions within which they work. One review of successful lobbying has found that, in addition to clear and focused policy goals, the key strategic capacities required are identifying natural allies, developing relationships and credibility with policy actors, and understanding the nature of the policy process and institutional access (Jones, 2011, p. 8).

The Araucaria Project sought to influence policy to improve working conditions in Chile and other countries and thus reduce exposure to psychosocial hazards. Given these objectives, the formation of natural alliances with research users to promote their capacity to intervene, either as researchers from the Dirección de trabajo (Labour Inspectorate), as occupational health professionals from the Ministry of Health, as labour inspectors, or as unions and community groups having a lobbying agenda, was seen to be the most effective strategy to ensure that research was translated into practices that could effect change.

Participating in the research process is one aspect that ensures better uptake by the research users. Our Chilean community partners were active throughout the research programme, from the initial conceptualisation of the objectives and research to be accomplished, through the data gathering process, and on to the conceptualisation of tools designed to best ensure accessibility for a broad range of potential users.

We also sought to increase the research capacity of all participants in the programme, including the Chilean and Canadian researchers, and the students funded by the programme. The Chilean researchers benefited from input from the Canadian scientists in fields such as ergonomics, public health and law, while the Canadian researchers, both professors and students, learned new skills in developing a research programme that was respectful of the needs of the Chilean partners. Chilean expertise was particularly strong with regard to women’s working conditions and Canadian researchers integrated new ways of including gender issues in Canadian tools, thanks to the input of the Chilean partners. In turn, specific skills related to ergonomic analysis and gender were shared by a Canadian expert in gender-based ergonomics, who provided training to the Chilean sociologists preparing workplace observations and designing the survey. This collaboration also led to the development of a new programme, funded by the CIDA, which was designed to train Chilean graduate students in ergonomic analysis applied through a gender lens.

Finally, it became clear that capacity building extended far beyond the specific processes we had initially identified in the grant application. For example, in the context of the policy stream (axis 2), we organised a series of seminars and special issues of journals, and researchers from 16 countries, as well as union representatives from three countries, either in person or through the scientific production process, to reflect on psychosocial hazards, gender and policy issues. Emphasis was placed on specific challenges raised by precarious employment and changing labour markets. One unexpected outcome of this process was that the participants (which included researchers and union representatives from Canada, Chile and elsewhere) needed to improve their own local understanding of certain aspects of the issues to be discussed in order to successfully participate in the process. Thus, a union representative, with vast experience in health and safety interventions, needed to reflect specifically on his union’s role in reducing psychosocial hazards, a process that led to new interventions by that Canadian union long after the seminar in Santiago was over. Similarly, seasoned academics who specialised in worker representation with unions with regard to more traditional health and safety hazards were required to reflect more specifically on psychosocial hazards and gender issues.

In the following section, we will provide a few concrete examples of the ways in which our researchers and research users were empowered by the research programme, in order to illustrate the preceding points.

T HE MANY FACETS OF CAPACITY BUILDING

More classical strategies contributed to research capacity building of the Chilean and Brazilian scholars who received doctoral scholarships through our programme. For example, academic training and research assistance was provided to several Latin American scholars studying at the University of Ottawa, mostly in law, but some in public health.

One bursary student was a professor of law at the Universidad Diego Portales in Santiago, and as she pursued her doctoral studies on regulatory effectiveness of sexual harassment policy in Chile, she in turn built new research capacity by transmitting novel ways of studying law to her own law students. As has been the case in many Canadian universities (Lajoie, 2003), classic legal scholarship in Chile does not usually include interdisciplinary approaches to law. Nor does it invite reflection on legal rules through a gender lens. Our scholarship student’s study of the regulatory effectiveness of Chilean sexual harassment legislation allowed her to transmit, to her students, knowledge and readings drawn from feminist legal theory. Her participation in Araucaria also allowed her to draw on knowledge from the social sciences in the pursuit of her analysis of effectiveness, and she included specific questions on sexual harassment in the Araucaria survey of working conditions and participated in the analysis of data relevant to her research question. By training the trainers in new ways of analysing law to make it more effective in protecting workers’ health, particularly the health of women workers, the project allowed for capacity-building activities that would continue long after the original project was completed.

A Brazilian scholarship student in law studying the regulatory effectiveness of anti-bullying legislation in Brazil, compared the effects of explicit legislation as opposed to the more general legal protections that exist in the Constitution and in general labour law. Empirical research by legal scholars is unusual in Brazil, so not only will the results of the study be useful to policy-makers but also the introduction of new methodological approaches will inform research practices in Brazilian law schools, where the candidate intends to work as a law professor upon the completion of her studies.

Finally, a third scholarship recipient, a Chilean doctoral student in public health, trained in occupational psychology, and a professor of psychology at the Universidad Diego Portales, pursued her doctoral studies under the supervision of the Canadian public health physician from the Araucaria team, adapting the Québec EQCOTESST survey of working conditions and workers’ health (Vézina et al, 2011) to the Chilean context. CEM also benefited from her participation in the development of the Chilean Araucaria survey, which included questions adapted from the EQCOTESST survey to the Chilean context.

Aside from capacity building for the students, numerous undertakings were designed to build the capacity of the community partners. The following example shows how the knowledge and skill of the Chilean research team served to empower a local community group to make change in light of research results.

La Asociación Nacional de Mujeres Rurales e Indígenas (ANAMURI) was one of our most active research users throughout the life of the project. Unlike the traditional rural NGOs active in the agricultural sector, which are run by men who own the land, this organisation is a women’s group comprised mostly of temporary agricultural workers. Chile is an exporter of agricultural products, particularly of fruit, and the development of the sector has had a negative impact on subsistence farming. This sector is comprised of a small permanent workforce that works all year long and a significant temporary workforce that is active at harvesting time. Women comprise almost two-thirds (64%) of this workforce (Díaz & Mauro, 2010).

In the context of the Araucaria project, in partnership with the Chilean research team, ANAMURI organised a vast campaign to sensitise its members to work-related mental health problems. Local meetings were organised in various parts of the country. These meetings allowed the group to document the experiences of hundreds of workers, experiences that were then synthesised in a brochure documenting working and employment conditions and their consequences for workers’ health. The knowledge transfer tool included ten chapters written in clear language and illustrated with concrete examples from the workers themselves. The brochure was broadly distributed and read during meetings of women workers. Specific discussion questions allowed for collective small group discussions. Those who took part in this experience spoke of it with great enthusiasm and related how this consciousness-raising approach allowed workers to organise around concrete objectives and, in those cases where the balance of power made it possible, to obtain a real improvement in their working conditions. This success is all the more remarkable considering the particular vulnerability of temporary female agricultural workers, who are among the poorest working people in Chile, and have few opportunities for paid employment other than seasonal work (Díaz & Mauro, 2010).

Another example was the proposed knowledge transfer project to mobilise a trade union that was a research user in our team. The Confederación Nacional de Funcionarios Técnicos en Enfermería de Chile was a union representing health sector workers. One of the activities involving that union was the development of a proposal to study workload and the work–family balance of paramedical nursing technicians in Chile. The objective of the grant application was to develop a tool for organisational empowerment concerning psychosocial hazards associated with workload and work–family balance via knowledge transfer activities. The union members mobilised around the activity preceding the grant application. Despite the fact that the funding was not forthcoming, workshops took place to develop a collective diagnosis of the specific challenges facing the participants with regard to work–family balance, which served to improve our understanding of how our research could be applied to the specific context of the workers involved.

State actors have also been involved in the research process in a way that has allowed for appropriation and adaptation of tools to suit the Chilean context. One example is drawn from the work undertaken with the Unidad de Condiciones y Medio Ambiente de Trabajo, Dirección del Trabajo, part of the Chilean Labour Inspectorate. Starting with a Québec-based tool developed to support occupational physicians working within the Institut national de santé publique du Québec, Chilean and Canadian members of the research team met with representatives of the Unidad de Condiciones y Medio Ambiente de Trabajo, Dirección del Trabajo, who helped to define the needs of their inspectors with regard to the protection of workers from psychosocial hazards. The process of adapting the tool to the Chilean context led to an ongoing series of collaborations between the Canadian and the Chilean researchers, and between the Chilean institutions that had not previously collaborated on this issue. The process led to the creation of an expert panel that included representatives of the Chilean public health department, the Departamento de Salud Ocupacional del Instituto de Salud Pública (whose director was a member of the research team), members of CEM and other institutional actors. Representatives of the Chilean inspection institutions were able to define their needs, with input from a variety of other groups, including CEM, who ensured that gender issues like sexual harassment and work–family balance were included in the final tool. Aside from linguistic issues, one of the adaptations of the tool that was required for the Chilean context focused on making it more like a checklist than a design to lead discussion in the workplace. The role of those responsible for using the tool in Chile, for the most part labour inspectors, required that they have a list of variables to examine, articulated in a way that would allow them to determine compliance in a binary fashion. The committee finalised the adaptation of the tool in July 2012 (see http://www.ispch.cl/sites/default/files/instrumento_de_evaluacion_de_medidas_para_la_prevencion_de_riesgos_psicosociales_en_el_trabajo.pdf ) after years of discussion that included interruptions attributable to the earthquake of 2010 which changed public health priorities in the short term.

The intervention of the Araucaria Project in the development of the tool in Chile had repercussions in Québec as well, since the original instrument did not include issues of importance from a gender perspective. The tool in Québec was thus improved by the input of the researchers in the Araucaria team, who helped to provide content that captured issues more specific to women, like sexual harassment, work–family balance and discrimination (Vézina & Chénard, 2011).

Training sessions for labour inspectors were provided by a Canadian member of the research team and the lead advisor in the development of the Québec tool. Other training sessions were given by the Canadian expert in gender and ergonomics to labour inspectors and inspectors from the Chilean Institute of Public Health on ergonomics and gender-based analysis of the working conditions of men and women.

These are only some examples of the capacity-building activities that took place in Chile and in the context of university training in Canada.

One of the most significant components of the research programme was a major survey of working conditions in Chile, and a preliminary analysis of those results was presented for the first time in January 2012. It is expected that the analysis of those results will inform policy recommendations specific to the Chilean context, which will also be informed by the results of the comparative policy research undertaken in axis 2 of the programme, as described below.

Beyond the activities specific to Chile, the very process of developing three special issues of international journals on psychosocial hazards and work provided an opportunity to place the issues explored locally in an international perspective, thus raising the profile of workplace psychosocial hazards in the international research community. Three special issues were produced in the first four years of the project, and others are either in press or have since been published (Lippel & Casas Becerra, 2012). The first, in the International Journal of Law and Psychiatry (Lippel, 2007), included a broad range of contributions, from many disciplines and many countries. It allowed those team members responsible for its production to mobilise researchers from different scientific communities and perspectives around the issue of psychosocial hazards, work and policy. The second special issue, in the Comparative Labor Law and Policy Journal (Lippel, 2010), focused specifically on policy and law governing workplace bullying. Contributions from nine countries, including Chile, set the stage for an overview of regulatory strategies, and also provided models for different research strategies to study regulatory effectiveness. This special issue has since been translated into Mandarin and is scheduled to be published by the Peking University Press. The third special issue, in Safety Science (Lippel & Quinlan, 2011), brought together several papers presented at the international seminar on the role of unions and labour inspectorates in the prevention of psychosocial hazards, organised in Santiago by Araucaria in January 2010. Contributions from scholars who had not attended the seminar were also included, so that the overview covered situations in the European Union and six other countries. The various publications provided a platform to learn from the successes of inspectorates or unions in other jurisdictions, like Spain, and led to further international exchanges that are still ongoing. The activities associated with the development of these publications and the seminars held in Santiago have allowed for the creation of an informal international network of scholars, from both the North and the South, interested in the policy issues governing psychosocial hazards, and these exchanges will continue beyond the life of Araucaria.

C HALLENGES

While we are stimulated by the successes of our project with regard to capacity building, it would be inaccurate to suggest that these results came easily and without challenges. Aside from the challenges specific to language (Spanish, French and English were all used in our project), the primary challenges were not specific to global health research and collaborations between Canadian researchers and researchers from a middle-income country. Instead, interdisciplinary differences in methodological approaches required more translation efforts to ensure the quality of communication between team members than did cultural or even linguistic differences. Among the issues that were the most challenging from an intercultural perspective were those related to Canadian institutional research requirements, particularly with regard to ethics. For example, the requirement of signed consent forms had an adverse effect on response rates to the survey conducted in Chile, a country where it is unusual to make such a request. In hindsight, we would have proactively insisted that signatures not be required on consent forms, but we followed Canadian practices without considering whether or not they were appropriate to the Chilean context. Only when we saw the poor response rate did we realise that we should not have proposed a signed consent form from the outset.

Another source of unanticipated challenge emerged during the recruitment of students, for whom we had designed bursaries to study in Canada. It was important to us to support students interested in gender issues, but we had not anticipated the difficulties in finding students who could speak sufficient English or French to participate in graduate studies in Canada. It was particularly challenging to recruit women, who, in the Chilean context, could not easily leave their homes to study abroad. Furthermore, male students with families required funding to relocate them, although no funding of this nature was available. We eventually recruited two doctoral students from Latin America to study in Canada, instead of four as originally planned, and reallocated the remaining funding to students pursuing studies in Chilean universities, under the supervision of a Canadian member of the team, in one case, and a Chilean member of the team, in the other.

A final set of difficulties, perhaps universal, was associated with the challenges for researchers who hoped to inform policy-makers and state actors who were necessarily influenced by their political masters. In a volatile political context, years of collaboration and building of trust can be jeopardised by a change in government, an event totally beyond the control of the researchers and the research users themselves. Because a key component of our project focused on policies regulating working conditions, it was necessary to involve both NGOs, whose mission addressed working conditions, as well as state actors, who were responsible for the implementation of legal frameworks governing working conditions and occupational health.

The delicate task of respecting the sovereignty of local regulators (Okuonzi & Macrae, 1995), particularly in a context of political transition, was facilitated by the active involvement of NGOs, whose involvement in the project was less vulnerable to political change associated with electoral agendas. Their continued enthusiasm for the research programme and its implications suggested that the impact of the programme would outlive not only current funding but also the political climate of the moment. In the context of the political changes in Chile brought about by a change in government in 2010, it became evident that some aspects of our policy work would be more difficult to pursue than others. For example, the promotion of rights of workers was more difficult than the promotion of gender issues, all political spectra acknowledging the equality rights of women, which was not so for the rights of workers. The best strategy when faced with political change is to take the space you can in light of the political context. Despite the change in government, collaboration by our team in the development of a tool to monitor psychosocial hazards in the workplace continued, and the finalisation of the Chilean instrument in July 2012 was a clear example of the successful influence on policy through the long process of collaborative research.

It may well be that the existence of the Araucaria Project also ensured the inclusion of gender issues in that tool. The initiative for the tool in Chile came about as a suggestion of the Araucaria Project, an exchange between public health researchers from Québec involved in our project and those in Chile. Because of the funding through Araucaria, gender became a necessary ingredient of the project, and when the Chilean version took on a life of its own, the inclusion of CEM as a legitimate interlocutor in the process ensured that the gender-based adaptations were integrated in the final product. It is likely that, were it not for the initial funding that created the opportunity for collaboration, no psychosocial hazard tool would have been developed in Chile, or were it to have been developed, gender issues would not have been included.

This chapter has sought to outline concrete examples of various ways in which the Araucaria Project has pursued the objectives of capacity building for all those involved in the programme. The transdisciplinary nature of the research team and the project’s inherent grassroots approach has encouraged bilateral and internal capacity building, much of which has been reciprocal. The team members, including the students, learned from each other, from the research users and from the project advisors and vice versa. We saw progress in the integration of a gender lens in the analyses of both the research users and the researchers from the international, Canadian and Chilean contexts, and we saw an increased interest in psychosocial hazards for both researchers and research users. We believe we succeeded insofar as the political and global contexts permit in supporting the empowerment of all those involved, from local workers to policy-makers and researchers, so that they were able to intervene more effectively for the promotion of workers’ mental health, while including a gender perspective in their interventions.

The impact of these capacity-building activities is ongoing, and can be illustrated by a few examples. Through a broad range of workshops, tools and cartillas (primers) that were produced, the project succeeded in sensitising many male and female workers with regard to the importance of the relationship between work and employment conditions and mental health. This is particularly significant for working women and their organisations. For instance, the Asociación Nacional de Mujeres Rurales e Indígenas presently includes, in its advocacy agenda, the improvement of working conditions affecting mental health. The vice-president of the women’s committee of the Agrupación Nacional de Empleados Fiscales (ANEF) and the Central Autónoma de Trabajadores (CAT) incorporated the mental health of public sector workers as one of its principal preoccupations, promoting the eradication of discriminatory practices towards women (psychological harassment, sexual harassment, schedules that are incompatible with work–family balance, etc). The Araucaria team, including researchers and research users, hopes to follow up with the development of broadly based training schools in which workers will be given the tools needed to recognise, prevent and collectively address pathogenic factors in their workplace as well as the legal tools needed to act effectively on these factors.

Lessons learned from our experience included the importance of acknowledging the limitations of a classic research programme designed to train graduate students from abroad. The usual indicators of success, such as publications in high-impact journals and the number of diplomas obtained by students, are not the only or the most appropriate indicators of successful capacity building. This is even truer if one of the objectives is to empower women researchers from countries in the global South and more flexible measures of success would ensure better involvement of scholars and research users from grassroots associations. Impact factors are often inappropriate measures of relevance or usefulness of publications, particularly when the objective of the research programme is to have an impact on scholars and research users whose language is not English.

In research that includes a policy component, much of the impact of research activities will only become evident in years to come. Quantitative measures of success are in large part inappropriate not only with regard to policy components but also in relation to capacity-building activities involving grassroots organisations. Training materials and training sessions in the community are often far more useful and meaningful than peer-reviewed articles in the context of programmes designed to build capacity for effective change on the ground. The number of training sessions is relevant, but says little about the quality of that training and the transformations brought about by that training.

A more meaningful evaluation tool would provide opportunities for qualitative analysis. The preparation of this chapter proved to be extremely useful in promoting collective reflections on the meaning of the capacity-building experiences of the team over the years. Collective books inviting teams to reflect on the usefulness and meaning of their work may well be more appropriate vehicles for providing evidence of success and failure than any quantitative questionnaire or other bureaucratic exercise ill adapted to understanding the real impact of the research and capacity-building programme.

Benach, J., Muntaner, C., & Santana, V. (2007, September 20). Employment conditions and health inequalities . Final report to the WHO Commission on Social Determinants of Health. Toronto: Employment Conditions Knowledge Network.

Cornish, M. (2008, July 31). Realising the right of women to safe work: Building gender equality into OSH governance. Concept note. Presented at the XVIII World Congress on Occupational Safety and Health, Seoul, South Korea.

De Smet, P., Sans, S., Dramiax, M., Boulenguez, C., De Backer, G., Ferraio, M. & Kornitzer, M. (2005). Gender and regional differences in perceived job stress across Europe. European Journal of Public Health , 15 (5):536–545.

Díaz, X., & Mauro, A. (2010). Reflexiones sobre salud mental y trabajo en Chile: Analisis de siete casos desde une perspectiva de género . Santiago, Chile: Centro de Estudios de la Mujer.

European Agency for Safety and Health at Work. (2003). Gender issues in safety and health at work: A review . Luxembourg: European Agency for Safety and Health at Work.

Jones, H. (2011, February). A guide to monitoring and evaluating policy influence. Background note. Overseas Development Institute. Available from: http://www.odi.org.uk/publications/5252-monitoring-evaluation-me-policy-influence (Accessed 24 May 2016).

Kortum, E. (2007). Work-related stress and psychosocial risks: Trends in developing and newly industrialized countries. GOHNET Newsletter (Special issue):3–6.

Lajoie, A. (2003). Le Rapport Arthurs vingt ans plus tard: ‘Commentaire sur les commentaires’. Canadian Journal of Law and Society/Revue Canadienne de Droit et Société , 18 (1):45–49.

Lippel, K. (2007). Research, policy and practice in work and mental health: A multi-disciplinary discussion. International Journal of Law and Psychiatry , 30 (4–5):269–271.

Lippel, K. (2010). The law of workplace bullying: An international overview. Comparative Labor Law & Policy Journal , 32 (1):1–13.

Lippel, K., & Casas Becerra, L. (2012). Derecho y acoso desde la perspectiva comparada. Revista Estudios Laborales , 7:5–24.

Lippel, K., & Quinlan, M. (2011). Regulation of psychosocial risk factors at work: An international overview. Safety Science , 49 (4):543–546.

Messing, K., & Silverstein, B.A. (2009). Gender and occupational health. Scandinavian Journal of Work, Environment and Health , 35 (2):81–83.

Okuonzi, S.A., & Macrae, J. (1995). Whose policy is it anyway? International and national influences on health policy development in Uganda. Health Policy and Planning , 10 (2):122–132.

Potter, C., & Brough, R. (2004). Systemic capacity building: A hierarchy of needs. Health Policy and Planning , 19 (5):336–345.

Theobald, S., Tolhurst, R., Elsey, H., & Standing, H. (2005). Engendering the bureaucracy? Challenges and opportunities for mainstreaming gender in ministries of health under sector-wide approaches. Health Policy and Planning , 20 (3):141–149.

Vézina, M., Cloutier, E., Stock, S., Lippel, K., Fortin, É., Delisle, A., & Prud’homme, P. (2011). Québec survey on working and employment conditions and occupational health and safety (EQCOTESST): Summary report. Québec: Institut de recherche Robert-Sauvé en santé et sécurité du travail, Institut national de santé publique du Québec, Institut de la statistique du Québec.

Vézina, M., & Chénard, C. (2011). Guide d’identification de risques psychosociaux au travail . Québec: Institut national de santé publique du Québec.

Chapter 9 Research Capacity Building and Dissemination Components of the HIV Prevention for Rural Youth Project in Nigeria

Andrew Onokerhoraye & Eleanor Maticka-Tyndale

Interest in evidence-based policy and practice has grown in development circles during the last three decades. Hennink and Stephenson (2004), in their discussion of health policy, observe that the ‘continuing trend towards evidence-based policy formation has increased the demand for research outputs that can provide clear, concise policy-relevant findings’. (WHO, 2000; ECDPM, 2000, p. 3). In developing countries, the increase in demand for policy-oriented research and communication of actionable, evidence-based policy recommendations has been driven, to a great degree, by the requirements of international donors. In the domain of health research, dissemination of research results to policy-makers is now regarded as a key component of any research programme (Askew, Matthews, & Partridge, 2002). Thus, capacity-building endeavours in developing countries need to include not only building research capacity, but also capacity in translation of research into actionable policy and in strategic communication of such to policy-makers and practitioners.

Capacity in the realms of research, knowledge translation and communication of programme and policy-relevant findings remains a key challenge in developing countries. During the colonial and early independence years in Nigeria, the question of the research capacity of Nigerian researchers was not an issue since the quality of training received in Nigerian universities generally met what can be called international standards. The neglect of higher education in sub-Saharan African countries following the economic crisis of the 1980s left Nigerian and other African universities struggling to retain staff. This had considerable negative effects on research capacity. Although efforts are now being made to revive and improve research capacity in sub-Saharan African countries, the prevailing situation requires urgent attention if local research is to play a significant role in socio-economic development in these countries. At the same time, attention must also be paid to bridging the gap between findings and their utilisation by those whose duty it is to solve development problems. Typically, the findings resulting from the limited research conducted in universities and research institutions in Nigeria are disseminated through academic publications (Ezinwa, 2010) or presentations at conferences, both of which are requirements for the advancement of academic careers and elevation of the reputations of universities. These findings end up in libraries, desk drawers and conference proceedings, shared among academics, but never reaching policy-makers and practitioners (Stephenson & Hennink, 2002). In effect, the evidence produced in research fails to contribute to evidence-based planning or implementation of development strategies, programmes or policies (Martin, Foreit, Ezcurra, & Vernon, 2002). It is in the context of the prevailing weak research and knowledge translation capacity that there is increasing need and interest in building the capacity of researchers in Nigeria to address key development challenges. There is also a need to raise the consciousness of Nigerian policy-makers and the wider public about the importance of social research and its relevance to health and socio-economic development.

This case study is based on HIV Prevention for Rural Youth: Mobilizing Nigerian Schools and Communities (HP4RY) (Maticka-Tyndale et al, 2007). HP4RY was conceived and implemented by a team of Canadian and Nigerian researchers, non-government organisations and the Edo State Ministry of Education to address the pressing health problem of HIV infection in Nigeria. The goal of HP4RY was to contribute to the reduction of vulnerability of rural youth to HIV by building and evaluating a research-based model that strengthened and expanded the influence of the Family Life and HIV Education (FLHE) initiative. FLHE was developed by the Nigerian Educational Research and Development Council (NERDC) together with Action Health Incorporated (AHI). It has been endorsed by federal and state ministries of education, as well as by faith-based leaders. Key strategic objectives of the National Education Sector HIV/AIDS Strategic Plan for Action (NESSP) commit Nigeria to implementing FLHE throughout the country. HP4RY targeted youth and the communities in which they live with HIV prevention programming that addressed the gendered nature of vulnerability and risk. It translated the knowledge gained in research to enhance the FLHE programme already approved for delivery in all junior secondary schools in Nigeria and to work with communities to raise their AIDS competence as outlined by Catherine Campbell (Campbell & MacPhail, 2002) and described in Maticka-Tyndale and HP4RY Team (2012). The goal was to develop, evaluate and produce effective and efficacious models of interventions and intervention delivery that are sustainable and deliverable in rural Nigerian communities.

The conception and implementation of the HP4RY project recognised the importance of research capacity building and knowledge translation and mobilisation by making both of them key components of the research programme. Capacity building of Nigerian team members, research staff and field data collection personnel in the implementation of all aspects of the research programme was a major component of the project. Knowledge gained through research was translated into enhancements of the FLHE programme and development of a community-based programme deliverable by members of the National Youth Service Corps (NYSC) to enhance AIDS competence of rural communities. These programmes were mobilised and delivered in schools and communities, and their procedures and impact evaluated. Through the research and evaluation components of HP4RY, new knowledge was generated that could contribute to the articulation of new, and modifications to existing, policies relevant to FLHE delivery in rural settings as well as to the use of the NYSC to address HIV vulnerability of rural communities. These policies cut across multiple levels, from local schools and communities, to ministries and NGOs working at the state level, and ultimately to federal offices and international funding bodies. A strategic approach to the dissemination of research results was central to HP4RY, ensuring that individuals and organisations at all levels were informed of salient findings. In this way, members of the general public, community workers, researchers, NGO staff, and state and federal ministries, as well as directors and ministers were all involved in knowledge transfer and exchange. This paper outlines the research capacity-building approach and outcomes as well as the dissemination strategy used during the implementation of the HP4RY project in Nigeria.

P ERSPECTIVES ON RESEARCH CAPACITY BUILDING AND DISSEMINATION IN THE SUB -S AHARAN A FRICAN CONTEXT

Today, research capacity includes not only skills in data collection and analysis, but also in the translation of research results into actionable strategies, programmes and policies, and their dissemination to civil society and government organisations for implementation. Capacity for health and socio-economic research has been further defined ‘as an ability of individuals, organisations or systems to perform [and utilise health research] effectively, efficiently, and sustainably’ (Milèn, 2001, p. 1). Such research capacity is still a major problem limiting the participation of researchers in Nigeria, as in other parts of sub-Saharan Africa (Lansang & Dennis, 2006). In an overview of capacity-building needs in the Global South, Nchinda (2002) outlines two inter-related, interdependent components of capacity strengthening: ‘providing institutional or organizational support, and improving researcher capacity through appropriate training’ (Mahmood, Hort, Ahmed, Salam, & Cravioto, 2011, p. 2). Mahmood et al (2011), based on research in Bangladesh, similarly points out that institutional research capacity suffers from inadequate support and poor management as well as other negative influences.

Among the factors influencing research capacity are: an enabling environment, physical infrastructure, adequate funds for research and training, access to scientific and technical information, effective interactions between researchers and policy-makers and good governance (Hyder, Akhter, & Qayyum, 2003; Mahmood et al, 2011; Nchinda, 2002; White, 2002). The absence of these is a major constraint to research capacity in a developing country such as Nigeria. Support from advanced countries is indispensable to strengthening research capacity (Lansang & Dennis, 2004; Simon, 2000).

Closely related to the challenge of research capacity building in sub-Saharan Africa is that of dissemination of research output to policy-makers. In view of the marked differences in the work domains of policy-makers, the results of research must be disseminated differently to each sector, reflecting the specific programmes, policies or development challenges they face. There is a broad consensus in the literature that successful communication between researchers and research users is crucial for the effective use of research in decision making in policy and practice (Hanney, Gonzales-Block, Buxton, & Kogan, 2003). A major objective of research communication and dissemination is to promote the use of research results for policy making and in the implementation of development programmes. To this end, there is a need for a coherent conceptual framework for research dissemination.

Attempts have been made within the domain of healthcare policy and the delivery of health services to conceptualise the dissemination and use of research (Buxton & Hanney, 1996; Hanney, Packwood, & Buxton, 2000; Hanney et al, 2003). The framework identifies the potential value of applying research results to policy formation in health, and also provides a description of the processes and stages involved in the application of research output to policy making. ‘The stages include the inputs to research, the research process, primary outputs from research, secondary outputs from research, practitioners application of research and final outcomes…. [T]he framework also outlines various feedback loops and forward leaps, recognizing that the process of research utilization is often multidirectional (Hanney et al 2002)’ (Hennink & Stephenson, 2004, p. 5).

A number of models of research utilisation by policy-makers and other stakeholders have been articulated. These focus on the nature of the relationship between researchers and policy-makers (Hanney et al, 2003; Stephenson & Hennink, 2002; Weiss, 1979). Hennink and Stephenson (2004) conceptualise the use of research results in policy making as falling ‘into three broad categories; rational models, incremental models and political models (Hanney et al 2002; Weiss 1979, 1980)’ (p. 5). The rational models view research as producing actionable information that can be passed directly to policy-makers (Walt, 1994a, 1994b; Weiss, 1977). Porter and Prysor-Jones (1997, cited in Hennink & Stephenson, 2004) argue that such communication of research results stimulates policy change and development strategy. The second category, incremental models, comprises interactive and enlightenment models that emphasise systematic consultations and interactions between researchers and policy-makers at multiple points during the research process. The third, political models, focuses on the fact that policy-makers use the output of research to support specific political positions (Martin et al, 2002), commissioning research to provide evidence to justify existing positions.

Several barriers and facilitators of research utilisation have been identified in the literature. A study by Funk, Tornquist and Champagne (1995) documented several barriers such as lack of staff, time and access to research output (Adamsen, Larsen, Bjerregaard, & Madsen, 2003; Hatcher & Tranmer, 1997; Parahoo, 2000). In another study, Glacken (2002) identified persistence of existing, ineffective practices and poor understanding of how to apply research output to policy. Documented facilitators using research results include ‘administrative commitment and support, knowledge of the research process, availability of research consultants, favourable research attitude, affiliation with a university, and financial resources’ (Tan, 2008, p. 13).

These perspectives on research capacity building and the dissemination of research output provided the conceptual framework for the examination of capacity-building and dissemination strategies of the HP4RY project in Nigeria.

B ACKGROUND TO THE P ROJECT ON HIV P REVENTION FOR R URAL Y OUTH IN E DO S TATE , N IGERIA

HIV and AIDS in Nigeria were first identified in 1985 and were reported in 1986 (Avert International, no date). The military government in Nigeria did not appreciate the challenge posed by the increasing rates of HIV transmission until 1991, at which point the federal government, through the Ministry of Health, made its first attempt to assess Nigeria’s AIDS situation. It was then reported that about 1.8% of the population in the country was infected. Reports of other surveys from the 1990s showed that HIV prevalence rose to 3.8% in 1993 and 4.5% in 1998 (Avert International, no date). As cited in Ikpang (2010), following the return of democracy in Nigeria in 1999, ‘HIV prevention, treatment and care became one of the government’s primary concerns. The [Nigerian] President’s Committee on AIDS and National Action Committee on AIDS (NACA) was created and in 2001, the government set a three year HIV/AIDS Emergency Action Plan (HEAP) ’ (p. 32).

Despite some increased efforts to pay attention to the HIV epidemic, the prevalence in Nigeria ranged between 4.5 and 5.5% between 2000 and 2008. In some geo-political regions of the country, the HIV prevalence was as high as 8–10% during the 2000 to 2008 period. Implementation of numerous preventive and care plans and institutions such as the National Action Committee on AIDS (NACA), the National Education Sector HIV/AIDS Strategic Plan (NESSP), HEAP and FLHE were largely ineffective. For example, while NESSP was designed specifically to address the role of the education sector as an agent of change and declared that every child had a right to participate in the Nigerian Family Life and HIV Education programme in school, programme delivery remained scattered and the impact of FLHE on participating students was not rigorously assessed and evaluated. In addition, while HEAP called for the elimination of barriers to a community-based response, there was little evidence of community-based activities that could support the learning of youth. While youth in Nigeria, as in many countries, might learn how to protect themselves against HIV, they continued to live in communities where acting on what they had learned was difficult, if not impossible (Onokerhoraye, Maticka-Tyndale, & HP4RY Team, 2012a).

Furthermore, although FLHE had been approved for delivery in all junior secondary schools in Nigeria, because of the cost and logistics associated with delivering this programme, it was still not accessible to the majority of youth. Where sexuality education was available, the bulk of the school programmes continued to use extracurricular methods, leaving many children without sexuality education. Additionally, whereas out-of-school youth were generally less informed about reproductive health and participated more often in risky sexual activities, most existing sexuality education programmes for young persons were school based. Consequently, the reproductive health needs of out-of-school youth were not being addressed. As well, due to funding and other constraints, many sexuality education programmes for youth in Nigeria had not been sufficiently sustained to ensure full positive impact.

It was within the context of the need to reach youth in rural schools and communities that HP4RY was conceived and implemented in Edo state in the southern part of Nigeria. Edo is one of 36 states in the Federal Republic of Nigeria. The state had an estimated population of over 3 million people in 2011, 30% of whom were youth between the ages of 12 and 20 years. The choice of Edo state for the project was motivated by four factors:

1)   As is common in most of Nigeria, over 70% of the population of the state lives in small rural communities of between 500 and 20 000 people. Thus lessons from this project might be relevant to a large number of the rural communities in Nigeria.

2)   The implementation of FLHE in the state was low when the project was conceived (Maticka-Tyndale et al, 2007).

3)   Edo state is located in the South-South geo-political zone of Nigeria where HIV prevalence is higher than in the other five geo-political zones in the country.

4)   Edo state is reputed to be a major region of international female migration to Europe, with a large proportion of the young female sex trade migrants coming from its rural communities (WHARC, no date).

The project was designed, therefore, to make maximum impact in terms of knowledge development and translation within Edo state specifically and, by extension, other parts of Nigeria.

R ESEARCH C APACITY B UILDING OF HIV P REVENTION FOR R URAL Y OUTH: M OBILISING N IGERIAN S CHOOLS AND C OMMUNITIES (HP4RY)

In view of the generally poor research infrastructure and human resources for contemporary research in Nigeria, coupled with the rather low quality of graduates from most institutions in the country in recent decades, HP4RY adopted a broad conception of capacity building. The research capacity building of HP4RY focused on the following:

•   the improvement of the research infrastructure and capacity of partner organisations, specifically the Centre for Population and Environmental Development (CPED)

•   the re-orientation and enhancement of skills of Nigerian research team members

•   the training of junior academics who were the project staff

•   the training of field research staff involved in data collection and the delivery of services in schools and communities.

Improvements in the research infrastructure and capacity of CPED

Of the three Nigerian institutions involved in the research partnership, CPED provided the base for implementing research activities in Nigeria. It was essential that the research infrastructure capacity of CPED be improved. An initial assessment of the facilities available at CPED was made during preparation of the research proposal. The assessment focused on identifying assets as well as weaknesses at both the individual and organisational level. The lack of adequate institutional resources at CPED was a major challenge that had to be tackled at the beginning of the project. This meant providing computers and computer peripherals, furnishing the project office and conference room, and maintaining the power generator.

The organisational capacity of CPED was also enhanced in terms of the considerable experience gained as host to the collaboration between researchers from different academic fields and institutions. The enhanced organisational capacity of CPED contributed to its selection as one of 24 African research centres being supported by the Canadian International Development Research Centre (IDRC) Think Tank Initiative. The University of Benin and CPED benefited from the project by acquiring considerable experience in international collaborative research. The experience has improved teaching and research on HIV and sex education at the university as well as generated new interests among Canadian and Nigerian researchers. Furthermore, CPED, through its participation in this project, enhanced its research administration capabilities in areas such as financial management, technology transfer and research ethics. Finally, CPED improved its ability to perform outreach activities aimed at the general public through service delivery activities carried out in target communities.

Enhanced research capacity of Nigerian project team members

Although most of the Nigerian project team members received their higher degrees from Western universities and have considerable experience in contract research, the decay in the Nigerian university system in the last two decades has had a negative effect on their research capacity. It was not, therefore, safe to assume that members of the research team were familiar with contemporary approaches to research such as the full range of quantitative and qualitative methods, or the differences between writing for academic publication, contract reports and policy papers. Through proposal writing meetings, project team meetings and workshops in Nigeria and Canada, the Nigerian team members were exposed to the action research model, which is not well understood or practised in Nigeria. Courses taught by Canadian team members at the University of Benin and the experiences gained on this project by individual team members paved the way for curriculum improvement in the social sciences and education and, ultimately, a transfer of knowledge to students who will become leaders and teachers themselves, thus ensuring the sustainability of the results.

The Canada–Nigeria research partners have made a conscious effort to ensure that research results are published in peer-reviewed journals with a wide international readership. To this end, the project contributed to an enhanced integration of Nigerian team members into the international scientific community by increasing their capacity to produce articles suitable for presentation and publication in international, peer-reviewed venues. A number of strategies were initiated to deal with the challenges of preparing and disseminating research results. In order to compensate for the insufficiencies of local libraries and access to international scientific literature, a massive literature review and annotation project was undertaken in Canada. Four graduate students located, reviewed and annotated over 1400 scientific articles related to the project. These were organised by topic and made available to all the team members and junior academics on the research project in order to facilitate their own literature reviews relevant to the publications and presentations they were preparing. Some of the main findings, conclusions and policy recommendations of HP4RY were published in a special issue of the African Journal of Reproductive Health and were posted electronically on the CPED website where they were freely accessible.

Research capacity building of junior academics

A key component of the capacity-building benefits of the research programme related to training junior academics to play significant roles in the project. Many junior academics in Nigeria lack the opportunity for further studies and collaborative research and university positions are few. To enhance career opportunities for junior academics, they were employed as project staff and trained in sexuality and sexual health, research methodology, data collection, data processing and analysis, writing for research reports and academic publications, and project management.

Training took different forms, from academic training leading to a degree, to short-term training to acquire specific skills and knowledge, to mentoring. Four junior academics who served as staff for HP4RY benefited from participating in the Sexuality Leadership Development programme run by Action Health Incorporated (AHI) over two-week periods in July of each year. They were exposed to ‘cutting-edge conceptual, theoretical and programmatic issues in sexuality, sexual health and sexual rights’ as well as ‘sexual diversity, pleasure, HIV/AIDS and vulnerabilities, gender and violence against women’ (Africa Regional Sexuality Resource Centre, 2012). All staff increased their report-writing ability, gaining knowledge in the use of surveys, interviewing and ethnographic methodologies, data analysis and the preparation of presentations for a variety of stakeholder and academic audiences. In Canada, the graduate students, recent graduates and postdoctoral research fellows who worked on the project gained experience in international, cross-cultural collaboration. Similar to their Nigerian counterparts, several contributed to and benefited from co-authorship of publications and presentations. Three of these Canadians also contributed to the training of their Nigerian peers in data management and analysis techniques and report writing, enhancing their own skills in cross-cultural training and collaboration. The speed and ability of the junior Nigerian academics to acquire the research skills imparted to them was slow and, at times, frustrating for their trainers; however, there was considerable improvement and some were able to make presentations at international conferences while others continue to work on publications based on the data collected during the research programme (Onokerhoraye, Maticka-Tyndale, & HP4RY Team, 2012b).

Apart from informal training and mentoring, some of the Nigerian and Canadian junior academics acquired master’s degrees over the course of the project. Of major importance is the fact that five (two from Nigeria and three from Canada) enrolled in PhD programmes. The proportion of non-PhD-qualified staff in most Nigerian institutions is quite high by international standards. Most holders of Doctorate degrees were trained from the 1960s to the early 1980s. As these older academics retire, it is the younger, less qualified researchers who fill the gap. Boosting the number of doctorates within Nigerian academic institutions is obviously a major priority in the next few years. Thus, this project’s facilitation of the ongoing doctoral training of two of the Nigerian project staff was an important contribution to research capacity building in Nigeria.

Research capacity building of graduate field research staff

As noted earlier, although the training of field research staff may not be strictly regarded as part of research capacity building, in the context of Nigeria in general and the HP4RY research project in particular, it is quite important to the research process. Fifty-five research assistants were trained in quantitative and qualitative data collection and some of them secured junior research appointments in both CPED and other Nigerian institutions on the basis of the skills they acquired. In addition, 40 graduate Youth Corps members were trained to work in the target communities. They completed a systematic training programme in sexuality and community mobilisation strategies, led by seasoned and professional personnel comprised of team members from Nigeria and Canada. This made it possible for them to have an impact in the 10 communities to which they were posted. Five of them are now employed by CPED as junior academics in training on other research projects. A few have made presentations at local workshops on community development in Nigeria and others are collaborating with senior researchers on writing papers.

D ISSEMINATION AND K NOWLEDGE T RANSLATION S TRATEGIES OF HP4RY

The HP4RY project in Nigeria recognised the value of research for policy, programme and service development from its inception. As such, a knowledge translation and mobilisation plan was incorporated during the planning stages. The incremental model described by Stephenson and Hennink (2002) was applied with the participation of knowledge-using stakeholders at various stages of the project’s development and implementation and attention was paid to the salient content and form of dissemination for different stakeholders and target audiences. Action-oriented knowledge was disseminated to participating schools (including teachers and principals), communities, youth in the NYSC, local agencies, education and health ministries, the NYSC directorate, researchers, policy-makers and HIV advocacy groups. Programme packages relevant to delivery of FLHE and a community-based programme using the AIDS Competent Community framework and delivered by members of the NYSC were prepared and made available to NGOs, relevant state and federal ministries and the NYSC directorate. The content and methods used for knowledge transfer and exchange were targeted to specific audiences and are discussed later in the chapter.

Participation of key stakeholders

A first step in the incremental model is the involvement of diverse stakeholders at salient phases of the project. Such participation may include formal partnership with researchers as project team members, performance as advisors on key issues, participation in knowledge translation and exchange, action as conduits of information from the project to broader audiences, and implementation of practical recommendations and outputs of the project.

Depending on the level of involvement, stakeholder participation may imply a certain level of joint responsibility for decision making, project implementation and eventual delivery and dissemination of results. Beyond such responsibilities, stakeholder participation involves empowerment by being included in issues in which they have a stake. Stakeholder participation is beneficial to all aspects of an action research project as it helps to ensure that the needs and practical realities of stakeholders are reflected in research, knowledge translation and actions. Such participation increases the likelihood that full, rich and relevant data will be collected and that interpretations appropriate to local conditions and contexts will be made. It also increases the likelihood that the actions taken based on the research evidence will reflect local needs and contexts, and be feasible and sustainable.

Stakeholder participation was present at all stages of the HP4RY project, from development of the letter of intent and eventual project proposal, to refinement of the research design and data collection instruments, to interpretation of research findings, translation of results into programmes and strategies, to evaluation of those programmes and strategies, and to the eventual delivery of results to a broader array of stakeholders and interested parties.

The project team included researchers with academic appointments, representatives of two NGOs (AHI and CPED) that focused their activities on sexual and reproductive health issues in Nigeria, and the AIDS officer from the Edo State Ministry of Education. These representatives were instrumental in identifying the types of project results and outputs that were needed to enhance their work developing and delivering programming that targets the sexual and reproductive health of youth and the reduction of vulnerability to HIV infection. For AHI, the desired output was a scientifically rigorous evaluation of FLHE as delivered in rural schools and recommendations for strengthening the programme in this context. For the Ministry of Education, this was a rigorous evaluation of FLHE training and delivery and the factors that contribute to or impede its effective delivery in state schools. For CPED, this was development of methods, programmes and strategies to enhance the AIDS competence of rural communities.

As the project progressed and translation of preliminary research results indicated how to deliver community-based programmes to enhance AIDS competency, a fourth stakeholder became involved, the Edo state NYSC directorate. The NYSC directorate is in charge of the one year compulsory national service for Nigerian university graduates. It was NYSC members who were trained to work in and with communities to enhance their AIDS competence. Although the NYSC directorate was not involved from the beginning of the project, officers and staff in the directorate worked with project team members and staff to make delivery of this HP4RY programme component possible. They also provided information on what would be needed in a programme package (HP4RY, 2012a) for continued NYSC involvement beyond the end of the HP4RY project.

Other stakeholders that were involved in specific stages of the project included local community leaders, master trainers of teachers, Youth Corps members and other NGOs and researchers in the region. The local community leaders provided crucial information for the development of community-based programming as well as feedback on the work of the Youth Corps members in their community, thereby influencing the final package for the community-based programme. Master trainers reviewed the knowledge gained from baseline research in schools and communities and worked with several team members to translate it for inclusion in the FLHE curriculum and teacher/principal training as well as for inclusion in the FLHE programme package (HP4RY, 2012b). The Youth Corps members provided ongoing information from the communities in which they worked as well as critical commentary on the community-based programme which they were implementing. This was also incorporated into the community-based programme package. Local NGOs and researchers reviewed research results at two stages of the project and provided advice and critical commentary based on their own research and experience.

The overall effect of different stakeholders participating was that they became key players in establishing project outputs and in the dissemination and use of the research results, by their organisations and by a wider array of knowledge users in Nigeria and beyond.

Capacity building of junior academics on research communication

HP4RY recognised the fact that effective dissemination requires dedicated human and material resources to support specialised communication skills, in addition to the resources needed for the research activity. It was in this context that several junior academics were trained and mentored in translation and communication of research results to different stakeholders. Over the course of the project, they learnt and acquired experience, including bringing knowledge gained from research back to community members, translating research results into components of the training for Youth Corps members, presenting research at academic conferences in the form of both oral and poster presentations, and preparing research reports for circulation to team members and the funding agency.

M ODES OF K NOWLEDGE D ELIVERY A PPROPRIATE TO T ARGET A UDIENCES

If the knowledge and recommendations that result from research are to be utilised by stakeholders, they must be delivered in a form that is readily accessible, understandable and useful to the stakeholder. This requires the production of knowledge and recommendations in different forms for different stakeholders.

Presentation of research findings to local researchers and NGOs

The research component of HP4RY entailed the collection of three waves of data, including brief ethnographies in the target communities, surveys and interviews in schools at baseline, and repeated surveys and interviews in schools in two additional waves. HP4RY’s dissemination strategy for research outputs entailed the preparation of summaries of findings from each wave. Reports were produced and circulated among team members. This was followed by seminars in which results were presented to the team members, project staff, representatives of the Edo State Ministry of Education and the state directorate of the NYSC, NGOs, the media and other academic researchers. Presentation of the second wave of data, which comprised a first look at evaluation results for school and community programming, followed a more formal course than the first, with representatives from the University of Benin and the Edo State Ministry of Education and other researchers speaking in addition to HP4RY members. The target audience was expanded to include Youth Corps members and several community representatives. In each of these seminars there was open, animated discussion of the research findings and their implications for HIV prevention in schools and communities.

Presentations to participating schools and communities

Following baseline and the final wave of data collection, meetings were held with school and community representatives to present and discuss research findings and their implications. The participants included teachers, heads of schools, parents, community-based civil society groups, students and out-of-school youth. The meetings were held in each of the three senatorial districts in Edo state where the programme operated (ten communities in each senatorial district). The results, lessons learned and the way forward as reflected in the findings of the research were presented at these meetings with the opportunity for feedback from those in attendance. The presentations were made jointly by representatives of the research team, other programme personnel and some community members. The feedback from the audience was documented and incorporated into the final reports and recommendations.

T RANSLATION AND M OBILISATION OF K NOWLEDGE THROUGH P ROGRAMMING

Community-based programming.

Translation and mobilisation of knowledge gained in research took place through activities undertaken by trained members of the Youth Corps posted to communities over a two-year period. The team members and junior academics with positions as HP4RY research staff worked to translate what was learned from the baseline research in communities and schools into community-based strategies for developing locally relevant programming to reduce youth vulnerability to HIV infection. Members of the Youth Corps were then trained to implement these strategies under the guidance and supervision of a staff member. Three cohorts of Youth Corps members were deployed to communities over a two-year period.

Upon gaining the confidence of community members, the Youth Corps members embarked on forming what they tagged ‘core groups’ of different age cohorts. Their aim was to build the capacity of community members to create activities to delay sexual initiation among youth and to spread the message that HIV exists and about its consequences, transmission and prevention to neighbouring communities. Such groups bore names such as HP4RY Club, Abstinence Club, HIV Club, Zip-up Club and Faithful Club. The empowered community-based core group members engaged in activities which they felt would be beneficial to members of their community. They staged plays on HIV/AIDS, unwanted/teenage pregnancies, parent–child communication, abuse and incest, among others, at open spaces in the target communities. Community chiefs, elders, women, youth and children came out to witness these activities. In some communities, the Youth Corps members also carried out vocational training for women and girls, which contributed toward empowering them to carry out income-generating activities and raising their self-esteem, independence and autonomy in making life decisions, including those related to behaviours that placed them at risk of acquiring HIV. A total of 180 women and girls graduated from this activity in just one community and most of the beneficiaries used the skills acquired to generate income. The Youth Corps members also negotiated with local and neighbouring health facilities to make HIV testing accessible to the community members. In most communities, Youth Corps members helped in the delivery of FLHE in schools and continued the lessons and activities outside of the school.

While implementing these activities, Youth Corps members also kept a journal of activities, experiences, successes and challenges which contributed to both revised programme guidelines and the final programme package (HP4RY, 2012a). Thus, Youth Corps members participated in knowledge translation and exchange at two reciprocal levels: in translating knowledge from baseline and delivering programming to communities, and in providing feedback on that programming to contribute to refinement of the programming model. Evaluation results demonstrated the strength of the contribution made by the community programming to reducing youth vulnerability to HIV infection (Arnold et al, 2012; Omorodion, Akpede, Maticka-Tyndale, Agbontean-Eghafona, & Onokerhoraye, 2012).

Empowerment of teachers and students to deliver FLHE

Teachers and students in 30 junior secondary schools were empowered to play key roles in the delivery of FLHE. The teachers were selected from those teaching three core subjects (English language, basic/integrated science and social studies) and providing guidance and counselling with the intention of integrating the FLHE curriculum. This was carried out across the three senatorial districts of Edo state – Edo South, Edo Central and Edo North. Teachers took FLHE back to their schools and delivered lessons to the students through classroom delivery, co-curricular activities and the establishment of anti-AIDS clubs which they tagged ‘FLHE clubs’. Students from each of the junior secondary school grades one to three were also trained as peer educators to help in the messaging of HIV prevention and in extracurricular activities. Both government teachers and community teachers, who complemented teaching in these schools, benefited from this programme. Master trainers who were initially trained by the Edo State Ministry of Education attended refresher courses as part of the HP4RY project, prior to carrying out training of teachers for FLHE. Remote schools, which were rarely included in such initiatives, were also given the opportunity to participate.

Evaluation results confirm the contribution the school-based programme made to reducing youth vulnerability to HIV (Arnold et al, 2012). The training has empowered students in these schools to take the lead in the spread of HIV prevention messaging and teachers now feel comfortable talking to students about sex, sexuality and HIV/AIDS (Dlamini, Okoro, Oni Ekhosuehi, Esiet, Lowik, & Metcalfe, 2012).

Newsletters

HP4RY recognised that distributing a regular newsletter summarising study findings and activities in schools and communities was an ideal way to update study participants, participating agencies and other stakeholders. While such newsletters can involve a fair amount of work, our experience suggests that dissemination benefits are well worth the effort. A project newsletter was established translating results from baseline research, particularly those on sexual scripting, into strategies for use in classrooms. Additional issues of the newsletter focused on the activities of Youth Corps members in communities, providing a view of what was going on in each community to Youth Corps members, teachers, schools and policy-makers in the education and Youth Service Corps sectors.

Programme packages

Both of these programming initiatives — delivery of FLHE in Edo state schools and Youth Corps members working to enhance the AIDS competence of rural communities — have been fully documented, making the models of programme delivery available to others (HP4RY, 2012a; 2012b). Although FLHE was delivered in only 30 junior secondary schools and community-based programming in only ten rural communities, the documentation of the models and processes used in programme packages provides a means to improve and promote HIV prevention and sex education for people living in other rural communities in Nigeria and elsewhere.

Knowledge translation for government and NGOs

HP4RY recognised the fact that utilisation of knowledge gained from research is predicated on policy-makers and other users having ready access to research findings. In a country such as Nigeria where policy-makers are yet to appreciate the value of research, results which are difficult to access may not be used. Often research users in Nigeria complain that traditional communication vehicles for research findings, including conference presentations and peer-reviewed publications, are not primary information sources for them. They point out that summaries of main findings communicated through other easily accessible channels are preferred. Such channels obviously allow for timely access to information and actively route documents directly to key stakeholders. HP4RY adopted the strategy of using multiple communication channels in order to reach various audiences, including policy-makers, associations, advocacy groups and the media. This justified the considerable attention paid to the use of policy briefs, newsletters, direct presentations of research results to policy-makers and the organisation of workshops. The use of these dissemination channels by HP4RY also responded to the need for timeliness in the communication of research results. HP4RY put into consideration users’ needs for immediate, relevant and quality research when the policies were being developed or reviewed rather than after the policies had been conclusively formulated. This was carried out by annually briefing the key stakeholders on the research progress and output during the research implementation period.

Networking with other programmes

The opportunities provided by the implementation of another project titled Building Civil Society Capacity for Advocacy on Sexual and Reproductive Health and Rights (SRHR) in Nigeria, funded by the European Commission, was used to carry out the dissemination of the results of the project to ten additional states across the different geo-political zones in the country. The aim of the European Commission’s funded project, which was implemented by one of the collaborating organisations, CPED, was to improve the reproductive and sexual health situation in Nigeria with a specific focus on adolescents through interventions designed to strengthen the capacity of NGOs, media personnel and youth groups to play key advocacy roles in policy dialogues. The training was designed to improve the beneficiaries’ knowledge about the SRHR situation and concerns in Nigeria, and empower them to publicise the problems and challenges facing SRHR as well as to advocate to governments at the federal, state and local levels to pay adequate attention to the reproductive health challenges facing Nigerians, especially adolescents.

Some of the outputs of the HP4RY research provided inputs into programme content and training materials. Furthermore, the experience and output of HP4RY’s Youth Corps activities in the communities provided inputs in the training and empowerment of adolescents in ten local government areas outside Edo state. Youth groups and organisations in the rural communities of Nigeria, previously not well organised or coordinated to play active roles in the promotion of the reproductive health of their peers, were thus empowered to design and deliver especially peer education and ‘youth friendly’ services. The empowered NGOs have since been carrying out advocacy activities on reproductive health in collaboration with trained media personnel. Free media coverage was used to get results out to as many people as possible. Using the experience of HP4RY, youth in 40 rural communities across Nigeria have been carrying out peer education and youth-friendly activities on reproductive health.

Redesigning CPED website for more effective communication

Facilities for the dissemination of HP4RY outputs at the CPED level were enhanced by the redesign of the CPED website. The aim of this activity was mainly to make research content available (especially that of studies) and to improve the navigation tools of the website to make it more dynamic. The website now includes brief summaries of the different project activities in which CPED is involved, including HP4RY. This has resulted in an increased number of visits to the website. The website will provide a location where programme packages, reports, presentations, publications and summaries of the HP4RY project may be accessed by interested parties from anywhere in the world.

Policy briefs

The outputs of HP4RY coupled with those of other reproductive health and HIV/AIDS programmes carried out by CPED are being used to advocate for policy change on reproductive health and HIV challenges at the local, state and national levels in Nigeria. The typical audience for a policy brief is not interested in the research methods used to produce the evidence, but is interested in potential solutions based on new evidence. These are presented in the form of CPED policy briefs, which outline the rationale for choosing a particular policy alternative or course of action in a current policy debate.

CPED produced six such policy briefs on reproductive health and youth, which were widely distributed to key stakeholders, especially government departments across the country. The policy briefs are concise and focus on the implications of new evidence for reproductive health policy and implementation in Nigeria.

Presentations to Edo State Ministry of Education and the Directorate of the NYSC

HP4RY appreciated the fact that in the context of Nigeria, the most effective means of disseminating research to policy users was through direct, interpersonal contact. In view of the critical issues examined in this project, policy-makers must trust that the information they receive is reliable and credible. In Nigeria, policy-makers will often rely on personal contact with researchers they trust. Sustained and substantive communication engenders trust. In the context of HP4RY, the points of contact included locally based users and policy-makers with jurisdictional responsibility in the topic area of the research, which in this case, was the Edo State Ministry of Education.

It was against this background that a presentation was made to the education sector in Edo state. The specific participants were set in collaboration with the State Ministry of Education. The focus of the presentation was on the knowledge and lessons learned that were particularly relevant to the delivery of the FLHE programme as well as methods for supplementing the basic programme to strengthen its impact. A similar presentation was planned for the Council on Education, a national body that includes representation from every state.

The active promotion of evidence-based decision-making for development challenges facing Nigeria is a necessary step toward meeting the Millennium Development Goals (Sustainable Development Goals). We observe in Nigeria what Nchinda (2002) has identified across countries in the Global South, a mismatch between the challenges facing key sectors of development and the technical and human capacity to use existing knowledge and to generate new knowledge to promote development. We concur with Nchinda (2002) that it is essential to build indigenous research capacity so that Nigerian researchers can undertake studies which can lead to the development of appropriate public policies. Building indigenous research capacity will enable Nigerian researchers to translate results of studies carried out elsewhere into the country’s national setting. Such research will, eventually, increase knowledge-based decision making by the country’s political leadership. Consistent with its capacity-building goal, the HP4RY project took steps to strengthen capacity for both school and community programme delivery and the conduct and presentation of research. In this way, HP4RY made a significant impact on research sustainability in Nigeria, including the importance of building up a suitable infrastructure for research.

The goal of all research dissemination should be utilisation of the output for policy change or programme implementation. The utilisation of research results is facilitated when individuals and organisations are able to assess and digest the new information thoroughly and critically, integrating it with prior understanding and experience. The dissemination and knowledge transfer strategy of HP4RY was governed by an incremental, interactive model that emphasised the importance of interactions between the researchers, policy-makers and other stakeholders throughout the research process. The approach has brought considerable benefits to the dissemination of the HP4RY processes and outputs, which we believe other social research projects in Nigeria should emulate.

Based on the experience of the dissemination activities of HP4RY, we conclude that the effectiveness of dissemination to influence policy and programme implementation requires a strengthening of the network among researchers, policy-makers and practitioners; an improvement in the dissemination of research for different audiences across the country, and a strengthening of Nigeria’s social science research capacity. This suggests a need to strengthen work at the research–practice boundary and to extend the tradition of critical enquiry beyond formal academic research programmes. The HP4RY project has made its contribution in this respect by the capacity-building and dissemination approaches adopted.

Adamsen, L., Larsen, K., Bjerregaard, L., & Madsen, J. K. (2003). Danish research-active clinical nurses overcome barriers in research utilization. Scandinavian Journal of Caring Science , 17 (1):57–65.

Africa Regional Sexuality Resource Centre. (2012). Sexuality Leadership Development Fellowship. Available from: http://arsrc.org/training/sldf/index.html (Accessed 24 May 2016).

Arnold, R., Maticka-Tyndale, E., Tenkorang, E., Holland, D, Gaspard, A., Luginaah, I., & the HP4RY team. (2012). Evaluation of school- and community-based HIV prevention interventions with junior secondary school students in Edo State, Nigeria. African Journal of Reproductive Health , 16 (2):103–125.

Askew, I., Matthews, Z., & Partridge, R. (2002). Going beyond research: A key issues paper raising discussion points related to dissemination, utilization and impact of reproductive and sexual health research. Paper presented at Moving Beyond Research to Influence Policy workshop, University of Southhampton, 23–24 January 2001. Available from: http://r4d.dfid.gov.uk/PDF/Outputs/HSafeGoodbeyondresearch.pdf (Accessed 24 May 2016)

Avert International. (n.d.). HIV and AIDS in Nigeria. Available from: http://www.avert.org/aids-nigeria.htm?&lang=en_us&output=json (Accessed 24 May 2016).

Buxton, M., & Hanney, S. (1996). How can payback from health research be assessed? Journal of Health Services Research and Policy , 1 (1):35–43.

Campbell, C., & MacPhail, C. (2002). Peer education, gender and the development of critical consciousness: Participatory HIV prevention by South African youth. Social Science and Medicine , 55 (2):331–345.

Dlamini, S.N., Okoro, F., Oni Ekhosuehi, U., Esiet, A., Lowik, A.J. & Metcalfe, K. (2012). Empowering teachers to change youth practices: Evaluating teacher delivery and responses to the FLHE program in Edo State, Nigeria. African Journal of Reproductive Health , 16 (2):87–102.

Ezinwa B.A. (2010). Barriers and facilitators to research utilization as perceived by business educators in tertiary institutions in Nigeria. African Journals Online , 18 (2). Available from: http://www.ajol.info/index.php/ifep/article/view/56644/ (Accessed 24 May 2016).

Funk, S.G., Tornquist, E.M., & Champagne, M.T. (1995). Barriers and facilitators of research utilization: An integrative review. Nursing Clinics of North America , 30 (3):395–407.

Glacken, M. (2002). Research and development in a Northern Ireland Trust. Nursing Standard , 16 (32):33–37.

Hanney, S., Packwood, T., & Buxton, M. (2000). Evaluating the benefits from health research and development centres: A categorization, a model, and examples of application. Evaluation: The International Journal of Theory, Research and Practice , 6 (2):137–160.

Hanney, S.R., Gonzales-Block, M.A., Buxton, M.J., & Kogan, M. (2003). The utilisation of health research in policy-making: Concepts, examples and methods of assessment. Health Research Policy and Systems, 1:2.

Hatcher, S., & Tranmer, J. (1997). A survey of variables related to research utilization in nursing practice in the acute care setting. Canadian Journal of Nursing Administration , 10 (3):31–53.

Hennink, M., & Stephenson, R. (2004). Using research to inform health policy: Barriers and strategies in developing countries. Opportunities and Choices Working Paper No. 9. Southampton: University of Southampton.

HP4RY. (2012a). Documentation of community intervention: A component of HIV prevention for rural youth, Nigeria . Canada: University of Windsor.

HP4RY. (2012b). Family life and HIV education, Edo State: Implementation report . Canada: University of Windsor.

Hyder, A.A., Akhter, T., & Qayyum, A. (2003). Capacity development for health research in Pakistan: The effects of doctoral training. Health Policy and Planning , 18 (3):338–343.

Ikpang, S. (2010). What the Church Can Do for Persons Living with HIV/AIDS . Raleigh, NC: Lulu.

Lansang, M., & Dennis, R. (2004). Building health research capacity in the developing world. Bulletin of the World Health Organization , 82 (10):764–770.

Lansang, M., & Dennis, R. (2006). The need to develop research capacity. Working document for the 10/90 Report on Health Research 2006. Geneva: Global Forum for Health Research.

Mahmood, S., Hort, K., Ahmed, S., Salam, M., & Cravioto, A. (2011). Strategies for capacity building for health research in Bangladesh: Role of core funding and a common monitoring and evaluation framework. Health Research Policy and Systems , 9:31.

Martin, A., Foreit, J.R., Ezcurra, E., & Vernon, R. (2002). Workshop on utilisation of research results, San Jose, Costa Rica, June 4–7, 2002: Report. LAC Regional Office: Population Council. Available from: http://www.popcouncil.org/pdfs/frontiers/Capacity_Bldg/CostaRicaUtiliWpshp.pdf (Not available).

Maticka-Tyndale, E., Onokerhoraye, A.G., Anucha, U., Arnold, R., Dlamini, S.N., Eghafona, K., & Oni Ekhosuehi, U. (2007). HIV prevention for rural youth: Mobilizing Nigerian schools and communities . Proposal to Teasdale-Corti team competition under the Global Health Research Initiative, International Research Development Centre, Ottawa.

Maticka-Tyndale, E., & HP4RY Team. (2012). Bridging theory and practice in HIV prevention for rural youth, Nigeria. African Journal of Reproductive Health , 16 (2):39–53.

Milèn, A. (2001). What do we know about capacity building? An overview of existing knowledge and good practice . Geneva, Switzerland: World Health Organization.

Nchinda, T.C. (2002). Research capacity strengthening in the South. Social Science and Medicine , 54 (11):1699–1711.

Omorodion, F., Akpede, E., Maticka-Tyndale, E., Agbontean-Eghafona, K., & Onokerhoraye, A. (2012). The use of National Youth Service Corp members to build AIDS competent communities in rural Edo State, Nigeria. African Journal of Reproductive Health , 16 (2):71–85.

Onokerhoraye, A.G., Maticka-Tyndale, E., & HP4RY Team. (2012a). HIV prevention for rural youth in Nigeria: Background overview. African Journal of Reproductive Health , 16 (2):25–38.

Onokerhoraye, A.G., Maticka-Tyndale, E., & HP4RY Team. (2012b). Meeting the challenges of North–South collaboration: The case of HIV prevention for rural youth, Edo State, Nigeria. African Journal of Reproductive Health , 16 (2):127–146.

Parahoo, K. (2000). Barriers to, and facilitators of, research utilization among nurses in Northern Ireland. Journal of Advanced Nursing , 31 (1):89–98.

Porter, R.W., & Prysor-Jones, S. (1997). (July). Making a difference to policies and programmes: A guide for researchers . Washington, DC: Support for Analysis and Research in Africa (SARA) Project, Academy for Educational Development.

Simon, J. (2000). Commentary: Does strengthening research capacity improve health equity? British Medical Journal , 321 (7264):816–817.

Stephenson, R., & Hennink, M. (2002). Moving beyond research to inform policy: Barriers and strategies in developing countries . Southampton: University of Southampton.

Tan, C.W. (2008). Staff Nurses’ Attitude and Perception Towards Nursing Research . Sarawak, Malaysia: Faculty of Medicine and Health Sciences, University of Malaysia Sarawak.

Walt, G. (1994a). Health policy: An introduction to process and power . London: Zed Books.

Walt, G. (1994b). How far does research influence policy ? European Journal of Public Health , 4 (4):233–235.

Weiss, C. (1977). Introduction. In C. Weiss (Ed.), Using social research in public policymaking (pp. 1–22). Lexington, Ky: Lexington.

Weiss, C. (1979). The many meanings of research utilization. Public Administration Review , 39 (5):426–431.

White, F. (2002). Capacity-building for health research in developing countries: A manager’s approach. Pan American Journal of Public Health , 12 (3):165–172.

WHARC (Women’s Health and Action Research Center) (nd). Profile of sexual and reproductive health of young adults and adolescents in Edo State of Nigeria. Benin City, Nigeria: WHARC.

Chapter 10 Indicators for Measuring and Evaluating Research Capacity Building

Dan Kaseje, Nancy Edwards & Natasha Mortley

In order to assess the progress towards achieving the objectives of an initiative certain indicators are required. These indicators provide the criteria against which the successes or achievements are measured and serve as a means to monitor the completion of interdependent activities. This chapter examines the indicators used by the Teasdale-Corti project teams to assess research capacity building and provides recommendations for the development and future application of research capacity-building indicators.

Although there is extensive literature on the use of indicators for programme evaluation, there are two important caveats of relevance to this case study. In the first instance, although indicators are often used for reporting and accountability purposes, based on contractual relationships between partners, they have not been adequately used for learning, strategic planning, or enhancing performance and decision making (Whyte, 2004). Thus, the inherent experimentation and learning-by-doing that characterises some capacity-building efforts have not been adequately captured or informed by indicators. Secondly, there are few reports of indicators that have been used to assess research capacity building in LMICs (Airhihenbuwa et al, 2011; Bates et al, 2011; Gadsby, 2011). Bates et al (2011) argued that in spite of a wealth of literature on the theory of evaluating capacity building ‘there is very little published evidence about how to monitor its effectiveness in practice’ (p. 1).

A number of authors have called for case studies to fill these gaps. Morgan (1997), for instance, looked at the issue of indicators from the perspective of funding agencies, noting the need for more case studies that would yield indicators to measure capacity-building results effectively. Some pertinent case studies have now been published. Gadsby (2011) provides an in-depth description of evaluation processes being implemented by six international funding agencies to improve the measurement of capacity building. Current initiatives include developing systems for long-term career tracking, and examining the use of research and the impact of research on policy and practice. Similar indicators are described by Mahmood, Holt, Ahmed, Salam and Cravioto (2011), who documented the challenges of sustainable capacity-building efforts faced by the International Centre for Diarrhoeal Disease Research in Bangladesh, given their many international partners and funding agreements. They suggest the need for indicators to assess ‘the ability and autonomy of the research institute to determine its own strategic growth and priorities’ (p. 8), and to make investments in line with these priorities, reporting on results using a monitoring and evaluation framework that is mutually agreed upon. In a case study of mental health policy development initiatives in four sub-Saharan African countries undertaken by the UK Department for International Development-funded Mental Health and Poverty Project, Thornicroft, Cooper, Van Bortel, Kakuma and Lund (2012) call for indicators to address the transitional phases of capacity building. For instance, they describe the importance of documenting a shift in roles of LMICs and higher-income partners with an increase in the number of LMIC first-authored publications and grant applications with an LMIC principal investigator. Similarly, in an analysis of four African case studies, Bates et al (2011) propose the need for indicators that are consistent with three strategies for sustainable capacity building, including the use of a phased approach, strengthening processes within existing rather than parallel systems and developing partnerships for local ownership and problem solving.

Other authors have suggested indicators that are more process oriented. Horton et al’s (2003) collection of evaluation studies from six LMIC provides managerial perspectives on organisational capacity development and its evaluation. They highlight some key principles to underpin the assessment of capacity development initiatives, including the following:

•   using assessment processes that provide a learning and growth process for those being evaluated

•   employing participatory approaches that are useful — first and foremost to those being evaluated

•   being sensitive to the context while recognising that it is often changing

•   systematically documenting assessment approaches to ensure integrity and transparency.

Several authors of case studies have discussed the importance of process indicators to capture building trust and strong relationships, being open to differences, being transparent and having the kind of dialogue needed to resolve differences (Airhihenbuwa et al, 2011; Mahmood et al, 2011; Thornicroft et al, 2012).

There is emerging consensus that research capacity building must be understood through a complex lens of individual, organisational and structural changes (Cooke, 2005; ESSENCE on Health Research, 2011; Gadsby, 2011; Morgan, 1997) and better matched with the stage of development of projects, organisations and wider systems (Bates et al, 2011; Frieden & Koplan, 2010; Morgan, 1997). Furthermore, authors have noted the importance of recognising socio-structural influences such as colonisation (Costello & Zumla, 2000) and legal frameworks (Hanlin, Chataway, & Smith, 2007) on approaches to building research capacity.

Research questions and design

The purpose of this study was to describe indicators that were purposefully used to assess research capacity building and to identify indicators that emerged from the experiences of building research capacity among the Teasdale-Corti teams. The project was not undertaken to evaluate the Teasdale-Corti programme. The intention was rather to enhance understanding by describing how research capacity-building efforts can be assessed.

The research took the form of a composite case study, with each funded Teasdale-Corti project considered a case, limited by its defined scope and objectives, team members, partners, geographic location and participating institutions. Mixed methods data collection involved data extraction and qualitative interviews.

Data extraction

Data on research capacity-building indicators were extracted from the original study protocols and from annual and final reports submitted to funders by teams. These documents were made available to us by the secretariat for the funders, once they had been given permission to use them from the research team who wrote the report.

For the original protocols, any indicators, specifically described as such by the original teams were identified. Indicators for the initial set of annual reports were also identified. However, we were also interested in emerging indicators. Thus, for both annual reports provided in the last couple of years of the projects and for final reports, we extracted data that seemed pertinent to capacity building, whether or not it was identified as an indicator by the reporting team.

Qualitative interviews

Qualitative interviews were guided by several domains of inquiry about research capacity building within each team’s project experience:

1)   approaches used and lessons learned

2)   successes and challenges

3)   benefits and beneficiaries

4)   monitoring and evaluating capacity building.

In the final year of the projects, a research assistant invited the Canadian and LMIC co-principal investigators from each project team to participate in a face-to-face or telephone interview at a time convenient for the interviewee. These co-principal investigators were also asked to inform eligible members of their team and its project beneficiaries of the option to participate in an interview for this case study. Beneficiaries included student researchers, academics, community members, government managers, decision-makers and policy-makers. Eligibility criteria for team members and project beneficiaries were as follows:

1)   directly involved in capacity-building and/or institutional-strengthening activities of the project

2)   speak and understand English, French or Spanish

3)   18 years and older.

We asked the co-principal investigators to identify individuals whom they felt were familiar with the different facets of capacity-building activities that were undertaken as part of their project and who may have divergent viewpoints about those activities. If the teams worked in more than one country, we requested that they identify team members or beneficiaries at different project sites. We aimed to recruit four to five participants per project team. At least half of the interviewees from each project had to be from LMICs. Co-principal investigators of each case study team were then asked to send an e-mail and/or to phone these individuals to ask if the case study project team could contact them. For those who agreed to be contacted, an initial invitation and two reminders were sent by the case study project coordinator. The study was explained to the potential participant and consent for the interview obtained.

Semi-structured interviews were then conducted in person or by phone and in the participant’s language of choice (English, French or Spanish). With permission, interviews were digitally recorded and transcribed. Identifying data such as names of individuals and institutions were removed from transcripts. Quotes were further anonymised by removing references to specific countries so that project teams would not be identifiable.

Ethics approval was obtained from the Research Ethics Board at the University of Ottawa, Canada; the Ethics Review Committee at the Great Lakes University of Kisumu, Kenya, and the UWI Ethics Committee University of the West Indies, Mona Campus, Jamaica.

Analysis of extracted protocol and report data

Protocols and reports were read in their entirety to identify descriptions of indicators. We defined applied indicators as measures used purposefully by teams to monitor and evaluate the research capacity-building activities and initiatives within their project. We defined emergent indicators as observed or expected changes in research capacity. For both sets of indicators, we identified research capacity building occurring at individual, team, organisational and network levels. Pertinent text about indicators was extracted and recorded in an Excel spreadsheet. To ensure that we identified emergent indicators, all information contained in reports was considered potential indicator data. In other words, text did not have to be labeled as indicator data in a report for us to include it in our extraction process. Indicator data were sorted by study time period and indicator type (qualitative or quantitative, and structure, process or outcome). They were also organised according to the modified categories in Cooke’s framework outlined in Chapter 2 .

Analysis of interviews

Interviews were transcribed. Spanish interviews were translated into English. The interview schedule and Cooke’s modified framework provided the initial coding structure for content analysis. This level of analysis not only categorised indicators but also yielded relevant context influences on the choice of indicators. Interview transcripts were then read and reread by two team members (NM and NE), using a constant comparative approach to identify emerging indicator themes. Matrices were used to compare responses across project teams using data from interviews and reports sorted by elements of the framework and when possible by level of indicator (structure, process and outcome).

Thirteen of the 14 teams gave consent for the review of their technical reports, while 12 teams consented to interviews. A summary of technical reports reviewed and interviews conducted is provided below. In total, we extracted data from 13 original protocols, 47 annual reports and 7 final reports. Final reports for 5 of the teams were either not available or not released to us. Thirty-six interviews were completed during the final year of the projects with a range of 1 to 5 members from 12 teams agreeing to be interviewed. The roles of team members who were interviewed included researcher, research manager or research assistant, graduate students and collaborator/decision-maker.

Table 10.1 Reports reviewed and interviews completed for the case study

Note: To maintain anonymity, the order of teams in this table does not correspond with the order of teams shown in Appendix A .

All teams were multidisciplinary and worked with multiple stakeholders; over half of the teams worked across multiple country settings.

Descriptions of indicators

The reports were uneven in detail and in the type of indicator content captured. Some teams opted for an external evaluation process; these reports were not accessible for our evaluation. In other cases, teams set out a rather extensive set of indicators as part of their project application and reported on these annually. More commonly, the teams were planning to develop a more formalised set of indicators for evaluation purposes as their project progressed.

Indicators in the reports were described in a number of ways. All teams listed some quantitative indicators with supporting data (eg, number of trainees and students supported, dissemination events, publications, library holdings on a particular topic, lab tests conducted in a region, annual review meetings held, etc). Qualitative indicators were also provided, although not necessarily identified by the teams as indicators, for example, descriptions of the dynamic involvement of researchers and research users, contextually based approaches to working in their setting(s) (eg, working in war-torn countries), or the experience of having successfully engaged ministry partners.

Indicators that were most prominent included those pertinent to training and mentoring. Indicators also focused on bridging the gap between research to policy and practice. All of these indicators reflected stipulated requirements outlined in the funding opportunity. During the interviews, some individuals indicated that they used traditional quantitative indicators for accountability purposes, but were of the view that true capacity-building success should be assessed through the individual experiences and narratives of their team members.

Interviews provided data on context, and explanations about the nature of the work, why deviations from original capacity-building plans took place and how priorities were shaped. The participants described process elements of capacity-building experiences that were not necessarily self-evident from the written reports. For instance, several teams described relationships among some team members (including former relationships), or the level of trust among the team members as factors that affected capacity-building work. In some cases, interviewees spoke about contentious issues their teams had grappled with and approaches used to address and manage conflict. Other team characteristics, such as differing language abilities or ethnicity of team members, which influenced these relational processes, were also described.

Although each team described indicators, the underlying logic model, theory of change and causal linkages among indicators received little attention in project reports. Teams were also hesitant in describing attribution. As one interviewee stated: ‘ I struggle with how … to measure social impacts … I think we have had significant influence but this has been happening within a larger context where there are other global national forces … I’m not really sure how to do a proportional attribution in the evaluation .’

Classifications of indicators by framework dimension and system level

We attempted to classify indicators described in the annual and final technical reports according to their level (individual, team, organisation or institution, and network) and Cooke’s framework dimensions (eg, infrastructure, skills and confidence, research application to policy and practice). Overall, most of these indicators focused on individual, team and organisational level achievements since the majority of capacity-building activities occurred at these two structural levels.

In the next section, indicators are presented using the dimensions of Cooke’s modified framework as the organising structure.

Infrastructure and resources

Almost all teams reported indicators assessing improvements in infrastructure, technology, research structures and/or mechanisms for ensuring quality. A number of teams reported the acquisition of office space, and/or the purchase of laboratory equipment and supplies or information technology and software for local, national or regional use. For instance, one team indicated that ‘ we have purchased lab equipment and lab standards and safety protocols [have been] established ’. Another team acquired office space, computers and data analysis technology. Open access to online libraries and databases was obtained by some teams for the first time. Most teams reported indicators assessing improvements in standard operating procedures, safety protocols, ethics guidelines and/or codes of conduct. In later reports, technological improvements were described as leading to increased numbers of well-functioning facilities, and facilitating high standards of research as well as research training. Some reported increased capacity for training other students using electronic means to access training materials, share information, network and search database archives. While these indicators reflect acquisitions at the level of the organisation, benefits to individuals and teams and in some cases to a wider network of collaborators were often highlighted and their longer term, post-project impact anticipated.

Enabling environments for research were described as creating an atmosphere conducive to research work, improving technical assistance and providing mechanisms to facilitate LMIC-led and joint research with Canadian and other international partners. Various processes and structures were set up by teams to support research and capacity-building activities. These included research triads, leadership hubs, working groups and annual review meetings.

The establishment of relevant degree programmes or other accredited training programmes to produce more graduates with research skills was mentioned by a few teams. A research leader from the South described how a ‘ master’s programme sets a platform for collaboration with other institutions and countries ’. With this in mind, in some instances, the infrastructure established, supported networking functions both regionally and internationally.

Some teams included indicators of increased ability to mobilise financial support for research or capacity building such as fellowships, scholarships and grants. A number of teams gave examples of additional sources of funding (local and international) that they had received, building on their success as a team. Within southern academic institutions, there were some examples of local resource investments in new areas of research that had been led by the Teasdale-Corti teams, including the establishment of research posts and organisational arrangements to offer protected research time for academics. Indicators in this subcategory were described as being indicative of administrators’ perceptions of the relevance and quality of research being led by project teams.

Linkages, partnerships, collaborations: Engaging stakeholders

Indicators under this dimension measured capacity building mainly at institutional and network levels. Reports included descriptions of the number, nature and/or quality of partnerships forged between Canadian universities and LMIC institutions and in some cases among LMIC institutions. The benefits of these partnerships included increased access to internationally recognised academics and experts, and better recognition by other research groups in Canada or internationally. The ways in which these partnerships extended the networks and broadened the perspective of both young and more senior researchers were identified as important outputs. In some cases, data on the number of trainees, researchers and research assistants benefitting from these partnerships were provided. Indicators to measure the multiplicity of individuals (eg, disciplines) and groups (eg, agencies, civil society) engaged were reported by several teams.

The teams emphasised how their projects had extended networks beyond traditional boundaries. Diverse, formal and informal partnerships with advisors, key stakeholders and research users were also described. These were often identified as essential for knowledge translation and exchange activities. As one researcher said, ‘ We have links, partnerships and collaborations with a wide range of users including NGOs, government, academia and civil society ’. Another team described the importance of state actors who were responsible for implementing legal frameworks, and local worker unions who were engaged in this process. Some teams described formal partnership agreements such as those negotiated with government officials or local academic institutions.

Interviewees described creating, building and linking to existing networks. Some of these were international and provided ready access to an existing set of linkages that teams could leverage. ‘ I mean it’s like a whole network; part of the thread that runs between the countries … You got an opportunity to link up with [senior researchers who] are well respected in the academic field and well connected. So you got … introduced to networks that you would not have had access to if you were just doing research at the local level .’

The teams themselves also provided network opportunities among both Canadian and LMIC partners and LMIC team members. As one of the multi-country team members described: ‘ It was an eye opener that people from a health service in one country had a chance to meet people in another continent that had similar issues to deal with ’.

The role of stakeholders in identifying training priorities was described by a number of teams. For example, one team noted requests for courses that arose from participating countries: ‘ We have given [numerous] international courses because the countries themselves started asking the … principal investigators to give specific courses to the countries. For example, in [country A] they wanted a workshop on how to write specific papers, in [country B], … they wanted a course on qualitative methods ’.

There were important underlying processes identified as critical to this domain. Linkages, collaborations and partnerships involved maintaining good working relationships, and establishing mutual trust and respect with partners and team members. One team member stated that ‘ there has been a lot of negotiating among the various interests and the various power dynamics trying to get people to work together towards these capacity outcomes ’. Mechanisms for coping with power, authority and control in the partnerships were reported as indicators. Involving external partners as key players in a research project, and facilitating ongoing dialogue and continued involvement among key players were identified as core processes for success.

Skills and confidence building

The teams had to describe all of the training formats that were used including courses, seminars and workshops. The number of training activities, their purpose, and the type of training materials prepared were detailed in these project reports. Novel features of the training approaches were often highlighted, such as e-learning, mentorship pairing, participatory training, experiential learning and reciprocal learning between Canadian and LMIC partners. The number and type of trainees (eg, graduate students, junior and senior researchers, academic researchers, clinicians, lab technicians and other service providers, and decision-makers) and their roles in training initiatives were also often described. The annual reports provided a snapshot of training activities, but in most cases did not yield a sense of the incremental trajectory of training activities and approaches. However, the latter was more apparent in final reports and also described in some interviews. For instance, one team began with an initial step of reworking a concept paper into a research protocol. ‘ The second year of training focused on preliminary results and thinking about knowledge transfer… the next year, we looked more on what had happened — publication and policy briefs and interviews by the media … It was quite an intensive process of following people on as they did research .’ Others pointed to characteristics of the training approach that appeared promising. These included training where time was not a factor, the commitment of senior mentors during and beyond a classroom training period, and the replication of training by trainees: ‘ I think that longer term placements are very useful … and to work on a project in a face-to-face environment really benefits from senior mentors for a period of time rather than a week or two days of training, then those people become trainers and go back to … where they are working and train people there ’.

The types and frequency of training and learning opportunities that were offered, and the number and characteristics of trainees were described by all teams. Common indicators in the reports included the number of researchers trained, the number of graduate students supported through bursaries and research grants, and the number of individuals involved as mentors or mentees. The teams described the reach of their programmes using numerator data (number of trainees, number of institutions involved), sometimes providing this data in relation to the original targets set for the programme, but rarely in relation to a denominator of potential trainees. The characteristics of trainees most frequently described included sex, country and role or stage of training (eg, graduate student, decision-maker). With few exceptions, only indirect measures of competencies acquired were documented in the reports. The most commonly described indicators of skill and confidence acquisition were in the domain of research outputs. These included conference presentations, peer-reviewed publications, completion of graduate training, submission and/or funding of research proposals, and number of trainees taking on the role of mentors as training programmes were replicated. Other types of research outputs, such as newsletters and policy briefs, were mentioned; these were consistent with the dissemination and knowledge translation objectives stipulated in the call for funding.

Various teams highlighted the relevance and responsiveness of their training and mentoring programmes to local contexts. Relevance concerned the extent to which the training materials, activities and interventions were geared towards local research problems, and contributed to solving health system problems, while responsiveness referred to how training initially fostered the researchers’ growth and professional maturity, and then in turn helped address health concerns of the target groups. There was some description of the team processes used to assess needs and gaps, to come to a better mutual understanding of issues like country needs and opportunities, and to gauge relevance. These included formal needs assessments, outcome mapping and reflective self-assessment processes. The teams addressed the challenges of and ongoing demands for training, noting the diversity of the trainees and contexts. One team said ‘ Capacity building cannot be a one-event activity; it has to be a continuous process and it needs a lot of resources .’ This team also described the different languages and cultural backgrounds they had to take into account: ‘ We have tried to be fair to different communities. We have been conducting training in the north and east, in Muslim speaking and Hindu cultures .’

Learning approaches described included the following:

•   informal training in the field or in the workplace setting, with mentors gradually stepping back as mentees began to apply their learning and skills

•   mentorship triads in which junior and senior researchers were paired with decision-makers

•   apprenticeship-style learning wherein team members were supported to build competencies and take on positions with more responsibility over time.

The teams described how these approaches improved competencies and confidence as illustrated by the following quotes. ‘ I believe that [the apprenticeship] process was successful because some people who started off as research assistants are now [formal research] collaborators .’ One team explained how ‘ learning from senior team members about diplomacy, in dealing with research partners, ethics and project management ’ helped advance skills, while another described the benefits of having ‘ subteams specifically set up to provide support to PhD students ’. When discussing mentorship, one team indicated that ‘ the youngest of all teams’ members had been able to interact with some of the expatriates at the global level ’. This interviewee went on to describe the working groups and structures for this project that also built capacity: ‘ We have project teams, data analysis teams … and manuscript writing teams. So working in those teams has been extremely useful ’.

Career advancements were documented in reports and discussed by some teams. These included advancements within the structure of a project (eg, research assistants becoming project collaborators or co-investigators; mentees becoming mentors) or within their employing institution (eg, promotion to a higher academic rank). In some cases, this was identified as a deliberate strategy, in other cases it was an outcome that was monitored and a hoped-for result.

Dissemination and knowledge translation, research applicability

Indicators for the framework dimension on research applicability were largely indistinguishable from indicators on appropriate dissemination and knowledge translation. Consequently, these two dimensions of the framework were merged. In part, this overlap may reflect the overall intention of the dissemination strategies described in the reports, which was to bridge research and practice, programmes and/or policy.

Indicators in this domain encompassed all four levels of interest. However, the most frequently reported indicators pertained to the type and frequency of tools used and their modes of dissemination and knowledge translation. There were three types of dissemination and knowledge translation strategies reflected in the indicators:

1)   passive and supply-driven dissemination, which often involved the one-way preparation and transfer of written information by researchers to decision-makers or other researchers (eg, research reports)

2)   active engagement, which included a mix of ‘push’ and ‘pull’ strategies involving researchers and decisions makers in both the development of dissemination tools (eg, interactive seminars)

3)   responsive and demand-driven approaches, in which decision-makers appeared to be driving requests to generate research findings (eg, a request for researchers to provide research evidence to a technical advisory committee).

Many teams highlighted the objectives of their knowledge translation approaches and gave emphasis, in their narrative descriptions, to those aimed at the active engagement of decision-makers or other stakeholders. They emphasised how these knowledge translation strategies were interactive and in some cases co-designed by researchers and decision-makers. Related outputs were discussed, as illustrated by the following: ‘ Research users have a much stronger understanding of the strengths and limitations of research and are much more competent now in terms of their use of research in their work and the researchers, in turn , have a much better understanding of the policy or programmatic context in which research is only one of many influences leading to decisions after ’.

In their reports, the teams provided descriptions of modes of dissemination and their intent. For example, one team described meetings held in schools and communities (with feedback and reflections from the audience) and other community activities that aimed to mobilise knowledge learned from research while empowering teachers and students to continue this work in their communities. Another team described how service providers shaped the content of a newsletter to update their managers regularly about their participation in the project and research findings.

The reach of dissemination strategies was described in myriad ways. References were made to targeted efforts to reach certain groups (eg, policy-makers, health workers, vulnerable populations), with dissemination tools and modes of delivery tailored to these audiences. Those to be reached were described in terms of disciplinary and gender mix, geography, social strata and sectors. While numbers reached were reported by some teams, a quantitative estimate of the proportion of a target group reached was rarely provided.

In the early phases of their projects, many teams adopted strategies to sensitise stakeholders on the objectives of and rationale for their research programmes. This took many forms, including one-on-one meetings and inviting stakeholders to join project advisory groups. In some cases, these activities were intended to mobilise the involvement of these actors in the programme of research. ‘ Having created enthusiasm for [the research topic] at the government level, the project was then able to garner national investment in this area of research ’.

In later phases, the teams described the use of policy briefs, communiqués, media interviews and interactive workshops and conferences to share results. Some teams created spaces for dialogue between researchers and decision-makers. In some cases, this involved leveraging relationships that researchers had within decision-making and policy-making organisations. For instance, in one project, a co-investigator was appointed to the Ministry of Health as state coordinator of a programme relevant to the research project. In another project, a co-investigator became advisor to the Ministry of Health and Ministry of Education. Both appointments provided opportunities to discuss research findings with policy-makers. While none of the teams explicitly detailed the ratio of effort put into different modes of dissemination or described how they chose the balance of strategies used, most teams appeared to use a combination of dissemination strategies for decision-makers. There seemed to be streams of both opportunistic and more strategic knowledge translation approaches.

There were many descriptions of how knowledge translation enhanced awareness, built interest in the research or generated in-kind and, in some instances, financial support for research, but fewer descriptions of what specific outcomes (eg, policy-related decisions) had arisen. The outputs and outcomes from the dissemination and knowledge translation activities did reflect efforts to bridge the divide between knowledge and practice, and programmes and policy. Flexibility within the projects to respond to researcher user needs was described as another way to create conditions for knowledge translation. For instance, a research user noted the following: ‘ Results from laboratory studies directly benefit government and country. In [country A], the Ministry of Agriculture was most interested in studies of water and mercury; data are useful to them ’. This team invited research users to provide input on what they perceived as the most significant environmental exposures; questionnaires were adapted accordingly. Another team used project funds to support priority evaluation projects that were identified jointly by local decision-makers and service providers.

Empowerment, local governance and leadership

Two dimensions are combined in this discussion of indicators, because disentangling empowerment and leadership was difficult. The dimension of governance emerged in the teams’ descriptions of their projects and has been added here.

Of relevance to this dimension of the framework were the teams’ descriptions of ownership of the programme of research by LMIC research investigators and local partners in terms of design, administration, management and decision making. The critical areas of decision-making authority considered important by LMIC partners included being directly accountable for how project funds were spent, reviewing and (as required) renewing project directions, and hiring and managing research staff. For example, one team described: ‘ Local ownership of the project in that management administration and implementation of the master’s programme were in the hands of the [LMIC project] team . In this project, a consequence of this ownership was that it changed the approach from using experts from abroad to using local experts ’.

Local ownership was described as empowering LMIC researchers to design their own research, tailor questions to local needs and priorities, and participate in all aspects of research, including its structures of governance. ‘ At the end [of the project], ownership by the country teams may be one of the most important outcomes to ensure capacity building .’ Among engaged communities and service providers, participation and empowerment were described as increasing capacity to use evidence in the design of interventions, to develop and initiate a research agenda based on local issues, and to contribute their knowledge and skills to knowledge translation activities. Local leadership for research, within academic institutions, service delivery organisations and policy shops, was an underlying element of ownership.

The teams described governance and management structures that were set up for purposes of advising on or steering research directions, and capacity-building and knowledge translation priorities. Examples included local, national, regional and international technical, advisory or steering committees. Most of these appeared to be set up specifically for the purposes of the project, but in a few instances, they were established structures within the institutions and/or countries where the project was taking place. Some teams described how project-specific entities replicated local structures such as ministerial technical resource groups. These structures were described as supporting better training, higher quality and more credible research, research that was relevant to decision-making, and knowledge translation strategies with a broader reach.

Functions of these committees were reflected in indicators about project ownership. For example, some interviewees described how the governance structure allowed local teams to make decisions about and control project funds, ‘ taking ownership of research in some area ’. Local engagement of stakeholders was described as a related function. As one team noted, ‘ Another benefit of local ownership was the involvement of local and regional decisions makers; thereby increasing investigators’ professional networks ’. Similarly another team reflected that ‘ members of the research team in [LMIC country] all became decision-makers in [Ministry X] [and this] was of course critical for capacity building ’.

When describing governance structures set up with the aims of building local ownership, the uneven readiness of different groups to take this on was discussed. One team described having ‘ worked very hard to improve the relationship between the universities in [country X] and the government. … We tried to negotiate our way between them by developing a project steering committee that has both groups involved [while] over the years, slowly trying to disentangle Canadian management and having more of the local ownership of the governance of the projects ’.

Some indicators of relevance to this dimension have been partially captured in the sections above. Examples include the funding received by LMIC team members for new research projects; infrastructure that would be used for future research; the continuation of staff who were able to undertake quality research and attract research funding; and the continuation of formal and informal collaborations, partnerships and networks to support research training and research production and use. Formal government and international accreditation and recognition of training programmes were presented as evidence of sustainability, as were institutional scientific, governance and management structures set up for these programmes.

Indicators to assess sustainability and continuity were indirectly described when teams made reference to their aspirations for longer-term impacts or noted plans that were being put in place to try and sustain project activities or build on project outcomes. These included descriptions of how the teams planned to retain trained researchers and service providers; adapt or integrate structures to bridge research and programmes and practice; or continue to engage decision-makers in new research initiatives, and integrate new training methods and content within existing curricula.

Cross-cutting themes

While the dimensions outlined in Cooke’s modified framework were pertinent to these projects, three additional overarching themes emerged across these dimensions. The first theme concerned stakeholder engagement and commitment. The teams described the importance of harnessing interest among stakeholders, building a sense of community and strengthening commitment either to particular fields of research (eg, mental health, obesity prevention and primary healthcare) or to the engagement of certain disciplines in research (eg, veterinarians, nurses, social workers, lawyers and educators). This often involved a variety of stakeholders including those in academia, government, and professional and non-governmental organisations. It required inter-agency collaboration and setting up structures, committees, meetings and networking platforms that provided a means for different local, national and/or regional groups to come together. The aims of this profiling varied and included better quality training in doing or using research, expanding practice roles, or shifting funding investments. For example, one participant described ‘ a strong focus on elevating the status of [type of research] on the health agenda and within the research community … to garner national investments in this area of research ’.

The second theme was leveraging. This was described in relation to several of Cooke’s dimensions and included leveraging investments, efforts and opportunities. Leveraging was identified as contributing to a foundational aim of systems’ strengthening. Thus, the projects provided a fulcrum or leverage point for driving home the need for change. Leveraging was also described in relation to extending efforts and impact.

There were some differences in how the LMIC and Canadian partners described what had been leveraged. While all teams described leveraging international partnerships, the LMIC partners emphasised the important profile, connections and credibility they had been able to build in their own countries and regions, and spoke about a wider array of potential international funding partners they might be able to access through connections they had built with other LMIC institutions. Some opportunities for leveraging appeared to have arisen serendipitously, but many teams described proactive attempts to leverage opportunities that came along.

The third and final theme that emerged in a number of projects was team resilience and adaptation to contextual dynamism. There were numerous changes in the political and policy context within which the projects operated. These presented both challenges and opportunities. Coupled with the descriptions of a dynamic context were indicators of how the teams made adjustments, persevered and/or rode a tumultuous period to achieve their project ends.

This case study provided a rich harvest of indicators, both traditional and new, from the Teasdale-Corti teams. The indicators reported fit well with the modified Cooke framework in terms of dimensions and structural levels. However, based on the kinds of indicators that emerged, we propose that Cooke’s framework be adapted and further consolidated into the following six dimensions: infrastructure and resources; linkages, partnerships, collaborations and networks; skills and confidence building; research applicability and knowledge translation; empowerment, governance and leadership, and sustainability and continuity. In this section, we offer some reflections on the types of indicators used; describe their strengths, gaps and limitations; summarise limitations of the case study design, and outline recommendations for the use of indicators to assess research capacity building.

Reflections on indicators used

We classified the indicators by principles of research capacity building, based on the modified version of Cooke’s model outlined in Chapter 2 , and the four structural levels of Neilson and Lusthaus (2007): individual, team, organisational and network. Others have proposed classification at three levels: individual, organisation and research system (ESSENCE on Health Research, 2011), or individual, team and organisational levels (Holden, Pager, Golenko, & Ware, 2012), while Chadwich-Parkes (2005) classifies indicators at only two levels — individual and organisational, arguing that it is only these two levels that really matter.

Based on our findings, it is our view that the four structural levels are useful to delineate. During preliminary stages of analysis, we augmented this four-level classification by adding the research system level proposed in the ESSENCE Framework, but there was a gap in data for this structure that emerged in our analysis. Some teams made reference to levels of the research system with which they directly interacted in their universities, such as ethics committees, peer review processes and supports for grant writing. However, indicators of the larger research system — including the presence of in-country funding councils or other research funding sources for operating grants and personnel awards, and country-specific or regional directions for research priorities — were largely absent from the written documentation. One could argue that indicators at the country or regional research system level were beyond the scope of the teams. However, if efforts are not concurrently directed at building these structural supports for research, sustained research production will be compromised. This is particularly the case for newer fields of research that some teams were trying to profile and build. Furthermore, some teams were extending the number of disciplines engaged in their work. Sustained impact of such initiatives may also require changes at the research system level, with funding councils making explicit the eligibility of a wider range of disciplines and/or endorsing interdisciplinary approaches.

We found that the indicators described by the teams, both traditional and new, and of all types (structure, process, outcome, and qualitative and quantitative), were concentrated within the individual, team and organisational levels. This may reflect the relatively short time periods for the studies (four to five years), and team perceptions of what indicators were most relevant to their monitoring and evaluation functions. The teams may have been concerned that using a wider set of indicators, which would potentially link their project results to system-level outcomes, would unduly suggest attribution. The pattern of indicators described is consistent with observations that Gadsby (2011) makes in her review of how donors assess the impact of research capacity strengthening in LMICs. She noted that indicators ‘to capture changes in research capacity at the national (systems) level are less often included in the tracking conducted by donors’ (p. 100).

There were many commonalities in the types of indicators described across projects. This is somewhat surprising given the diverse foci of the projects and the wide mix of disciplines, institutions and countries involved. It may reflect the influence of international trends and norms regarding how to assess research productivity (eg, bibliometrics, institutional rating of academic institutions on research productivity) and common parameters for the teams that were set out in the initial request for proposals (eg, requirements for knowledge translation, working with decision-making partners, mentorship and building research capacity).

While the teams used some generic indicators and thus potentially could have provided inter-team descriptions of progress made, this was thwarted by two factors. First, teams were asked to report exclusively on their project and its achievements against project-specific goals. The summative reflections across teams were framed as the work of the GHRI secretariat. Second, the teams were carrying out their research in many different countries and on many different topics. While interactions among the teams were fostered through occasional joint meetings, thus providing opportunities for reflective learning on matters like research capacity building, gender analysis and ethics, the wide distribution of funding across geographic settings and topics did not lend itself to jointly reporting on and/or reflecting learning across teams and contexts. Had funding been concentrated on particular populations (eg, indigenous peoples) and/or in specific regions of the world, this might have generated concerted efforts to use common indicators pertinent to research system strengthening.

Authors have identified a number of challenges affecting sustainable research capacity development. For example, Segrott, Mclvor and Green (2006) describe material constraints, unsupportive organisational contexts and lack of funding, while Sadana, Lee-Martin and Lee (2006) identify poor wages and inattention to career nurturing as factors. Not surprisingly, these challenge areas were addressed unevenly by the Teasdale-Corti teams. With respect to organisation contexts, there was little reference to using or leveraging written commitments from academic and institutional boards or signed agreements other than what was outlined in the original protocols. Thus, it was difficult to ascertain whether these commitments had served their intended purpose or had been or would likely be ignored as staffing and leadership changes occurred.

The indicators which surfaced on partnerships and linkages are consistent with those of Joffres et al (2004). However, our case study results add the dimension of extending existing networks and creating interlocutions between emerging networks of researchers (in this case study, those within the Teasdale-Corti teams) and mature existing networks focused not only on research activities but also on functions such as advocacy. The importance of collaborative dimensions of partnerships, with different types of organisations joining up their strengths to improve research capacity building, was highlighted by the teams. This is a partnership attribute that has been emphasised by others (Crookes & Bradshaw, 2002; Fyffe & Hanley, 2002; Waisbord, 2006).

 Teams reported many indicators that are widely used to document skills and confidence building through graduate training, mentorship programmes and continuing education (Chan, Gardner, Webster, & Geary, 2010; Crisp, Swerissen, & Duckett, 2000; Deckelbaum, Ntambi, & Wolgemuth, 2011). The process indicators reported by the teams appeared to be more novel as compared to outcome indicators. Examples of new process indicators included within-team equity and reciprocal learning, purposeful staging of incremental learning and mentorship continuity. The outcome indicators tended to be those more commonly reported in the literature, such as the number of people trained, number of articles published and increased skills and confidence. Both indicator types attempted to measure improvements in the quantity and quality of research capacity as described by Farmer and Weston (2002). Since these projects were relatively short in duration, and the period of data collection did not extend beyond the project period, only a few indicators to measure sustained achievement surfaced.

Waisbord (2006) emphasises that when initiatives are donor-driven, building local ownership is extremely difficult. This author suggests that external actors should only catalyse the process(es) of achieving commonly defined goals of action in research capacity development. Along the same lines, LaFond and Brown (2003) stress that indicators should be designed to assess ownership of capacity-building processes and outcomes, and Gadsby (2011) summarises indicators in this area such as local control of the research agenda and of research funding. The catalytic process of capacity building was indirectly assessed by the teams. For instance, references were made to how particular approaches (eg, participatory or experiential training) spawned the involvement of others, how building research capacity sparked locally determined and locally driven evaluation projects, and how capacity-building efforts catalysed a paradigm shift in thinking about a particular area of research. These catalytic processes were often described as having been co-generated by both the LMIC and Canadian partners, although the distinctive and critical roles of team members in catalysing changes in their home institutions were also identified. Generally, indicators and their descriptions were not refined enough to parse out and characterise the nature of catalytic processes used by internal and external actors. There were some descriptions that attempted to disentangle the nature of project ownership by the Canadian and LMIC partners, but the teams tended to emphasise the joint and partnered aspects of their efforts.

Building on the principle of ownership, Waisbord (2006) insists that projects need to find ways to strengthen institutional commitment, working with ‘project champions’ to increase and solidify institutional support. There were some descriptions of organisational and system champions (beyond research team members), and where they were positioned within the system. However, for the most part, indicators did not capture whether these individuals were likely to continue in their champion role in the absence of project funds.

Gaps and limitations of indicators used

The vast majority of quantitative indicators involved basic counts (eg, the number of trainees, courses, decision-makers and organisations involved), reflecting an output orientation to capturing data. Few teams provided a denominator for the proposed reach or scale up of their capacity-building initiatives. Thus, while it was obvious that substantial reach had been achieved, and the characteristics of those involved were consistent with efforts to reach vulnerable populations or other disciplines, for example, the numbers could not be assessed against a population target and the residual capacity-building gap was not described.

Additionally, the use of robust measures to collect data for indicators was infrequently reported. For instance, few teams generated indicator data using validated measures or tools such as those available to assess relevant outputs such as social networks, team functioning, stakeholder engagement or research capacity at individual, team and organisational levels (Holden et al, 2012). While these types of measures may only have been pertinent to some teams, their application could be fostered in future evaluations. Common measurement tools for a subset of indicators would enhance opportunities for cumulative learning across teams. When possible, baseline assessments of indicators should be undertaken to provide pertinent comparators for project achievements.

 The network level, according to Neilson and Lusthaus (2007), is about creating a critical mass of local capacity to carry out research in a particular field. The notion of critical mass within institutions did not directly emerge from the cases. Rather, the wide reach of activities across a number of countries that was a feature of several teams pointed to possibilities for indicators assessing distributed and joined-up capacity. This is an important dimension that needs to be better developed.

Measuring and evaluating capacity-building activities and strategies does not come without its challenges. Some have argued that an important challenge stems from the lack of a standardised understanding of the term ‘research capacity building’. While this is a pertinent concern, we do not see it as a primary one. It was clear that each of the 13 projects had a strong capacity-building focus (as per funding requirements), but the priorities within projects were determined by a wide array of contextual and project factors. LaFond and Brown (2003) argued that indicators should be context specific. Thus a standardised definition of capacity building may not have brought any more consistency to the pool of indicators used across projects.

Generally, the indicators were presented in a way that did not reveal a causal chain or mechanisms of influence. In the final reports and in some interviews, project-specific mechanisms were described, but an a priori, coherent logic framework with harmonised indicators for the Teasdale-Corti programme was not provided to the teams. We believe this raises an important question for funders. Accountability frameworks emphasise measurements that document return on investment. While the underlying logic of a project (as reflected in logical framework analysis approaches) provides the programme theory that underpins causal linkages for accountability frameworks, this logic was not manifest in the presentation of indicator results in the project reports. Furthermore, comparative data using counterfactuals or baseline comparisons within the actual study sites were largely absent.

The complex nature of multi-level research capacity building, and the influence of many other contextual factors on these types of initiative, makes it difficult to describe the purported causal chain that underlies outcomes. This in turn makes for tentative conclusions about attribution, a concern that has been raised in an examination of evaluation frameworks used by funding councils to assess the impact of research capacity building (Boyd et al, 2013). Nevertheless, an attempt to report on indicators in ways that reflect the expected causal chain would be more informative than discrete lists of indicators.

Although the descriptions of project activities within and across the teams certainly reflected contextual relevance and specificity, this comparative case study suggests that many generic indicators can be adapted for use across diverse settings. Contextual specificity is especially important in the interpretation of indicators and this is where those who are engaged in project activities can offer insights. Boyd et al (2013) contend that contextual constraints are important to capture. Our findings suggest that a wider examination of context is necessary, since the teams described how context influenced capacity-building efforts and outcomes in a number of different ways. A range of constraints were indeed evident in descriptions of research capacity building among project teams. These included both anticipated and unanticipated events such as elections, natural disasters and healthcare restructuring. However, context also produced serendipitous opportunities that teams were able to harness, such as the assignment of decision-makers to new roles. Some teams indicated that context interacted with capacity-building strategies in synergistic ways.

Implications

A number of issues arose that warrant further attention. The time required for individual and systems change to occur and the extent to which these take place concurrently rather than in succession entails important considerations in terms of sequencing indicators. The utility of indicator data is affected by the challenge of capturing critical dimensions and processes of capacity building as they unfold. This entails both foreseeable and unforeseeable processes and may require just-in-time measures to capture critical but fleeting processes that create important pivot points in projects.

There was a general under-utilisation of established and well-validated measures for indicators. It would be useful for teams and funders to identify and adopt such indicators across projects. These types of measures might be most relevant in the domains of Cooke’s framework related to networking, leadership and governance structures. These would allow for benchmarking across projects.

Findings indicate that capacity-building interventions took place primarily within and across three of the four structural levels. In an extension of the earlier point made about the causal chain, vertical linkages across systems levels are important to capture and critical to provide a systems view of capacity building. Funding agencies themselves play an important role here as they may forge links among funded teams and develop purposefully aligned funding calls aimed at capacity building.

Limitations

For this case study, we included individuals who were described as being part of the Teasdale-Corti teams as well as key stakeholders engaged in the programmes of research. Had a wider set of individuals been interviewed, including research funders, other indicators of capacity development may have emerged.

Although we had technical reports across the years, it was difficult to capture the temporal nature of the project indicators. In part, this was because the funder changed the emphasis and expectations of what teams were to report from one year to the next. For instance, in the first year, reporting was largely administratively oriented with a lot of activity reporting. By the final year, teams were expected to emphasise key findings and provide examples of outcomes and impact.

There were also a number of indicators that the teams were asked to report at the end of the project, including publications and presentations. This likely drove the teams to emphasise some indicators over others in their reports. Importantly, the reports were not written to inform this case study. While the teams were aware of the case study and could decide whether or not to participate, there was no overarching reporting structure provided to teams to augment their regular reporting on capacity-building indicators. This also relates to the diversity of research funded under this initiative, which we consider to be both a strength and limitation of this case study. While this diversity likely enhances the external generalisability of findings, it made for some limitations in the depth of data in particular fields of research. Specifically, most teams were working in areas of health services and policy research, with fewer teams having either a biomedical or clinical orientation.

There were some difficulties in categorising the level of indicators. In some instances, a single indicator was considered a fit with more than one level of the framework. For instance, mentorship is a capacity-building activity which took place at individual, team, organisational and network levels. Thus an indicator such as ‘number of persons successfully mentored’ reflects more than a single level. Similarly, distinctions between structure, process and outcome indicators were not straightforward. This stems in part from the internal logic of a particular initiative; what is designated a process indicator in one project may be an outcome indicator in another, relative to the aims of that project. Thus, it was the overall pattern of indicators within and across projects that was considered.

There are some gaps in our data. Not all teams agreed to be interviewed and we were not able to access all of their final reports. The projects were conducted over four or five years; longer-term follow-up would be needed for a more in-depth examination of indicators related to succession and sustainability. Furthermore, the length of the projects drove the indicators selected by teams, given accompanying requirements for the reporting period.

Gadsby (2011) describes indicators capturing information on how much money is spent, on what and to what effects. Sadana et al (2006) identify transparency of the funding process as an important assessment area for supportive environments in research. Although comments about funding transparency did surface, we did not have access to the budgetary expenditure information for these projects. This limits our observations and reflections in this realm.

While shifts in the use and application of indicators across project years were evident, it was difficult to ascertain whether indicators had guided decision-making about changes to projects. Further examination of these indicators to determine their utility as markers of performance would have been useful, but this would have taken us into the realm of project evaluation, which was specifically outside the parameters of this composite case study.

Dimensions from the Cooke (2005) framework have been useful in contextualising the findings from this case study research and led us to elaborate a more comprehensive framework to monitor and evaluate the success of research capacity-building activities in different contexts. Developing an evaluation model that captures both processes and end results, while reflecting the uniqueness of research capacity-building elements within various fields of research and different system contexts, remains a challenge. Commonalities allow for broad-based applicability and comparability, while the combination of research fields and unique implementation contexts draw attention to areas where flexibility in applying the framework is necessary.

Using a common framework to guide selection of core indicators would enhance and accelerate development of a stronger body of knowledge about research capacity building across system levels. But the challenge of finding a common set of indicators remains and some authors (Gadsby, 2011) have questioned the utility of developing a definitive and comprehensive list of indicators for this purpose. While efforts to reach agreement on a framework and common indicators for research capacity building remain important, we conclude that there are four other emergent issues requiring discussion: what indicators should be used by LMIC and higher-income partners to examine complementary or divergent system changes within partnered countries? How can shifts in indicators (both expected and unexpected) be used to guide decision-making? What indicators are most aligned with system changes that yield improvements in health outcomes in the population? And finally, what indicators would provide the most useful benchmarks for research capacity building across teams, countries and donors? A coordinated and concerted approach is required to carry this discussion forward. International coalitions of research funding agencies, such as the Global Alliance for Chronic Diseases and ESSENCE, and international collaborations of research institutes such as Schools of Public Health are well poised to continue this work.

Using a composite case study and mixed methods data collection, this chapter describes indicators to assess research capacity building and identifies indicators that emerged from the experiences of research capacity building by 13 research teams funded through the GHRI.

Overall, most of the indicators focused on individual-, team- and organisational-level achievements, because this was where the majority of the capacity-building activities occurred.

Indicators for each dimension of Cooke’s framework were identified by the teams (see Chapter 2 for an overview of the framework). New process indicators reported included the following:

•   stakeholder engagement and commitment

•   leveraging investments, efforts and opportunities

•   team resilience and adaptation to contextual dynamism

•   within team equity and reciprocal learning

•   purposeful staging of incremental learning

•   mentorship continuity.

Several indicator gaps and limitations were identified. Funding agencies should consider these when stipulating reporting requirements:

•   Many quantitative indicators would have improved utility if denominator data were provided to assess the reach and scale-up of capacity-building initiatives.

•   Assessing critical mass for specific fields of research requires the use of common indicators that can be aggregated to describe distributed and joined-up capacity across teams, institutions and countries.

•   Funded teams should be encouraged to report on indicators in ways that reflect the underlying programme theory for capacity-building approaches.

Recommendations for funding agencies:

•   Agreement among funders to adopt well-validated measures for indicators across funded initiatives would support benchmarking and a deeper understanding of capacity-building initiatives that work. Priority indicators for this purpose include those targeting networking, leadership and governance structures, since indicators in these domains are largely absent from accountability frameworks.

•   Descriptions of context are important to document constraints and enablers, and the ways in which context interacts with capacity-building strategies.

•   Additional attention should be given to the following questions and a coordinated and concerted approach is required to carry this discussion forward:

•    What indicators should be used by LMIC and higher-income partners to examine complementary or divergent system changes within partnered countries?

•    What indicators of capacity building are most aligned with system changes that yield improvements in health outcomes in the population?

•    What indicators would provide the most useful benchmarks for research capacity building across teams, countries and donors?

Airhihenbuwa, C.O, Shisana, O., Zungu, N., BeLue, R., Makofani, D.M., Shefer, T., Smith, E., & Simbay, L. (2011). Research capacity building: A US–South African partnership. Global Health Promotion , 18 (2): 27–35. doi:10.1177/1757975911404745.

Bates, I., Taegtmeyer, M., Squire, S.B., Ansong, D., Nhlema-Simwaka, B., Baba, A., & Theobald, S. (2011). Indicators of sustainable capacity building for health research: Analysis of four African case studies. Health Research Policy and Systems , 9: 14. doi: 10.1186/1478-4505-9-14.

Boyd, A., Cole, D.C., Cho, D.B., Aslanyan, G., & Bates, I. (2013). Frameworks for evaluating health research capacity strengthening: A qualitative study. Health Research Policy and Systems , 11: 46. doi: 10.1186/1478-4505-11-46.

Chadwich-Parkes, S.J. (2005). Capacity building in community based organizations. Caribbean Journal of Social Work , 4, 39–57.

Chan, R., Gardener, G, Webster, J., & Geary, A. (2010). Building research capacity in the nursing workforce: The design and evaluation of the nurse researcher role. Australian Journal of Advanced Nursing , 27 (4): 62–69.

Cooke, J. (2005). A framework to evaluate research capacity building in health care. BMC Family Practice , 6: 44. doi: 10.1186/1471-2296-6-44.

Costello, A., & Zumla, A. (2000). Moving to research partnerships in developing countries. BMJ , 321 (7264): 827–829.

Crisp, B., Swerissen, H., & Duckett, S. (2000). Four approaches to capacity building in health: Consequences for measurement and accountability. Health Promotion International , 15 (2): 99–107.

Crookes, P., & Bradshaw, P. (2002). Developing scholarship in nursing-steps within a strategy. Journal of Nursing Management , 10: 177–181.

Deckelbaum, R., Ntambi, J., & Wolgemuth, D. (2011). Basic science research and education: A priority for training and capacity building in developing countries. Infectious Disease Clinics of North America , 25: 669–676. doi: 10.1016/j.idc.2011.05.009.

ESSENCE on Health Research. (2011). Planning, monitoring and evaluation: Framework forcapacity strengthening in health research. ESSENCE Good Practice Document Series . Retrieved from http://whqlibdoc.who.int/hq/2011/tdr_essence_11.1_eng.pdf .

Farmer, E., & Weston, K. (2002). A conceptual model for capacity building in Australian primary health care research. Australian Family Physician , 21 (12): 1139–1142.

Frieden, T.R., & Koplan, J.P. (2010). Stronger national public health institutes for global health. The Lancet , 376 (9754): 1721–1722.

Fyffe, T., & Hanley, J. (2002). Scoping the nursing and midwifery research and development capacity in Scotland to inform the development of a future strategy. Nursing Times , 7 (4): 255–262.

Gadsby, E.W. (2011). Research capacity strengthening: Donor approaches to improving and assessing its impact in low- and middle-income countries. International Journal of Health Planning and Management , 26: 89–106. doi: 10.1002/hpm.1031.

Hanlin, R., Chataway, J., & Smith, J. (2007). Global health public–private partnerships: IAVI, partnerships and capacity building. African Journal of Medicine and Medical Science , 36 (S1), Suppl: 69–75.

Holden, L., Pager, S., Golenko, X., & Ware, R. (2012). Validation of the research capacity and culture (RCC) tool: Measuring RCC at individual, team and organisation levels. Australian Journal of Primary Health , 18 (1): 62–67.doi: 10.1071/PY10081.

Horton, D., Alexaki, A., Bennett-Lartey, S., Brice, K.N., Campila, D., Carden, F. & Watts, J. (2003). Evaluating capacity development: Experiences from research and development organizations around the world . International Service for National Agricultural Research (ISNAR), The Netherlands; International Development Research Centre (IDRC), Canada; and ACP-EU Technical Centre for Agricultural and Rural Cooperation (CTA), the Netherlands. Retrieved from http://www.idrc.ca/EN/Resources/Publications/Pages/IDRCBookDetails.aspx?PublicationID=233 .

Joffres, C., Heath, S., Farquharson, J., Barkhouse, K., Latter, C., & MacLean, D. (2004). Facilitators and challenges to organizational capacity building in heart health promotion. Qualitative Health Research , 14 (1): 39–60. doi: 10.1177/1049732303259802.

LaFond, A., & Brown, L. (2003). A guide to monitoring and evaluation of capacity-building interventions in the health sector in developing countries. MEASURE Evaluation Manual Series , ( No. 7 ). Retrieved from http://gametlibrary.worldbank.org/FILES/610_M&E%20of%20Capacity%20Building%20Interventions.pdf .

Mahmood, S., Holt, K., Ahmed, S., Salam, M., & Cravioto, A. (2011). Strategies for capacity building for health research in Bangladesh: Role of core funding and a common monitoring and evaluation framework. Health Research Policy and Systems , 9: 31.

Morgan. P. (1997, December). The design and use of capacity development indicators. Paper prepared for the Policy Branch of CIDA. Retrieved from http://www.oecd.org/development/governance-development/1919953.pdf .

Neilson S., & Lusthaus C. (2007, February). IDRC-supported capacity building: Developing a framework for capturing capacity changes . Universalia. Retrieved from http://web.idrc.ca/uploads/user-S/11762347991CBDeveloping_Framework_Capturing_Capacity_Changes_FINAL.pdf .

Sadana, R., Lee-Martin, S-P., & Lee, J. (2006). Health Research System Analysis (HRSA) Initiative: Methods for Collection Benchmarks and Systems Analysis Toolkit. Tool #1: A brief overview of WHO/HRSA Initiative and an overview of core indicators and descriptive variables . Geneva: WHO Press.

Segrott, J., McIvor, M., & Green B. (2006). Challenges and strategies in developing nursing research capacity: A review of the literature. International Journal of Nursing Studies , 43: 637–651.

Thornicroft, G., Cooper, S., Van Bortel, T., Kakuma, R., & Lund, C. (2012). Capacity building in global mental health research. Harvard Review of Psychiatry , 20 (1): 13–24. doi: 10.3109/10673229.2012.649117 .

Waisbord, S. (2006). When training is insufficient: Reflections on capacity development in health promotion in Peru. Health Promotion International, 2 1 (3): 230–237.

Teasdale-Corti team grants

Literature search strategy tables

Five databases were searched to identify literature and theoretical frameworks pertaining to capacity building. Key words used in the search are shown in Table 1 .

Table 1: Key words used in the literature search

Search terms used and the yield of articles for each combination are shown in Table 2 . The number of full articles retrieved for each combination of terms is also shown. Articles retrieved were those relevant to research capacity building, which contained practical descriptions of capacity-building initiatives and/or their evaluations.

Table 2: Articles identified through a combination of search terms

accountability 75

Accra Agenda for Action 4 , 27

Action Health Incorporated (AHI) 190 , 198 , 201

Action Research Journal 54

adaptability 235 see also flexibility

African Journal of Reproductive Health 198

AHI see Action Health Incorporated

AIDS see HIV/AIDS

Alma Ata Declaration 39

AML see Atlantis Mobile Laboratory

ANAMURI (La Asociación Nacional de Mujeres Rurales e Indígenas) 178

applicability of research 21 –22, 24

apprenticeship 228

Araucaria Project (Proyecto Araucaria) (Chile)

ANAMURI 178

background of project 170 –173

capacity building, components 176 –181

capacity building, objectives 173 –176

Centro de Estudios de la Mujer (CEM) 170 –171, 179 –180, 183 –184

challenges 182 –184

Chile, working conditions in 171 –172, 180 –181

Chilean Labour Inspectorate 179 –180

CINBIOSE 170 –171

community partners 175

context, natural disasters 180

context, political 171 –172, 183

continuity 184 –185

cultural differences 182

empowerment 176 –181, 184

gender issues 172 , 175 –176, 183 –184

knowledge transfer 179 , 181

language barriers 182

methodological approaches 182

outcomes of project 176 –181, 183 –186

policy interventions 174 –175, 183

survey of working conditions 180 –181

‘systemic capacity building’ 174

tool to monitor psychosocial hazards 183 –184

trade unions 179

assessments, importance of 25 –26, 40 –41, 45 –47, 87

Atlantis Mobile Laboratory (AML) 134 –135, 138 , 139 , 141 –142, 143 , 144 –145, 147

Bamako Call to Action on Research for Health 4 , 11

brain drain 29

Brazil, anti-bullying legislation study in 177

Building Civil Society Capacity for Advocacy on Sexual and Reproductive Health and Rights (SRHR) 207

burden of disease

childhood obesity, rise in 101 –102

foodborne burden of illness (FBOI) study 134 , 135 , 137 , 139 –140, 142

HIV/AIDS 61 , 63 , 65 , 68 –71, 154 , 193 –195

impact of 13

CAMBIO programme (Mexico)

assignments 112 –113, 117 , 121 –122, 125 –126

background of programme 101 –104

challenges 125 –126, 126–127

course description 104 –105, 105 –111 , 112 –113

capacity building, dose delivered or received 102 , 115 -116, 118 , 119

evaluation components 102 –103, 113 , 114 –115 , 115 –117, 118 –122, 119 , 120 , 121 , 122 –124

knowledge uptake 116 , 119 –120, 120 , 124 –125

limitations of study 127 –128

methods 104 –105, 105 –111 , 112 –113, 114 –115 , 115 –118

participants 104 , 118

personal stories 117 , 122 , 122 – 124

questionnaire 117 , 120 –121, 121

reach 102 , 115 , 118

results 118 –122, 119 , 120 , 121 , 122 –124

statistical analysis 117 –118, 119 –121, 119 , 120 , 121

value of study 128 –129

Canada, obesity research in 102

Canada–China Project (Canada/China)

background, healthcare in China 153 –154

background to project 150 –155

capacity building, approach 158 –159

capacity building, dimensions 159

capacity building, focus 158 –159

capacity exchange 165 –167

challenges 162 –165

collaborations 160 –161

communication styles 166 –167

continuity 155 –156

cultural differences 162 –164, 165 , 166 –167

flexibility 150 , 167

knowledge transfer 159 , 164 , 165

National Innovation Capacity policy 150 –151, 156 –158

partnerships 160 –161

resources 164 –165

successes in field 161 –162

sustainability 155 –156

value of project 152 –153, 167

Canadian International Development Agency (CIDA) 4 , 176

capacity building, definitions

challenges with 2 , 13 , 28 , 240

concept-specific 16 –17

generic 14 –15

iterative 16 –17

research-specific 15 –16

systems orientation 17 –19

working definition for study 17 , 31

capacity building, importance of 1 –2

capacity building indicators

classifications of indicators 223 –236, 243

collaborations 224 –226, 238

comparative case study

cross-cutting themes 234 –235

data analysis 220

data collection 218 –219, 220 –221, 221

design of study 217 –218

findings of study 220 –235, 221

implications of study 241 –242, 244 –246

interviews 218 –219, 220 , 222

limitations of study 242 –244

methodology of study 217 –220

recommendations for funding agencies 246

reflections on indicators used 235 –239

research questions of study 217 –218

confidence building 226 –229

context 241 , 246

continuity 233

Cooke’s framework 235 , 244 , 245

dissemination of research output 229 –231

empowerment 231 –233

indicators, descriptions of 221 –223

limitations of indicators 3 , 215 –217, 239 –241, 244 –245

gaps of indicators 239 –241, 245

infrastructure 223 –224

sustainability 233 , 237

knowledge translation 229 –231

leadership 231 –233

linkages 224 –226, 238

local governance 231 –233

mentorship 228 –229, 238

networking 224 –226, 240

partnerships 224 –226, 238

relationship building 226

resources 223 –224

skills building 226 –229

stakeholders 224 –226, 230 , 234

training 224 , 226 –229, 238

career advancements 29 , 228 –229

Caribbean EcoHealth Programme (CEHP) (Caribbean)

background of programme 131 –132

capacity building, guiding principles 144 –148

capacity building, individual 137 –140, 138 , 139 , 140

capacity building, organisational 141 –142

capacity building, structural levels 137 –144, 138 , 139 , 140

collaborations 135 , 136 – 137 , 142 –144

continuity 144 –146

Cooke’s framework 137 –148

description of 133 –135, 136 –137 , 137

eco-toxicological water (ETW) studies 134 , 137

empowerment 146 –147

food safety training (FST) programme 134 , 137 , 143

leadership 147 –148

networks 135 , 136 – 137 , 142 –144

objectives of programme 132 –135, 136 –137 , 137

persistent organic pollutants (POPs) study 133 , 136 , 137 , 142

rainwater harvesting (RWH) study 134 , 135 , 137 , 142

recreational water quality (RWQ) study 134 , 137 , 142

supra-organisational capacity-building 135 , 136 – 137 , 142 –144

sustainability 144 –146

team capacity-building 141

value of programme 148 –149

zoonotic infections (ZI) study 133 , 136

catalytic processes 239

CEHP see Caribbean EcoHealth Programme

Centre de recherche interdisciplinaire sur la biologie, la santé et l’environnement (CINBIOSE) 170 –171

Centre for Population and Environmental Development (CPED) 196 –197, 198 , 199 , 201 , 207 , 208 –209

Centro de Estudios de la Mujer (CEM) (Centre for Women’s Studies) 170 –171, 179 –180, 183 –184

champions 239

change 80 –87, 98 –99

Chile, working conditions in 171 –172, 180 –181 see also Araucaria Project

background of healthcare in 153 –154

HIV/AIDS in 154

see also Canada–China Project

CIDA see Canadian International Development Agency

CINBIOSE (Centre de recherche interdisciplinaire sur la biologie, la santé et l’environnement) 170 –171

commitment 91 –92

communication 70 –71, 88 , 166 –167

communication of research outcomes see knowledge transfer or translation

communities 175 , 203 , 204 –205

Comparative Labor Law and Policy Journal 181

conceptual frameworks for research capacity building

approaches 25 –28

Cooke’s framework 20 –23, 26 , 31

Cooke’s modified framework 23 –25, 23 , 31

value of 19 –20

Confederación Nacional de Funcionarios Técnicos en Enfermería de Chile (trade union) 179

Araucaria Project 171 –172, 180 , 183

elections 63 –64, 66 , 69 , 71 –72, 241

HIV Prevention for Rural Youth (HP4RY) 191 –193

importance of 26 –27, 28 –29

indicators 241 , 246

Revitalizing Health for All (RHFA) project 40 –41

Strengthening Nurses’ Capacity programme 87 –92, 99

Araucaria Project 184 –185

Canada–China Project 155 –156

Caribbean EcoHealth Programme (CEHP) 144 –146

Cooke’s framework 22

indicators 233

Cooke’s framework 20 –23, 26 , 31 , 244 , 245

modified 23 –25, 23 , 31 , 137 –148, 235

Corti, Pierre 4

CPED see Centre for Population and Environmental Development

critical subjectivity 55

cultural differences

Araucaria Project 182

Canada–China Project 162 –164, 165 , 166 –167

indicators 228

Strengthening Nurses’ Capacity programme 72

definitions of capacity building see capacity building, definitions

democratisation 56 , 97 –98

Department for International Development (DFID) 26

digital divide see infrastructure

dissemination of research output

Cooke’s framework 22 , 24

HIV Prevention for Rural Youth (HP4RY) 192 –193, 197 –198, 199 –202, 210

indicators 229 –231

see also knowledge transfer or translation

documentation, value of 77 –78

ecohealth 133 see also Caribbean EcoHealth Programme

Edo state, Nigeria 195 see also HIV Prevention for Rural Youth (HP4RY)

empowerment

Araucaria Project 176 –181, 184

Caribbean EcoHealth Programme (CEHP) 146 –147

Cooke’s modified framework 25

indicators 231 –233

Strengthening Nurses’ Capacity programme 75 –80

see also power

EQCOTESST study 171 , 177

ESSENCE on Health Research 2 , 26 –27

ethics 112 , 182 , 219

expectations 95 –96

experiential understanding and learning 28 , 55 , 56 , 58 –59, 85 , 93 , 97 , 150 , 226 , 239

Family Life and HIV Education (FLHE) initiative 189 –190, 194 –195, 201 –202, 205 –206, 209

field research staff 199

flexibility 28 , 150 , 167 , 235

FLHE initiative see Family Life and HIV Education (FLHE) initiative

frameworks see conceptual frameworks for research capacity building

funding 11 –12, 29 , 224 , 242 , 243 , 246

funding agencies 71 , 246

gender issues 172 , 175 –176, 183 –184, 204 see also Araucaria Project

Global Foodborne Infections Network 147

Global Forum for Health Research 16

Global Health Research Initiative 3 –6

Global Ministerial Forum on Research for Health 11

GLUK see Great Lakes University of Kisumu

Great Lakes University of Kisumu (GLUK) 60 , 63 , 64 –65, 93

Guide for Useful Interventions for Activity in Brazil and Latin America (GUIA) 103

handbooks 69 –70, 98

HEAP see HIV/AIDS Emergency Action Plan

in China 154

in Jamaica 61

in Kenya 63

in Nigeria 193 –195

in South Africa 65

in Uganda 68 –71

see also burden of disease; Canada–China Project; HIV Prevention for Rural Youth (HP4RY); Strengthening Nurses’ Capacity programme

HIV/AIDS Emergency Action Plan (HEAP) 194

HIV Prevention for Rural Youth (HP4RY) (Nigeria)

background to project 188 –191, 193 –195

communities 203 , 204 –205

context, sub-Saharan African 191 –193

definition of research capacity 191

dissemination of research output 192 –193, 197 –198, 199 –202, 210

Edo state 195

factors influencing research capacity 191 –193

government 206 –207, 209

junior academics 198 –199, 202

knowledge delivery modes 202 –203, 204 –209

knowledge translation strategies 199 –202, 204 –209

local researchers 203

National Education Sector HIV/AIDS Strategic Plan (NESSP) 190 , 194

National Youth Service Corps (NYSC) 190 , 200 , 201 , 204 –205, 206 , 209

networking 207 –208

newsletters 206

NGOs 203 , 206 –207

policy interventions 188 –189, 190 –191, 192 –193, 206 –207, 208 –209

policy-makers 206 –207, 209

programme packages 206

programming 204 –209

research capacity building of project 195 –199

schools 203

stakeholders 200 –202

team members 197 –198

value of project 209 –210

vocational training for women 204

human resources 61 , 62

IDRC see International Development Research Centre

IICBA see International Institute for Capacity Building in Africa

indicators see capacity building indicators

individual capacity-building 137 –140, 138 , 139 , 140

infrastructure

CAMBIO programme 127 , 128

Canada–China Project 162 , 164

Caribbean EcoHealth Programme (CEHP) 134 –135, 138 , 139 , 141 –142, 143 , 144 –145, 147

Cooke’s modified framework 22 , 24

HIV Prevention for Rural Youth (HP4RY) 195 –197

indicators 223 –224

Strengthening Nurses’ Capacity programme 63 –64, 89

innovation 150 –151, 156 –158

institutional commitment 239

institutional cultures and politics 67 –68, 73

International Development Research Centre (IDRC) 4 , 43 , 45 , 196

International Institute for Capacity Building in Africa (IICBA) 15

international network 69 –71, 72 –73 see also networking

Jamaica 53 , 59 , 60 –63, 72 , 81 , 82 –83, 88 , 89

Kenya 53 , 60 , 63 –65, 71 –72, 77 , 81 , 82 –83, 85 , 88 , 91 , 93

‘know–do gap’ 12 see also ‘research-to-action’

knowledge transfer or translation

Araucaria Project 179 , 181

Canada–China Project 159 , 164 , 165

Cooke’s modified framework 24

HIV Prevention for Rural Youth (HP4RY) 199 –209

Revitalizing Health for All (RHFA) project 40 –41, 45

see also dissemination of research output

La Asociación Nacional de Mujeres Rurales e Indígenas (ANAMURI) 178

language barriers 163 , 166 , 182 , 228

leadership 24 –25, 97 –98, 147 –148, 231 –233

leadership awards 5

leadership hubs 53 , 56 , 57 –58, 61 , 66 –68

learning, use of term 97

leveraging 234

linkages 22 , 224 –226, 238

literature review 6 , 14 , 252

local context see context

mental health see Araucaria Project

CAMBIO programme 128

indicators 228 –229, 238

Revitalizing Health for All (RHFA) project 40 , 43 –44

Strengthening Nurses’ Capacity programme 57

triads 40 , 45 –47, 48 , 51

see also training

methodological approaches, differences in 182

Mexico see CAMBIO programme

national research bodies 24

NESSP see National Education Sector HIV/AIDS Strategic Plan

Caribbean EcoHealth Programme (CEHP) 135 , 136 – 137 , 142 –144

HIV Prevention for Rural Youth (HP4RY) 207 –208

indicators 224 –226, 240

Strengthening Nurses’ Capacity programme 69 –71, 72 –73

NGOs see non-governmental organisations

Nigeria, HIV/AIDS in 193 –195 see also HIV Prevention for Rural Youth (HP4RY)

non-governmental organisations (NGOs) 203 , 206 –207

nurses see Strengthening Nurses’ Capacity programme

NYSC see National Youth Service Corps

occupational health see Araucaria Project

online journals, access to 127

organisational capacity-building 141 –142

organisational culture 19

ownership 238 –239

Pan-American Health Organization 147

PAR see participatory action research

Paris Declaration on Aid Effectiveness 4 , 27

participatory action research (PAR) 53 –54, 73 –96

participatory worldview 55 –56, 96 –98

partnerships

Canada–China Project 160 –161

Cooke’s framework 22 , 27 –28

indicators 224 –226, 238

Revitalizing Health for All (RHFA) project 50

policy interventions

Araucaria Project 174 –175, 183

HIV Prevention for Rural Youth (HP4RY) 188 –189, 190 –191, 192 –193, 206 –207, 208 –209

indicators 230 –231

political context

Araucaria Project 171 –172, 183

Strengthening Nurses’ Capacity programme 63 –64, 66 , 69

power 75 –80, 96 see also empowerment

practical knowing 55

presentational knowing 55

primary healthcare, importance of 39 see also Revitalizing Health for All (RHFA) project

professional politics 62 –63

‘project champions’ 239

propositional knowing 55

Proyecto Araucaria see Araucaria Project

psychosocial hazards, tool to monitor 183 –184

Public Health Agency of Canada 4 , 133

published articles see knowledge transfer or translation

relationship building 71 , 75 , 226

research applicability 21 –22, 24

research capacity, definition of 191

research capacity building see capacity building

research capacity building indicators see capacity building indicators

‘research-to-action’ 39 –40, 51 see also ‘know–do gap’

resources 164 –165, 223 –224 see also infrastructure

Revitalizing Health for All (RHFA) project

challenges 47

feedback from teams 47 –51

goals of project 39 –40

mentorship 40 , 43 –44

methodology 45 –47

partnerships 50

‘research-to-action’ 39 –40, 51

role distinction 51

selection of teams 42 –43

self-assessments 40 –41, 45 –47

suggestions 48 –51

training sessions 45 –46

Ross University 143 , 144 , 145

rural areas see HIV Prevention for Rural Youth (HP4RY)

Safety Science 181

scale up 239

SCDC see Shanghai Centre for Disease Control

sexually transmitted infections (STIs) 154 see also Canada–China Project

SGU see St. George’s University

Shanghai, China see Canada–China Project

Shanghai Centre for Disease Control (SCDC) 152 , 154 , 156 , 159 , 160 –161, 164

skills building see training

social dimensions of nursing 69

South Africa 53 , 60 , 65 –68, 72 , 82 , 88 , 89

South—South partnerships 29

SRHR see Building Civil Society Capacity for Advocacy on Sexual and Reproductive Health and Rights

stakeholders

approaches 26 –27

HIV Prevention for Rural Youth (HP4RY) 200 –202

indicators 224 –226, 230 , 234

St. George’s University (SGU) 133 , 143

STIs see sexually transmitted infections

Strengthening Nurses’ Capacity programme (Jamaica/Kenya/South Africa/Uganda)

assumptions 92 –96

background to project 54

case descriptions 60

collaborations 55 –56, 95 , 97 , 99

communication 70 –71, 88

comparative case study 57 –73

context 87 –92, 99

data analysis 59 –60

data collection 58 –59

elections 63 –64, 66 , 69

empowerment 75 –80

equity 93 , 95

research questions 57 –58

HIV/AIDS 56 –57, 61 , 63 , 65 , 68 –71

international network 69 –71, 72 –73

interviews 58 –59

leadership 97 –98

mentoring 57

objectives of project 56 –57, 87

policy-makers 68

power 75 –80, 96

relationship building 71 , 75

shortage of nurses 61 , 62

theoretical perspective 55 –56

Uganda 53 , 60 , 68 –69, 72 , 81 , 82 –83, 88 , 91

values 92 –96

vertical relationships 84 –86, 97

workplace conditions 89 –90

subjectivity 55

support 78 –79, 239

sustainability

indicators 233 , 237

systems orientation 17 –19, 20 –21, 31

teams 141 , 197 –198

Teasdale, Lucille 4

Teasdale-Corti Initiative 4 –6, 11 –13, 249 –251

time challenges 87 –92

trade agreements 173

Cooke’s framework 21

indicators 224 , 226 –229, 238

see also mentorship

transparency 75

United Nations Development Programme (UNDP) 26

United Nations Educational Scientific and Cultural Organization (UNESCO) 15

urbanisation 153 –154

vertical relationships 84 –86, 97 , 242

weather considerations 89 , 180

Windward Islands Research and Education Foundation (WINDREF) 143

women, vocational training for 204 see also Araucaria Project; gender issues

work-related mental health see Araucaria Project

World Bank 14 –15

World Health Organization 1 , 172

youth see HIV Prevention for Rural Youth (HP4RY)

• DOI: 10.1186/s12913-018-3518-7
• Corpus ID: 52280140

Research capacity building frameworks for allied health professionals – a systematic review

• Janine Matus , Ashlea Walker , S. Mickan
• Published in BMC Health Services Research 15 September 2018

83 Citations

Research education and training for nurses and allied health professionals: a systematic scoping review, exploring research capacity and culture of allied health professionals: a mixed methods evaluation, developing a consolidated research framework for clinical allied health professionals practising in the uk, a practical toolkit of strategies for building research capacity in allied health.

• Highly Influenced

Building Heath Research Capacity: The Impact of a United Kingdom Collaborative Programme

Developing a consolidated research career competency framework for allied health professionals, rethinking health professionals’ motivation to do research: a systematic review, allied health professional research engagement and impact on healthcare performance: a systematic review protocol..

• 15 Excerpts

The health service perspective on determinants of success in allied health student research project collaborations: a qualitative study guided by the Consolidated Framework for Implementation Research

Describing the evidence-base for research engagement by health care providers and health care organisations: a scoping review, 47 references, frameworks for embedding a research culture in allied health practice: a rapid review, research culture in allied health: a systematic review., motivators, enablers, and barriers to building allied health research capacity.

• Highly Influential

Utilising implementation science in building research and evaluation capacity in community health.

Key factors influencing allied health research capacity in a large australian metropolitan health district, a framework to evaluate research capacity building in health care, a thematic analysis of the role of the organisation in building allied health research capacity: a senior managers’ perspective, building research capacity in the allied health professions, research capacity and culture of the victorian public health allied health workforce is influenced by key research support staff and location., a systematic review of the individual determinants of research evidence use in allied health, related papers.

Showing 1 through 3 of 0 Related Papers

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• For authors
• Browse by collection
• BMJ Journals

You are here

• Volume 11, Issue 7
• Measuring research capacity development in healthcare workers: a systematic review
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-8765-7384 Davide Bilardi 1 , 2 ,
• http://orcid.org/0000-0001-8818-8148 Elizabeth Rapa 3 ,
• http://orcid.org/0000-0001-7628-8408 Sarah Bernays 4 , 5 ,
• http://orcid.org/0000-0003-2273-5975 Trudie Lang 1
• 1 Nuffield Department of Medicine , University of Oxford Centre for Tropical Medicine and Global Health , Oxford , UK
• 2 Fondazione Penta Onlus , Padova , Italy
• 3 Department of Psychiatry , University of Oxford , Oxford , UK
• 4 School of Public Health , University of Sydney–Sydney Medical School Nepean , Sydney , New South Wales , Australia
• 5 Public Health and Policy , London School of Hygiene & Tropical Medicine , London , UK
• Correspondence to Dr Davide Bilardi; davide.bilardi{at}gtc.ox.ac.uk

Objectives A key barrier in supporting health research capacity development (HRCD) is the lack of empirical measurement of competencies to assess skills and identify gaps in research activities. An effective tool to measure HRCD in healthcare workers would help inform teams to undertake more locally led research. The objective of this systematic review is to identify tools measuring healthcare workers’ individual capacities to conduct research.

Design Systematic review and narrative synthesis using Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist for reporting systematic reviews and narrative synthesis and the Critical Appraisals Skills Programme (CASP) checklist for qualitative studies.

Data sources 11 databases were searched from inception to 16 January 2020. The first 10 pages of Google Scholar results were also screened.

Eligibility criteria We included papers describing the use of tools/to measure/assess HRCD at an individual level among healthcare workers involved in research. Qualitative, mixed and quantitative methods were all eligible. Search was limited to English language only.

Data extraction and synthesis Two authors independently screened and reviewed studies using Covidence software, and performed quality assessments using the extraction log validated against the CASP qualitative checklist. The content method was used to define a narrative synthesis.

Results The titles and abstracts for 7474 unique records were screened and the full texts of 178 references were reviewed. 16 papers were selected: 7 quantitative studies; 1 qualitative study; 5 mixed methods studies; and 3 studies describing the creation of a tool. Tools with different levels of accuracy in measuring HRCD in healthcare workers at the individual level were described. The Research Capacity and Culture tool and the ‘Research Spider’ tool were the most commonly defined. Other tools designed for ad hoc interventions with good generalisability potential were identified. Three papers described health research core competency frameworks. All tools measured HRCD in healthcare workers at an individual level with the majority adding a measurement at the team/organisational level, or data about perceived barriers and motivators for conducting health research.

Conclusions Capacity building is commonly identified with pre/postintervention evaluations without using a specific tool. This shows the need for a clear distinction between measuring the outcomes of training activities in a team/organisation, and effective actions promoting HRCD. This review highlights the lack of globally applicable comprehensive tools to provide comparable, standardised and consistent measurements of research competencies.

PROSPERO registration number CRD42019122310.

• organisational development
• organisation of health services
• medical education & training
• public health

Data availability statement

Data are available upon reasonable request. All data relevant to the study are included in the article. The complete data set generated by the systematic review and included in the extraction log is available upon request.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjopen-2020-046796

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

Thoroughly conducted systematic review collecting data from all major existing databases and grey literature.

Topic not previously addressed in other reviews searching for tools to measure health research capacity building at individual level.

Brief overview of the identified tools to measure health research capacity building at individual level highlighting strengths and weaknesses of them.

Complex identification of relevant studies due to the lack of clarity on a common definition and terminology to identify health research capacity development.

None of the studies use the standard reporting procedures for qualitative or quantitative research.

Introduction

In 2004, the Global Forum for Health Research highlighted the challenge for low and middle-income countries to have the capacity to perform effective and locally led health research which address the major health problems affecting their own populations. 1–3 Twenty years later, low and middle-income countries still carry 90% of the global disease burden, but only 10% of global funding for health research is devoted to addressing these persistent health challenges. 4 Health research capacity development (HRCD) for healthcare workers has been recognised as a critical element to overcoming global health challenges, especially in low and middle-income countries. 5 For too long HRCD in low and middle-income countries has been documented through training programmes which enable local teams to participate in externally sponsored trials, creating a false appearance of growth and generating dependence on foreign support. 6 7

The process of progressive empowerment is usually referred to as capacity development. 8 This term has been used in multiple areas and applied in different sectors to develop new or existing competencies, skills and strategies at a macro or individual level. 9 In the field of health, research capacity development should support healthcare workers in generating local evidence-based results to inform policy and improve population health. The three health-related Millennium Development Goals, and more recently the targets ‘B’ and ‘C’ of the Sustainable Development Goals, all support the adoption of new strategies to strengthen the capacity of healthcare workers in all countries in performing their job and engaging in research. 10–12 One of the critical barriers in supporting HRCD is the lack of empirical measurement of competencies in relation to the performance of research activities. Existing frameworks and tools have been developed for a particular purpose in a particular context. 13 14 Others have identified barriers that healthcare workers encounter in engaging in research or have monitored and evaluated targeted training activities. 15 This systematic review aims to identify tools to measure individual healthcare workers’ capacities to conduct research.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist 16 for reporting systematic reviews and narrative synthesis and Critical Appraisals Skills Programme (CASP) checklist 17 on critical appraisal for qualitative studies were used to design this systematic review and to refine the extraction log according to recognised guidelines.

Inclusion and exclusion criteria

The aim of the systematic review was to identify existing tools which measure individual capacities in conducting research in healthcare workers. The inclusion and exclusion criteria were defined in advance and documented using an adapted version of a SPIDER table ( table 1 ). The primary population of interest were all health-related professionals or healthcare workers involved in research activities. Healthcare workers delivering health services when research was not considered as the focus of the study were excluded. Occupational health research was excluded. Studies about volunteers, defined as people offering their services to support health activities with no specific training as health professionals, were also excluded. Initially, only healthcare workers working in low and middle-income countries were included, but this limitation was removed to identify any tool measuring HRCD in any setting. The Phenomenon of Interest was defined as: assessing HRCD; or identifying tools, frameworks and templates designed to assess HRCD. A comprehensive range of terms including synonyms for ‘assess’, ‘tool’ or ‘development’ was used. Studies were excluded which mentioned components that could be considered to assess, measure and ‘give evidence to’ research capacity development, but were not presented in any capacity development context. In addition, since the concept of capacity development is widely applied to different settings, studies on areas unrelated to health, such as ‘air pollution’, ‘financial capacity’ or ‘tobacco’, were also excluded. The study design criteria were broad to include qualitative, quantitative and mixed methods papers. Further criteria of eligibility included in the SPIDER table refer to the quality of the study (Evaluation) and the Research type.

• View inline

SPIDER diagram—inclusion and exclusion criteria

Information sources and search strategy

Eleven databases were searched from inception to 16 January 2020: Ovid MEDLINE; Ovid Embase; Ovid PsycINFO; Ovid Global Health; EBSCO CINAHL; ProQuest Applied Social Sciences Index & Abstracts (ASSIA); ProQuest Sociological Abstracts; ProQuest Dissertations & Theses Global; Scopus; Web of Science Core Collection; and the WHO Global Index Medicus Regional Libraries. The first 10 pages of results from Google Scholar were also screened. The search strategies used free text terms and combinations of the relevant thesaurus terms, limited to English language publications only, to combine terms for capacity building, measuring and health research. The ‘NOT’ command was used to exclude papers about students, postgraduate students, tobacco, air pollution and a variety of other concepts to minimise the number of irrelevant results (see box 1 for a full set of search strategies).

Search strategy

Database: medline (ovid medline epub ahead of print, in-process & other non-indexed citations, ovid medline daily and ovid medline) 1946 to present.

Capacity Building/ (1965)

(capacit* adj2 build*).ti,ab. (5789)

(capacit* adj2 develop*).ti,ab. (3591)

(capacit* adj2 strengthen*).ti,ab. (924)

(competenc* adj2 improv*).ti,ab. (1460)

((professional* adj2 develop*) and (competenc* or capacit*)).ti,ab. (1747)

1 or 2 or 3 or 4 or 5 or 6 (13649)

Mentoring/ (820)

mentor*.ti,ab. (13369)

(assess* or measur* or evaluat* or analys* or tool* or equip*).ti,ab. (9653076)

“giv* evidence”.ti,ab. (3814)

framework*.ti,ab. (231138)

8 or 9 or 10 or 11 or 12 (9763562)

Research/ (196782)

clinical.ti,ab. (3158817)

(health* and research*).ti,ab. (337604)

14 or 15 or 16 (3588891)

7 and 13 and 17 (3433)

limit 19 to English language (3346)

(student* or graduate or graduates or postgraduate* or “post graduate*” or volunteer* or communit* or tobacco or “climate change” or “air pollution” or occupational or “financial capacity” or informatics or “IT system” or “information system” or transport or “cultural competenc*” or disabili* or trauma).ti,ab. (1828113)

20 not 21 (1673)

Google Scholar—screen the first 10 pages of results

Sorted by relevance:.

(“capacit* build*”|“build* capacit*”|“capacit* develop*”|“develop* capacit*”|“capacit* strengthen*”|“strengthen* capacit*”|“professional* develop*”|“completenc* improv*”|“improv* competenc*”)(“health* research*”|clinical) https://scholar.google.co.uk/scholar?q= (%22capacit*+build*%22%7C%22build*+capacit*%22%7C%22capacit*+develop*%22%7C%22develop*+capacit*%22%7C%22capacit*+strengthen*%22%7C%22strengthen*+capacit*%22%7C%22professional*+develop*%22%7C%22completenc*+improv*%22%7C%22improv*+competenc*%22)(%22health*+research*%22%7Cclinical)&hl=en&as_sdt=0,5

Study selection

Two researchers, DB and ER, independently screened and reviewed studies using the Covidence systematic review software. 18 In case of disagreement, DB and ER discussed the abstracts in question. After consensus on inclusion was reached, the full texts of all included studies were rechecked for inclusion by DB and confirmed by ER.

Study analysis procedure

Data from selected papers were extracted, and quality assessments performed using an extraction log created and validated against the CASP checklist 17 on critical appraisal for qualitative studies. Macro areas of interest in the log were: general information on the paper such as author and title, main focus and study design. The source of funding, conflict of interests and ethics approval were also recorded. A separate section of the extraction log recorded the characteristics of the tool used or described in each selected paper ( figure 1 ). The extraction log also included specific sections considering the study design, the methodology and the main findings of each paper. Furthermore, a dedicated section of the log collected data on the quality of each study, analysing selection biases and a critical appraisal derived from the CASP checklist. If a definition of capacity development was given, the definition was collected. Some of these sections of the extraction log are not present in figure 1 since it focuses on the description of the identified tool. The content method was used to define a narrative, described in the Discussion section.

• Download figure
• Open in new tab
• Download powerpoint

Extraction log.

Patient and public involvement

Patients and/or the public were not involved inthe design, or conduct, or reporting, or dissemination plans of this research.

Database search and results screening

In December 2018, the first round of the search was performed in 11 different databases and in Google Scholar using the search strategy described in box 1 . A total of 13 264 suitable records were found. A total of 6905 duplicates were removed, resulting in 6359 unique records for inclusion screening by title and abstract ( table 2 ), which was performed throughout 2019. In January 2020, an additional search for papers published or included in publication databases in 2019 was performed using the same search strategy and resulted in 15 775 papers and after removal of duplications, a total of 1118 papers were found. These papers were then added to the 6359 papers identified from the first search. A total of 7474 unique papers were included for title and abstract inclusion screening (three duplicate records were removed in the Covidence software).

Search results

The 7474 unique relevant studies identified were uploaded to the Covidence systematic review software. Two researchers, DB and ER, independently screened the studies, including or excluding according to the criteria in the SPIDER table ( table 1 ). A total of 7280 studies were considered irrelevant. The full-text papers for the remaining 178 references were reviewed. Reasons for exclusion were identified by streamlining the SPIDER table criteria into three main criteria: wrong setting, irrelevant study design and wrong focus of the study. A reason for exclusion was assigned to each paper. All 178 studies described some form of activity to measure the competencies related to performing health research. Thirty were excluded because they were literature reviews on a different aspect of health research or because they described a general perspective on the topic of health capacity development without offering any specific measurement or without reference to research. In addition, 42 studies were excluded because of the wrong setting, since competencies were measured at the level of research institutions or within a specific network. An additional 90 studies were excluded because the study design did not match the inclusion criteria: 38 studies described the use of a measurement tool tailored to the context (eg, specific profession, intervention or setting) and not at the individual level; the remaining 34 studies were excluded because there was no mention of a specific tool to measure HRCD. The final 18 papers reported the use of an evaluation tool, but the tool was an ad hoc pre/postintervention questionnaire with low potential of applicability in a context different from the one described in the paper. A total of 162 studies were therefore excluded, leaving 16 studies for this review ( figure 2 ).

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) screening diagram.

Analysis of the findings across the selected papers

A total of 16 studies met the inclusion criteria set for this systematic review. 19–34 The 16 articles were analysed using the extraction log created and validated against the CASP qualitative checklist.

The results are summarised in table 2 . None of the papers were published before 2006 and only nine of them were published after 2014. 20 21 23–26 31 33 34 The majority (n=13) applied a tool in high-income settings. 19 20 22–24 26–32 34 Seven papers described the use of tools in Australia, 20 22 24 26 28 29 34 three in low and middle-income countries (one in Ghana, Kenya, Malawi, Sri Lanka, 25 one in the Pacific Islands 21 and one in the Philippines 33 ), one in Europe (Norway), 19 one in the USA 32 and one measured HRCD in a group linked to a specific intervention located in multiple areas of the world. Three of them described the creation of a tool without applying it to any specific context, 23 30 31 but they were all designed by research groups in high-income countries (one in the USA and two in the UK).

All of the selected studies applied quantitative, qualitative and mixed methods analyses. The preferred approach (n=7) was to generate quantitative data using an HRCD tool. 20 24 26–28 33 34 One-third of the studies (n=5) used a mixed methods approach 19 21 22 25 29 ; quantitative tools were associated with semistructured interviews, or in some cases qualitative questions were added to the questionnaire. The three studies describing the creation of a tool were not analysed under this methodological category.

Of the 16 selected studies, three used the term ‘capacity development’, 23 25 30 and two included a definition of the concept. 25 30 Seven papers used ‘capacity building’, 20–22 24 26 28 31 of which four also included a definition. 20 22 24 28 In two papers, the capacity building definition was associated with the definition of ‘research culture’. 20 22 Two additional papers used alternative generic terms like ‘research capacity’ 33 or ‘research self-efficacy’. 32 Four papers did not refer to any specific term and therefore no definition was given. 19 27 29 34

Five of the 16 selected papers openly declared no conflict of interests. 22–24 28 31 Eight stated the source of funding used to carry out the activities described. 19 21 23 27–31 The number of participants in the studies varied from 28 enrolled participants for a qualitative study 21 to 3500 users of an online measurement tool. 27

Analysis of the tools from the selected papers

The tools described or used in the 16 selected papers varied in nature, length and applicability. In general, even when there were similarities, each paper described a different perspective on the use of a tool. Four papers applied a questionnaire-type tool to assess research competencies and skills. 19 21 25 33 The length of these questionnaires varied from 19 21 to 59 19 health research capacity-related questions, with the addition of open-ended qualitative questions in two studies, 19 21 and a structured interview in another study. 25

Three studies 22 24 34 used, with a range of adaptations, the Research Capacity and Culture tool and one study 20 revised this tool into a Research Capacity and Context tool referencing Research Capacity and Culture tool as a primary source. Another recurrence in the papers was the use of the ‘Research Spider’ tool. 28 29 Again, the original tool had been adapted to the context, and in one case, 29 the tool was used as a base for qualitative research on HRCD. Two additional papers described tools designed ad hoc to measure the impact of an intervention (CareerTrac 27 and Cross-Sectional Electronic Survey 26 ). These last two papers were not excluded under pre/postintervention since the action was wider, at a programme level and the tool used to measure HRCD was the main focus of the paper. Furthermore, another paper described a tool for a specific category of healthcare workers (Nursing Research Self-Efficacy Scale—NURSES). 32 Three papers 23 30 31 focused on the creation of a new tool and described the process of identifying a set of competencies required to run health research. The outcome of two of them was defined as a ‘core competency framework’. 23 31 The third defined the outcome of the analysis as a ‘set of indicators’. 30

In terms of the target population, the identified tools aimed to measure HRCD in a range of different healthcare worker professions. One-third of the papers (n=5) focused on measuring HRCD on allied health professionals (AHPs). 20 22 24 26 34 Nurses were the main focus in two other studies, 19 32 and four studies applied a tool to a range of health professions (ranging from laboratory scientists to data managers). 21 25 28 29 Two other papers focused on groups linked to a specific intervention. 27 33 All 16 papers included, alongside healthcare workers, representatives of technical professions in health such as managers, directors, faculty members and consumer organisation representatives. In the case of the three papers describing the creation of a new tool, they suggest that these tools would be applicable to all research roles. 23 30 31

As per inclusion criteria, the main level of measurement of the tools was at the individual level. Seven papers only measured HRCD at the individual level. 19 23 28 29 31–33 Three papers added to the individual level of measurement by including information on the perceived barriers in performing health research 21 26 29 ; of these three, two also focused on understanding what motivates healthcare workers to become involved in health research. 26 29 The five studies, which used the Research Capacity and Culture tool and its variants, included the measurement of HRCD at the individual level, and at the team and organisational level. 20 22 24 25 34 One paper described the creation of a tool designed to be used at the organisational level, but embedded a measurement of HRCD at the individual level as well. 30

The most common way a selected tool was validated was by referencing the main paper that described the selected tool and its validation process (n=6). 23 28 29 32–34 This was the case for some of the ad hoc questionnaires, 23 33 34 of the ‘Research Spider’ tool 28 29 and of the NURSES tool. 32 Papers which described an original process or used modified versions of an original tool validated the tool through a contextual validation process described in the paper. 21 22 24 25 31 These validation processes included a consultation of a panel of experts 22 24 31 or a reiterative process of validity. 21 25 One paper stated that the tool used was a validated tool without referencing the process or tool. 20

Overall, only two papers 23 31 focused specifically on tools to measure HRCD on a wider level, without linking the measurement to a specific group or a geographical area which was done in the majority of papers. 19 24 25 28 29 33 In four cases, the tools described were adapted to identify determinants or barriers of HRCD in a defined setting 20 30 34 or to promote HRCD in relation to a specific disease or research topic. 21 In other cases, the papers focused on a tool aiming to assess the impact of specific interventions or programmes on HRCD. 26 27

Summary of evidence

This systematic review aimed to identify tools which measure individual capacities in conducting research in healthcare workers; the 16 included articles 19–34 which demonstrated that tools to measure HRCD in healthcare workers are available, even if they are limited in number. In most cases, the identified tools do not originate from the need to measure and foster HRCD as a necessary strategy to promote research capacity. There is, therefore, a need to design more comprehensive tools which are globally applicable and able to provide comparable, standardised and consistent measurements of research competencies.

The importance of measuring HRCD has only been recognised recently. 15 As the date of publication of the identified papers shows, the appreciation of the contribution that health research can offer in capacity development at a personal level only began in the first decade of this new millennium. Almost half of the selected papers (n=7) refer to studies whose data have been collected after 2014. 20 21 24 26 31 33 34 Of note is the high number of new publications which were retrieved from the academic databases (1118 papers) when the search strategy was rerun in 2020.

Questionnaires were the most commonly used method for assessing research skills and competencies. Almost two-thirds of the papers (n=10) 19 20 22 24 26 28 29 32–34 based the measuring system of different research skills at a personal level using a 5-point Likert scale (n=6) 19 26 28 29 32 33 or a 10-point scale (n=4). 20 22 24 34 This choice highlights the need for a validated quantitative tool based on a set of competency-related questions that can bring standardisation, comparability and consistency across different roles and contexts. However, the extensive use of mixed methods, combining quantitative questionnaires with other qualitative instruments, reflects that HRCD depends on a complex series of components that need to be identified both qualitatively and quantitatively.

By not limiting the selection of articles for this review to those tools used in low and middle-income countries, this review has revealed that most of the tools identified were used in high-income settings. It is important to note that excluding pre/postintervention assessments significantly reduced the inclusion of studies performed in low and middle-income countries. This finding highlights that although health systems in low and middle-income countries may benefit from providing evidence for HRCD, 5 they are rarely the focus of the HRCD literature. Most of the measurements of HRCD in lower income settings appear, in fact, to be narrowly linked to the measurement of the effectiveness of training offered for a specific study or limited to a particular disease. Even when the perspective is broader than a particular study, it is mostly limited to the evaluation and sustainability of training programmes and not linked to a plan of career progression and research competency acquisition. More attention should therefore be given in creating tools which are able to measure, support and promote long-lasting research capabilities in the perspective of professional growth for healthcare workers.

Three essential findings of this systematic review support a change in the perception of HRCD and the tools needed to measure it. First, many of the excluded papers (42 out of 162 excluded papers from the last round of analysis) focused exclusively on the institutional level of measuring research capacity. This is mostly because training interventions are designed to prepare a team to run a study and rarely to promote individual HRCD. 1 35 36 In some cases, the measurement via a tool is also an exercise to demonstrate the investment in training activities for reporting purposes. 37 38 It is therefore important to start promoting a more effective research culture which is independent of specific diseases or roles. This progression could be achieved by championing systems which measure the changes in research capacities at a team and personal level using a globally applicable tool. Most of the tools excluded were evaluation tools designed for, or used in, a specific setting and thus not suitable for a comparable, standardised and consistent analysis of long-term research competency acquisition strategies.

Second, papers that focused on measuring HRCD at the individual level confirmed that research is seen as an opportunity to learn the cross-cutting skills needed in healthcare. A defined set of standardised competencies required to conduct research could be used to measure an individual, team and organisation’s abilities. This was the focus of two papers 23 31 which identified a framework of core competencies. Most of the tools (n=7) were designed to be applied to a wider variety of health professions. 21 23 25 28–31 HRCD can be accessed at different entry points depending on the specific job title, but the set of skills acquired is common and shared among the research team. 1 The approach on assessing these inter-related competencies should therefore be global and not role or disease based. 39 The measurement at an individual level is essential to promote a consistent and coherent career progression for each person and role. 40 However, the overall capability in running research programmes should be measured at a team level where all roles and competencies complement each other, skills are made visible, and measurable as a whole against an overall competency framework. Individual and institutional/team levels are therefore two aspects of HRCD that grow together supported by a common comparable, standardised and consistent tool.

Third, the lack of a standard definition for HRCD can lead to post-training evaluations being categorised as HRCD activities. Although pre/post-training evaluations are important, it might be helpful to define what a ‘structured action’ is to promote HRCD. As previously mentioned, the term ‘capacity development’ is not universally used, with many synonyms such as ‘research capacity’ or ‘capacity strengthening’, creating the possibility of different interpretations. Furthermore, inconsistent terminology was found in describing activities in support of HRCD that in reality were very similar (eg, workshop, training, course). Steinert et al 41 suggest that there should be a standard definition in the context of educational capacity development. This suggestion, alongside a common taxonomy to describe health professions, would support the identification of HRCD as a defined process with specific characteristics and not with a general effort for research training.

The most common tool identified in this review was the Research Capacity and Culture tool. 20 22 24 34 The Research Capacity and Culture tool consists of 52 questions that examine participants’ self-reported success or skill in a range of areas related to research capacity or culture across three domains including the organisation (18 questions), team (19 questions) and individual (15 questions). The Research Capacity and Culture tool includes questions on perceived barriers and motivators for undertaking research. The respondents of the Research Capacity and Culture tool are asked to rate a series of statements relevant to these three domains on a scale of 1–10, with 1 being the lowest and 10 being the highest possible skill or success level. It represents a good example of a comprehensive tool. As confirmed by the review findings, a potential limitation is its application mainly in an Australian context and almost exclusively to measure HRCD in AHPs. 22 24 34 The generalisability of the tool should thus be confirmed. Nevertheless, the Research Capacity and Culture tool represents a strong example of how having a tool refined around a context, and a specific health profession can be an incentive in measuring HRCD.

Another tool highlighted by this review was the ‘Research Spider’ tool. 28 29 42 This tool collects information on individual research experience and interest in research skill development in 10 core areas. These include ‘writing a research protocol’, ‘using quantitative research methods’, ‘publishing research’, ‘finding relevant literature’ and ‘applying for research funding’. In each area, the level of experience is measured on a 5-point Likert scale, from 1 (no experience) to 5 (high experience). The primary aim of the ‘Research Spider’ is to be a flexible tool. This flexibility is confirmed in two studies 28 29 which used the ‘Research Spider’, with one 28 using it as the main measurement, and the other 29 as a quantitative base for qualitative semistructured interviews. The advantage of this tool is that it provides a visual overview of personal research competencies. However, although the limited number of measurement areas (n=10) makes the tool a good initial evaluation instrument, it does not offer a specification of the subskills of each area.

A critical mention should be reserved for the two papers which described the creation of a comprehensive research core competency framework. 23 31 Despite no specific tool being described and the competency scores being visualised by using a spider diagram, these studies present the most accurate overview of the skills required in running research programmes related to health. As mentioned before, a tool which applies a scoring system to the list of competencies identified by these frameworks has the potential of being widely applicable and reliable. This wide applicability and the absence of explicit biases in measuring research skills improvement can foster a more robust approach to research in health. The measurement of HRCD unrelated to specific interventions would maximise the benefit of research at every level. At a personal level, it would clarify a potential career progression path highlighting possible gaps; at the team level, it would support a multidisciplinary approach to health challenges; and at an institutional level, the measurement of HRCD would make the know-how generated by the international scientific community accessible to a broader group of local health workers. Overall, health practice at a global scale would benefit from the incentive of getting involved in research derived from measuring the impact of it on improving competencies. Thus positive outcomes of measuring HRCD could place the issue of universal transferability, and applicability of research methodology and results at a higher level of priority in the design of health research projects.

Limitations of the systematic review

Methodological limitations are recognised for this systematic review. First, there is a lack of clarity on a common definition and terminology to identify HRCD which complicates the search strategy. A long reiteration process was necessary when developing the search strategies for the databases to try and include all the possible variants used to define ‘tool’, ‘capacities’ and ‘development’. Despite this effort, some studies may have been missed. Second, there was a lack of studies which referenced a standard reporting procedure, despite the presence of standards available for reporting qualitative or quantitative research 43–45 as well as for mixed methods research. 46 Other limitations typical for reviews may also apply. Third, while this review has attempted to be as comprehensive as possible, some sources might not have been detected due to the challenge in finding all the relevant grey literature, and the restriction to English language sources only. Finally, it was not possible to analyse the psychometric aspects of each identified tool due to inconsistent reporting.

Conclusions

Sixteen studies using or describing tools to measure HRCD were identified and analysed in this systematic review. 19–34 Identifying capacity development with pre/postintervention evaluations or to generically evaluate capacity development without using a tool was common. There is a need for a clear distinction between simply measuring training activity outcomes in healthcare workers and effective action promoting HRCD for healthcare workers.

The most recurrent tools described were the Research Capacity and Culture tool 20 22 24 34 and the ‘Research Spider’ tool. 28 29 A variety of other tools, mostly questionnaire based, were identified, and in most cases, a broader applicability than described in the specific context of the paper may be possible. Two frameworks systematising research core competencies were identified. 23 31 The potential of tools derived from these frameworks could be significant. The applicability of each tool depends on the context and on the level of accuracy needed. Such tools could be routinely incorporated into standard personal development reviews in order to consistently support capacity development in research studies and organisations.

Future directions for HRCD include the design of a standardised, comparable and consistent tool to measure individual HRCD not linked to training evaluation, but support a long-term research competencies acquisition strategy. In addition, the harmonisation of definitions and terminologies used in identifying HRCD actions and processes could facilitate standardisation and comparability of HRCD strategies.

Ethics statements

Patient consent for publication.

Not required.

Acknowledgments

Authors are sincerely thankful for the immense and competent support of Elinor Harriss, Librarian of the Bodleian Health Care Libraries; Rebekah Burrow who instructed on the different steps and tool needed to perform the present systematic review; and Filippo Bianchi, first DPhil colleague who provided the basic knowledge on systematic reviews.

• Franzen SRP ,
• Chandler C ,
• Research GFfH, editor
• Hoelscher M , et al
• Laabes EP ,
• Zawedde SM , et al
• Lusthaus C ,
• Adrien M-H ,
• Perstinger M
• Meyers DC , et al
• Lansang MA ,
• UN General Assembly
• Smith H , et al
• Bauer D , et al
• Liberati A , et al
• The Critical Appraisal Skills Programme UK
• Akerjordet K ,
• Severinsson E
• Alison JA ,
• Zafiropoulos B ,
• Ekeroma AJ ,
• Kenealy T ,
• Shulruf B , et al
• Golenko X , et al
• Furtado T ,
• Boggs L , et al
• Hughes I , et al
• Njelesani J ,
• Dacombe R ,
• Palmer T , et al
• Petersen M ,
• Raghavan R ,
• Farmer EA ,
• Sonstein SA ,
• Namenek Brouwer RJ ,
• Gluck W , et al
• Swenson-Britt E ,
• Torres GCS ,
• Estrada MG ,
• Sumile EFR , et al
• Williams C ,
• Miyazaki K ,
• Borkowski D , et al
• Siribaddana S
• Lyytikainen M
• Siribaddana S , et al
• Ijsselmuiden C ,
• Marais DL ,
• Becerra-Posada F , et al
• Reeder JC ,
• DeGruttola V ,
• Dixon D , et al
• Steinert Y ,
• Centeno A , et al
• Morgan S , et al
• Lipsey MW ,
• Frambach JM ,
• van der Vleuten CPM ,

Contributors DB and ER designed and conducted the systematic review. DB wrote the draft of the systematic review and revised it according to the commentaries of ER, SB and TL. DB provided the final version of the manuscript. ER critically reviewed the manuscript and substantially contributed to the final version of the manuscript. SB critically reviewed both the design of the systematic review and the manuscript, and was involved in the development of meaningful inclusion criteria. TL critically reviewed the design of the study, made important suggestions for improvement, critically reviewed the manuscript and substantially contributed to the final version of the manuscript. All authors approved the final version of the manuscript.

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

• National Institute of General Medical Sciences

Capacity Building

The Division for Research Capacity Building (DRCB)

DRCB administers the Institutional Development Award (IDeA) program that supports research, research workforce development, and research infrastructure improvements in states where levels of NIH research funding have historically been low. The division also oversees the Native American Research Centers for Health (NARCH) program, the Support for Research Excellence (SuRE) program, and the Science Education Partnership Awards (SEPA) program.

Search NIGMS Funding Opportunities by Keyword(s)

View Funding Opportunities

Division for research capacity building news.

​Latest Program News/Events

​  ​

• Biophysics, Biomedical Technology, and Computational Biosciences
• Genetics and Molecular, Cellular, and Developmental Biology
• Pharmacology, Physiology, and Biological Chemistry
• Research Capacity Building
• Training, Workforce Development, and Diversity
• Contacts by Research Area
• Funding Opportunities and Notices
• Post Award Information
• Submitting an Application
• NIH RePORTER
• Dashboard of TWD Funded Programs
• High School and Undergraduate
• Postbaccalaureate and Graduate Students
• Postdoctoral, Early Career, and Faculty
• Workforce Development
• Contact Information
• Division Structure and Programs
• Enhancing Diversity in Training Programs
• Evaluation Resources
• Laboratory Safety and Guidelines
• Training Resources
• Institutional Development Award (IDeA)
• Native American Research Centers for Health (NARCH)
• Science Education Partnership Awards (SEPA)
• Support for Research Excellence (SuRE)
• DRCB Staff Contacts
• Current NIGMS Funding Opportunities
• Parent Announcements for Investigator-Initiated Applications
• Research Project Grants (NIH Parent R01)
• Research With Activities Related to Diversity (ReWARD)
• Maximizing Investigators' Research Awards (MIRA)
• Instrumentation Grant Program for Resource-Limited Institutions (RLI-S10)
• Undergraduate-Focused Institutions
• Small Business Research
• Multidisciplinary Teams/Collaborative Research
• Technology Development
• Research Resources
• Clinical Studies and Trials
• Conferences and Scientific Meetings
• Administrative Supplements
• All Funding Opportunities
• Which Research Grant Is Right for Me?
• How to Apply
• Grant Application and Review Process
• NIGMS Funding Policies
• Post-Award Information
• Talking to NIH Staff About Your Application and Grant
• Considerations for Multiple Principal Investigator (MPI) Applications
• Attribution of NIH/NIGMS Support
• Message to NIGMS Investigators
• Research Using Human Subjects or Specimens
• Data Management and Sharing Plan Expectations for NIGMS Grantees
• STEM Teaching Resources
• Coloring Pages
• Educator's Corner
• Image & Video Gallery
• NIGMS-Supported Resources
• Protein Alphabet
• Biomedical Beat Blog
• Featured Topics
• Past Campaigns
• Past Releases and Announcements
• Media Coverage
• Feedback Loop Blog
• NIGMS Events
• Webinars for the NIGMS Training Community
• Face to Face with Program Directors
• Grant Writing Webinar Series for Institutions Building Research and Research Training Capacity
• Image and Video Gallery
• Director's Corner
• Organization and Staff
• Staff Directory
• Budget, Financial Management, and Congressional Material
• Strategic Plans
• Data Integration, Modeling, and Analytics
• Advisory Council
• Communications and Public Liaison Branch
• Job Vacancies
• Visitor Information

Connect With Us: Instagram Linkedin Subscriptions YouTube

• Your Privacy
• Accessibility
• Disclaimers
• HHS Vulnerability Disclosure
• U.S. Department of Health and Human Services
• National Institutes of Health: NIH...Turning Discovery Into Health®
• 45 Center Drive MSC 6200
• Bethesda, MD 20892-6200
• 301-496-7301

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMC Fam Pract

A framework to evaluate research capacity building in health care

1 Primary Care and Social Care Lead, Trent Research and Development Unit, formerly, Trent Focus Group, ICOSS Building, The University of Sheffield, 219 Portobello, Sheffield S1 4DP, UK

This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Building research capacity in health services has been recognised internationally as important in order to produce a sound evidence base for decision-making in policy and practice. Activities to increase research capacity for, within, and by practice include initiatives to support individuals and teams, organisations and networks. Little has been discussed or concluded about how to measure the effectiveness of research capacity building (RCB)

This article attempts to develop the debate on measuring RCB. It highlights that traditional outcomes of publications in peer reviewed journals and successful grant applications may be important outcomes to measure, but they may not address all the relevant issues to highlight progress, especially amongst novice researchers. They do not capture factors that contribute to developing an environment to support capacity development, or on measuring the usefulness or the 'social impact' of research, or on professional outcomes.

The paper suggests a framework for planning change and measuring progress, based on six principles of RCB, which have been generated through the analysis of the literature, policy documents, empirical studies, and the experience of one Research and Development Support Unit in the UK. These principles are that RCB should: develop skills and confidence, support linkages and partnerships, ensure the research is 'close to practice', develop appropriate dissemination, invest in infrastructure, and build elements of sustainability and continuity. It is suggested that each principle operates at individual, team, organisation and supra-organisational levels. Some criteria for measuring progress are also given.

This paper highlights the need to identify ways of measuring RCB. It points out the limitations of current measurements that exist in the literature, and proposes a framework for measuring progress, which may form the basis of comparison of RCB activities. In this way it could contribute to establishing the effectiveness of these interventions, and establishing a knowledge base to inform the science of RCB.

The need to develop a sound scientific research base to inform service planning and decision-making in health services is strongly supported in the literature [ 1 ], and policy [ 2 ]. However, the level of research activity and the ability to carry out research is limited in some areas of practice, resulting in a low evidence base in these areas. Primary Care, for example, has been identified as having a poor capacity for undertaking research [ 3 - 5 ], and certain professional groups, for example nursing and allied health professionals, lack research experience and skills [ 5 - 7 ]. Much of the literature and the limited research on research capacity building (RCB) has therefore focused on this area of practice, and these professional groups. Policy initiatives to build research capacity include support in developing research for practice, where research is conducted by academics to inform practice decision making, research within or through practice, which encompasses research being conducted in collaboration with academics and practice, and research by practice, where ideas are initiated and research is conducted by practitioners [ 3 , 8 ].

The interventions to increase research capacity for, within, and by practice incorporates initiatives to support individuals and teams, organisations and networks. Examples include fellowships, training schemes and bursaries, and the development of support infrastructures, for example, research practice networks [ 9 - 13 ]. In the UK, the National Coordinating Centre for Research Capacity Development has supported links with universities and practice through funding a number of Research and Development Support Units (RDSU) [ 14 ]which are based within universities, but whose purpose is to support new and established researchers who are based in the National Health Service (NHS). However, both policy advisers and researchers have highlighted a lack of evaluative frameworks to measure progress and build an understanding of what works[ 15 , 16 ].

This paper argues for a need to establish a framework for planning and measuring progress, and to initiate a debate about identifying what are appropriate outcomes for RCB, not simply to rely on things that are easy to measure. The suggested framework has been generated through analysis of the literature, using policy documents, position statements, a limited amount of empirical studies on evaluating research RCB, and the experience of one large RSDU based in the UK.

The Department of Health within the UK has adopted the definition of RCB as 'a process of individual and institutional development which leads to higher levels of skills and greater ability to perform useful research". (pp1321) [ 17 ]

Albert & Mickan cited the National Information Services in Australia [ 18 ] who define it as

" an approach to the development of sustainable skills, organizational structures, resources and commitment to health improvement in health and other sectors to multiply health gains many times over.'

RCB can therefore be seen as a means to an end, the end being 'useful' research that informs practice and leads to health gain, or an end in itself, emphasising developments in skills and structures enabling research to take place.

A framework for measuring capacity building should therefore be inclusive of both process and outcome measures [ 19 ], to capture changes in both the 'ends' and 'means'; it should measure the ultimate goals, but also measure the steps and mechanisms to achieve them. The notion of measuring RCB by both process and outcome measures is supported within the research networks literature [ 12 , 20 ], and capacity building in health more generally [ 19 , 21 ]. Some argue we should acknowledge 'process as outcome', particularly if capacity building is seen as an end in itself [ 21 ]. In this context process measures are 'surrogate' [ 12 ], or 'proxy' outcome measures[ 16 ]. Carter et al [ 16 ]stress caution in terms of using 'proxy' measures in the context of RCB, as there is currently little evidence to link process with outcome. They do not argue against the notion of collecting process data, but stress that evaluation work should examine the relationship of process to outcome. The proposed framework discussed in this paper suggests areas to consider for both process and outcome measurement.

The most commonly accepted outcomes for RCB cited in the literature includes traditional measures of high quality research including publications, conference presentations, successful grant applications, and qualifications obtained. Many evaluations of RCB have used these as outcomes [ 9 , 10 , 22 , 23 ]. Some argue that publications in peer reviewed journals are a tall order for the low research skills base in some areas of health care practice [ 5 ], and argue for an appropriate time frame to evaluate progress. Process measures in this context could measure progress more sensitively and quickly.

However, using traditional outcomes may not be the whole story in terms of measuring impact. Position statements suggest that the ultimate goal of research capacity building is one of health improvement [ 17 , 18 , 24 ]. In order for capacity building initiatives to address these issues, outcomes should also explore the direct impact on services and clients: what Smith [ 25 ]defines as the social impact of research.

There is a strong emphasis within the primary care literature that capacity building should enhance the ability of practitioners to build their research skills: to support the development of research 'by' and 'with' practice [ 3 , 26 ], and suggests 'added value' to develop such close links to practice. A framework to measure RCB should explore and try to unpack this 'added value', both in terms of professional outcomes,[ 10 ] which include increasing professional enthusiasm, and supporting the application of critical thinking, and the use of evidence in practice. Whilst doing research alongside practice is not the only way these skills and attitudes can be developed, it does seem to be an important impact of RCB that should be examined.

The notion of developing RCB close to practice does not necessarily mean that it is small scale just because it is close to the coal face. Obviously, in order for individuals and teams to build up a track record of experience their initial projects may justifiably be small scale, but as individual's progress, they may gain experience to be able to conduct large scale studies, still based on practice problems, working in partnership with others. Similarly networks can support large scale studies as their capacity and infrastructure is developed to accommodate them.

The framework

The framework is represented by Figure ​ Figure1. 1 . It has two dimensions

Research Capacity Building: A Framework for Evaluation.

• Four structural levels of development activity . These include individual, team, organisational, and the network or supra- organisational support level (networks and support units). These are represented by the concentric circles within the diagram.

• Six principles of capacity building . This are discussed in more detail below but include: building skills and confidence, developing linkages and partnerships, ensuring the research is 'close to practice', developing appropriate dissemination, investments in infrastructure, and building elements of sustainability and continuity. Each principle is represented by an arrow within the diagram, which indicates activities and processes that contribute towards capacity building. The arrows cut across the structural levels suggesting that activities and interventions may occur within, and across, structural levels. The arrow heads point in both directions suggesting that principles applied to each structural level could have an impact on other levels.

The framework acknowledges that capacity building is conducted within a policy context. Whilst this paper focuses on measurement at different structural levels, it should be acknowledged that progress and impact on RCB can be greatly nurtured or restricted by the prevailing policy. Policy decisions will influence opportunities for developing researchers, can facilitate collaborations in research, support research careers, fund research directed by practice priorities, and can influence the sustainability and the very existence of supportive infrastructures such as research networks.

The paper will explain the rationale for the dimensions of the framework, and then will suggest some examples of measurement criteria for each principle at different structural levels to evaluate RCB. It is hope that as the framework is applied, further criteria will be developed, and then used taking into account time constraints, resources, and the purpose of such evaluations.

Structural levels at which capacity building takes place

The literature strongly supports that RCB should take place at an individual and organisational level [ 8 , 15 , 27 , 28 ]. For example, the conceptual model for RCB in primary care put forward by Farmer & Weston [ 15 ] focuses particularly on individual General Practitioners (GPs) and primary care practitioners who may progress from non participation through participation, to become academic leaders in research. Their model also acknowledges the context and organisational infrastructure to support RCB by reducing barriers and accommodating diversity through providing mentorship, collaborations and networking, and by adopting a whole systems approach based on local need and existing levels of capacity. Others have acknowledged that capacity development can be focussed at a team level [ 11 , 29 ]. Jowett et al [ 30 ] found that GPs were more likely to be research active if they were part of a practice where others were involved with research. Guidance from a number of national bodies highlights the need for multiprofessional and inter-professional involvement in conducting useful research for practice [ 3 , 4 , 6 , 31 ] which implies an appropriate mix of skills and practice experience within research teams to enable this [ 32 ]. Additionally, the organisational literature has identified the importance of teams in the production of knowledge [ 18 , 33 , 34 ].

Developing structures between and outside health organisations, including the development of research networks seems important for capacity building [ 12 , 24 , 34 ]. The Department of Health in the UK [ 14 ] categorizes this supra-organisational support infrastructure to include centres of academic activity, Research & Development Support Units, and research networks.

As interventions for RCB are targeted at different levels, the framework for measuring its effectiveness mirrors this. However, these levels should not be measured in isolation. One level can have an impact on capacity development at another level, and could potentially have a synergistic or detrimental effect on the other.

The six principles of research capacity building

Evaluation involves assessing the success of an intervention against a set of indicators or criteria [ 35 , 36 ], which Meyrick and Sinkler [ 37 ] suggest should be based on underlying principles in relation to the initiative. For this reason the framework includes six principles of capacity building. The rationale for each principle is given below, along with a description of some suggested criteria for each principle. The criteria presented are not an exhaustive list. As the framework is developed and used in practice, a body of criteria will be developed and built on further.

Principle 1. Research capacity is built by developing appropriate skills, and confidence, through training and creating opportunities to apply skills

The need to develop research skills in practitioners is well established [ 3 , 4 , 6 ], and can be supported through training [ 14 , 26 ], and through mentorship and supervision [ 15 , 24 , 28 ]. There is some empirical evidence that research skill development increases research activity [ 23 , 38 ], and enhances positive attitudes towards conducting and collaborating in research [ 39 ]. Other studies cite lack of training and research skills as a barrier to doing research [ 30 , 31 ]. The need to apply and use research skills in practice is highlighted in order to build confidence [ 40 ]and to consolidate learning.

Some needs assessment studies highlight that research skills development should adopt 'outreach' and flexible learning packages and acknowledge the skills, background and epistemologies of the professional groups concerned [ 7 , 15 , 39 , 41 , 42 ]. These include doctors, nurses, a range of allied health professional and social workers. Developing an appropriate mix of professionals to support health services research means that training should be inclusive and appropriate to them, and adopt a range of methodologies and examples to support appropriate learning and experience [ 15 , 31 , 41 ]. How learning and teaching is undertaken, and the content of support programmes to reflect the backgrounds, tasks and skills of participants should therefore be measured. For example, the type of research methods teaching offered by networks and support units should reflect a range and balance of skills needed for health service research, including both qualitative and quantitative research methods.

Skills development also should be set in the context of career development, and further opportunities to apply skills to practice should be examined. Policy and position statements [ 14 , 26 ] support the concept of career progression or 'careers escalator', which also enables the sustainability of skills. Opportunities to apply research skills through applications for funding is also important [ 9 , 10 , 22 , 43 , 44 ].

At team and network level Fenton et al [ 34 ]suggest that capacity can be increased through building intellectual capacity (sharing knowledge), which enhances the ability to do research. Whilst there is no formal measure for this, an audit of the transfer of knowledge would appear to be beneficial. For example teams may share expertise within a project to build skills in novice researchers [ 45 ]which can be tracked, and appropriate divisions of workload through reading research literature and sharing this with the rest of the team/network could be noted.

The notion of stepping outside of a safety zone may also suggest increased confidence and ability to do research. This may be illustrated at an individual level by the practitioner-researcher taking on more of a management role, supervising others, or tackling new methodologies/approaches in research, or in working with other groups of health and research professionals on research projects. This approach is supported by the model of RCB suggested by Farmer and Weston [ 15 ] which supports progress from participation through to academic leadership.

Some examples of criteria for measuring skills and confidence levels are give in table ​ table1 1 .

Building skills and confidence

Principle 2. Research capacity building should support research 'close to practice' in order for it to be useful

The underlying philosophy for developing research capacity in health is that it should generate research that is useful for practice. The North American Primary Care Group [ 24 ] defined the 'ultimate goal' of research capacity development as the generation and application of new knowledge to improve the health of individuals and families (p679). There is strong support that 'useful' research is that which is conducted 'close' to practice for two reasons. Firstly by generating research knowledge that is relevant to service user and practice concerns. Many argue that the most relevant and useful research questions are those generated by, or in consultation with, practitioners and services [ 3 , 11 , 24 ], policy makers [ 46 ] and service users [ 47 , 48 ]. The level of 'immediate' usefulness [ 49 ] may also mean that messages are more likely to taken up in practice[ 50 ]. Empirical evidence suggests that practitioners and policy makers are more likely to engage in research if they see its relevance to their own decision making [ 31 , 39 , 46 ]. The notion of building research that is 'close to practice' does not necessarily mean that they are small scale, but that the research is highly relevant to practice or policy concerns. A large network of practitioners could facilitate large scale, experimental based projects for example. However, the adoption of certain methodologies is more favoured by practice because of their potential immediate impact on practice [ 47 ] and this framework acknowledges such approaches and their relevance. This includes action research projects, and participatory inquiry [ 31 , 42 ]. An example where this more participatory approach has been developed in capacity building is the WeLREN (West London Research Network) cycle [ 51 ]. Here research projects are developed in cycles of action, reflection, and dissemination, and use of findings is integral to the process. This network reports high levels of practitioner involvement.

Secondly, building research capacity 'close to practice' is useful because of the skills of critical thinking it engenders which can be applied also to practice decision making [ 28 ], and which supports quality improvement approaches in organisations [ 8 ]. Practitioners in a local bursary scheme, for example, said they were more able to take an evidence-based approach into their every day practice [ 9 ].

Developing a 'research culture' within organisations suggests a closeness to practice that impacts on the ability of teams and individuals to do research. Lester et al [ 23 ] touched on measuring this idea through a questionnaire where they explored aspects of a supportive culture within primary care academic departments. This included aspects around exploring opportunities to discuss career progression, supervision, formal appraisal, mentorship, and junior support groups. This may be a fruitful idea to expand further to develop a tool in relation to a health care environment.

Some examples of criteria for measuring the close to practice principle are give in table ​ table2 2

Close to practice

3. Linkages, partnerships and collaborations enhance research capacity building

The notion of building partnerships and collaborations is integral to capacity building [ 19 , 24 ]. It is the mechanism by which research skills, and practice knowledge is exchanged, developed and enhanced [ 12 ], and research activity conducted to address complex health problems [ 4 ]. The linkages between the practice worlds and that of academia may also enhance research use and impact [ 46 ].

The linkages that enhance RCB can exist between

• Universities and practice [ 4 , 14 , 43 ]

• Novice and experienced researchers [ 22 , 24 , 51 ].

• Different professional groups [ 2 , 4 , 20 , 34 ]

• Different health and care provider sectors [ 4 , 31 , 47 , 52 ]

• Service users, practitioners and researchers [ 47 , 48 ]

• Researchers and policy makers [ 46 ]

• Different countries [ 28 , 52 ]

• Health and industry [ 53 , 54 ]

It is suggested that it is through networking and building partnerships that intellectual capital (knowledge) and social capital (relationships) can be built, which enhances the ability to do research [ 12 , 31 , 34 ]. In particular, there is the notion that the build up of trust between different groups and individuals can enhance information and knowledge exchange[ 12 ]. This may not only have benefits for the development of appropriate research ideas, but may also have benefits for the whole of the research process including the impact of research findings.

The notion of building links with industry is becoming progressively evident within policy in the UK [ 54 ] which may impact on economic outcomes to health organisations and the society as a whole[ 55 , 56 ].

Some examples of criteria for measuring linkages and collaborations are given in table ​ table3 3 .

Linkages, collaborations and partnerships.

4. Research capacity building should ensure appropriate dissemination to maximize impact

A widely accepted measure to illustrate the impact of RCB is the dissemination of research in peer reviewed publications, and through conference presentations to academic and practice communities [ 5 , 12 , 26 , 57 ]. However this principle extends beyond this more traditional method of dissemination. The litmus test that ultimately determines the success of capacity building is that it should impact on practice, and on the health of patients and comminutes[ 24 ] that is; the social impact of research [ 25 ]. Smith [ 25 ]argues that the strategies of dissemination should include a range of methods that are 'fit for purpose'. This includes traditional dissemination, but also includes other methods, for example, instruments and programmes of care implementation, protocols, lay publications, and publicity through factsheets, the media and the Internet.

Dissemination and tracking use of products and technologies arising from RCB should also be considered, which relate to economic outcomes of capacity building [ 55 ]. In the UK, the notion of building health trusts as innovative organisations which can benefit economically through building intellectual property highlights this as an area for potential measurement [ 56 ].

Some examples of criteria for measuring appropriate dissemination are given in table ​ table4 4

Appropriate dissemination and impact

5. Research capacity building should include elements of continuity and sustainability

Definitions of capacity building suggest that it should contain elements of sustainability which alludes to the maintenance and continuity of newly acquired skills and structures to undertake research [ 18 , 19 ]. However the literature does not explore this concept well [ 19 ]. This in itself may be partly due problems around measuring capacity building. It is difficult to know how well an initiative is progressing, and how well progress is consolidated, if there are no benchmarks or outcomes against which to demonstrate this.

Crisp et al [ 19 ] suggests that capacity can be sustained by applying skills to practice. This gives us some insight about where we might look for measures of sustainability. It could include enabling opportunities to extend skills and experience, and may link into the concept of a career escalator. It also involves utilizing the capacity that has been already built. For example engaging with those who have gained skills in earlier RCB initiatives to help more novice researchers, once they have become 'experts', and in finding an appropriate place to position the person with expertise with the organisation. It could also be measured by the number of opportunities for funding for continued application of skills to research practice.

Some examples of criteria for measuring sustainability and continuity are gibe in table ​ table5 5

Continuity and sustainability

6. Appropriate infrastructures enhance research capacity building

Infrastructure includes structures and processes that are set up to enable the smooth and effective running of research projects. For example, project management skills are essential to enable projects to move forward, and as such should be measured in relation to capacity building. Similarly, projects should be suitably supervised with academic and management support. To make research work 'legitimate' it may be beneficial to make research a part of some job descriptions for certain positions, not only to reinforce research as a core skill and activity, but also to review in annual appraisals, which can be a tool for research capacity evaluation. Information flow about calls for funding and fellowships and conferences is also important. Hurst [ 42 ] found that information flow varied between trusts, and managers were more aware of research information than practitioners.

The importance of protected time and backfill arrangements as well as funding to support this, is an important principle to enable capacity building [9, 15, 24, 58]. Such arrangements may reduce barriers to participation and enable skills and enthusiasm to be developed[ 15 ]. Infrastructure to help direct new practitioners to research support has also been highlighted[ 14 ]. This is particularly true in the light of the new research governance and research ethics framework in the UK [59]. The reality of implementing systems to deal with the complexities of the research governance regulations has proved problematic, particularly in primary care, where the relative lack of research management expertise and infrastructure has resulted in what are perceived as disproportionately bureaucratic systems. Recent discussion in the literature has focused on the detrimental impact of both ethical review, and NHS approval systems, and there is evidence of serious delays in getting research projects started [60]. Administrative and support staff to help researchers through this process is important to enable research to take place [61].

Some examples of criteria for measuring are given in table ​ table6 6 .

Infrastructure

This paper suggests a framework which sets out a tentative structure by which to start measuring the impact of capacity building interventions, and invites debate around the application of this framework to plan and measure progress. It highlights that interventions can focus on individuals, teams, organisations, and through support infrastructures like RDSUs and research networks. However, capacity building may only take place once change has occurred at more than one level: for example, the culture of an organisation in which teams and individuals work may have an influence of their abilities and opportunities to do research work. It is also possible that the interplay between different levels may have an effect on the outcomes at other levels. In measuring progress, it should be possible to determine a greater understanding of the relationship between different levels. The framework proposed in this paper may be the first step to doing this.

The notion of building capacity at any structural level is dependent on funding and support opportunities, which are influenced by policy and funding bodies. The ability to build capacity across the principles developed in the framework will also be dependent of R&D strategy and policy decisions. For example, if policy fluctuates in its emphasis on building capacity 'by', 'for' or 'with' practice, the ability to build capacity close to practice will be affected.

In terms of developing a science of RCB, there is a need to capture further information on issues of measuring process and outcome data to understand what helps develop 'useful' and 'useable' research. The paper suggests principles whereby a number of indicators could be developed. The list is not exhaustive, and it is hoped that through debate and application of the framework further indicators will be developed.

An important first step to building the science of RCB should be debate about identifying appropriate outcomes. This paper supports the use of traditional outcomes of measurement, including publications in peer reviewed journals and conference presentations. This assures quality, and engages critical review and debate. However, the paper also suggests that we might move on from these outcomes in order to capture the social impact of research, and supports the notion of developing outcomes which measure how research has had an impact on the quality of services, and on the lives of patients and communities. This includes adopting and shaping the type of methodologies that capacity building interventions support, which includes incorporating patient centred outcomes in research designs, highlighting issues such as cost effectiveness of interventions, exploring economic impact of research both in terms of product outputs and health gain, and in developing action oriented, and user involvement methodologies that describe and demonstrate impact. It also may mean that we have to track the types of linkages and collaborations that are built through RCB, as linkages that are close to practice, including those with policy makers and practitioners, may enhance research use and therefore 'usefulness'. If we are to measure progress through impact and change in practice, an appropriate time frame would have to be established alongside these measures.

This paper argues that 'professional outcomes' should also be measured, to recognize how critical thinking developed during research impacts on clinical practice more generally.

Finally, the proposed framework provides the basis by which we can build a body of evidence to link process to the outcomes of capacity building. By gathering process data and linking it to appropriate outcomes, we can more clearly unpack the 'black box' of process, and investigate which processes link to desired outcomes. It is through adopting such a framework, and testing out these measurements, that we can systematically build a body of knowledge that will inform the science and the art of capacity building in health care.

• There is currently little evidence on how to plan and measure progress in research capacity building (RCB), or agreement to determining its ultimate outcomes.

• Traditional outcomes of publications in peer reviewed journals, and successful grant applications may be the easy and important outcomes to measure, but do not necessarily address issues to do with the usefulness of research, professional outcomes, the impact of research activity on practice, or on measuring health gain.

• The paper suggests a framework which provides a tentative structure by which measuring the impact of RCB could be achieved, shaped around six principles of research capacity building, and includes four structural levels on which each principle can be applied.

• The framework could be the basis by which RCB interventions could be planned, and progress measured. It could act as a basis of comparison across interventions, and could contribute to establishing a knowledge base on what is effective in RCB in healthcare

Competing interests

The author(s) declare that they have no competing interests.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2296/6/44/prepub

Acknowledgements

My warm thanks go to my colleagues in the primary care group of the Trent RDSU for reading and commenting on earlier drafts of this paper, and for their continued support in practice.

• Muir Gray JA. Evidence-based Healthcare. How to make health policy and management decisions. Edinburgh, Churchill Livingstone; 1997. [ Google Scholar ]
• Department of Health . Research and Development for a First Class Service. Leeds, DoH; 2000. [ Google Scholar ]
• Mant D. National working party on R&D in primary care. Final Report. London, NHSE South and West.; 1997. [ Google Scholar ]
• Department of Health . Strategic review of the NHS R&D Levy (The Clarke Report) , Central Research Department, Department of Health; 1999. p. 11. [ Google Scholar ]
• Campbell SM, Roland M, Bentley E, Dowell J, Hassall K, Pooley J, Price H. Research capacity in UK primary care. British Journal of General Practice. 1999; 49 :967–970. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Department of Health . Towards a strategy for nursing research and development. London, Department of Health; 2000. [ Google Scholar ]
• Ross F, Vernon S, Smith E. Mapping research in primary care nursing: Current activity and future priorities. NT Research. 2002; 7 :46–59. [ Google Scholar ]
• Marks L, Godfrey M. Developing Research Capacity within the NHS: A summary of the evidence. Leeds, Nuffield Portfolio Programme Report.; 2000. [ Google Scholar ]
• Lee M, Saunders K. Oak trees from acorns? An evaluation of local bursaries in primary care. Primary Health Care Research and Development. 2004; 5 :93–95. doi: 10.1191/1463423604pc197xx. [ CrossRef ] [ Google Scholar ]
• Bateman H, Walter F, Elliott J. What happens next? Evaluation of a scheme to support primary care practitioners with a fledgling interest in research. Family Practice. 2004; 21 :83–86. doi: 10.1093/fampra/cmh118. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Smith LFP. Research general practices: what, who and why? British Journal of General Practice. 1997; 47 :83–86. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Griffiths F, Wild A, Harvey J, Fenton E. The productivity of primary care research networks. British Journal of General Practice. 2000; 50 :913–915. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Fenton F, Harvey J, Griffiths F, Wild A, Sturt J. Reflections from organization science of primary health care networks. Family Practice. 2001; 18 :540–544. doi: 10.1093/fampra/18.5.540. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Department of Health . Research Capacity Development Strategy. London, Department of Health; 2004. [ Google Scholar ]
• Farmer E, Weston K. A conceptual model for capacity building in Australian primary health care research. Australian Family Physician. 2002; 31 :1139–1142. [ PubMed ] [ Google Scholar ]
• Carter YH, Shaw S, Sibbald B. Primary care research networks: an evolving model meriting national evaluation. British Journal of General Practice. 2000; 50 :859–860. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Trostle J. Research Capacity building and international health: Definitions, evaluations and strategies for success. Social Science and Medicine. 1992; 35 :1321–1324. doi: 10.1016/0277-9536(92)90035-O. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Albert E, Mickan S. Closing the gap and widening the scope. New directions for research capacity building in primary health care. Australian Family Physician. 2002; 31 :1038 –10341. [ PubMed ] [ Google Scholar ]
• Crisp BR, Swerissen H, Duckett SJ. Four approaches to capacity building in health: consequences for measurement and accountability. Health Promotion International. 2000; 15 :99–107. doi: 10.1093/heapro/15.2.99. [ CrossRef ] [ Google Scholar ]
• Ryan , Wyke S. The evaluation of primary care research networks in Scotland. British Journal of General Practice. 2001. pp. 154–155.
• Gillies P. Effectiveness of alliances and partnerships for health promotion. Health Promotion International. 1998; 13 :99–120. doi: 10.1093/heapro/13.2.99. [ CrossRef ] [ Google Scholar ]
• Pitkethly M, Sullivan F. Four years of TayRen, a primary care research and development network. Primary Care Research and Development. 2003; 4 :279–283. doi: 10.1191/1463423603pc167oa. [ CrossRef ] [ Google Scholar ]
• Lester H, Carter YH, Dassu D, Hobbs F. Survey of research activity, training needs. departmental support, and career intentions of junior academic general practitioners. British Journal of General Practice. 1998; 48 :1322–1326. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• North American Primary Care Research Group What does it mean to build research capacity? Family Medicine. 2002; 34 :678–684. [ PubMed ] [ Google Scholar ]
• Smith R. Measuring the social impact of research. BMJ. 2001; 323 :528. doi: 10.1136/bmj.323.7312.528. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sarre G. Capacity and activity in research project (CARP): supporting R&D in primary care trusts. 2002.
• Del Mar C, Askew D. Building family/general practice research capacity. Annals of Family Medicine. 2004; 2 :535–540. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Carter YH, Shaw S, Macfarlane F. Primary Care research team assessment (PCRTA): development and evaluation. Occasional paper (Royal College of General Practitioners) 2002; 81 :1–72. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Jowett S, Macleod J, Wilson S, Hobbs F. Research in Primary Care: extent of involvement and perceived determinants among practitioners for one English region. British Journal of General Practice. 2000; 50 :387–389. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Cooke J, Owen J, Wilson A. Research and development at the health and social care interface in primary care: a scoping exercise in one National Health Service region. Health and Social Care in the Community. 2002; 10 :435 –4444. doi: 10.1046/j.1365-2524.2002.00395.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Raghunath AS, Innes A. The case of multidisciplinary research in primary care. Primary Care Research and Development. 2004; 5 :265–273. [ Google Scholar ]
• Reagans RZER. Networks, Diversity and Productivity: The Social Capital of Corporate R&D Teams. Organisational Science. 2001; 12 :502–517. doi: 10.1287/orsc.12.4.502.10637. [ CrossRef ] [ Google Scholar ]
• Ovretveit J. Evaluating Health Interventions. Buckingham, Open University; 1998. [ Google Scholar ]
• Meyrick J, Sinkler P. An evaluation Resource for Healthy Living Centres. London, Health Education Authority; 1999. [ Google Scholar ]
• Hakansson A, Henriksson K, Isacsson A. Research methods courses for GPs: ten years' experience in southern Sweden. British Journal of General Practice. 2000; 50 :811–812. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Bacigalupo B, Cooke J, Hawley M. Research activity, interest and skills in a health and social care setting: a snapshot of a primary care trust in Northern England. Health and Social Care in the Community.
• Kernick D. Evaluating primary care research networks - exposing a wider agenda. British Journal of General Practice. 2001; 51 :63. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Owen J, Cooke J. Developing research capacity and collaboration in primary care and social care: is there enough common ground? Qualitative Social Work. 2004; 3 :398–410. doi: 10.1177/1473325004048022. [ CrossRef ] [ Google Scholar ]
• Hurst Building a research conscious workforce. Journal of Health Organization and management. 2003; 17 :373–384. [ PubMed ] [ Google Scholar ]
• Gillibrand WP, Burton C, Watkins GG. Clinical networks for nursing research. International Nursing Review. 2002; 49 :188–193. doi: 10.1046/j.1466-7657.2002.00124.x. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Campbell J, Longo D. Building research capacity in family medicine: Evaluation of the Grant Generating Project. Journal of Family Practice. 2002; 51 :593. [ PubMed ] [ Google Scholar ]
• Cooke J, Nancarrow S, Hammersley V, Farndon L, Vernon W. The "Designated Research Team" approach to building research capacity in primary care. Primary Health Care Research and Development. [ PMC free article ] [ PubMed ]
• Innvaer S, Vist G, Trommald M, Oxman A. Health policy- makers' perceptions of their use of evidence: a systematic review. Journal of Health Services Research and Policy. 2002; 7 :239–244. doi: 10.1258/135581902320432778. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• NHS Service Delivery Organisation . NHS Service Delivery and Against National R&D programme, National listening exercise. London, NHS SDO; 2000. [ Google Scholar ]
• Hanley J, Bradburn S, Gorin M, Barnes M, Evans C, Goodare HB. Involving consumers in research and development in the NHS: briefing notes for researchers. Winchester, Consumers in NHS Research Support Unit,; 2000. [ Google Scholar ]
• Frenk J. Balancing relevance and excellence: organisational responses to link research with decision making. Social Science and Medicine. 1992; 35 :1397–1404. doi: 10.1016/0277-9536(92)90043-P. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• National Audit Office. An international review on Governments' research procurement strategies. A paper in support of Getting the evidence: Using research in policy making. London, The Stationary Office.; 2003. [ Google Scholar ]
• Thomas P, While A. Increasing research capacity and changing the culture of primary care towards reflective inquiring practice: the experience of West London Research Network (WeLReN) Journal of Interprofessional Care. 2001; 15 :133–139. doi: 10.1080/13561820120039865. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rowlands G, Crilly T, Ashworth M, Mager J, Johns C, Hilton S. Linking research and development in primary care: primary care trusts, primary care research networks and primary care academics. Primary Care Research and Development. 2004; 5 :255–263. doi: 10.1191/1463423604pc201oa. [ CrossRef ] [ Google Scholar ]
• Davies S. R&D for the NHS- Delivering the research agenda: ; London. National Coordinating Centre for Research Capacity Development; 2005. [ Google Scholar ]
• Department of Health. Best Research for Best Health: A New National Health Research Strategy. The NHS contribution to health research in England: A consultation. London, Department of Health; 2005. [ Google Scholar ]
• Buxton M, Hanney S, Jones T. Estimating the economic value to societies of the impact of health research: a critical review. Bulletin of the World Health Organisation. 2004; 82 :733–739. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Department of Health. The NHS as an innovative organisation: A framework and guidance on the management of intellectual property in the NHS. London, Department of Health; 2002. [ Google Scholar ]
• Sarre G. Trent Focus Supporting research and development in primary care organisations: report of the capacity and activity in research project (CARP) 2003.
• Department of Health . Research Governance Framework for Health and Social Care. London, Department of Health.; 2001. [ Google Scholar ]
• Hill J, Foster N, Hughes R, Hay E. Meeting the challenges of research governance. Rheumatology. 2005; 44 :571–572. doi: 10.1093/rheumatology/keh579. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Shaw S. Developing research management and governance capacity in primary care organizations: transferable learning from a qualitative evaluation of UK pilot sites. Family Practice. 2004; 21 :92–98. doi: 10.1093/fampra/cmh120. [ PubMed ] [ CrossRef ] [ Google Scholar ]

U.S. Government Accountability Office

Evidence-Based Policymaking: Agencies Need Additional Guidance to Assess Their Capacity

Federal decision-makers need evidence—such as data and the results of studies—to determine if programs are working as intended and to identify potential improvements.

As part of a government-wide push to improve the collection and use of evidence, agencies are required to assess their related efforts every 4 years.

Of the 23 agencies we reviewed, 21 said the assessment process helped them to better understand what they were doing to collect and use evidence, and their capacity to do so. But 18 agencies also had trouble interpreting guidance and doing their assessments.

Our recommendations are to help improve the assessment process.

What GAO Found

The Foundations for Evidence-Based Policymaking Act of 2018 (Evidence Act) requires agencies, every 4 years, to assess aspects of their statistics, evaluation, research, and analysis efforts by addressing five topics, such as quality and methods. Agencies published capacity assessments for the first time in 2022.

GAO identified 12 themes related to six topics across agencies' assessments.

Agency officials identified common benefits and challenges related to developing capacity assessments. Officials expect some challenges will not persist. For example, officials stated it was challenging to identify all evidence-building activities because they are dispersed around their agencies, but they have now developed a better understanding of where those activities occurred.

Agency officials also identified unresolved challenges:

• Guidance . Agency officials said they faced challenges understanding Office of Management and Budget (OMB) guidance and how the assessments would be used. GAO found variation in what agencies assessed and how they presented their findings. Additional guidance could help agencies provide more comparable information in future iterations. This could also help decision-makers identify and address common issues across agencies.
• Methods . Agency officials said it was a challenge to identify appropriate approaches for conducting the assessment. They used different methodologies to assess their evidence-building capacity, which sometimes did not result in useful information. Identifying, documenting, and sharing lessons learned could help agencies select appropriate methodologies to ensure future capacity assessments consistently provide useful information.

The interagency Evaluation Officer Council, chaired by OMB, has responsibilities for sharing information and helping agencies with Evidence Act implementation.

Why GAO Did This Study

Federal decision-makers need evidence about whether federal programs and activities are achieving intended results. The Evidence Act aims to enhance federal agencies' capacity to build and use evidence.

The Evidence Act includes provisions for GAO to report on findings and trends in agencies' capacity assessments. This report describes (1) common themes in agencies' capacity assessments, and (2) benefits and challenges related to conducting capacity assessments identified by agency officials.

To address these objectives, GAO conducted a content analysis of 23 agencies' capacity assessments. GAO also interviewed OMB and Evaluation Officer Council staff as well as officials at the 24 agencies directed by OMB to conduct capacity assessments—those covered by the Chief Financial Officers Act of 1990.

Recommendations

GAO recommends that OMB should leverage the Evaluation Officer Council to (1) identify agency officials' needs for additional guidance on capacity assessments and address them accordingly and (2) identify, document, and share lessons learned on capacity assessment methods. OMB neither agreed nor disagreed with the recommendations and stated that it would take them into consideration moving forward. The Departments of Agriculture and the Treasury, Nuclear Regulatory Commission, Social Security Administration, and U.S. Agency for International Development also provided comments. The remaining agencies did not comment.

Recommendations for Executive Action

Full Report

Gao contacts.

Dawn Locke Director [email protected] (202) 512-6806

Office of Public Affairs

Sarah Kaczmarek Acting Managing Director [email protected] (202) 512-4800

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

A framework to evaluate research capacity building in health care

2005, BMC family practice

Building research capacity in health services has been recognised internationally as important in order to produce a sound evidence base for decision-making in policy and practice. Activities to increase research capacity for, within, and by practice include initiatives to support individuals and teams, organisations and networks. Little has been discussed or concluded about how to measure the effectiveness of research capacity building (RCB) DISCUSSION: This article attempts to develop the debate on measuring RCB. It highlights that traditional outcomes of publications in peer reviewed journals and successful grant applications may be important outcomes to measure, but they may not address all the relevant issues to highlight progress, especially amongst novice researchers. They do not capture factors that contribute to developing an environment to support capacity development, or on measuring the usefulness or the &#39;social impact&#39; of research, or on professional outcomes. T...

Related Papers

PLoS Medicine

Daniel Ansong , George Bedu-addo , Tsiri Agbenyega

Kathryn Weston

Background The South Australian Research Network \u27SARNet\u27 aims to build research capacity in primary health care, as part of a national government-funded strategy to integrate research into clinical practice. Internationally, research networks have been a fundamental part of research culture change, and a variety of network models exist. The \u27SARNet\u27 model uses a whole system, multidisciplinary approach to capacity building and supports individuals and groups. We undertook a descriptive baseline survey in order to understand the background and needs of SARNet members and to tailor network activities towards those needs. Methods A questionnaire survey, assessing members\u27 professional background, research experience, and interest in research development and training, was sent to all members who joined the network in its first year. The visual \u27research spider\u27 tool was used to ascertain members\u27 experience in ten core research skills, as well as their interest ...

BMC Health Services Research

Ashlea Walker

Nova Southeastern University

Jennie Popay

Eulalia K Kahwa

Background The South Australian Research Network 'SARNet' aims to build research capacity in primary health care, as part of a national government-funded strategy to integrate research into clinical practice. Internationally, research networks have been a fundamental part of research culture change, and a variety of network models exist. The 'SARNet' model uses a whole system, multidisciplinary approach to capacity building and supports individuals and groups. We undertook a descriptive baseline survey in order to understand the background and needs of SARNet members and to tailor network activities towards those needs. Methods A questionnaire survey, assessing members' professional background, research experience, and interest in research development and training, was sent to all members who joined the network in its first year. The visual 'research spider' tool was used to ascertain members' experience in ten core research skills, as well as their interest in developing these skills. Individuals were asked to classify themselves into one of four categories of researchers, based on previous research experience. These self-assessment categories ranged from non-participant to academic. Results Network membership was diverse. Of the 89 survey participants, 55% were general practitioners or allied health professionals. Overall, most survey respondents indicated little to moderate experience in 7 out of the 10 skills depicted in the 'research spider'. In comparison, respondents were generally highly interested in developing their research skills in all areas. Respondents' research skills correlated significantly with their self-assessed category of research participation (Spearman rank correlation, r = 0.82, p < 0.0005). Correlations between research category and publication record (Gamma association, γ = 0.53, p < 0.0005) or funding record (Gamma association, γ = 0.62, p < 0.0005) supported the internal validity of the survey instrument. Conclusion Literature describing evaluation of the impact of networks is scarce. Our survey questionnaire could provide a useful instrument for evaluation of both networks and capacity building initiatives. The survey including the 'research spider' tool provided valuable information about members' needs and interest in strategies to develop their research skills. Initial needs analyses as well as on-going evaluation of network activities are important to include into the business plans of research networks, in order to ensure the network's effectiveness and support of its membership.

Internet Journal of Allied Health Sciences and Practice

Purpose: Strengthening research capacity (RC) amongst health professionals has both organisational and individual benefits. It can increase the quality of research and support the transfer of evidence into practice and policy. However there is little evidence on what works to develop and strengthen RC. This paper contributes to the evidence base by reporting findings from an evaluation of a programme that aimed to build capacity to use and do research amongst NHS and local authority organisations and their staff in a large english research partnership organisation. Methods: The evaluation used multiple qualitative methods including semi-structured interviews, focus groups and workshops (n=131 respondents including public advisers, university, NHS, and local government partners). Results: The RC building programme provided a range of development opportunities for NHS and local authority staff resulting in increased confidence and skills to undertake, participate in, and use research....

Asia-Pacific Journal of Health Management

rachel Wenke

Objectives: The objectives of this project were firstly to develop a practical toolkit of evidence-informed strategies for building research capacity in allied health, and secondly to disseminate and apply this toolkit to inform tailored research capacity building plans for allied health teams. Design: This project used a plan, do, study, act (PDSA) service quality improvement methodology to develop, disseminate and apply a toolkit which was based on the results of a recent systematic review of allied health research capacity building frameworks and a narrative review of other interventions and theoretical recommendations. Setting: Eight allied health professional teams in a publicly funded tertiary health service were supported to develop tailored research capacity building plans based on their specific needs, goals and context. Main outcome measures: The outcomes of this project were evaluated using process measures including whether a research capacity building plan was develo...

Elizabeth Farmer , Kathryn Weston

Background: Many general practitioners and primary health care practitioners lack research and evaluation skills. In response, the Australian Government has funded important capacity building initiatives. Aim: To propose a conceptual model to assist these initiatives. Model: Four groups of research involvement are suggested: nonparticipants; participating (as part of a research team); managing/training (either leading research, or in formal training to

Australian Journal of Primary Health

Emma L Friesen

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Susan Nancarrow

BMC Family Practice

Martin Williams

Asia Pacific Journal of Health Management

Rachael Beldham-Collins

Journal of Research in Nursing

Annie Topping

Idongesit Ibanga Akpabio

Libby Holden

Sphindile Magwaza

Australian Journal of Rural Health

Giuliana Fuscaldo

Journal of Nursing Management

Evidence &# 38; Policy: A Journal of Research, Debate and Practice

Wesley Vernon

Africa Journal of Nursing and Midwifery

Gisela van Rensburg

Australia and New Zealand health policy

Stuart Peacock

International Journal for Equity in Health

David Maclaren

BMJ Global Health

Nadia Tagoe

Health Policy and Planning

Donald C Cole

Johanna Huber , Thembisa Abby T Sigauke

Global Public Health

Lê Nữ Quỳnh Giang

Anne Whitworth

Journal of Health Services Research & Policy

Lisa Simpson

International Journal of Environmental Research and Public Health

Yuwadee Wittayapun

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

A framework to evaluate research capacity building in health care

Affiliation.

• 1 Primary Care and Social Care Lead, Trent Research and Development Unit, The University of Sheffield, 219 Portobello, Sheffield S1 4DP, UK. [email protected]
• PMID: 16253133
• PMCID: PMC1289281
• DOI: 10.1186/1471-2296-6-44

Background: Building research capacity in health services has been recognised internationally as important in order to produce a sound evidence base for decision-making in policy and practice. Activities to increase research capacity for, within, and by practice include initiatives to support individuals and teams, organisations and networks. Little has been discussed or concluded about how to measure the effectiveness of research capacity building (RCB) DISCUSSION: This article attempts to develop the debate on measuring RCB. It highlights that traditional outcomes of publications in peer reviewed journals and successful grant applications may be important outcomes to measure, but they may not address all the relevant issues to highlight progress, especially amongst novice researchers. They do not capture factors that contribute to developing an environment to support capacity development, or on measuring the usefulness or the 'social impact' of research, or on professional outcomes. The paper suggests a framework for planning change and measuring progress, based on six principles of RCB, which have been generated through the analysis of the literature, policy documents, empirical studies, and the experience of one Research and Development Support Unit in the UK. These principles are that RCB should: develop skills and confidence, support linkages and partnerships, ensure the research is 'close to practice', develop appropriate dissemination, invest in infrastructure, and build elements of sustainability and continuity. It is suggested that each principle operates at individual, team, organisation and supra-organisational levels. Some criteria for measuring progress are also given.

Summary: This paper highlights the need to identify ways of measuring RCB. It points out the limitations of current measurements that exist in the literature, and proposes a framework for measuring progress, which may form the basis of comparison of RCB activities. In this way it could contribute to establishing the effectiveness of these interventions, and establishing a knowledge base to inform the science of RCB.

PubMed Disclaimer

Research Capacity Building: A Framework…

Research Capacity Building: A Framework for Evaluation.

Similar articles

• An evaluation of the 'Designated Research Team' approach to building research capacity in primary care. Cooke J, Nancarrow S, Dyas J, Williams M. Cooke J, et al. BMC Fam Pract. 2008 Jun 27;9:37. doi: 10.1186/1471-2296-9-37. BMC Fam Pract. 2008. PMID: 18588685 Free PMC article.
• Primary Care Research Team Assessment (PCRTA): development and evaluation. Carter YH, Shaw S, Macfarlane F. Carter YH, et al. Occas Pap R Coll Gen Pract. 2002 Feb;(81):iii-vi, 1-72. Occas Pap R Coll Gen Pract. 2002. PMID: 12049028 Free PMC article.
• A thematic analysis of the role of the organisation in building allied health research capacity: a senior managers' perspective. Golenko X, Pager S, Holden L. Golenko X, et al. BMC Health Serv Res. 2012 Aug 27;12:276. doi: 10.1186/1472-6963-12-276. BMC Health Serv Res. 2012. PMID: 22920443 Free PMC article.
• New ways of working in mental health services: a qualitative, comparative case study assessing and informing the emergence of new peer worker roles in mental health services in England. Gillard S, Edwards C, Gibson S, Holley J, Owen K. Gillard S, et al. Southampton (UK): NIHR Journals Library; 2014 Jul. Southampton (UK): NIHR Journals Library; 2014 Jul. PMID: 27466663 Free Books & Documents. Review.
• Evidence Brief: The Effectiveness Of Mandatory Computer-Based Trainings On Government Ethics, Workplace Harassment, Or Privacy And Information Security-Related Topics [Internet]. Peterson K, McCleery E. Peterson K, et al. Washington (DC): Department of Veterans Affairs (US); 2014 May. Washington (DC): Department of Veterans Affairs (US); 2014 May. PMID: 27606391 Free Books & Documents. Review.
• Capacity building in migration and health in higher education: lessons from five European countries. Diaz E, Gimeno-Feliu LA, Czapka E, Suurmond J, Razum O, Kumar BN. Diaz E, et al. Lancet Reg Health Eur. 2024 May 28;41:100818. doi: 10.1016/j.lanepe.2023.100818. eCollection 2024 Jun. Lancet Reg Health Eur. 2024. PMID: 39119097 Free PMC article. Review.
• Benchmarking Drug Regulatory Systems for Capacity Building: An Integrative Review of Tools, Practice, and Recommendations. Shi J, Chen X, Hu H, Ung COL. Shi J, et al. Int J Health Policy Manag. 2023;12:8100. doi: 10.34172/ijhpm.2023.8100. Epub 2023 Oct 16. Int J Health Policy Manag. 2023. PMID: 38618782 Free PMC article. Review.
• Models and approaches for building knowledge translation capacity and capability in health services: a scoping review. King O, West E, Alston L, Beks H, Callisaya M, Huggins CE, Murray M, Mc Namara K, Pang M, Payne W, Peeters A, Pithie M, Sayner AM, Wong Shee A. King O, et al. Implement Sci. 2024 Jan 29;19(1):7. doi: 10.1186/s13012-024-01336-0. Implement Sci. 2024. PMID: 38287351 Free PMC article. Review.
• Investigating the research capacity and productivity of Canadian sports chiropractors. Lee AD, Muir BJ, Oh D, Chung K, Debusschere R, Kissel J, Richer N, Poulin C, Murnaghan K, Stuber K. Lee AD, et al. J Can Chiropr Assoc. 2023 Dec;67(3):202-225. J Can Chiropr Assoc. 2023. PMID: 38283160 Free PMC article.
• Research translation mentoring for emerging clinician researchers in rural and regional health settings: a qualitative study. King OA, Sayner AM, Beauchamp A, West E, Aras D, Hitch D, Wong Shee A. King OA, et al. BMC Med Educ. 2023 Oct 31;23(1):817. doi: 10.1186/s12909-023-04786-0. BMC Med Educ. 2023. PMID: 37907938 Free PMC article.
• Muir Gray JA. Evidence-based Healthcare. How to make health policy and management decisions. Edinburgh, Churchill Livingstone; 1997.
• Department of Health . Research and Development for a First Class Service. Leeds, DoH; 2000.
• Mant D. National working party on R&D in primary care. Final Report. London, NHSE South and West.; 1997.
• Department of Health . Strategic review of the NHS R&D Levy (The Clarke Report) , Central Research Department, Department of Health; 1999. p. 11.
• Campbell SM, Roland M, Bentley E, Dowell J, Hassall K, Pooley J, Price H. Research capacity in UK primary care. British Journal of General Practice. 1999;49:967–970. - PMC - PubMed
• Search in MeSH

Related information

• Cited in Books

LinkOut - more resources

Full text sources.

• BioMed Central
• Europe PubMed Central
• PubMed Central
• White Rose Research Online

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Cisco Blogs / High Tech Policy / Why the UN Convention Against Cybercrime Requires a Second Look

High Tech Policy

Why the un convention against cybercrime requires a second look, eric wenger.

Cybercrime is escalating globally. Criminal groups are leveraging advanced technology to operate across borders, necessitating that law enforcement agencies have the capabilities to prevent, investigate, and prosecute these crimes while also protecting human rights. Effective international cooperation is essential to combat these threats and uphold shared values. While the recently adopted UN Convention against Cybercrime addresses important issues around the need for increased cyber capacity building, it raises concerns around the protection of shared rights and values, and requires more attention before ratification by member states.

Concerns for human rights and liberal democracies values in the UN Convention against Cybercrime

Given the power of networks to transform how we live, work, learn, and play, it is natural that criminal groups also use more advances in technology to operate more efficiently than in the past without regard for national borders. In response, we need to ensure law enforcement agencies have the necessary capabilities to prevent, investigate, and prosecute transnational cybercrimes. We must also uphold and protect the importance of basic human rights and the rule of law.

Unfortunately, the UN Convention, as it stands, does not sufficiently protect basic human rights and poses risks to the rule of law.

Rather than specifically focusing on hacking and cybercrimes, it broadly aims at the misuse of computer networks to disseminate objectionable information. This represents a misalignment with the values of free speech in liberal democracies, which should be addressed via an amendment before the Convention is taken up by member states for adoption.

As providers of foundational products and services vital to economic growth and stability, technology leaders like Cisco have our own responsibilities to drive adoption of concrete practices that improve the safety and security of the shared network we increasingly rely upon. We share concerns about how best to enable governments, law enforcement, and national security officials to protect their citizens against crime and terror even as the necessary information requires the assistance of other governments. At the same time, our societies must effectively balance the legitimate needs of governments to pursue cybercriminals across borders with our shared values and long-standing commitments.

More alignment with existing international law enforcement frameworks needed

We should also not forget the importance of the original cybercrime agreement. The Council of Europe Cybercrime Treaty, sometimes referred to as the Budapest Convention , has been around for 20-plus years. This existing agreement should be more widely adopted, and it is regrettable that the new UN agreement does not align more tightly with existing international law enforcement frameworks, which reflect carefully negotiated balances between competing equities.

For example, the Budapest Convention ensures that signatory nations have a 24/7 point of contact that can be reached in the event of an emergency. The agreement serves as a default framework for cross-border cooperation where there is no existing mutual or bilateral legal assistance treaty. Importantly it allowed the US government to not sign on to parts of it that would have required the prosecution of crimes that would violate the First Amendment of the US Constitution. Even if we need a separate UN treaty, it should parallel previously agreed norms of governmental behavior.

Capacity building effort: a welcome addition to the fight against cybercrime

The UN treaty includes an important capacity building effort that is not contemplated by the Budapest Convention. That’s a welcome addition to international cybercrime conversations. Training and education can help establish a common understanding about the types of information available and how to investigate and prosecute crimes.

These programs also help ensure that there is not only education but also resources to be able to do those kinds of things. The result will be greater confidence, agreement, and certainty about what kinds of acts are criminal, assuming the challenges about the scope I outlined above are addressed.

Finding the right balance for the pursuit of cybercrime while protecting human rights and due process

There is certainly a path forward where the capabilities of an effective tool for cross border tool pursuit of cyber criminals is combined with capacity building resources for developing and emerging economies within a framework that honors shared values and prior commitments to protect human rights and due process.

Cisco stands ready to partner with governments on getting the answer to these tough questions right. We urge UN member states to take a careful look at the text of the UN Convention and consider how to better align it with the authorities and protections already in the Budapest Convention before proceeding to ratify its current version.

Senior Director, Technology Policy

Global government affairs, leave a comment cancel reply.

We'd love to hear from you! Your comment(s) will appear instantly on the live site. Spam, promotional and derogatory comments will be removed and HTML formatting will not appear.

CONNECT WITH US