policy making research topics

• IPR Intranet

INSTITUTE FOR POLICY RESEARCH

• What We Study

Trending Policy Topics

Bipartisanship and Partisanship

Housing Discrimination

School Spending

Food Insecurity

Policing and Violence

COVID-19 and Public Opinion

Research Topics & Ideas: Politics

100+ Politics-Related Research Ideas To Fast-Track Your Project

Finding and choosing a strong research topic is the critical first step when it comes to crafting a high-quality dissertation or thesis. If you’ve landed on this post, chances are you’re looking for a politics-related research topic , but aren’t sure where to start. Here, we’ll explore a variety of politically-related research ideas across a range of disciplines, including political theory and philosophy, comparative politics, international relations, public administration and policy.

NB – This is just the start…

The topic ideation and evaluation process has multiple steps . In this post, we’ll kickstart the process by sharing some research topic ideas. This is the starting point, but to develop a well-defined research topic, you’ll need to identify a clear and convincing research gap , along with a well-justified plan of action to fill that gap.

If you’re new to the oftentimes perplexing world of research, or if this is your first time undertaking a formal academic research project, be sure to check out our free dissertation mini-course. Also, be sure to sign up for our free webinar that explores how to find a high-quality research topic from scratch.

Overview: Politics-Related Topics

• Political theory and philosophy
• Comparative politics
• International relations
• Public administration
• Public policy
• Examples of politics-related dissertations

Topics & Ideas: Political Theory

• An analysis of the impact of feminism on political theory and the concept of citizenship in Saudi Arabia in the context of Vision 2030
• A comparative study of the political philosophies of Marxism and liberalism and their influence on modern politics
• An examination of how the Covid-19 pandemic affected the relationship between individual freedom and collective responsibility in political philosophy
• A study of the impact of race and ethnicity on French political philosophy and the concept of justice
• An exploration of the role of religion in political theory and its impact on secular democracy in the Middle East
• A Review of Social contract theory, comparative analysis of the political philosophies of Hobbes, Locke, and Rousseau
• A study of the concept of the common good in political philosophy and its relevance to the ongoing refugee crisis in Europe
• An examination of the relationship between political power and the rule of law in developing African countries
• A study of the impact of postmodernism on political theory and the concept of truth, a case study of the US
• An exploration of the role of virtue in political philosophy and its impact on the assessment of moral character in political leaders

Topics & Ideas: Comparative Politics

• A comparative study of different models of federalism and their impact on democratic governance: A case Study of South American federalist states
• The impact of ethnic and religious diversity on political stability and democracy in developing countries, a review of literature from Africa
• An analysis of the role of civil society in promoting democratic change in autocratic regimes: A case study in Sweden
• A comparative examination of the impact of globalization on political institutions and processes in South America and Africa.
• A study of the factors that contribute to successful democratization in authoritarian regimes, a review of the role of Elite-driven democratization
• A comparison of the political and economic systems of China and India and their impact on social development
• The impact of corruption on political institutions and democracy in South East Asia, a critical review
• A comparative examination of the impact of majoritarian representation (winner-take-all) vs proportional representation on political representation and governance
• An exploration of Multi-party systems in democratic countries and their impact on minority representation and policy-making.
• A study of the factors that contribute to successful decentralization and regional autonomy, a case study of Spain

Topics & Ideas: International Relations

• A comparative analysis of the effectiveness of diplomacy and military force in resolving international conflicts in Central Africa.
• The impact of globalization on the sovereignty of nation-states and the changing nature of international politics, a review of the role of Multinational Corporations
• An examination of the role of international aid organizations in promoting peace, security, and development in the Middle East.
• A study of the impact of economic interdependence on the likelihood of conflict in international relations: A critical review of weaponized interdependence
• A comparative analysis of the foreign policies of the EU and the US and their impact on international stability in Africa
• An exploration of the relationship between international human rights and national sovereignty during the Covid 19 pandemic
• A study of the role of decentralized autonomous organizations (DAO)s in international politics and their impact on state behaviour
• A comparative analysis of the effectiveness of international regimes in addressing global challenges such as climate change, arms control, and terrorism in Brazil
• An examination of the impact of the rise of BRICS on the international system and global governance
• A study of the role of ideology in shaping the foreign policies of states and the dynamics of international relations in the US

Tops & Ideas: Public Administration

• An analysis of the impact of digital technology on public administration and the delivery of public services in Estonia
• A review of models of public-private partnerships and their impact on the delivery of public services in Ghana
• An examination of the role of civil society organizations in monitoring and accountability of public administration in Papua New Guinea
• A study of the impact of environmentalism as a political ideology on public administration and policy implementation in Germany
• An exploration of the relationship between public administration and citizen engagement in the policy-making process, an exploration of gender identity concerns in schools
• A comparative analysis of the efficiency and effectiveness of public administration, decentralisation and pay and employment reform in developing countries
• A study of the role of collaborative leadership in public administration and its impact on organizational performance
• A systematic review of the challenges and opportunities related to diversity and inclusion in police services
• A study of the impact of corrupt public administration on economic development and regional growth in Eastern Europe
• An exploration of the relationship between public administration and civil rights and liberties, including issues related to privacy and surveillance, a case study in South Korea

Topics & Ideas: Public Policy

• An analysis of the impacts of public policy on income inequality and poverty reduction in South Sudan
• A comparative study of the effectiveness of legal and regulatory, economic and financial, and social and cultural instruments for addressing climate change in South Korea
• An examination of the role of interest groups in shaping public policy and the policy-making process regarding land-use claims
• A study of the impact of globalization on the development of public policies and programs for mitigating climate change in Singapore
• An exploration of the relationship between public policy and social justice in tertiary education in the UAE
• A comparative analysis of the impact of health policies for the management of diabetes on access to healthcare and health outcomes in developing countries
• Exploring the role of evidence-based policymaking in the design and implementation of public policies for the management of invasive invertebrates in Australia
• An examination of the challenges and opportunities of implementing educational dietary public policies in developing multicultural countries
• A study of the impact of public policies on urbanization and urban development in rural Indonesia
• An exploration of the role of media and public opinion in shaping public policy and the policy-making process in the transport industry of Malaysia

Examples: Politics Dissertations & Theses

While the ideas we’ve presented above are a decent starting point for finding a politics-related research topic, they are fairly generic and non-specific. So, it helps to look at actual dissertations and theses to see how this all comes together.

Below, we’ve included a selection of research projects from various politics-related degree programs to help refine your thinking. These are actual dissertations and theses, written as part of Master’s and PhD-level programs, so they can provide some useful insight as to what a research topic looks like in practice.

• We, the Righteous Few: Immoral Actions of Fellow Partisans are Judged as Less Possible (Varnam, 2020)
• Civilizing the State: Civil Society and the Politics of Primary Public Health Care Provision in Urban Brazil (Gibson, 2012)
• Political regimes and minority language policies: evidence from Taiwan and southeast Asia (Wu, 2021)
• The Feminist Third Wave: Social Reproduction, Feminism as Class Struggle, and Contemporary Women’s Movements (Angulo, 2019)
• The Politics of Immigration under Authoritarianism (Joo, 2019)
• The politics of digital platforms: Sour Dictionary, activist subjectivities, and contemporary cultures of resistance (Okten, 2019)
• Vote choice and support for diverse candidates on the Boston City Council At-Large (Dolcimascolo, 2022)
• The city agenda: local governance and national influence in the policy agenda, 1900-2020 (Shannon, 2022)
• Turf wars: who supported measures to criminalize homelessness in Austin, Texas? (Bompiedi, 2021)
• Do BITs Cause Opposition Between Investor Rights and Environmental Protection? (Xiong, 2022)
• Revealed corruption and electoral accountability in Brazil: How politicians anticipate voting behavior (Diaz, 2021)
• Intersectional Solidarity: The Political Consequences of a Consciousness of Race, Gender and Sexuality (Crowder, 2020)
• The Congressional Hispanic Caucus and the Coalitional Representation of Latinxs in the U.S. House of Representatives (Munoz, 2019)

Looking at these titles, you can probably pick up that the research topics here are quite specific and narrowly-focused , compared to the generic ones presented earlier. In other words, to create a top-notch research topic, you must be precise and target a specific context with specific variables of interest . In other words, you need to identify a clear, well-justified research gap.

Get 1:1 Help

If you’re still feeling a bit unsure about how to find a research topic for your dissertation or research project, check out our Topic Kickstarter service below.

You Might Also Like:

Interesting thesis.

I really appreciate your work which will greatly help me rethink on my topic

Please how can I get the full thesis?

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Library electronic resources outage May 29th and 30th

Between 9:00 PM EST on Saturday, May 29th and 9:00 PM EST on Sunday, May 30th users will not be able to access resources through the Law Library’s Catalog, the Law Library’s Database List, the Law Library’s Frequently Used Databases List, or the Law Library’s Research Guides. Users can still access databases that require an individual user account (ex. Westlaw, LexisNexis, and Bloomberg Law), or databases listed on the Main Library’s A-Z Database List.

• Georgetown Law Library

Policy Research Guide

Getting started.

• Government Resources
• Think Tanks and Public Opinion

Key to Icons

• Georgetown only
• On Bloomberg
• More Info (hover)
• Preeminent Treatise

Introduction

This research guide is designed to serve as a starting point for conducting research in policy analysis. It will include both print and electronic resources available in the Georgetown Law Library. 

The Georgetown Law Library also has a collection of research guides on many topics closely impacted by public policy:

• Banking Law Research Guide by Daniel Donahue Last Updated Aug 29, 2023 99 views this year
• Education Law Research Guide by Jill Smith Last Updated Nov 2, 2023 3192 views this year
• Environmental Law Research Guide by Rachel Jorgensen Last Updated Apr 23, 2024 683 views this year
• Health Law Research Guide by Jill Smith Last Updated Mar 13, 2024 540 views this year
• National Security Law Research Guide by Law Library Reference Last Updated Apr 10, 2023 1523 views this year
• Poverty Law Research Guide by Law Library Reference Last Updated Feb 7, 2024 623 views this year
• Securities Law (U.S. and International) Research Guide by Jill Smith Last Updated Apr 4, 2024 1966 views this year
• Statistics and Empirical Legal Studies Research Guide by Sara E. Burriesci Last Updated Apr 15, 2024 2283 views this year
• Tax Research - Federal Guide by Law Library Reference Last Updated Apr 6, 2023 4905 views this year

Please also refer to the Lauinger Library's research guides related to public policy:

• Environmental Policy
• Public Opinion and Polling
• Public Policy
• PAIS Index Covers issues in the public debate through a wide variety of international sources including journal articles, books, government documents, statistical directories, grey literature, research reports, conference papers, web content, and more.
• Policy File Index Indexes research and publications from public policy think tanks, university research programs, research organizations, and publishers. Each item includes an abstract and, where available, access to the full-text of the report and/or the homepages and email addresses of the authoring institution.
• ProQuest Congressional Bills & Public Laws 1987-present; Committee Prints & misc. publications 1817-present; Congressional Record Bound Edition & predecessors 1789-2001; Congressional Record Daily Edition 1985-present; Congressional Research Service (CRS) Reports 1916-present; Hearings 1817-present; House & Senate Documents 1817-present; House & Senate Reports 1789-present; Serial Set publications 1789-present; Serial Set Maps 1789-1969; CIS Legislative Histories 1969-present.
• ProQuest Political Science Covers the literature of political science and international relations, including such topics as comparative politics, political economy, international development, environmental policy, and hundreds of related topics.

Williams Library Reference

Reference Desk : Atrium, 2nd (Main) Floor (202) 662-9140 Request a Research Consultation  

Update history.

Links 6/14 (MK) Updated 9/15 (MK) Updated 3/21 (CMC) Updated 4/23 (SB)

• Next: Books >>
• © Georgetown University Law Library. These guides may be used for educational purposes, as long as proper credit is given. These guides may not be sold. Any comments, suggestions, or requests to republish or adapt a guide should be submitted using the Research Guides Comments form . Proper credit includes the statement: Written by, or adapted from, Georgetown Law Library (current as of .....).
• Last Updated: Oct 2, 2023 2:21 PM
• URL: https://guides.ll.georgetown.edu/policyresearch

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• 15 April 2024
• Correction 22 April 2024

Revealed: the ten research papers that policy documents cite most

• Dalmeet Singh Chawla 0

Dalmeet Singh Chawla is a freelance science journalist based in London.

You can also search for this author in PubMed   Google Scholar

Policymakers often work behind closed doors — but the documents they produce offer clues about the research that influences them. Credit: Stefan Rousseau/Getty

When David Autor co-wrote a paper on how computerization affects job skill demands more than 20 years ago, a journal took 18 months to consider it — only to reject it after review. He went on to submit it to The Quarterly Journal of Economics , which eventually published the work 1 in November 2003.

Autor’s paper is now the third most cited in policy documents worldwide, according to an analysis of data provided exclusively to Nature . It has accumulated around 1,100 citations in policy documents, show figures from the London-based firm Overton (see ‘The most-cited papers in policy’), which maintains a database of more than 12 million policy documents, think-tank papers, white papers and guidelines.

“I thought it was destined to be quite an obscure paper,” recalls Autor, a public-policy scholar and economist at the Massachusetts Institute of Technology in Cambridge. “I’m excited that a lot of people are citing it.”

The most-cited papers in policy

Economics papers dominate the top ten papers that policy documents reference most.

Data from Overton as of 15 April 2024

The top ten most cited papers in policy documents are dominated by economics research; the number one most referenced study has around 1,300 citations. When economics studies are excluded, a 1997 Nature paper 2 about Earth’s ecosystem services and natural capital is second on the list, with more than 900 policy citations. The paper has also garnered more than 32,000 references from other studies, according to Google Scholar. Other highly cited non-economics studies include works on planetary boundaries, sustainable foods and the future of employment (see ‘Most-cited papers — excluding economics research’).

These lists provide insight into the types of research that politicians pay attention to, but policy citations don’t necessarily imply impact or influence, and Overton’s database has a bias towards documents published in English.

Interdisciplinary impact

Overton usually charges a licence fee to access its citation data. But last year, the firm worked with the publisher Sage to release a free web-based tool , based in Thousand Oaks, California, that allows any researcher to find out how many times policy documents have cited their papers or mention their names. Overton and Sage said they created the tool, called Sage Policy Profiles, to help researchers to demonstrate the impact or influence their work might be having on policy. This can be useful for researchers during promotion or tenure interviews and in grant applications.

Autor thinks his study stands out because his paper was different from what other economists were writing at the time. It suggested that ‘middle-skill’ work, typically done in offices or factories by people who haven’t attended university, was going to be largely automated, leaving workers with either highly skilled jobs or manual work. “It has stood the test of time,” he says, “and it got people to focus on what I think is the right problem.” That topic is just as relevant today, Autor says, especially with the rise of artificial intelligence.

Most-cited papers — excluding economics research

When economics studies are excluded, the research papers that policy documents most commonly reference cover topics including climate change and nutrition.

Walter Willett, an epidemiologist and food scientist at the Harvard T.H. Chan School of Public Health in Boston, Massachusetts, thinks that interdisciplinary teams are most likely to gain a lot of policy citations. He co-authored a paper on the list of most cited non-economics studies: a 2019 work 3 that was part of a Lancet commission to investigate how to feed the global population a healthy and environmentally sustainable diet by 2050 and has accumulated more than 600 policy citations.

“I think it had an impact because it was clearly a multidisciplinary effort,” says Willett. The work was co-authored by 37 scientists from 17 countries. The team included researchers from disciplines including food science, health metrics, climate change, ecology and evolution and bioethics. “None of us could have done this on our own. It really did require working with people outside our fields.”

Sverker Sörlin, an environmental historian at the KTH Royal Institute of Technology in Stockholm, agrees that papers with a diverse set of authors often attract more policy citations. “It’s the combined effect that is often the key to getting more influence,” he says.

Has your research influenced policy? Use this free tool to check

Sörlin co-authored two papers in the list of top ten non-economics papers. One of those is a 2015 Science paper 4 on planetary boundaries — a concept defining the environmental limits in which humanity can develop and thrive — which has attracted more than 750 policy citations. Sörlin thinks one reason it has been popular is that it’s a sequel to a 2009 Nature paper 5 he co-authored on the same topic, which has been cited by policy documents 575 times.

Although policy citations don’t necessarily imply influence, Willett has seen evidence that his paper is prompting changes in policy. He points to Denmark as an example, noting that the nation is reformatting its dietary guidelines in line with the study’s recommendations. “I certainly can’t say that this document is the only thing that’s changing their guidelines,” he says. But “this gave it the support and credibility that allowed them to go forward”.

Broad brush

Peter Gluckman, who was the chief science adviser to the prime minister of New Zealand between 2009 and 2018, is not surprised by the lists. He expects policymakers to refer to broad-brush papers rather than those reporting on incremental advances in a field.

Gluckman, a paediatrician and biomedical scientist at the University of Auckland in New Zealand, notes that it’s important to consider the context in which papers are being cited, because studies reporting controversial findings sometimes attract many citations. He also warns that the list is probably not comprehensive: many policy papers are not easily accessible to tools such as Overton, which uses text mining to compile data, and so will not be included in the database.

The top 100 papers

“The thing that worries me most is the age of the papers that are involved,” Gluckman says. “Does that tell us something about just the way the analysis is done or that relatively few papers get heavily used in policymaking?”

Gluckman says it’s strange that some recent work on climate change, food security, social cohesion and similar areas hasn’t made it to the non-economics list. “Maybe it’s just because they’re not being referred to,” he says, or perhaps that work is cited, in turn, in the broad-scope papers that are most heavily referenced in policy documents.

As for Sage Policy Profiles, Gluckman says it’s always useful to get an idea of which studies are attracting attention from policymakers, but he notes that studies often take years to influence policy. “Yet the average academic is trying to make a claim here and now that their current work is having an impact,” he adds. “So there’s a disconnect there.”

Willett thinks policy citations are probably more important than scholarly citations in other papers. “In the end, we don’t want this to just sit on an academic shelf.”

doi: https://doi.org/10.1038/d41586-024-00660-1

Updates & Corrections

Correction 22 April 2024 : The original version of this story credited Sage, rather than Overton, as the source of the policy papers’ citation data. Sage’s location has also been updated.

Autor, D. H., Levy, F. & Murnane, R. J. Q. J. Econ. 118 , 1279–1333 (2003).

Article   Google Scholar  

Costanza, R. et al. Nature 387 , 253–260 (1997).

Willett, W. et al. Lancet 393 , 447–492 (2019).

Article   PubMed   Google Scholar  

Steffen, W. et al. Science 347 , 1259855 (2015).

Rockström, J. et al. Nature 461 , 472–475 (2009).

Download references

Reprints and permissions

Related Articles

India’s 50-year-old Chipko movement is a model for environmental activism

Correspondence 23 APR 24

The Middle East’s largest hypersaline lake risks turning into an environmental disaster zone

More work is needed to take on the rural wastewater challenge

CERN’s impact goes way beyond tiny particles

Spotlight 17 APR 24

The economic commitment of climate change

Article 17 APR 24

Last-mile delivery increases vaccine uptake in Sierra Leone

Article 13 MAR 24

Postdoctoral Fellow

The Dubal Laboratory of Neuroscience and Aging at the University of California, San Francisco (UCSF) seeks postdoctoral fellows to investigate the ...

San Francisco, California

University of California, San Francsico

Postdoctoral Associate

Houston, Texas (US)

Baylor College of Medicine (BCM)

Postdoctoral Research Fellow

Description Applications are invited for a postdoctoral fellow position at the Lunenfeld-Tanenbaum Research Institute, Sinai Health, to participate...

Toronto (City), Ontario (CA)

Sinai Health

Postdoctoral Research Associate - Surgery

Memphis, Tennessee

St. Jude Children's Research Hospital (St. Jude)

Open Rank Faculty Position in Biochemistry and Molecular Genetics

The Department of Biochemistry & Molecular Genetics (www.virginia.edu/bmg) and the University of Virginia Cancer Center

Charlottesville, Virginia

Biochemistry & Molecular Genetics

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Articles on Policymaking

Displaying 1 - 20 of 28 articles.

Connecting researchers and legislators can lead to policies that reflect scientific evidence

Taylor Scott , Penn State

To get to net zero, policymakers need to listen to communities. Here’s what they can learn from places like Geelong

Amanda Tattersall , University of Sydney

Politics urgently needs more imagination. Competence alone will not save us from this ‘polycrisis’

Geoff Mulgan , UCL

Canadian ethicists recognize the critical importance of science and research

Judy Illes , University of British Columbia ; Bartha Knoppers , McGill University ; Eric M. Meslin , University of Toronto ; Jennifer Chandler , L’Université d’Ottawa/University of Ottawa ; Ross Upshur , University of Toronto ; Steven J. Hoffman , York University, Canada ; Tania Bubela , Simon Fraser University , and Vardit Ravitsky , Université de Montréal

Kenya doesn’t have a stand-alone tax on sugary drinks: we set out to find out why

Milka Wanjohi , African Population and Health Research Center and Gershim Asiki , African Population and Health Research Center

We can build a more inclusive government and economy out of the pandemic — this blueprint shows us how

Obesity strategy: policies placing responsibility on individuals don’t work – so why does the government keep using them?

Charlotte Godziewski , Aston University

Coronavirus: record ethnicity on all death certificates to start building a clearer picture

Siobhan Morris , UCL and Olivia Stevenson , UCL

Coronavirus: the pressures governments face in balancing safety and liberty

Christoph Meyer , King's College London

How simple policy changes can help us age better and prevent cognitive decline

Mauricio Avendano Pabon , King's College London and Ludovico Carrino , King's College London

Analysis: Indonesian policymaking is not supported by quality research and academic freedom

Inaya Rakhmani , Universitas Indonesia and Zulfa Sakhiyya , Universitas Negeri Semarang

West African states have a science and technology plan. But it’s going nowhere

Joseph Ato Forson

The White House is upending decades of protocol for  policy-making

Stuart Shapiro , Rutgers University

South Africa’s “new dawn” should be built on evidence-based policy

Laurenz Langer , University of Johannesburg and Promise Nduku , University of Johannesburg

Life in South Africa’s economic hub is improving – but big challenges remain

Julia de Kadt , Gauteng City-Region Observatory ; Alexandra Parker , Gauteng City-Region Observatory , and Christina Culwick Fatti , Gauteng City-Region Observatory

What it takes for Indonesia to create, share and use knowledge to grow its economy

Arnaldo Pellini , Tampere University

How soil scientists can do a better job of making their research useful

Keith Shepherd , Center for International Forestry Research – World Agroforestry (CIFOR-ICRAF)

Market v government? In fact, hybrid policy is the best fit for the 21st century

Mark Fabian , Australian National University and Robert Breunig , Crawford School of Public Policy, Australian National University

Why health research rarely influence policy in Indonesia

Meiwita Budiharsana , Universitas Indonesia

Rejection of subsidies for coal and nuclear power is a win for fact-based policymaking

Ellen Hughes-Cromwick , University of Michigan

Related Topics

• Evidence-based policy
• Peacebuilding
• Public policy
• Science communication

Top contributors

Senior Researcher, University of Johannesburg

Associate Professor and Host of ChangeMakers Podcast, University of Sydney

Assistant professor of public policy, University of Warwick

Associate Professor of Earthquake Science, The University of Melbourne

Professor of Economics and Director, Tax and Transfer Policy Institute, Crawford School of Public Policy, Australian National University

Lecturer in applied mathematics, University of Adelaide

Senior Lecturer in Social Policy, Centre for Social Research and Methods, Australian National University

Honorary Fellow in the School of Social and Political Sciences, The University of Melbourne

Emeritus Professor, Centre for Urban Research, RMIT University

Senior Manager, Planning & Research, RMIT University

Professor of Economics and Public Policy, ANU College of Arts and Social Sciences, Australian National University

Professor of Planning and Public Policy, Rutgers University

Professor of Anthropology, New York University

Vice Dean and Professor of Business Law, New York University

Professor of Urban Policy and Director, Centre for Urban Research, RMIT University

• X (Twitter)
• Unfollow topic Follow topic

Public Policy and Research in Africa pp 63–85 Cite as

Research Methods for Public Policy

• Susan Mbula Kilonzo 3 &
• Ayobami Ojebode 4  
• Open Access
• First Online: 19 October 2022

25k Accesses

This chapter examined the nature of public policy and role of policy analysis in the policy process. It examines a variety of research methods and their use in public policy engagements and analysis for evidence-informed policymaking. It explains qualitative methods, quantitative methods, multiple and mixed-method research. Other issues addressed include causal research in public policy, report writing and communication and related issues in public policy research.

You have full access to this open access chapter,  Download chapter PDF

Introduction

As implied by the topic, this chapter focuses on research methods applied or applicable in public policy research. Though the overriding focus is on specific research methods, we deemed it necessary to preface these with a brief discussion of the nature of public policy research and the nature of policy-engaged research problem or question. These are then followed by the specific research approaches or traditions and methods as applied to public policy. Given that public policy research deals with issues that have important implications for the society, the mixed-method research is often preferred as a means of arriving at findings and conclusion concrete and reliable enough to serve as a basis for policy. For this reason, we devoted a section to mixing methods in public policy research. This chapter is thus divided into four sections:

Nature of public policy research

The policy-engaged research problem or question,

Specific public policy research approaches and methods, and

Mixing methods in public policy research.

In the first section, we focus on the definitive characteristics of the kind of research that supports or evokes public policy, especially the solution-orientation of such research. In the second section, we focus on what it means for research to be policy-engaged—which is different from being policy-relevant. We propose the nature and source of a good problem or question for policy-engaged research and its basic design. In the third section, we focus on the two broad traditions of research: qualitative and quantitative traditions, and the specific methods under these traditions. We explain how these methods are used in public policy research using both hypothetical and existing examples. In the last section, we discuss mixing research methods in public policy research, stressing the reasons for it and summing up the process of doing it.

Nature of Public Policy Research

Public policy research is one whose primary aim is to understand or explain social, political, economic, cultural and other issues that are significant to the society and which require the intervention or attention of policy actors. In providing an understanding of such issues, the research also presents itself as a trustable basis for the actions and interventions of these policy actors. It must, therefore, be a piece of research based on sound evidence, produced out of convincing rigour and woven from start to finish around a societal issue of concern.

In addition to being thorough and trustable, public policy research must also go beyond describing a problem or situation into engaging the how and why of things (Osifo 2015 : 149) for it to establish causality with reference to a given problem and the options of addressing such a problem. Descriptive studies do sometimes provide an important basis for policy; however, causal studies often interest and command the attention of policy actors more than descriptive ones do.

A good public policy research is sensitive to both the policy and political agenda. These two environments or elements determine action or inaction. Howlett ( 2012 : 451) argues for an approach that encourages absorption of research outputs at two levels: enhancing instrumental arguments about policy programme content and ensuring a deeper political engagement experience.

Though policy makers do not entirely depend on research to make decisions on policy options (Edwards 2004 ; UK Cabinet Office 2009 ), the role of research, and specifically field-based research, in public policy remains critical (Mead 2005 ; Young 2005 ). Since scholarly research competes with expert knowledge, domestic and international policy, stakeholder consultations and evaluation of previous policies, among other sources (UK Cabinet 1999), evidence generated from research that is meant to inform public policy needs a strong basis for argument on the problem under scrutiny, as well as a variety of policy options from field evidence.

Recent studies show that research in policymaking over the last four decades plays a less direct role than is often assumed and expected (Howlett 2012 ). Nevertheless, the role of research in public policy is not to be downplayed, and as Mead ( 2005 : 535) explains, field research is essential to realistic policy research that ties governmental action to good outcomes. However, we need to take cognizance of the fact that, as Tierney and Clemens ( 2011 ) argue, many of today’s most pressing policy issues are extraordinarily complex and will benefit from carefully conceived and analysed studies utilizing multiple methodological approaches. Public policy researchers should understand this complexity of policy problems. This complex web determines, to a great extent, what forms of research and/or research methods a researcher should consider.

Literature shows that in the history of public policy research, statistical evidence was very important (Mead 2005 ). Studies meant to inform policy were therefore mostly, if not always, survey-based (Mead 2005 ). Survey-based research, as Mead ( 2005 : 544) shows, is good at generating accurate depictions of the clientele served by a given policy. Social problems and their correlates can be clearly captured. Earlier approaches to policy research favoured output that could be generalized across settings that were validated and reliable. In those early approaches, quantitative research, especially survey, was given priority. Qualitative research did not so much move into policy arena and research evidence from qualitative studies did not seem to find a place in policy discussion tables (Tierney and Clemens 2011 : 59).

Over the years, survey-based approach has been criticized for its narrow economistic approach because social problems are complex. The argument is that survey-based policy research projects onto its subjects, the psychology assumed by the quantitative researcher. Simply put on its own, the approach lacks the ability to explain why and how complex social problems arise, and what public policies would best be suited to address them in their complexity. Surveys, for instance, may not give the full range of information required to account for the behaviours of the poor, needy and dependent persons in certain circumstances. These people, though challenged by certain economic factors, can survive in difficult circumstances, but the how and why of their survival would be beyond the easy reach of survey. Thus, as Mead ( 2005 ) argues, there is need for a more complex and robust approach that incorporates those factors that are beyond the statistics. We argue that for a public policy research to claim authenticity of findings that capture the attention of policy makers, and subsequently inform the policy process, an integration of research methods, that is, mixed-method design, is important.

Public policy research is meant to provide solutions to social and public problems that are in many ways complex. Establishing causes and effects of these problems run beyond analysis of existing policies. Mead ( 2013 ), for instance, argues:

[Where] texts in public policy devote attention to both policy analysis and political analysis; they fail to capture the intimate connection between them. The two subjects appear as separate worlds, when they are really two sides of the same coin. The texts do not consider that political constraints should really be part of policy argument or that the policy-making process can sharply limit what best policy means. And in research on public policy, there is even less sense of policy and politics shaping and reshaping each other. Typically, the usual division prevails where economists recommend best policy while political scientists explain what government does. (p. 393)

These views relate to the policy and politics dichotomy, and how political analysis is good in reshaping policy analysis (Mead 2013 : 392). While it is important to pay attention in public policy research to how these two influence each other, it is also important to pay careful attention to the stakeholders. Good research methods for public policy should engage stakeholders in the research process to enhance the use of the research findings and recommendations for effective policies. Besides the policy makers, policy actors include the public, which is always at the receiving end of the end products of public policy research are important. Consultations with them at most, if not all levels, help researchers to articulate policies that include their ideas or address their concerns (Oxman et al. 2009 ) and result in the good policy performance.

The Policy-Engaged Research Problem/Question

With reference to their level of policy engagement, public policy research in Africa can be categorized into three: public policy-appended research, commissioned policy research and public policy analysis. Public policy-appended research is the most common of the three. For most African researchers, there is a mandatory section of their article or thesis that presents policy recommendations. In that section, researchers attempt to point out how their research findings can be applied to real-life policy situations and consequently change those situations for the better. Efforts are made by experienced researchers to ensure a close fit between the recommendations and the findings that precede it in the article or thesis. As common as this genre of public policy research is, it is a flawed approach for many reasons. The approach treats policy not as the centre of the research but as an appendage. Put differently, the researcher decides her or his research problem and question and decides on the methods most suitable for this. At the conclusion of the research, she or he then turns to policy actors with recommendations. Since the research was not informed by a policy need or gap, it can hardly fit into the existing agenda and conversations among policy actors. It neither speaks the language of policy actors nor considers their priorities. The researcher would not have attempted to include policy actors at most, if not all, stages of the research, and as we will discuss shortly, there are consequences of not doing this. It also assumes that policy actors (i.e. policy makers, civil society and other stakeholders, including citizens) are on the lookout for policy recommendations from researchers and can wade through the different sections of the research to find these recommendations. As Oyedele, Atela and Ojebode ( 2017 ) opined, this is hardly so. The researcher’s research is her or his business, not that of the policy actors. As a result, policy actors do not access the tonnes of policy recommendations made by researchers.

Commissioned public policy research projects are initiated by government agencies and non-governmental organizations to address specific policy or implementation problem. The driving research question and the nature of the expected findings are articulated by the commissioning organization. A critical objection to this genre of public policy research is researcher’s autonomy on crucial fronts. To what extent can a researcher turn out findings that conflict with the political aspirations and public image of the funding government or its agency? How can the researcher be sure that his or her findings are not spun or twisted in favour of government? Therefore, while the findings and recommendations of this genre of public policy research are likely to be more easily accepted by policy actors than the findings of public policy-appended research, there is usually a cloud of doubt around its objectivity and integrity.

A third genre of public policy research deals with policy analysis . These studies take on an existing policy and subject its components to critical analysis often conjecturing whether it would produce expected results. They explore inconsistencies, systemic barriers and feasibility of a policy, and then draw conclusions as to why a policy works or does not. They may serve as formative or summative studies depending on when they are conducted in the life cycle of a policy. The challenge of this approach to public policy research has been that the researcher/analyst is basically tied to the outcomes of policies in existence—policies that he or she did not play a role in formulating.

The foregoing genres of public policy research are, at best, only partially policy-engaged. They may be policy-relevant, but they are not policy-engaged. So, the questions for us here are: What is policy-engaged research? How does it differ from policy analysis, commissioned public policy research and public policy-appended research? What is it that the other three misses out that policy-engaged research is good for? And how do we then design research in a way that the methods used are relevant in informing the public policymaking processes?

A policy-engaged piece of research derives its roots from the questions that are being asked in policy circles. As a response to current public policy issues, it is driven by a research question that explores, extends or clarifies a policy question or problem. Policy-engaged research therefore means bringing on board the stakeholders relevant in the development of a given public policy (Lemke and Harris-Wai 2015 ), whether their role is interest or influence. This means that there is an all-round way of understanding the problem that the policy is intended to solve and the politics surrounding the decision-making process.

It is important for a researcher to understand in policy-engaged research, is the need to tailor the research in a way that the policy options suggested are practical. This is because, a policy attempts to solve or prevent a problem, or scale up progress, and policy actors are interested in “what works”. In other words, they are keen about what causes an outcome or makes things happen. A piece of public policy research would, therefore, do well if it were causal, rather than descriptive.

There are two fundamental characteristics of a public policy research problem or question: First, it should explore cause, outcome, and/or causal mechanism in relation to an existing policy or a policy action it intends to propose. In exploring these, the researcher can tease out the specific factors that are responsible for a certain policy problem/issue (outcome) and have conclusive findings from which to confidently suggest specific points of intervention in a policy progression. For instance, if the researcher discovers that misinformation is the cause of vaccine rejection, then he or she knows better than to suggest increased procurement of vaccines but would rather suggest media campaigns or community meetings to increase citizens’ awareness of that vaccination. If, in exploring the mechanism between misinformation and rejection, she discovers that misinformation leads to cognitive dissonance which then leads citizens to seek clarification from traditional birth attendants who then counsels them to abstain from vaccination and whom they then obey by rejecting the vaccination, she is further equipped to make pointed suggestion on which point in the chain to focus intervention or “tweaking”. Public policy research without such causal information can easily become a shot in the dark.

Second, the public policy research problem should resonate with the questions that policy actors are asking as well as the questions that they should be asking. While it is important for the public policy research question to evolve from policy questions, it is also important to note that policy questions are sometimes wrong or inadequate. Put bluntly, policy actors sometimes do not ask the right questions. It is, therefore, important for the researcher to identify these policy questions and give them the needed redirection. Policy actors, for instance, may be asking if the gap between male and female children about access to education is narrowing or widening following the adoption of an affirmative action policy in favour of the girl child. Whereas this is an important question, it is not likely to reveal information that is specific enough to be a basis for the right adjustment of the policy. It is not only simply descriptive but also narrow and unworthy of much research. The researcher should push harder with questions of cause, outcome and causal mechanism about the male-female disparity in access to education in this case. Has the policy produced a narrowing of the gap? If not, why has it not? What skills or resources are lacking that account for this lack of narrowing? Or what historical, religious or cultural factors combine or act alone to ensure continuity of the gap despite the policy? The public policy research question may not be the exact one that policy actors are asking, but it is indeed a vital extension and reflection of the policy question.

When we have public policy research problems that are unrelated to the problems that policy actors have, the consequence can be predicted. We will come up with findings that may be scientifically sound but unattractive to policy actors. Such findings will have little or no uptake. This approach speaks to the disconnection which a vast amount of literature points out—the disconnection between researchers and policy makers (Edwards 2004 : 2; Young 2005 : 730–1; Saetren 2005 ). When we ask public policy research questions that are not causal, the consequence can as well be predicted—our findings will not be convincing or informing enough to move policy actors to targeted action. Ultimately, questions that are not in line with the policy makers, and non-causal questions, render our research simply as just another piece of research for its sake.

A research question largely dictates its own research design. The type of research question we advocate above implies an iterative approach that begins with policy actors and finally returns to them. It also implies a specific kind of methods. It is a back-and-forth movement that considers the concerns of the actors as the fulcrum. In addition to being iterative, the design is also causal. The stages given below may apply (Fig. 4.1 ).

Approach to designing policy research

The way in which research is designed determines the ability of the researcher to claim causal conclusions (Bachman 2007 ). This is important for it gives indication to policy makers on what influential factors lead to what outcomes. If this is not known, making relevant policy decisions is always not possible.

Qualitative and Quantitative Methods in Public Policy Research

In this section, we explain the commonly used qualitative and quantitative research methods for public policy.

Qualitative Methods in Public Research

Briefly stated, qualitative methods aim at providing deeper perspectives, attitudes, perceptions and contextual insights that surround the issue under investigation as experienced and understood by those living through it. The outcome of qualitative methods is usually the verbalized thoughts and viewpoints of the subjects of investigation rather than numbers or statistics. The following are some of the research methods used in qualitative research. Note that each of these methods applies a wide range of tools to collect data.

Historical and Archival Research

Libraries and archives store historical information in many forms including diaries, pictures, documents, minutes and artefacts, among others. These mean that they might have been stored as primary or secondary data. Historical or archival information that can be considered as primary is that which was collected from the author or field and stored in its original form without undergoing any form of analysis and change. Such may include minutes, diaries, pictures, artefacts, personal memoirs, autobiographies and others of the same nature. Any historical information that has gone through any studies or analysis then becomes secondary data. These may include journals, books and magazines, among others.

When a researcher wants to use historical and archival data, the aim is to research on the past and already existing information. However, historical and archival research does not always mean deriving data from the archives. A policy researcher may design a historical study in which they endeavour to visit the field and collect data from knowledgeable individuals concerning a certain historical issue of policy concern. They may partly engage documents from archives or libraries to historicize, contextualize and corroborate the issue under research. It is also the nature of many parliamentary researchers to “mine” data from parliamentary libraries/archives, some of which contain data that is classified as primary data.

Historical data is important in public policy, for it helps researchers situate their arguments within existing narratives, contexts and prior solutions suggested for policy problems. Roche ( 2016 ) argues that making assumptions about the ease with which historical research can be done is misleading. He advises that knowledge of context and a sequential approach should be given ascendance in the researcher’s priority. The researcher should be aware of chronology of information to clearly provide a coherent picture of the policy issue at hand. This implies that the past information should be relatable to the most current. With the advent in technology, most data are now digitalized, and as such, it is easy to get information from the Internet.

Archives are used to store vital government records such as personal letters, diaries, minutes, logbooks, plans, maps, photographs, among others, that easily qualify to be analysed as primary data (Roche 2016 : 174). Roche ( 2016 : 183–4) notes the challenge of fragmentation and partial availability of archival documents. He further alludes to technical challenges of the clarity of some of archival data. He cites examples of materials that were handwritten a while back and which may be ineligible. Historical and archival research apply both desk-based methods and interview techniques of data collection. Photography can also be used.

Ethnographic Methods

Ethnographic approach to research studies communities in their natural setting to understand their activities, behaviour, attitudes, perspectives and opinions within their social surrounding (Brewer 2000 ). To do so, ethnography entails close association with the research communities and sometimes participation in their activities (Brewer 2000 : 17). In fact, the commonly used methods of data collection in ethnography are participant (and sometimes non-participant) observation. The former allows for the researcher to get involved in the activities of the communities, while the latter is designed for the researcher to observe from the periphery. As Brewer argues, it is this day-to-day involvement in people’s activities that enable the researcher to make sense of the social worldviews of the research participants.

Non-participant observation describes a research situation where a researcher does not take part in the processes, events or activities that he or she is observing but removes himself or herself from the happenings to critically observe from a distance. This has challenges especially if the observed become aware of intrusion and subsequently alter their behaviour (Hawthorne effect). Sometimes the researcher may structure the observations or decide to use unstructured observations. The two differ in the sense of planning on the observation activities. For the structured type, the researcher has in mind what they want to observe and as such have a list and indications of what they would like to see. Take, for instance, a study on access to water meant to contribute to a water policy. A researcher may choose to observe how (many) times is water served at certain water points; how many people queue for the water in each of these servings; and this is likely to tell the researcher whether the water points are enough or otherwise. In unstructured observation, the researcher gets into the field with a research idea but without the specifics of that nature of data they expect from the field. Qualitative interview methods such as oral interviews and focus group discussions (FGDs) may apply where necessary during ethnography. Note taking is often applied as well.

Phenomenology

This method focuses on lived experiences of a given phenomenon by an individual or a group of individuals. Individuals can describe their views and opinions about the phenomenon in question (Johnson and Christensen 2014 ). Research on fertility issues can target women who either have or do not have children, depending on what the researcher wants to unearth, with individual women providing their lived experiences on the issue under investigation. Phenomenology is also applicable when writing biographies (an account of someone’s life written by someone else). Generally, life histories, personal testimonies and experiences are best collected through this method. This implies that oral in-depth interviews and desk-based methods of data collection are important in understanding the stories in question.

Narrative Method

A narrative is a story that structures human activity to give it some form of meaning (Elçi and Devran 2014 ). Research that applies the narrative method encourages the research participants to tell their stories around a certain issue. The researcher listens to the stories and uses them to make informed analysis on the issue at hand. A researcher concerned about experiences of people living in zones of conflicts may ask questions that elicit stories of the victims or perpetrators of violence and present these in narrative form. Researchers who use phenomenology method often apply use of narratives, but not always. Phenomenological research may not rely on story telling alone. A researcher may use desk-based method to gain perspectives of the target communities as well.

Case Studies

A case study is an intensive analysis of a small number of phenomena (events, actors, activities, processes, organizations, communities, among others) in each context. Though one can use a mix of qualitative or quantitative data within a case study, meaning that case studies can also take quantitative route, a case study is always a detailed analysis of the relationships between the contextual factors and a visible occurrence. Case studies are therefore considered when there is need for detailed information on the issue(s) under investigation. A single case study aims at providing details on the variables of interest. A comparative case study has two or more cases (what literature refers to as small-N) for the purpose of making comparative causal explanations. A researcher uses comparative case studies when they want to tease out the similarities and/or differences between or among the cases, usually for the purpose of explaining causation.

Action Research

Action research is problem-solution focused. It falls under the category of applied research and subsequently, uses practical approach to solve an immediate problem. In this case, the researcher works together with a community or practitioners to identify a challenging issue within the community that requires a possible solution. They formulate the problem together and design the research in a way that the aim is to work towards getting a solution to the problem. Once the data collected is analysed and recommendations given, a plan of action is drawn and applied to the problem that the research was designed for. The community (and researcher) reflects on the effectiveness of the solutions applied to take appropriate measures. In a nutshell, Huang ( 2010 : 99) explains that action research proceeds from a praxis of participation guided by practitioners’ concerns for practicality; it is inclusive of stakeholders’ ways of knowing and helps to build capacity for ongoing change efforts. This form of research requires money and time. As Huang ( 2010 ) notes, action research can take a qualitative, quantitative or mixed-method perspective. Various methods of data collection including oral interviews, surveys, community mapping, observation, among others, may be applied in action research.

Grounded Theory Research

A researcher may apply two approaches, inductive or deductive, to do research. The deductive approach means that one has a theoretical basis from where hypotheses can be formulated and tested. Inductive approach, on the other hand, is grounded or bottom-up. The researcher in this case starts by making observations that then provide him or her with patterns from where conclusions and theory can be drawn. Grounded research therefore moves from the point of poor or no theory up to where a researcher can deduce an informed hypothesis and towards theory building, all from the observations and analysis made from data. It is similar with other qualitative methods in the use of the various methods of data collection including oral interviews, observation and use of all forms of documents (Strauss and Corbin 1994 ).

Quantitative Methods in Public Policy Research

Quantitative research generates numerical data using such research instruments as the questionnaire, tests, code sheets for content analysis and similar other sources. The data is then subjected to mathematical or statistical analysis (Muijs 2004 ).

Literature divides quantitative research methods into two—experimental and non-experimental methods. Experimental methods are the quantitative approaches that are mainly concerned with manipulation situations with an aim of establishing cause and effect. Bachman ( 2007 : 151) argues that “the experimental design provides the most powerful design for testing causal hypotheses about the effect of a treatment or some other variable whose values can be manipulated by the researchers”. Experiments allow us to explain causality with some confidence because of the use of treatment and control. The basic and elementary type of experimental research involves setting up two groups (treatment and control groups) and introducing change to the treatment but nothing to the control. The effect of the change is measured in the differences in the behaviour or performance of the two groups after the treatment.

Experimental research has been criticized for their weakness in reflecting reality in that they take people out of their natural settings into a laboratory or pseudo-labs. Despite this, they can make important input to policymaking. For instance, micro-level policies on classroom instruction and curriculum have been largely influenced by experimental research.

Non-experimental methods do not manipulate. They are aimed and providing a descriptive picture of what is being studied. Non-experimental methods, as Muijs ( 2004 ) indicates, are more varied and may range from surveys to historical research, observations and analysis of existing data sets (applied quantitative methods). We will briefly look at the experimental and non-experimental quantitative research in the following sections.

Experimental Methods

The different types of experiments can range from randomized control trials (RCTs) to quasi-experiments, and sometimes, natural experiments.

Randomized Control Trials (RCTs)

In their simplest form, RCTs involve assigning individuals, groups, communities or settlements to experimental/treatment and control groups. The experimental group receives treatment—school feeding—while the control group receives no treatment (no school feeding). The difference in school attendance rates between these groups could then be attributed to the treatment, that is, school feeding. If statistics shows that attendance increases in the treatment group but stays the same or decreases in the control group, other things being equal, the researcher can make claims about school feeding causing increase in school attendance. Randomized control trials are expensive and are usually beyond the budget reach of most researchers. Public policy researchers therefore embark on other forms of experimental methods generally described as quasi-experimental methods.

Quasi-experiments

There is an unending controversy as to what constitutes a quasi-experiment. Given the little profit accruing from such a controversy, we would take a simple definition of that concept: any experiment that mimics as closely as possible the advantages of RCT (Muijs 2004 : 27). In quasi-experiments randomization is not possible (Muijs 2004 ). This makes it difficult to eliminate bias. The experimental group is already determined—they are the ones enjoying or experiencing the treatment of concern to the researcher. What the researcher does is to compare this group with another that is not experiencing the treatment. Often, the treatment is a government programme or some other kind of intervention out of the researcher’s control. Where it is possible to have another group to compare with, the researcher might work with data before treatment comparing that with data after treatment.

Take, for instance, the introduction of government-funded public examinations in some Nigerian prisons in 2019. Would the incidence of violence reduce in prisons because of this policy? A few years into the policy, a researcher might compare incidence of violence in Prison A where the policy is being implemented with Prison B where it is not being implemented. Or, where, for certain reasons that two-prison comparison is not possible, she might compare data on the incidence violence in Prison A before the policy with data on the incidence of violence in the same prison after the policy has been implemented.

Quasi-experiment templates consider space (spatial variation) and time (temporal variation) as important aspects that influence setting up of experimental research designs. Gerring ( 2007 ) and other scholars provide a variety of these templates. For instance, a researcher might be interested in explaining if and how a certain programme or policy, say a school feeding programme, increases students’ performance in national examinations. She can select two local government areas or sub-counties—one with a school feeding programme and the other without—and then compare school performances of students in both sub-counties and local government areas in national examination. It is important to ensure that the two cases (i.e. sub-counties or local government areas in this example) are similar in all other factors that might influence students’ performance in a national examination, the only difference being the presence of a school feeding programme in one and its absence in the other. The data can be collected by a variety of means—questionnaire, secondary data such as attendance registers, observation guide or any other that suits the research objective and question. A fruitful study of this type does not stop at showing that students in local government A where there is a school feeding programme perform better than their counterparts in local government B. That would be an interesting finding, but it leaves a lot unsaid. Rather, it should press on with an explanation of the causal mechanism—the pathway or trajectory by which the school feeding programme leads to better grades. This implies that what is largely categorized as quantitative study may require aspects of qualitative data to allow the researcher to get a complete picture of the issue under investigation.

Ojebode et al. ( 2016 ) attempted to explain the (in)effectiveness of community-based crime prevention practices in Ibadan, Nigeria. They selected two communities—one with a successful community-based crime prevention programme and another with a clearly unsuccessful one. These communities are similar in all the factors that matter to community-based crime prevention—population, ethnic mix, youth population, socio-economic status, and both have community-based crime prevention practices. Their puzzle was: why did the practice work so well in one community and fail so woefully in the other despite the similarities in these communities. Through different rounds of data collection and different instruments, their quantitative and qualitative analysis shows that the settlement patterns in the communities—dating several hundred years—perhaps explained the variation in the outcomes of the crime prevention practices.

Natural Experiments

Natural experiments take advantage of exogenous effect, that is, an intervention that is outside of the control of the researcher, which was also not intended to affect the outcome/dependent variable. The exogenous effect can be in the form of natural (such as a natural disaster), physical (like in the case of the colonial/government border) or historical event. They may also be a policy intervention. These were not intended for research or academic purposes. In other words, what becomes the treatment or causal factor happens through some “natural” occurrence or unplanned event. In some ways, these events may allow for observation of before and after they occurred. An example is Friedman et al. ( 2001 ) who carried out a kind of natural experiment during the 1996 Olympic games in Atlanta, Georgia. The researchers wanted to find out whether heavy traffic in the city was a cause of asthma in children. They made observations on how the city was organized during the 17 days of Olympics where the traffic rules changed. Small cars were forced onto alternative routes to leave main routes for mass transport, and this reduced traffic congestion on the major roads of the city. Through paediatric records (before and after Olympics), the study discovered 40% reduction in asthma attacks and emergency hospitalization. The researchers made a conclusion that traffic congestion contributes to paediatric asthma. This can be classified as a natural experiment, where the Olympics (manipulation/treatment) was not planned by the researcher and was exogenous (not related in any known way) to asthma. Such critically thought-out research can easily contribute to change in transport policies. Other studies, for instance, Daniel Posner’s on Chewas and Timbukas of Zambia and Malawi ( 2004 ), have used borders artificially created by colonial governments as boundaries of study groups. In his case, Posner shows how governments in two countries differently exploit similar ethnic compositions and the effect of this exploitation on inter-ethnic relations.

Non-experimental Quantitative Methods

Most quantitative researchers collect data using a standard questionnaire containing mostly close-ended questions. Some researchers may use a questerview, which combine both closed-ended and open-ended question. The latter is applicable when corroborative data or explanations to the closed-ended questions are needed. Survey questionnaires for this reason provide some standardized data that can be keyed into software for organization and analysis. The type of survey questionnaire depends on the nature of data that the researcher requires, the reach of the study population and ways in which the data is to be collected. One can decide to do telephone interviews, post the questionnaire, administer it online or have an ordinary written questionnaire.

Survey research considers a variety of factors including samples and sampling procedures, characteristics of the study population, among other issues. Surveys mainly apply probability sampling with an aim of giving all the elements a chance to be included in the study sample. This is the opposite of non-probability sampling those centres on purposive and convenient sampling. There are various sampling techniques in probability sampling, and these are available in various research methods books. Just to mention, some of the probability sampling approaches include simple random, stratified random, cluster, quota and multistage (see Muijs 2004 , 2011 ; Babbie 2004 ; Kothari 2004 ; Kumar 2011 ). For sample sizes, there are suggested formulas that researchers can apply for both finite and infinite populations.

Observational Studies

Observations are important for both qualitative and quantitative research. In quantitative research, observation is applied both as a research method and as a method of data collection. In qualitative research, observation is mostly categorized as a method of data collection and features in various research methods including ethnography, case study and action research. In quantitative studies, observational methods are important, for they enable a researcher to interact with the study environment and participants in a way that the questionnaire would not. Observational data for quantitative research is collected using standardized/structured observation schedules. A researcher can develop a descriptive observational record or a rating scale to help them collect observational data. This enables the researcher to observe and record the behaviour and activities in the selected study sites in a standardized way. Observations can also be made on existing reports within the institutions being studied, say for instance, school performance and statistical data collected from such reports (see Muijs 2004 ). In the end, the different methods may generate descriptive data of various types, that is, from open-ended and closed-ended descriptions. The selection of participants is also randomized to give all a chance to participate, and subsequently, those falling within the sample size are meant to represent the study population on which generalizations can be made.

Applied Quantitative Method

This method makes use of existing data sets. It applies analytical methods to facilitate description of data that has already been recorded and stored. Different research institutes store varied forms of data sets. These could be useful if a researcher is interested in analysing them with the purpose of achieving a certain research objective. For instance, one might be interested in understanding and describing the population growth trends. In such instances, one does not need to go to the field to collect fresh information when the national bureaux or offices of statistics have the data sets. All one needs is to get permission from relevant authorities to access such information. The challenge with using such data sets is that if they are erroneous in any way, then the errors are carried forth in the analysis. As Muijs ( 2004 ) indicates, the various quantitative research methods can be combined in a single study if this is necessary.

Mixed Methods in Public Policy Research

The advent of mixed-method research and the place that it currently occupies in social science research reinforce the arguments for the use of both traditions of qualitative and quantitative methods in public policy research. Statistics should be complemented and explained by meaning-making concepts, metaphors, symbols and descriptions from qualitative research to make sense of hard data. On the other hand, narratives on their own are not enough. Jones and McBeth ( 2010 : 330) show that despite the apparent power of stories in public policy, public policy studies have largely remained on the side-lines of the use narratives. The two scholars suggest the relevance of using a narrative policy framework as a methodological complement for positivists in the study of policy. Some scholars have also shown that for policy problems to be clearly defined, a narrative structure is needed. Narration, as Fischer ( 1998 ) and Stone ( 2002 : 138) explain, helps make sense of the socially constructed world that requires tangible solutions. Since qualitative approach may not be able to engage hypothesis testing to allow for replication and falsification (Jones and McBeth 2010 : 339), they should complement or be complemented by quantitative data.

Qualitative and quantitative methods have their own separate strengths. As noted above, qualitative research is about depth and qualitative is about breadth. This means, if a study requires both, then mixing the methods is important. Mixing methods therefore means a research problem requires both qualitative and quantitative data. Morse ( 1991 ) argued that triangulation of methods not only maximizes the strengths and minimizes those weaknesses of each approach, but also strengthens research results and contributes to theory and knowledge development.

Mixing research methods does not just imply mixing methods of data collection. A researcher must intentionally clarify which research methods (as discussed above) are applicable in their research to speak to qualitative and quantitative aspects, and by extension what methods of data collection will be used. Note that one research method may have many methods and tools data collection. If one is using ethnography, then participant observation, oral in-depth interviews, observations and focus group discussions are examples of applicable data collection methods. The various methods of data collection have their instruments/tools.

Mixing of methods entirely depends on the purpose for which the methods are mixed. This is determined by the research problem. Mixed research methods books provide a wide range of typologies of designing mixed-method research (see, for instance, Greene et al. 1989 ; Creswell and Clark 2011 ; Schoonenboom and Johnson 2017 ). Below is a simple illustration of the continuum for mixing methods (Fig. 4.2 ). A researcher can move from a purely quantitative or qualitative research method (A and E), towards integrating either quantitative (B) or qualitative (D) methods to the dominant method. A researcher can also design a fully mixed-method research (C). This is a simplified way of understanding how mixing can happen; there are other more complex typologies.

The mixed-method continuum. (Source: Teddlie and Yu 2007 : 84)

In public policy research, the mixing is important for various reasons. One might require results for complementary purpose, explanations to the statistical results, expansion of results from one domain (qualitative or quantitative) or confirmation of results. The dictates of mixing are found within the research problem and by extension research questions/objectives.

There is subtle blame game between bureaucrats and policy makers, on the one hand, and researchers, on the other hand, in Africa. While the latter accuse the former of not using the research they conduct, the former responds by claiming that many of the research do not speak to policy or societal issues and are thus not usable. They add that many of them are rendered in a language that is not accessible to non-academic actors. As a result, not a few policy decisions are based on political and other judgements rather than on sound research.

Our discussion so far suggests that the bureaucrats and policy makers may not be totally right in their accusation, but they are not totally wrong either. The preponderance of policy-appended research, and of solo-method research which offers little as a basis for policy, seems to justify their accusation. It is, therefore, important that public policy researchers weave their research around societal issues that are not only significant but also contemporary and topical, craft their design with the aim of policy engagement and stakeholder involvement, and adopt mixed methods as and when necessary, to provide findings and conclusion that command and compel policy actors’ attention.

Babbie, E. 2004. The Practice of Social Research . 10th ed. Belmont: Wandsworth.

Google Scholar  

Bachman, R. 2007. Causation and Research Design. In The Practice of Research in Criminology and Criminal Justice , ed. R. Bachman and R. Schutt, 3rd ed., 141–169. Thousand Oaks, CA: Sage Publications.

Brewer, J. 2000. Ethnography . Buckingham: Philadelphia. Open University Press.

Creswell, John W., and Vicki L. Plano Clark. 2011. Designing and Conducting Mixed Methods Research . 2nd ed. Los Angeles: Sage.

Edwards, M. 2004. Social Science Research and Public Policy: Narrowing the Divide. Occasional Paper 2/2004. Policy Paper #2. Academy of the Social Sciences in Australia. Canberra. ASSA.

Elçi, A., and Devran, B.C. (2014). A Narrative Research Approach: The Experiences of Social Media Support in Higher Education, in P. Zaphiris (Eds.): Human-Computer Interaction, Part I, HCII 2014, LNCS 8523, pp. 36–42.

Fischer, F. 1998. Beyond Empiricism: Policy Inquiry in Postpositivist Perspective. Policy Studies Journal 26 (1): 129–146.

Article   Google Scholar  

Friedman, M., K. Powell, L. Hutwagner, L. Graham, and W. Teague. 2001. Impact of Changes in Transportation and Commuting Behaviours During the 1996 Summer Olympic Games in Atlanta on Air Quality and Childhood Asthma. JAMA. 285 (7): 897–905.

Gerring, J. 2007. Case Study Research Principles and Practices . Cambridge: Cambridge University Press.

Greene, J.C., V.J. Caracelli, and W.F. Graham. 1989. Toward a Conceptual Framework for Mixed-Method Evaluation Designs. Educational Evaluation and Policy Analysis 11: 255–274.

Howlett, M. 2012. The Lessons of Failure: Learning and Blame Avoidance in Public Policymaking. International Political Science Review 33 (5): 539–555.

Huang, B.H. 2010. What Is Good Action Research? Why the Resurgent Interest? Action Research 8 (1): 93–109.

Johnson, Burke, and Larry B. Christensen. 2014. Educational Research: Quantitative, Qualitative, and Mixed Approaches . Thousand Oaks, Calif.: Sage Publications.

Jones, M., and M. McBeth. 2010. A Narrative Policy Framework: Clear Enough to Be Wrong. The Policy Studies Journal 38 (2): 329–353.

Kothari, C. 2004. Research Methodology, Methods and Techniques . New Delhi: Wishwa Prakashan.

Kumar, R. (2011). Research Methodology: A Step by Step Guide for Beginners. London: Sage Publications Ltd. Third Edition.

Lemke, A., and J. Harris-Wai. 2015. Stakeholder Engagement in Policy Development: Challenges and Opportunities for Human Genomics. Genetics in Medicine 17 (12): 949–957.

Mead, L. 2005. Policy Research: The Field Dimension. Policy Studies Journal 33 (4): 535–557.

———. 2013. Teaching Public Policy: Linking Policy and Politics. JPAE 19 (3): 389–403.

Morse, M. 1991. Approaches to Qualitative and Quantitative Methodological: Triangulation. Qualitative Research 40 ( 1 ): 120–123.

Muijs, D. 2004. Doing Quantitative Research in Education . London: Sage Publications.

Book   Google Scholar  

———. 2011. Doing Quantitative Research in Education with SPSS . 2nd ed. London: SAGE Publications.

Ojebode, A., Ojebuyi, B. R., Onyechi, N. J., Oladapo, O., Oyedele, O., and Fadipe, I. 2016. Explaining the Effectiveness of Community-Based Crime Prevention Practices in Nigeria. Institute of Development Studies, University of Sussex. http://www.ids.ac.uk/publication/explaining-the-effectiveness-of-community-based-crime-prevention-practices-in-ibadan-nigeria .

Osifo, C. 2015. Public Management Research and a Three Qualitative Research Strategy. Review of Pub. Administration and Management 3 (1): 149–156.

Oxman, A., S. Lewin, J. Lavis, and A. Fretheim. 2009. Support Tools for Evidence-Informed Health Policymaking (STP) 15: Engaging the Public in Evidence-Informed. Health Research Policy and Systems 7 (1): S15 Policymaking.

Oyedele, O., M. Atela, A. Ojebode. 2017. Two lessons for early involvement of stakeholders in research. https://i2insights.org/2017/11/14/early-stakeholder-involvement/

Posner, D. 2004. The Political Salience of Cultural Difference: Why Chewas and Tumbukas Are Allies in Zambia and Adversaries in Malawi. The American Political Science Review 98 (4): 529–545.

Roche, M. 2016. Historical Research and Archival Sources. In Qualitative Research Methods in Human Geography , ed. Iain Hay, 4th ed., 225–245. New York: Oxford University Press.

Saetren, H. 2005. Facts and Myths About Research on Public Policy Implementation: Out-of-Fashion, Allegedly Dead, But Still Alive and Relevant. Policy Studies Journal 33 (4): 559–582–559–582.

Schoonenboom, J., and B. Johnson. 2017. How to Construct a Mixed Methods Research Design. Köln Z Soziol 69 (Suppl 2): 107–131.

Stone, D. 2002. Policy Paradox: The Art of Political Decision Making, Revised Edition . 3rd ed. New York: W. W. Norton.

Strauss, A., and J. Corbin. 1994. Grounded Theory Methodology: An Overview. In Handbook of Qualitative Research , ed. N.K. Denzin and Y.S. Lincoln, 273–285. Thousand Oaks: Sage.

Teddlie, C., and F. Yu. 2007. Mixed Methods Sampling: A Typology with Examples. Journal of Mixed Methods Research. 1 (1): 77.

Tierney, W.G., and R.F. Clemens. 2011. Qualitative Research and Public Policy: The Challenges of Relevance and Trustworthiness. In Higher Education: Handbook of Theory and Research. Higher Education: Handbook of Theory and Research , ed. J. Smart and M. Paulsen, vol. 26, 57–83. Dordrecht: Springer.

Chapter   Google Scholar  

UK Cabinet Office. 2009. Professional Policy Making for the 21st Century. Report by Strategic Policy Making Team. September, 7(1).

Young, J. 2005. Research, Policy and Practice: Why Developing Countries Are Different. Journal of International Development 17: 727–734.

Download references

Author information

Authors and affiliations.

Department of Religious Studies, Maseno University, Maseno, Kenya

Susan Mbula Kilonzo

Department of Communication and Language Arts, University of Ibadan, Ibadan, Nigeria

Ayobami Ojebode

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Susan Mbula Kilonzo .

Editor information

Editors and affiliations.

Department of Political Science, University of Ibadan, Ibadan, Nigeria

E. Remi Aiyede

PASGR, Nairobi, Kenya

Beatrice Muganda

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

© 2023 The Author(s)

About this chapter

Cite this chapter.

Kilonzo, S.M., Ojebode, A. (2023). Research Methods for Public Policy. In: Aiyede, E.R., Muganda, B. (eds) Public Policy and Research in Africa. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-99724-3_4

Download citation

DOI : https://doi.org/10.1007/978-3-030-99724-3_4

Published : 19 October 2022

Publisher Name : Palgrave Macmillan, Cham

Print ISBN : 978-3-030-99723-6

Online ISBN : 978-3-030-99724-3

eBook Packages : Political Science and International Studies Political Science and International Studies (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Open access
• Published: 06 March 2021

How to bring research evidence into policy? Synthesizing strategies of five research projects in low-and middle-income countries

• Séverine Erismann 1 , 2 ,
• Maria Amalia Pesantes 3 ,
• David Beran 4 ,
• Andrea Leuenberger 1 , 2 ,
• Andrea Farnham 1 , 2 ,
• Monica Berger Gonzalez de White 1 , 2 , 5 ,
• Niklaus Daniel Labhardt 1 , 2 , 6 ,
• Fabrizio Tediosi 1 , 2 ,
• Patricia Akweongo 7 ,
• August Kuwawenaruwa 1 , 2 , 8 ,
• Jakob Zinsstag 1 , 2 ,
• Fritz Brugger 9 ,
• Claire Somerville 10 ,
• Kaspar Wyss 1 , 2 &
• Helen Prytherch 1 , 2  

Health Research Policy and Systems volume  19 , Article number:  29 ( 2021 ) Cite this article

24k Accesses

17 Citations

13 Altmetric

Metrics details

Addressing the uptake of research findings into policy-making is increasingly important for researchers who ultimately seek to contribute to improved health outcomes. The aims of the Swiss Programme for Research on Global Issues for Development (r4d Programme) initiated by the Swiss National Science Foundation and the Swiss Agency for Development and Cooperation are to create and disseminate knowledge that supports policy changes in the context of the 2030 Agenda for Sustainable Development. This paper reports on five r4d research projects and shows how researchers engage with various stakeholders, including policy-makers, in order to assure uptake of the research results.

Eleven in-depth interviews were conducted with principal investigators and their research partners from five r4d projects, using a semi-structured interview guide. The interviews explored the process of how stakeholders and policy-makers were engaged in the research project.

Three key strategies were identified as fostering research uptake into policies and practices: (S1) stakeholders directly engaged with and sought evidence from researchers; (S2) stakeholders were involved in the design and throughout the implementation of the research project; and (S3) stakeholders engaged in participatory and transdisciplinary research approaches to coproduce knowledge and inform policy. In the first strategy, research evidence was directly taken up by international stakeholders as they were actively seeking new evidence on a very specific topic to up-date international guidelines. In the second strategy, examples from two r4d projects show that collaboration with stakeholders from early on in the projects increased the likelihood of translating research into policy, but that the latter was more effective in a supportive and stable policy environment. The third strategy adopted by two other r4d projects demonstrates the benefits of promoting colearning as a way to address potential power dynamics and working effectively across the local policy landscape through robust research partnerships.

Conclusions

This paper provides insights into the different strategies that facilitate collaboration and communication between stakeholders, including policy-makers, and researchers. However, it remains necessary to increase our understanding of the interests and motivations of the different actors involved in the process of influencing policy, identify clear policy-influencing objectives and provide more institutional support to engage in this complex and time-intensive process.

Peer Review reports

Increasingly, research funders are asking their grantees to address the uptake of research findings into decision-making processes and policy-making [ 1 , 2 ]. This growing trend is a response to a need for real-world and context-sensitive evidence to respond to and address complex health systems and health service delivery bottlenecks faced by policy-makers, health practitioners, communities and other actors that require more than single interventions to induce large-scale change [ 3 ]. Moreover, there is growing pressure for applied and implementation research to be relevant, demonstrate value for money and result in high-impact publications. The relevance of ensuring the translation of research into practice is also reflected in growing support for research projects with concrete requirements regarding the evaluation of their impact of science on society [ 4 ].

One example of the above is the Swiss Programme for Research on Global Issues for Development (r4d Programme) initiated by the Swiss National Science Foundation (SNSF) and the Swiss Agency for Development and Cooperation (SDC) covering the period 2012–2022. The r4d Programme is aimed at researchers in Switzerland and low-and middle-income countries (LMICs) conducting projects that specifically focus on poverty reduction and the protection of public goods in developing countries. Its specific objectives are to create and disseminate knowledge that supports policy-making in the area of global development and foster research on global issues in the context of the 2030 Agenda for Sustainable Development [ 5 , 6 ].

While the linkage of research to policy is strongly encouraged by research funding agencies, the uptake of research evidence by policy-makers to establish new laws and regulations or to improve policies to solve a problem or enhance implementation effectiveness, especially in LMICs, remains weak [ 2 , 7 ]. This is often referred to as the gap between research and policy [ 8 ]. One of the factors that was identified with the dearth of research uptake in previous studies is a lack of evidence that is context sensitive, timely and relevant for policy-makers; other factors include difficulties in accessing existing evidence, challenges with correctly interpreting and using existing evidence [ 7 , 9 ] and also a lack of interest from policy-makers in the use and uptake of evidence [ 10 ]. Using the SNSF r4d funding scheme, our aim is to show how researchers have engaged with stakeholders, including policy-makers, from the onset of a research project, in order to identify strategies for evidence uptake and use.

As part of the r4d Programme, several synthesis initiatives have been launched to disseminate the research evidence from the r4d projects and increase its impact ( http://www.r4d.ch/r4d programme/synthesis ). The aim of one of these synthesis initiatives is to support knowledge translation and exchange, as well as knowledge diffusion and dissemination among 15 r4d projects focusing on public health. More specifically, the aim is to facilitate the uptake of findings for the benefit of societies in LMICs, especially with regards to social inclusion and gender equity in the drive towards universal health coverage (UHC) and the 2030 Agenda for Sustainable Development [ 6 ]. The present study and resulting article are part of this synthesis initiative.

In this article, we present—through five case studies—strategies to translate and bridge evidence emerging from research into policy-making and decision-making. We rely on the experiences of five public health projects within the r4d research initiative. This paper describes these experiences, reports on the lessons learnt and outlines important features and challenges of engaging in this process using the researchers’ perspectives. This paper contributes to the body of literature on research translation by highlighting concrete examples and successful strategies for the uptake of research evidence in policy formulation.

Invitations were sent out to researchers working on projects within the r4d Programme to share their experiences with the project. Based on the interest shown by researchers, five projects were selected by the authors to demonstrate the different approaches and strategies used in the r4d projects with the aim to influence policy. Researchers were asked to share descriptions of the different approaches used in seeking to influence the uptake of research results by policy-makers. Each project represents a case study with emphasis on the main features of their translational approaches and the challenges, enablers and successes encountered.

The different research–policy engagement strategies were identified through data analysis of the interviews conducted within the framework of the five r4d case studies and were inspired by the work conducted by Uzochukwu and colleagues in Nigeria [ 2 ], who described four detailed strategies to support evidence-informed policy-making: (1) policy-makers and stakeholders seeking evidence from researchers; (2) involving stakeholders in designing objectives of a research project and throughout the research period; (3) facilitating policy-maker–researcher engagement in optimizing ways of using research findings to influence policy and practice; (4) active dissemination of own research findings to relevant stakeholders and policy-makers (see Table 1 ).

In using the term stakeholder, we apply the following definition by Brinkerhoff and Crosby [ 11 ]: “A stakeholder is an individual or group that makes a difference or that can affect or be affected by the achievement of the organization’s objectives”. Hence, individual stakeholders can include politicians (heads of state and legislators), government bureaucrats and technocrats from various sectors (e.g. health), but also representatives of civil society organizations and support groups [ 12 ].

Data collection

Eleven in-depth interviews with principal investigators and their research partners from five r4d projects were conducted by the first author, using a semi-structured interview guide. The interview guide covered the following themes: (1) How were stakeholders involved in the research project? (2) Was there uptake of research evidence in national/international policies? (3) How were research results disseminated? (4) What were the challenges or obstacles encountered in disseminating and translating evidence from research to policy? The interview duration was between 30 and 45 min. Seven interviews were conducted with researchers based in Switzerland and four with researchers in LMICs. At least two interviews were conducted for each r4d case study.

Data management and analysis

Of the 11 interviews, nine were audio recorded and notes taken. Audio files were transcribed verbatim by the same researcher. Two interviews were not recorded, but detailed notes were taken during the interview.

A qualitative content analysis method was used in order to organize and structure both the manifest and latent content [ 13 ]. Aligned to overall study questions, essential content was identified by the first author, which involved a process of generating a provisional list of themes of interest that were based on the study objectives, including stakeholder involvement in the generation of research questions, research process, generation of results and dissemination of research findings, as well as challenges to research dissemination and policy uptake. In a next step, the transcripts were sorted and grouped by the first author according to the coding scheme for analysis. This involved using the content summary analysis method, which consists of reducing the textual content and preserving only the essential content in order to produce a short text [ 14 ]. As several co-authors were interviewed, they validated that their perspective was not misinterpreted or misrepresented.

Three key strategies were identified for research uptake into policy and practice throughout the data collection of this synthesis initiative: (S1) stakeholders directly engaged with and sought evidence from researchers; (S2) stakeholders were involved in the design and throughout the implementation of the research project; and (S3) stakeholders engaged in participatory and transdisciplinary research approaches to co-produce knowledge and inform policy. The first two strategies (S1, S2) are in line with Uzochukwu and colleagues’ work [ 2 ], and the third strategy (S3) is an additional category based on the experiences of researchers in r4d projects [ 2 ]. Each r4d project is described in more detail as a case study in one of these three strategies (Table 2 ).

S1: stakeholders directly engaged with and sought evidence from researchers

In this strategy, international stakeholders requested evidence from the research team. This is a unique (and rare) strategy, as stated by Uzochukwu et al. [ 2 ], and often involves a policy window of opportunity in which stakeholders, including policy-makers, are looking to solve a particular problem, which coincides with the publishing of a scientific report or paper and the interests of these same groups [ 15 , 16 ].

Improving the HIV care cascade in Lesotho: towards 90-90-90—a research collaboration with the Ministry of Health of Lesotho

In this r4d project, the research team was approached by the International Aids Society (IAS) and the World Health Organization (WHO) in Geneva, based on the publication of their study protocol [ 17 ], introducing their innovative research approach of same-day antiretroviral therapy (ART) initiation in rural communities in Lesotho:

“They [international stakeholders] were all keen of getting the results out and requested evidence of the randomized controlled trials. We shared the results confidentially with WHO as soon as we had the data and thereafter published the results in a journal with a wide reach. WHO as well as other international guidelines and policy committees took up the recommendation of same-day ART initiation and informed global guidelines” (Researcher 1).

As a result, many HIV programmes in sub-Saharan Africa as well as in the global north have adopted the practice of offering rapid-start ART to persons who test HIV positive even outside a health facility. In this example, the policy window and direct stakeholder engagement was crucial for the effective translation and uptake of research evidence.

Furthermore, by closely collaborating with national policy-makers, the research team advocated for the setting up of a research database and of knowledge management units within the Ministry of Health (MoH) of Lesotho, which have been successfully established. The members of the research project consortia have also initiated a national research symposium on a bi-annual basis, which is chaired by the MoH with the aim of facilitating the dissemination and uptake of research findings.

S2: Stakeholders were involved in the design and throughout the implementation of the research project

In this strategy, policy uptake is facilitated through stakeholder engagement from the beginning as well as during the conduct of research activities, through participating at workshops or functioning in the governance of the projects. Two r4d projects illustrate this strategy.

Health system governance for an inclusive and sustainable social health protection in Ghana and Tanzania

This project established a Country Advisory Group (CAG) at the start that included representatives of the main stakeholders of the social health protection systems. The CAGs were involved in all phases of the project, from the definition of the research plans to the dissemination of the results. The specific research questions addressed by the project emerged from the interactions with these main stakeholders, i.e. national policy-makers, healthcare providers and members of the social health protection schemes (the NHIS and the Livelihood Empowerment Against Poverty schemes in Ghana; and the National Health Insurance Fund, the Community Health Funds and the Tanzania Social Action Fund in Tanzania). Specifically in Ghana, the following stakeholders played a major role in shaping the research plan: the Ministry of Gender Children and Social Protection (MGCSP), the Ghana Health Service (Policy Planning and Monitoring and Evaluation Division; Research and Development Division), the National Health Insurance Authority (NHIA) and the Associations of Private Health Care Providers. In Tanzania, a major role was played by the Ministry of Health, Community, Development, Gender, Elderly and Children, the President’s Office—Regional Administration and Local Government, by representatives of civil society organizations, such as Sikika, by the SDC (Swiss Agency for Development and Cooperation) Health Promotion and System Strengthening project and by the SDC-supported development programme.

These stakeholders were subsequently involved in steering the research, as captured by a researcher:

“In Ghana, it was a balanced relationship. They were involved since the very beginning of the project in articulating what the information gap at policy level is, formulating the research questions and understanding the methods/what is feasible. In Tanzania, where the policy landscape is more fragmented, it was very important to listen to the voices of several different stakeholders” (Researcher 2).

The stakeholder consultations in Ghana and Tanzania initially involved discussions on the relevance of the research plans to address the existing gaps in strengthening the social health protection scheme, the synergies with other research initiatives and the feasibility of implementing the proposed research. Later on in the project, the consultation process involved reviewing and discussing the focus of the research and the appropriateness of the research aims in light of decisions and reforms that were under discussion by the government but not in the public domain. This led to revision of the research questions as they would have become redundant when such reforms were made public, especially in Ghana. These consultation processes were more formal in Ghana and more informal in Tanzania, but they were very informative and had a tangible impact on the research plans, which were revised according to the feedback received. However, the research teams were always independent in deciding on the research methodology and in interpreting the results. The in-country dissemination of the results at the end of the first phase of the project informed the decisions to be made on the research plan for the second phase and provided the opportunity to discuss policy implications based on the results of the first phase. Because of this close collaboration and engagement with stakeholders, the results of the studies were widely disseminated in Ghana. Two of the main findings of the project were particularly considered by these stakeholders. According to the researcher:

“First, the study results showed that even though people registered with the NHIS they continued to pay out of pocket for health services. The reasons for this were delays in reimbursement by NHIS, escalating prices of drugs and medical products, low tariffs, lack of trust between providers and NHIA and inefficiencies. Secondly, the results showed that the current system of targeting the poor is not working properly, with more than half of people registered in the NHIS as indigents being in the non-poor socio-economic groups. These results contributed to inform decisions regarding the revision of the NHIA reimbursement tariffs, and to improve the identification of the poor to be exempted from paying the NHIS premium, in collaboration with the MGCSP” (Researcher 3).

In Tanzania, research was conducted to assess the effects of the public private partnership, referred as the Jazia Prime Vendor System (Jazia PVS), on improving access to medicines in the Dodoma and Morogoro regions in Tanzania. This is one of the reforms in the area of supply chain management taking place in the country. Results showed that a number of accountability mechanisms (inventory and financial auditing, close monitoring of standard operating procedures) implemented in conjunction with Jazia PVS contributed positively to the performance of Jazia PVS. Participants’ acceptability of Jazia PVS was influenced by the increased availability of essential medicines at the facilities, higher-order fulfilment rates and timely delivery of the consignment [ 18 , 19 , 20 ].

The findings from this study were disseminated during the national meeting attended by various stakeholders, including CAG members, government officials and policy-makers. In addition, the findings were used to inform the national scale-up of the Jazia PVS intervention as the government of Tanzania decided to scale up the Jazia PVS to all the 23 regions in 2018. Moreover, the results/manuscripts were published or submitted to peer-reviewed journals [ 18 , 19 , 20 ], enabling other countries intending to adopt such innovate public–private partnerships for improvement of the in-country pharmaceutical supply chain to learn from Jazia PVS in Tanzania.

Health impact assessment for engaging natural resource extraction projects in sustainable development in producer regions (HIA4SD)

In this r4d project, stakeholders were involved from the outset through their participation in the project launch meeting and in regular consortium meetings. The project is a collaboration between the Swiss Tropical and Public Health Institute (Swiss TPH), the Center for Development and Cooperation (NADEL) at the Swiss Federal Institute of Technology in Zurich/Switzerland and national research institutes, namely the Institut de Recherches en Sciences de la Santé in Burkina Faso, the University of Health and Allied Sciences in Ghana, the Centro de Investigação em Saúde de Manhiça in Mozambique and the Ifakara Health Institute in Tanzania [ 21 ]. The involvement of key stakeholders from the government, civil society, private sector and research community in an engaged dialogue from the beginning iss of central importance in this project, as in most cases mining is a highly politicized topic. To promote the immediate integration of research findings into policy, the project is organized into two streams, namely an “impact research stream” and a “governance stream”, that work in parallel. While the impact research stream is focused on evidence generation to support the uptake of health impact assessment (HIA) in Africa, the governance stream is focused on understanding the policy terrain and consequently the pathways that need to be utilized to support translation of the evidence into policy and practice. The second phase of the study is devoted to the dissemination of research findings into policy at the national and local levels, including capacity-building activities for national stakeholders. As the HIA4SD project examines operational questions of relevance for guiding both policy-making and decision-making, team members sought to regularly engage with and inform the national stakeholders. According to the researcher:

“Strategies employed to influence policy vary according to the country, but included regular stakeholder workshops, participation in a new national platform launched to discuss issues around mining in Mozambique, development of policy briefs, strengthened collaborations with national ministries of health, discussion of results and advocacy with policy makers, and conference presentation of findings” (Researcher 4).

In these two case examples, continuous stakeholder engagement was considered essential to translate and disseminate research evidence. Thus, beyond the stage of setting the objectives, contact with stakeholders was active and maintained on a regular basis through regular exchanges with stakeholder groups during workshops or meetings, which facilitated the dissemination and uptake of the research results. While the time and level of meaningful interaction varied across the countries and workshops, all meetings were well attended by participants from varied levels of government, MoHs, nongovernmental organizations and private industry, prompting spirited discussion and insight from these groups. All stakeholders were willing to attend these workshops as part of the scope of their professional duties.

S3: stakeholders engaged in participatory and transdisciplinary research approaches to co-produce knowledge and inform policy

In the two examples presented in this section, the research questions and approaches arose through community and stakeholder participation in the research and intervention design itself. The methodology adopted allowed them to engage, design research, act, share and sustain partnerships between the communities, the involved stakeholders and researchers [ 22 ]. These participatory research approaches facilitated grassroot-level policy and practice changes which were not researcher nor policy maker led, and that show promising approaches for developing culturally aligned solutions [ 23 ]. Policy makers at both the regional and national levels were invited to be part of the participatory research approach: they were interviewed during the initial stage, then the research results were presented and discussed with them; thereafter, we had several meetings to co-create potential interventions to address the identified problems, with the aim to directly engage in the research and intervention design itself in partnerships with the community stakeholders, including local leaders, and the researchers.

Surveillance and response to zoonotic diseases in Maya communities of Guatemala: a case for OneHealth

The research was embedded in a collaboration between the Universidad del Valle in Guatemala, the MoH, the Ministry of Animal Production and Health, the Maya Qéqchi’ Council of Elders, TIGO Telecommunications Foundation and the community development councils. The objective of this r4d programme was to set up integrated animal–human disease surveillance (OneHealth) in Maya communities in Guatemala. The research approach arose from a context of medical pluralism, where communities have access to and use two different medical systems: (1) the modern Western medical system and (2) traditional Maya medicine [ 24 ].

Researchers and community members collaborated at all stages of the research process, including the planning stage. Even before the grant proposal was finalized, researchers met with the communities that, should the funding come through, would be invited to participate in the research. According to the researchers:

“The project was set up through a transdisciplinary process, with academic and non-academic actors—including national, local and traditional authorities—involved in the problem through a collaborative design, analysis, dissemination and research translation. It was a co-producing transformative process—transferring knowledge between academic and non-academic stakeholders in plenary sessions and through group work. These meetings were held every year to continuously follow up the progress of the process” (Researcher 7).

The active engagement and collaboration by the community and stakeholders facilitated the acceptability of the study results and hence its dissemination, captured by the researchers as follows:

“The main result was that they allowed a frank discussion between Maya medical exponents in human–animal health and Western medicine, which allowed the patients and the animal holders to avoid the cognitive dissonance and so that the patients or the animal holders can choose freely what they want. Cognitive dissonance exists if one system dominates the other—or refutes the other” (Researcher 7).

“After all stakeholders discussed the research evidence produced jointly, an unprecedented process of collaboration between Government authorities and communities followed to develop three joint responses: a) education campaigns led by local teachers in tandem with the Ministry of Education, b) communication strategies at regional levels led by the Human and Animal Health authorities along with traditional Maya Ajilonel (medicine specialists), and c) a policy framework for producing a OneHealth approach led by Central Government authorities” (Researcher 8).

The process of mutual learning throughout the project produced a new level of awareness, facilitating culturally pertinent and socially robust responses that overcame a historical tendency of unilateral policy making based solely on Western values and preferences. As the project implemented a new approach to monitoring animal and human populations, the involvement of regional teams from the different ministries (Health, Livestock and Agriculture) throughout all the phases of methodological design, data collection, posterior data analysis and design of specific interventions for the local population (transformation of scientific results into actions for public health improvement) was essential to ensuring that the approach used secured the regional authorities’ commitment to defining new policies for immediate application in their territory. Accordingly, this also contributed towards the development of a OneHealth national strategy for Guatemala in which ministries start to cooperate to take up priority issues.

Addressing the double burden of disease: improving health systems for non-communicable and neglected tropical diseases (Community Health System Innovation [COHESION])

Together with three Swiss academic partners, this r4d project examined the challenges of a double burden of non-communicable and neglected tropical diseases at the primary healthcare level in vulnerable populations in Mozambique, Nepal and Peru. Community participation and co-creation were key elements of the project’s approach. The work conducted in Peru illustrates this approach:

“At the beginning, the people who were involved were respondents, but then they became active participants. So it was this active engagement and the changing of roles, giving feedback not just from the research responses but also from being involved in the process, which helped to design and create interventions together with the research team” (Researcher 5).

This participatory approach to co-creation actively sought a diverse range of stakeholders, including community members, primary healthcare workers, and regional and national health authorities. The co-creation approach to participatory research enables context-specific variation in methodological design, a critical element when studying three very different countries and health systems. Central to all aspects was a feedback loop whereby early findings were shared with research participants for further elaboration and iteration.

As active co-creators of the research process, local communities developed high levels of trust in the methodology and data, with the result that researchers achieved deeper “buy-in” which in turn is known to enhance the uptake of findings by decision-makers [ 25 ] as communities in which research is being undertaken play a central role in the decision-making process [ 26 ].

Challenges to research uptake in health policy identified by r4d researchers

During the interviews, r4d researchers identified several challenges to research utilization and uptake into policy. These challenges are summarized and highlighted in Table 3 .

Three key strategies identified for research uptake in policy and practice are described in this paper, namely: (S1) stakeholders directly engaged with and sought evidence from researchers; (S2) stakeholders were involved in the design and throughout the implementation of the research project; and (S3) stakeholders engaged in participatory and transdisciplinary research approaches to co-produce knowledge and inform policy. These strategies are in line with the overall objectives of the r4d projects, which are to generate scientific knowledge and research-based solutions to reduce poverty and global risks in LMICs, and also to offer national and international stakeholders integrated approaches to solving problems [ 5 ]. In the course of our synthesis work, we found that several lessons could be learned from the three strategies identified for research uptake in policy and practice.

S1: raising awareness of planned research to attract stakeholder involvement

The actual uptake of research findings in policy was most direct in the case of the first strategy (S1), in which IAS and WHO stakeholders were wanting new knowledge on HIV and same-day ART initiation, and were actively seeking new evidence on these specific topics. The findings published in peer-reviewed journals were then taken up by these stakeholders to update international policies and guidelines on rapid ART initiation [ 27 ]. This was also found in other studies, highlighting the importance of the timeliness and relevance of findings and the production of credible and trustworthy reports, among others, as key factors in promoting the use of research evidence in policy [ 2 , 28 ].

S2: sustainable collaborations in a supportive policy environment with stakeholder engagement from early on and throughout the research process

With regards to the second strategy (S2), we found that constant collaboration with an advisory and steering group composed of diverse stakeholders, including policy-makers, from early on promotes the uptake and use of research evidence. In line with findings from other studies [ 2 ], the experiences encountered in the r4d public health projects show that early involvement of stakeholders in the processes to identify the research problem and set the priorities facilitated the continuous exchange of information that might ultimately influence policy. The r4d project on social governance mechanisms in Ghana highlight that the evidence produced influenced policy documents (identification of the poor and tariff adjustments), but that frequent changes government officials made it difficult to maintain a close relationship between the researchers and the governmental agencies/policy stakeholders. From this, we draw the conclusion that research approaches need to be more adaptive and flexible to be successful in an unsupportive or unstable policy environment to ensure continuity in promoting the dissemination and uptake of research evidence in policy-making. One possible manner to secure this transformation is for researchers to apply for additional funding after the grant is finished. Other studies have also come to this conclusion, thereby demonstrating the key role of a supportive and effective policy environment that includes some degree of independence in governance and financing, strong links to stakeholders that facilitate trust and influence and also the capacity within the government workforce to process and apply policy advice developed by the research findings [ 29 ]. By involving stakeholders in the process of identifying research objectives and designing the project, as seen particularly in the r4d case studies on social health protection in Ghana and Tanzania and the HI4SD, but also in the HIV care cascade in Lesotho, the research approach responded to the need of locally led and demand-driven research in these countries, strengthening local research capacities and institutions, but also investing in research that is aligned with the national research priorities. As highlighted by other authors, advantages of this “demand-driven” approach is that it tailors research questions to local needs, helps to strengthen local individual and organizational capacities and provides a realized stringent framework on which a research project should deliver outcomes [ 30 , 31 ].

S3: co-creation and equal partnerships

The third strategy with a strong participatory approach, such as that adopted by two r4d projects, OneHealth in Guatemala and COHESION, demonstrates benefits to promoting co-learning as a way to minimize the impact of unequal power dynamics and to work effectively across the local policy landscape through equal partnerships. It also facilitates identifying solutions that are culturally pertinent, socially more robust and implementable.

The approaches of co-creation, equal participation and stakeholder involvement used in the research projects raise questions of ‘governance’, that is the way rules, norms and actions are structured, sustained and regulated by public and para-public actors to condition the engagement and impact of public involvement activities [ 32 , 33 ]. Through stakeholder involvement in setting the agenda and designing the research projects, as shown in the case studies on social protection in Ghana and Tanzania and the HI4SD project, but particularly in the two projects using a co-creation approach, the engagement of a range of stakeholders serves to make the health research systems a participaant in the endeavor that then has the capacity to promote changes in the healthcare system it aims to serve. By establishing a shared vision with a public involvement agenda and through the collaborative efforts of various stakeholders, as we found particularly in the co-creation approach, supportive health research systems are established. This leads to greater public advancement through collaborative actions, thereby tackling the stated problems of the health systems [ 34 ].

There were four key challenges mentioned by the respondents during the interviews to research uptake in policy making. The first was the necessary time investment by researchers to translate the result and develop policy advocacy products for the different audiences. This challenge is all the more difficult because research evidence and tangible products only become available towards the end of a research project, leaving only a short window of opportunity for exchange and engagement. There seems to be a need for wider discussion on the role of researchers in influencing policy. The concerns raised included whether influencing policy is actually a role for researchers and whether researchers have the right skills to be effective in persuasion or network formation [ 35 ]. Conversely, researchers may be in a good position to engage in the policy process if they enjoy finding solutions to complex problems while working with diverse and collaborative groups in partnerships [ 36 , 37 ]. The rationale for engaging in such a process needs to be clarified in advance: is the aim to frame an existing problem, or is it to simply measure the issues at stake and provide sound evidence according to an existing frame? Regarding the the former, how far should researchers go to be useful and influential in the policy process or to present challenges faced by vulnerable populations [ 37 ]? While fully engaging in the policy process may be the best approach for researchers to achieve credibility and impact, there may also be significant consequences, such as the risk of political interests undermining the methodological rigour of academic research (being considered as academic ‘lightweight’ among one’s peer group) [ 38 , 39 , 40 , 41 ]. For researchers there is also considerable opportunity costs because engaging in the policy-influencing process is a time-consuming activity [ 35 ], with no clear guarantee of the impact of success [ 37 ]. It is therefore crucial to consider the investment and overall time researchers may have to spend to engage [ 35 ], and how this time and investment can be distributed between actual research and the production of outreach products, such as policy briefs, presentation of research findings as policy narratives [ 35 ] and the setting-up of alliances, building of networks and exploitation of windows of opportunity for policy change [ 37 ].

The second challenge included the issue of scale and objectivity, as most of the projects are not scaled or national-level studies and thus are highly context specific. The difficulty to measure the contributions of a single research project or study in terms of policy outcomes was also highlighted, particularly in view of the different understandings among researchers and funders on the possible policy impacts of the research, which can range from guiding policy-makers to understand a situation or problem (awareness raising) to influencing a particular course of action by establishing new or revising existing policies. This has also been emphasized in the Evidence Peter Principle [ 42 ], showing that single studies are often inappropriately used to make global policy statements for which they are not suitable. To make global policy statements, an assessment of the global evidence in systematic reviews is needed [ 42 , 43 ].

The third challenge mentioned was the frequent changes in staff at the governmental level, which demanded continuous interactions between r4d researchers and stakeholders, highlighting the need for more adaptive and flexible research approaches. These should include a thorough analytical process prior to implementation in historical, sociopolitical and economic aspects, power differentials and context; backward planning exercises to check assumptions; and conflict transformation and negotiation skills in order to be able to constantly adapt to changing contexts. In line with our research findings, when researchers make the time investment needed to engage in the policy-influencing process, an opportunity is provided to getting know the involved stakeholders better and improve their understanding of the policy world in practice, but also to build diverse and longer-term networks [ 37 , 44 ] and to identify policy problems and the appropriate stakeholders to work with [ 45 , 46 ]. Engaging a diverse range of stakeholders through co-designing the research is widely held to be practically the best way to guarantee the uptake and use of evidence in policy through a more dynamic research approach [ 47 ]. However, the development of networks and contacts for collaboration, as well as the skills to do so, takes time and effort and is an ongoing process [ 48 ], factors which need to be acknowledged more widely.

Lastly, the fourth challenge related to research uptake was the diverging interests between researchers, research funding bodies and stakeholders. Time was identified as a limiting factor from the perspective of the design of the research project. Most research projects, including the r4d projects, are funded for 3–4 years [ 5 ]. It takes a considerable amount of time to generate new research results, and often these are more likely to be produced for further use at the end of a project. If researchers should engage more fully in the policy process to secure meaningful impact, it is critical to discuss the extent to which they have the skills, resources and institutional support to do so [ 37 ], as well as how projects could be set up differently. This could be done either by the funders in providing the necessary support that allows researchers to have the means to impact policy, or by the researchers in the design of their project to take on board the different strategies to influence evidence use and uptake. In moving forward, defining shared goals from the outset between funders and the researchers might translate to more achievable milestones in terms of which policy issue, theme or process a research project aims to change in order to effectively influence policy [ 49 ]. This would help to identify the resources and budget needed by the funders in order for the researchers to engage with more resources over a longer time span in this process.

Limitations

Interviews were limited to researchers of the r4d projects and did not include local stakeholders. Therefore, the synthesis work, including the analysis and results, reflects solely the perspective of researchers. We are aware that had we included a range of stakeholders, including policy-makers, in the sample, we would have potentially been able to identify additional factors relating to social, cultural and political barriers to the use and uptake of research findings in politics and practice. However, constraints such as access to local stakeholders, language barriers and time zones drove our decision to focus on researchers. A future synthesis effort would need to include the other voices.

There is ever growing awareness of how critical it is to close the gap between policy-makers, practitioners and researchers. Using the researchers’ perspectives, in this article we give insight into three different strategies that can facilitate this process, with the first strategy requiring proactive searching for the latest findings on the part of well-informed policy-makers, the second requiring researchers to take steps to ensure an active exchange of ideas and information with diverse stakeholders when designing the research project and ensuring the latter’s involvement throughout; and the third using a transdisciplinary and/or co-creation approach to establish equal partnerships and trust among all involved stakeholders.

The five case studies reported here also show some of the difficulties that prevail for research to be taken up into policy and practice, despite everyone’s best intentions and efforts. Researchers may not always be best placed for communication, dissemination and advocacy work, all activities which are very time intensive or become important only towards the end of a research project when clear and high-quality evidence is produced. Moreover, it takes a strong body of evidence, advocacy and coalition building with appropriate stakeholders to influence policy, and then a further major effort of resources to see policy followed through into practice. It is through experiences such as this synthesis initiative that precious insights and learning can be gained for the common good of all involved moving forward, and it is crucial that funders continue to support and/or adapt their funding schemes to ensure some of these strategies are implemented.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Antiretroviral therapy

Country Advisory Group

Community Health System Innovation

Health impact assessment

Health impact assessment for engaging natural resource extraction projects in sustainable development in producer regions

Human immunodeficiency virus

International Aids Society

Jazia Prime Vendor System

Low- and middle-income countries

Ministry of Health

Ministry of Gender Children and Social Protection

Center for Development and Cooperation at the Swiss Federal Institute of Technology

National Health Insurance Authority

National Health Insurance Scheme

Swiss Programme for Research on Global Issues for Development

Swiss Agency for Development and Cooperation

Swiss National Science Foundation

Swiss Tropical and Public Health Institute

Universal health coverage

World Health Organization

Court J, Young J. Bridging research and policy in international development: an analytical and practical framework. J Dev Pract. 2006;16(1):85–90.

Article   Google Scholar  

Uzochukwu B, Onwujekwe O, Mbachu C, Okwuosa C, Etiaba E, Nyström ME, Gilson L. The challenge of bridging the gap between researchers and policy makers: experiences of a Health Policy Research Group in engaging policy makers to support evidence informed policy making in Nigeria. Glob Health. 2016;12(1):67.

Di Ruggiero E, Edwards N. The interplay between participatory health research and implementation research: Canadian research funding perspectives. Biomed Res Int. 2018;2018:1519402.

Rau H, Goggins G, Fahy F. From invisibility to impact: recognising the scientific and societal relevance of interdisciplinary sustainability research. Res Policy. 2018;47(1):266–76.

Swiss National Science Foundation: Swiss Programme for Research on Global Issues for Development (r4d programme). http://www.r4d.ch/r4d programme/portrait . Accessed 20 Jan 2020.

United Nations: Sustainable Development Goals. https://sustainabledevelopment.un.org/ . Accessed 17 Dec 2019.

Shroff ZC, Javadi D, Gilson L, Kang R, Ghaffar A. Institutional capacity to generate and use evidence in LMICs: current state and opportunities for HPSR. Health Res Policy Syst. 2017;15(1):94.

McKee M. Bridging the gap between research and policy and practice Comment on “CIHR health system impact fellows: reflections on ‘driving change’ within the health system.” Int J Health Policy Manag. 2019;8(9):557–9.

World Health Organization. Sound choices: enhancing capacity for evidence-informed health policy. In: Bennett S, Green A, editors. Geneva: WHO; 2007.

Stoker G, Evans M. Evidence-based policy making in the social sciences: methods that matter. Bristol: Policy Press; 2016.

Book   Google Scholar  

Brinkerhoff DW, Crosby B. Managing policy reform: concepts and tools for decision-makers in developing and transitioning countries. Sterling: Kumarian Press; 2002.

Google Scholar  

Hardee KFI, Boezwinkle J, Clark B. A framework for analyzing the components of family planning, reproductive health, maternal health, and HIV/AIDS policies. Wilmette: The Policy Circle; 2004.

Mayring P. Qualitative Inhaltsanalyse. In: Mey G, Mruck K, editors. Handbuch qualitative Forschung in der Psychologie. Wiesbaden: Springer; 2010. pp. 601–613.

Mayring P, Fenzl T. Qualitative Inhaltsanalyse. In: Baur N, Blasius J, editors. Handbuch Methoden der empirischen Sozialforschung. Wiesbaden: Springer; 2014. pp. 543–556.

Rose DC, Amano T, González-Varo JP, Mukherjee N, Robertson RJ, Simmons BI, Wauchope HS, Sutherland WJ. Calling for a new agenda for conservation science to create evidence-informed policy. Biol Conserv. 2019;238:108222.

Shiffman J, Smith S. Generation of political priority for global health initiatives: a framework and case study of maternal mortality. Lancet. 2007;370(9595):1370–9.

Labhardt ND, Ringera I, Lejone TI, Klimkait T, Muhairwe J, Amstutz A, Glass TR. Effect of offering same-day ART vs usual health facility referral during home-based HIV testing on linkage to care and viral suppression among adults with HIV in Lesotho: the CASCADE randomized clinical trial. JAMA. 2018;319(11):1103–12.

Kuwawenaruwa A, Wyss K, Wiedenmayer K, Metta E, Tediosi F. The effects of medicines availability and stock-outs on household’s utilization of healthcare services in Dodoma region, Tanzania. Health Policy Plan. 2020;35(3):323–33.

Kuwawenaruwa A TF, Metta E, Obrist B, Wiedenmayer K, Msamba V, Wyss K. Acceptability of a prime vendor system in public healthcare facilities in Tanzania. Int J Health Policy Manag. 2020. (in press).

Kuwawenaruwa ATF, Obrist B, Metta E, Chiluda F, Wiedenmayer K, Wyss K. The role of accountability in the performance of Jazia prime vendor system in Tanzania. J Pharm Policy Pract. 2020;2020(13):25.

Farnham A, Cossa H, Dietler D, Engebretsen R, Leuenberger A, Lyatuu I, Nimako B, Zabre HR, Brugger F, Winkler MS. Investigating health impacts of natural resource extraction projects in Burkina Faso, Ghana, Mozambique, and Tanzania: protocol for a mixed methods study. JMIR Res Protoc. 2020;9(4):e17138.

Beran D, Lazo-Porras M, Cardenas MK, Chappuis F, Damasceno A, Jha N, Madede T, Lachat S, Perez Leon S, Aya Pastrana N, Pesantes MA, Singh SB, Sharma S, Somerville C, Suggs LS, Miranda JJ. Moving from formative research to co-creation of interventions: insights from a community health system project in Mozambique, Nepal and Peru. BMJ Gob Health. 2018;3(6):e001183.

Mertens DM. Advancing social change in South Africa through transformative research. S Afr Rev Sociol. 2016;47(1):5–17.

Berger-González M, Stauffacher M, Zinsstag J, Edwards P, Krütli P. Transdisciplinary research on cancer-healing systems between biomedicine and the Maya of Guatemala: a tool for reciprocal reflexivity in a multi-epistemological setting. Qual Health Res. 2016;26(1):77–91.

Theron LC. Using research to influence policy and practice: the case of the pathways-to-resilience study (South Africa). In: Abubakar A, van de Vijver FJR, editors. Handbook of applied developmental science in sub-Saharan Africa. New York: Springer; 2017. pp. 373–87.

Baum F, MacDougall C, Smith D. Participatory action research. J Epidemiol Community Health. 2006;60(10):854–7.

WHO. Guidelines for managing advanced HIV disease and rapid initiation of antiretroviral therapy. Geneva: World Health Organization; 2017.

Lavis JN, Oxman AD, Lewin S, Fretheim A. SUPPORT Tools for evidence-informed health policymaking (STP) 3: setting priorities for supporting evidence-informed policymaking. Health Res Policy Syst. 2009;7(S1):I3.

Bennett S, Corluka A, Doherty J, Tangcharoensathien V, Patcharanarumol W, Jesani A, Kyabaggu J, Namaganda G, Hussain AMZ, de-Graft Aikins A. Influencing policy change: the experience of health think tanks in low- and middle-income countries. Health Policy Plan. 2011;27(3):194–203.

Kok MO, Gyapong JO, Wolffers I, Ofori-Adjei D, Ruitenberg EJ. Towards fair and effective North–South collaboration: realising a programme for demand-driven and locally led research. Health Res Policy Syst. 2017;15(1):96.

Wolffers I, Adjei S. Research-agenda setting in developing countries. Lancet. 1999;353(9171):2248–9.

Article   CAS   Google Scholar  

Dodgson R, Lee K, Drager N. Global health governance: a conceptual review. London: Centre on Global Change and Health, London School of Hygiene & Tropical Medicine/World Health Organization; 2018.

Saltman RB, Ferroussier-Davis O. The concept of stewardship in health policy. Bull World Health Organ. 2000;78(6):732–9.

CAS   PubMed   PubMed Central   Google Scholar  

Miller FA, Patton SJ, Dobrow M, Marshall DA, Berta W. Public involvement and health research system governance: a qualitative study. Health Res Policy Syst. 2018;16(1):87.

Lloyd J. Should academics be expected to change policy? Six reasons why it is unrealistic for research to drive policy change. https://blogs.lse.ac.uk/impactofsocialsciences/2016/05/25/should-academics-be-expected-to-change-policy-six-reasons-why-it-is-unrealistic/ . Accessed 28 May 2020.

Petes LE, Meyer MD. An ecologist’s guide to careers in science policy advising. Front Ecol Environ. 2018;16(1):53–4.

Oliver KCP. The dos and don’ts of influencing policy: a systematic review of advice to academics. Palgrave Commun. 2019;5(1):21.

Hutchings JA, Stenseth NC. Communication of science advice to government. Trends Ecol Evol. 2016;31(1):7–11.

Maynard A. Is public engagement really career limiting?. https://www.timeshighereducation.com/blog/public-engagement-really-career-limiting . Accessed 27 May 2020.

Haynes AS, Derrick GE, Chapman S, Redman S, Hall WD, Gillespie J, Sturk H. From “our world” to the “real world”: exploring the views and behaviour of policy-influential Australian public health researchers. Soc Sci Med. 2011;72(7):1047–55.

Crouzat E, Arpin I, Brunet L, Colloff MJ, Turkelboom F, Lavorel S. Researchers must be aware of their roles at the interface of ecosystem services science and policy. Ambio. 2018;47(1):97–105.

White H. The Evidence Peter Principle: the misuse and abuse of evidence Reflections on the evidence architecture. https://www.campbellcollaboration.org/blog/the-evidence-peter-principle-the-misuse-and-abuse-of-evidence.html?utm_source=Campbell+Collaboration+newsletters&utm_campaign=4dfa01ec7d-Newsletter+September+2019&utm_medium=email&utm_term=0_ab55bacb0c-4dfa01ec7d-199138457 . Accessed 29 Jan 2020.

Caird J, Sutcliffe K, Kwan I, Dickson K, Thomas J. Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach? Evid Policy J Res Debate Pract. 2015;11(1):81–97.

Evans MC, Cvitanovic C. An introduction to achieving policy impact for early career researchers. Palgrave Commun. 2018;4(1):88.

Echt L. Context matters:” a framework to help connect knowledge with policy in government institutions. LSE Impact Blog. https://blogs.lse.ac.uk/impactofsocialsciences/2017/12/19/context-matters-a-framework-to-help-connect-knowledge-with-policy-in-government-institutions/ . Accessed 13 July 2020.

Lucey JM, Palmer G, Yeong KL, Edwards DP, Senior MJM, Scriven SA, Reynolds G, Hill JK. Reframing the evidence base for policy-relevance to increase impact: a case study on forest fragmentation in the oil palm sector. J Appl Ecol. 2017;54(3):731–6.

Green D: How academics and NGOs can work together to influence policy: insights from the InterAction report. LSE Impact blog https://blogs.lse.ac.uk/impactofsocialsciences/2016/09/23/how-academics-and-ngos-can-work-together-to-influence-policy-insights-from-the-interaction-report/ . Accessed 13 July 2020.

Boaz A, Hanney S, Borst R, O’Shea A, Kok M. How to engage stakeholders in research: design principles to support improvement. Health Res Policy Syst. 2018;16(1):60.

Tilley HSL, Rea J, Ball L, Young J. 10 things to know about how to influence policy with research. London: Overseas Development Institute; 2017.

Download references

Acknowledgements

The authors would like to acknowledge the contribution of Dr Claudia Rutte from the r4d programme/SNSF for her inputs to the history and background of the r4d programme.

The r4d synthesis initiative is implemented by the Swiss Tropical and Public Health Institute, which funded the costs of publishing this paper.

Author information

Authors and affiliations.

Swiss Tropical and Public Health Institute, Basel, Switzerland

Séverine Erismann, Andrea Leuenberger, Andrea Farnham, Monica Berger Gonzalez de White, Niklaus Daniel Labhardt, Fabrizio Tediosi, August Kuwawenaruwa, Jakob Zinsstag, Kaspar Wyss & Helen Prytherch

University of Basel, Basel, Switzerland

CRONICAS Centre of Excellence in Chronic Diseases, Universidad Peruana Cayetano Heredia, Lima, Peru

Maria Amalia Pesantes

Division of Tropical and Humanitarian Medicine, University of Geneva and Geneva University Hospitals, Geneva, Switzerland

David Beran

Centro de Estudios en Salud, Universidad del Valle de Guatemala, Guatemala, Guatemala

Monica Berger Gonzalez de White

Department of Infectious Diseases and Hospital Epidemiology, University Hospital Basel, Basel, Switzerland

Niklaus Daniel Labhardt

School of Public Health, College of Health Sciences, University of Ghana, Accra, Ghana

Patricia Akweongo

Ifakara Health Institute, Plot 463, Kiko Avenue Mikocheni, Dar es Salaam, Tanzania

August Kuwawenaruwa

Swiss Federal Institute of Technology Zurich, Zurich, Switzerland

Fritz Brugger

Gender Centre, Graduate Institute of International and Development Studies, Geneva, Switzerland

Claire Somerville

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the writing of this manuscript. Each author contributed with synthesizing their project experiences and with the discussion and recommendations. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Séverine Erismann or Helen Prytherch .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Erismann, S., Pesantes, M.A., Beran, D. et al. How to bring research evidence into policy? Synthesizing strategies of five research projects in low-and middle-income countries. Health Res Policy Sys 19 , 29 (2021). https://doi.org/10.1186/s12961-020-00646-1

Download citation

Received : 15 July 2020

Accepted : 15 October 2020

Published : 06 March 2021

DOI : https://doi.org/10.1186/s12961-020-00646-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Evidence-based policy-making
• Research for development

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Balancing Federalism: The Impact of Decentralizing School Accountability

Education policy, while primarily the responsibility of the state governments, involves complicated decision making at the local, state, and federal levels. The federal involvement dramatically increased with the introduction of test-based accountability under the No Child Left Behind Act of 2001. But, reflecting resistance to various parts of this law, the involvement of federal policy making was substantially reduced when Congress passed the Every Student Succeeds Act in 2015. This change in policy allows estimation of the impact of altered federalism. By looking at how states reacted to their enhanced decision-making role, we see a retreat from the use of output-based policy toward teachers, and this retreat was associated with significantly lower student achievement growth. As a result, this readjustment of federalism to decision making by lower levels appeared to lower national achievement. The snapshot of federalism impacts here is a lower bound on the effects as more states will very likely react to the flexibility of ESSA and as more school districts change their teacher force.

Paper prepared for the annual meetings of the Association for Public Policy Analysis and Management in Atlanta. We benefitted from the overall support of the National Council on Teacher Quality and the assistance of Kelli Lakis and Lisa Staresina in developing the cross-walk between the elements of teacher policies and the provisions of NCLB and ESSA. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.

MARC RIS BibTeΧ

Download Citation Data

More from NBER

In addition to working papers , the NBER disseminates affiliates’ latest findings through a range of free periodicals — the NBER Reporter , the NBER Digest , the Bulletin on Retirement and Disability , the Bulletin on Health , and the Bulletin on Entrepreneurship  — as well as online conference reports , video lectures , and interviews .

Choose Your Test

Sat / act prep online guides and tips, 113 great research paper topics.

General Education

One of the hardest parts of writing a research paper can be just finding a good topic to write about. Fortunately we've done the hard work for you and have compiled a list of 113 interesting research paper topics. They've been organized into ten categories and cover a wide range of subjects so you can easily find the best topic for you.

In addition to the list of good research topics, we've included advice on what makes a good research paper topic and how you can use your topic to start writing a great paper.

What Makes a Good Research Paper Topic?

Not all research paper topics are created equal, and you want to make sure you choose a great topic before you start writing. Below are the three most important factors to consider to make sure you choose the best research paper topics.

#1: It's Something You're Interested In

A paper is always easier to write if you're interested in the topic, and you'll be more motivated to do in-depth research and write a paper that really covers the entire subject. Even if a certain research paper topic is getting a lot of buzz right now or other people seem interested in writing about it, don't feel tempted to make it your topic unless you genuinely have some sort of interest in it as well.

#2: There's Enough Information to Write a Paper

Even if you come up with the absolute best research paper topic and you're so excited to write about it, you won't be able to produce a good paper if there isn't enough research about the topic. This can happen for very specific or specialized topics, as well as topics that are too new to have enough research done on them at the moment. Easy research paper topics will always be topics with enough information to write a full-length paper.

Trying to write a research paper on a topic that doesn't have much research on it is incredibly hard, so before you decide on a topic, do a bit of preliminary searching and make sure you'll have all the information you need to write your paper.

#3: It Fits Your Teacher's Guidelines

Don't get so carried away looking at lists of research paper topics that you forget any requirements or restrictions your teacher may have put on research topic ideas. If you're writing a research paper on a health-related topic, deciding to write about the impact of rap on the music scene probably won't be allowed, but there may be some sort of leeway. For example, if you're really interested in current events but your teacher wants you to write a research paper on a history topic, you may be able to choose a topic that fits both categories, like exploring the relationship between the US and North Korea. No matter what, always get your research paper topic approved by your teacher first before you begin writing.

113 Good Research Paper Topics

Below are 113 good research topics to help you get you started on your paper. We've organized them into ten categories to make it easier to find the type of research paper topics you're looking for.

Arts/Culture

• Discuss the main differences in art from the Italian Renaissance and the Northern Renaissance .
• Analyze the impact a famous artist had on the world.
• How is sexism portrayed in different types of media (music, film, video games, etc.)? Has the amount/type of sexism changed over the years?
• How has the music of slaves brought over from Africa shaped modern American music?
• How has rap music evolved in the past decade?
• How has the portrayal of minorities in the media changed?

Current Events

• What have been the impacts of China's one child policy?
• How have the goals of feminists changed over the decades?
• How has the Trump presidency changed international relations?
• Analyze the history of the relationship between the United States and North Korea.
• What factors contributed to the current decline in the rate of unemployment?
• What have been the impacts of states which have increased their minimum wage?
• How do US immigration laws compare to immigration laws of other countries?
• How have the US's immigration laws changed in the past few years/decades?
• How has the Black Lives Matter movement affected discussions and view about racism in the US?
• What impact has the Affordable Care Act had on healthcare in the US?
• What factors contributed to the UK deciding to leave the EU (Brexit)?
• What factors contributed to China becoming an economic power?
• Discuss the history of Bitcoin or other cryptocurrencies  (some of which tokenize the S&P 500 Index on the blockchain) .
• Do students in schools that eliminate grades do better in college and their careers?
• Do students from wealthier backgrounds score higher on standardized tests?
• Do students who receive free meals at school get higher grades compared to when they weren't receiving a free meal?
• Do students who attend charter schools score higher on standardized tests than students in public schools?
• Do students learn better in same-sex classrooms?
• How does giving each student access to an iPad or laptop affect their studies?
• What are the benefits and drawbacks of the Montessori Method ?
• Do children who attend preschool do better in school later on?
• What was the impact of the No Child Left Behind act?
• How does the US education system compare to education systems in other countries?
• What impact does mandatory physical education classes have on students' health?
• Which methods are most effective at reducing bullying in schools?
• Do homeschoolers who attend college do as well as students who attended traditional schools?
• Does offering tenure increase or decrease quality of teaching?
• How does college debt affect future life choices of students?
• Should graduate students be able to form unions?

• What are different ways to lower gun-related deaths in the US?
• How and why have divorce rates changed over time?
• Is affirmative action still necessary in education and/or the workplace?
• Should physician-assisted suicide be legal?
• How has stem cell research impacted the medical field?
• How can human trafficking be reduced in the United States/world?
• Should people be able to donate organs in exchange for money?
• Which types of juvenile punishment have proven most effective at preventing future crimes?
• Has the increase in US airport security made passengers safer?
• Analyze the immigration policies of certain countries and how they are similar and different from one another.
• Several states have legalized recreational marijuana. What positive and negative impacts have they experienced as a result?
• Do tariffs increase the number of domestic jobs?
• Which prison reforms have proven most effective?
• Should governments be able to censor certain information on the internet?
• Which methods/programs have been most effective at reducing teen pregnancy?
• What are the benefits and drawbacks of the Keto diet?
• How effective are different exercise regimes for losing weight and maintaining weight loss?
• How do the healthcare plans of various countries differ from each other?
• What are the most effective ways to treat depression ?
• What are the pros and cons of genetically modified foods?
• Which methods are most effective for improving memory?
• What can be done to lower healthcare costs in the US?
• What factors contributed to the current opioid crisis?
• Analyze the history and impact of the HIV/AIDS epidemic .
• Are low-carbohydrate or low-fat diets more effective for weight loss?
• How much exercise should the average adult be getting each week?
• Which methods are most effective to get parents to vaccinate their children?
• What are the pros and cons of clean needle programs?
• How does stress affect the body?
• Discuss the history of the conflict between Israel and the Palestinians.
• What were the causes and effects of the Salem Witch Trials?
• Who was responsible for the Iran-Contra situation?
• How has New Orleans and the government's response to natural disasters changed since Hurricane Katrina?
• What events led to the fall of the Roman Empire?
• What were the impacts of British rule in India ?
• Was the atomic bombing of Hiroshima and Nagasaki necessary?
• What were the successes and failures of the women's suffrage movement in the United States?
• What were the causes of the Civil War?
• How did Abraham Lincoln's assassination impact the country and reconstruction after the Civil War?
• Which factors contributed to the colonies winning the American Revolution?
• What caused Hitler's rise to power?
• Discuss how a specific invention impacted history.
• What led to Cleopatra's fall as ruler of Egypt?
• How has Japan changed and evolved over the centuries?
• What were the causes of the Rwandan genocide ?

• Why did Martin Luther decide to split with the Catholic Church?
• Analyze the history and impact of a well-known cult (Jonestown, Manson family, etc.)
• How did the sexual abuse scandal impact how people view the Catholic Church?
• How has the Catholic church's power changed over the past decades/centuries?
• What are the causes behind the rise in atheism/ agnosticism in the United States?
• What were the influences in Siddhartha's life resulted in him becoming the Buddha?
• How has media portrayal of Islam/Muslims changed since September 11th?

Science/Environment

• How has the earth's climate changed in the past few decades?
• How has the use and elimination of DDT affected bird populations in the US?
• Analyze how the number and severity of natural disasters have increased in the past few decades.
• Analyze deforestation rates in a certain area or globally over a period of time.
• How have past oil spills changed regulations and cleanup methods?
• How has the Flint water crisis changed water regulation safety?
• What are the pros and cons of fracking?
• What impact has the Paris Climate Agreement had so far?
• What have NASA's biggest successes and failures been?
• How can we improve access to clean water around the world?
• Does ecotourism actually have a positive impact on the environment?
• Should the US rely on nuclear energy more?
• What can be done to save amphibian species currently at risk of extinction?
• What impact has climate change had on coral reefs?
• How are black holes created?
• Are teens who spend more time on social media more likely to suffer anxiety and/or depression?
• How will the loss of net neutrality affect internet users?
• Analyze the history and progress of self-driving vehicles.
• How has the use of drones changed surveillance and warfare methods?
• Has social media made people more or less connected?
• What progress has currently been made with artificial intelligence ?
• Do smartphones increase or decrease workplace productivity?
• What are the most effective ways to use technology in the classroom?
• How is Google search affecting our intelligence?
• When is the best age for a child to begin owning a smartphone?
• Has frequent texting reduced teen literacy rates?

How to Write a Great Research Paper

Even great research paper topics won't give you a great research paper if you don't hone your topic before and during the writing process. Follow these three tips to turn good research paper topics into great papers.

#1: Figure Out Your Thesis Early

Before you start writing a single word of your paper, you first need to know what your thesis will be. Your thesis is a statement that explains what you intend to prove/show in your paper. Every sentence in your research paper will relate back to your thesis, so you don't want to start writing without it!

As some examples, if you're writing a research paper on if students learn better in same-sex classrooms, your thesis might be "Research has shown that elementary-age students in same-sex classrooms score higher on standardized tests and report feeling more comfortable in the classroom."

If you're writing a paper on the causes of the Civil War, your thesis might be "While the dispute between the North and South over slavery is the most well-known cause of the Civil War, other key causes include differences in the economies of the North and South, states' rights, and territorial expansion."

#2: Back Every Statement Up With Research

Remember, this is a research paper you're writing, so you'll need to use lots of research to make your points. Every statement you give must be backed up with research, properly cited the way your teacher requested. You're allowed to include opinions of your own, but they must also be supported by the research you give.

#3: Do Your Research Before You Begin Writing

You don't want to start writing your research paper and then learn that there isn't enough research to back up the points you're making, or, even worse, that the research contradicts the points you're trying to make!

Get most of your research on your good research topics done before you begin writing. Then use the research you've collected to create a rough outline of what your paper will cover and the key points you're going to make. This will help keep your paper clear and organized, and it'll ensure you have enough research to produce a strong paper.

What's Next?

Are you also learning about dynamic equilibrium in your science class? We break this sometimes tricky concept down so it's easy to understand in our complete guide to dynamic equilibrium .

Thinking about becoming a nurse practitioner? Nurse practitioners have one of the fastest growing careers in the country, and we have all the information you need to know about what to expect from nurse practitioner school .

Want to know the fastest and easiest ways to convert between Fahrenheit and Celsius? We've got you covered! Check out our guide to the best ways to convert Celsius to Fahrenheit (or vice versa).

These recommendations are based solely on our knowledge and experience. If you purchase an item through one of our links, PrepScholar may receive a commission.

Christine graduated from Michigan State University with degrees in Environmental Biology and Geography and received her Master's from Duke University. In high school she scored in the 99th percentile on the SAT and was named a National Merit Finalist. She has taught English and biology in several countries.

Student and Parent Forum

Our new student and parent forum, at ExpertHub.PrepScholar.com , allow you to interact with your peers and the PrepScholar staff. See how other students and parents are navigating high school, college, and the college admissions process. Ask questions; get answers.

Ask a Question Below

Have any questions about this article or other topics? Ask below and we'll reply!

Improve With Our Famous Guides

• For All Students

The 5 Strategies You Must Be Using to Improve 160+ SAT Points

How to Get a Perfect 1600, by a Perfect Scorer

Series: How to Get 800 on Each SAT Section:

Score 800 on SAT Math

Score 800 on SAT Reading

Score 800 on SAT Writing

Series: How to Get to 600 on Each SAT Section:

Score 600 on SAT Math

Score 600 on SAT Reading

Score 600 on SAT Writing

Free Complete Official SAT Practice Tests

What SAT Target Score Should You Be Aiming For?

15 Strategies to Improve Your SAT Essay

The 5 Strategies You Must Be Using to Improve 4+ ACT Points

How to Get a Perfect 36 ACT, by a Perfect Scorer

Series: How to Get 36 on Each ACT Section:

36 on ACT English

36 on ACT Math

36 on ACT Reading

36 on ACT Science

Series: How to Get to 24 on Each ACT Section:

24 on ACT English

24 on ACT Math

24 on ACT Reading

24 on ACT Science

What ACT target score should you be aiming for?

ACT Vocabulary You Must Know

ACT Writing: 15 Tips to Raise Your Essay Score

How to Get Into Harvard and the Ivy League

How to Get a Perfect 4.0 GPA

How to Write an Amazing College Essay

What Exactly Are Colleges Looking For?

Is the ACT easier than the SAT? A Comprehensive Guide

Should you retake your SAT or ACT?

When should you take the SAT or ACT?

Stay Informed

Get the latest articles and test prep tips!

Looking for Graduate School Test Prep?

Check out our top-rated graduate blogs here:

GRE Online Prep Blog

GMAT Online Prep Blog

TOEFL Online Prep Blog

Holly R. "I am absolutely overjoyed and cannot thank you enough for helping me!”

Harvard Kennedy School faculty members get creative and collaborative in a new artificial intelligence course module

“The Science and Implications of Generative AI” equips learners with the skills to use AI technology responsibly for societal benefit.

Since Open AI’s ChatGPT arrived at the end of 2022, generative artificial intelligence has been big news, with many companies scrambling to develop their own tools. The technology is already changing the way people work and learn, provoking excitement about its potential and anxiety about misuse.

To help Harvard Kennedy School students better understand generative AI — technology that can generate images or text based on prompts, such as ChatGPT—faculty members Sharad Goel, Dan Levy, and Teddy Svoronos developed an interdisciplinary course module, DPI-681M , “The Science and Implications of Generative AI ,” which they are teaching for the first time this semester. The course provides a background in how the technology works, plenty of hands-on exercises, and a curriculum that emphasizes how HKS students—future policymakers and public leaders—“can harness AI technology responsibly for the benefit of society.” They have also made much of the module materials public —including short videos, readings, and exercises—so that more people can benefit from these lessons.

“It’s important that when people leave the Kennedy School to go into policy positions, they have knowledge and informed opinions about generative AI.”

Teddy svoronos.

Sharad Goel , a professor of public policy, recalls that the idea for the course started in the early fall 2023. A number of HKS faculty members were experimenting with generative AI in the core courses, including an AI tool they called StatGPT that helped students in the core MPP courses practice and learn statistics. Goel found students were coming to him in office hours looking to learn about generative AI, and he realized there weren’t many opportunities at Harvard to do so. 

The hope, Goel says, is that HKS students “become sophisticated and responsible users of AI.”

Goel worked with Levy , a senior lecturer in public policy, and Svoronos , a lecturer in public policy, to develop the module quickly, despite full teaching loads. It was important and timely. Svoronos says he was concerned about people brushing off the technology and underestimating it. “If policymakers have a perspective that this is not a big deal, we are in deep trouble,” he says. “A lot of people making these tools see the potential. If we have a divide where the people who were going to potentially regulate it or think about the public good are not really paying attention to it, that’s quite troubling. It’s important that when people leave the Kennedy School to go into policy positions, they have knowledge and informed opinions about generative AI.”

To provide students with a thorough grounding of the technology and its implications, the course is divided into units on the science of how generative AI works, how individuals and organizations can use the technology, and its implications for society. “Designing this course represented a really exciting challenge. The field is evolving so rapidly that it is hard to keep up,” Levy says. “So, we sought to strike a balance between helping students learn things that are likely to be helpful regardless of how AI evolves while at the same time adapting in real time to the changes that might make some course ideas obsolete or irrelevant.”

Much of the classroom experience is hands-on. For example, to help understand the science, the instructors have an exercise with students acting as neurons in a deep neural network, a layered machine learning algorithm that mimics the way the human brain processes information. In class, students get their computers out, experiment with prompts to generate interesting results, build chatbots, and document what they are seeing. “We’re focusing on collaborative activities to get people to experiment,” Svoronos says, “because the goal is for people to shift their mindsets toward experimenting more and being comfortable enough with the tools to see what they can do and then decide whether they should use them.”

“Our hope is that they become sophisticated and responsible users of AI.”

Sharad goel.

Levy says that teaching with Goel and Svoronos was a special experience. “All three of us are in the classroom in every class session, with one of us at the front of the room at any one time,” he says. “This means that there are sessions where one or two of us gets to experiment what it feels like to be a student in the classroom. It is an incredible privilege and joy to be in a classroom to learn, especially about a field as exciting as this one.”

While “The Science and Implications of Generative AI” is a new module this spring, the teaching team hopes to develop it into a semester-long course and bring similar lessons into HKS Executive Education programming. A new HKS webpage also pulls together information on courses, events, and other resources on artificial intelligence.

Faculty-created chatbots and AI tools

Beyond the course Goel, Levy, and Svoronos teach, experimentation on AI abounds at HKS, with faculty members using machine learning in their teaching and research in a variety of ways. Instructors are using the latest version of StatGPT, which is now dubbed PingPong, to help their students learn, ask questions, and walk through problems—along with other customized bots. These tools give students additional support, complementing the work of the teaching teams.

For students—or anyone, really—hoping to make their writing more effective, there is an AI tool created by Todd Rogers , the Weatherhead Professor of Public Policy. Rogers, who studies the science of behavior change, built a free “ AI for Busy Readers ” email coaching tool. It edits emails so they are easy to skim by applying the principles from his book Writing for Busy Readers , coauthored with Jessica Lasky-Fink. You can submit any email and the AI tool will suggest a revision. “We developed this AI tool to help my students see what their emails could look like if they were written specifically for busy readers,” Rogers says. “To my surprise, students keep using the tool—and sharing it! In just the last few months we’ve exceeded 100,000 uses—and it’s still growing exponentially.”

“We sought to strike a balance between helping students learn things that are likely to be helpful regardless of how AI evolves while at the same time adapting in real time to the changes that might make some course ideas obsolete or irrelevant.”

And Julia Minson , an associate professor of public policy, is using the power of artificial intelligence to roleplay and take on personas. Minson, who studies the psychology of disagreement, is developing a bot to simulate conversations with someone with whom you might disagree. This tool will give people the opportunity to practice the hard skills of constructive conversation in a low-stakes environment. “One of the greatest challenges of improving your skills around disagreement is willingness to practice,” Minson says. “But practice is hard when there are serious interpersonal stakes attached. A chatbot can really take that pressure off.”

While the technology behind artificial intelligence is becoming increasingly sophisticated at an astonishing rate, School faculty members are experimenting to help students become more thoughtful, knowledgeable, and responsible future policy professionals.

Photographs by Jessica Scranton; Portraits by Martha Stewart

More from HKS

Artificial intelligence at hks, hks experts discuss how to harness, and how to rein in, artificial intelligence, using ai to combat medical misdiagnosis and improve patient care.

Get smart & reliable public policy insights right in your inbox. 

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

Americans who have traveled internationally stand out in their views and knowledge of foreign affairs

Do people who travel think differently about the world? A new Pew Research Center survey suggests they do.

Americans who have traveled internationally are more interested in and knowledgeable about foreign affairs, feel closer to others around the world, and favor a more active foreign policy, according to the survey of 3,576 U.S. adults conducted in spring 2023. We also surveyed people in 23 other countries about their international travel habits.

This analysis examines international travel with a focus on Americans’ travel, including which Americans travel abroad and how their interest in the world and views of international affairs differ from others.

For this analysis, we surveyed 3,576 U.S. adults from March 20 to March 26, 2023; 3,581 U.S. adults from March 21 to March 27, 2022; and 10,606 U.S. adults from June 14 to June 27, 2021. Everyone who took part in these surveys is a member of the Center’s American Trends Panel (ATP), an online survey panel that is recruited through national, random sampling of residential addresses. This way nearly all U.S. adults have a chance of selection. The survey is weighted to be representative of the U.S. adult population by gender, race, ethnicity, partisan affiliation, education and other categories. Read more about the ATP’s methodology .

For non-U.S. data, this report draws on nationally representative surveys of 27,285 adults conducted from Feb. 20 to May 22, 2023. All surveys were conducted over the phone with adults in Canada, France, Germany, Greece, Italy, Japan, the Netherlands, South Korea, Spain, Sweden and the United Kingdom. Surveys were conducted face-to-face in Hungary, Poland, India, Indonesia, Israel, Kenya, Nigeria, South Africa, Argentina, Brazil and Mexico. In Australia, we used a mixed-mode probability-based online panel.

Here are the June 2021 survey questions and responses used in this analysis. Those for the March 2022 survey may be found here , as well as those for the March 2023 survey .

How many Americans have traveled internationally?

Roughly three-quarters of Americans (76%) have visited at least one other country, including 26% who have been to five or more. About a quarter (23%) have not traveled internationally, though most in this group say they would if they had the opportunity.

Related:  How experience with international travel varies across 24 countries

To analyze how Americans’ travel experiences relate to their attitudes on other questions, we placed people into three categories:

• Globe-trotters have traveled to at least five other countries. About a quarter of the U.S. public (26%) falls into this category.
• Casual travelers have traveled to between one and four other countries. Half of Americans fall into this category.
• Nontravelers have never left the United States. This category includes 23% of Americans.

Compared with Americans, people in many European nations are more likely to have traveled to five or more other countries. For instance, 88% of Swedes have done so.

However, international travel is much less common in many middle-income nations. It is strongly correlated with a nation’s gross domestic product per capita. (For more on international travel and views about global engagement, read “Attitudes on an Interconnected World.” )

Who travels internationally?

Perhaps unsurprisingly, older people are more likely than younger people to have traveled internationally. Americans ages 65 and older are more than twice as likely as adults under 30 to fall into our globe-trotter category (37% vs. 17%).

Income is even more strongly related to travel than age. Two-thirds of upper-income Americans have traveled to at least five countries, compared with 9% of Americans with lower incomes.

Similarly, Americans with a postgraduate degree are far more likely to be globe-trotters than those with a high school education or less (59% vs. 10%).

Residents of suburban and urban areas generally have more international travel experience than people who live in rural areas.

There are no significant partisan differences when it comes to international travel: 26% of Democrats and Democratic-leaning independents qualify as globe-trotters, as do 28% of Republicans and GOP leaners.

Do travelers know more about the world?

Globe-trotters are especially likely to say they are interested in foreign affairs and follow international news. Casual travelers, in turn, are more likely than nontravelers to do so.

Globe-trotters are also the most knowledgeable about international affairs. In 2022, we conducted an international affairs quiz , asking Americans 12 questions related to international news. On average, globe-trotters got 8.2 of the 12 questions correct, compared with 6.4 for casual travelers and 4.2 for nontravelers.

Is international travel related to views of global engagement?

International travel experience is also linked to Americans’ views about international affairs and their feelings of connection to other people around the world.

When asked which comes closest to their view, 57% of globe-trotters say the U.S. should be active in world affairs, while 43% say the U.S. should pay less attention to problems in other countries and concentrate on problems at home. In contrast, most casual travelers and nontravelers say the U.S. should focus on problems at home.

In all three groups, at least half of respondents say that when the U.S. is making foreign policy, it should take other countries’ interests into account – even if that means making compromises. But globe-trotters are especially likely to hold that view.

Globe-trotters are also particularly likely to say they feel close to people around the world, with 42% saying so. By comparison, 34% of casual travelers and 30% of nontravelers say this.

• International Affairs

Richard Wike is director of global attitudes research at Pew Research Center

Janell Fetterolf is a senior researcher focusing on global attitudes at Pew Research Center

Fewer Americans view the United Nations favorably than in 2023

What are americans’ top foreign policy priorities, rising numbers of americans say jews and muslims face a lot of discrimination, younger americans stand out in their views of the israel-hamas war, how u.s. muslims are experiencing the israel-hamas war, most popular.

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy

• Open the Search Form

Publications

• Congressional Testimony

Research Areas

• Hard Choices in Defense
• The Future of Warfare
• Strengthening Deterrence
• The Gaming Lab
• Defense Discussions
• The China Challenge
• Regional Alliances and Partnerships
• The India Opportunity
• The North Korea Threat
• Confronting Threats to Democracy
• NATO and European Security
• Strengthening Alliances
• Israel-Hamas War
• Evolving the Mission–Iraq/Syria/ISIS
• Iran Futures
• Constructing Regional Partnerships and Seizing Emerging Opportunities
• Security Assistance
• Targeted Sanctions: Russia and Iran
• Economic Statecraft
• Energy & Geopolitics
• Sanctions by the Numbers
• Artificial Intelligence
• Technology Strategy
• Critical Digital Infrastructure
• Biotechnology
• Enhancing DHS Oversight & Accountability
• Congress and National Security
• Renewing the National Security Consensus

Resident Experts

• All Resident Experts
• Arona Baigal
• Vivek Chilukuri
• Carrie Cordero
• Lisa Curtis
• Hannah Dennis
• Michael Depp
• Bill Drexel
• Joshua Fitt
• Richard Fontaine
• Noah Greene
• Kate Johnston
• Hannah Kelley
• Andrea Kendall-Taylor
• Emily Kilcrease
• Katherine L. Kuzminski
• Nicholas Lokker
• Jonathan Lord
• Gibbs McKinley
• Andrew Metrick
• Carisa Nietsche
• Stacie Pettyjohn
• Paul Scharre
• Philip Sheers
• Jacob Stokes
• Taren Sylvester
• Adam H. Tong
• Josh Wallin
• Becca Wasser
• Caleb Withers
• Robert O. Work
• Evan Wright

Adjunct Experts

• All Adjunct Experts
• Manpreet Singh Anand
• Brandon J. Archuleta
• Lt. Gen. Robert Ashley, Jr. (Ret.)
• Christian Beckner
• Samuel Bendett
• Paul Benfield
• Rachel Brandenburg
• Dr. Jonathan Brewer
• Josh Campbell
• Pablo Chavez
• Richard Connolly
• John Costello
• Richard J. Danzig
• Anthony DeMartino
• Jason Dempsey
• Robin Dickey
• Billy Fabian
• Yaya J. Fanusie
• Ryan Fedasiuk
• David Feith
• Edward Fishman
• Ben FitzGerald
• Dr. Erik Lin-Greenberg
• Michael Greenwald
• Mikhail Grinberg
• Hamzeh Hadad
• Hon. Robert F. Hale
• Heli Hautala
• Franz-Stefan Gady
• Dr. Jeannette Gaudry Haynie
• Jacob Helberg
• Michelle Holko, PhD, PMP
• General Mike Holmes, U.S. Air Force (Ret.)
• John Hughes
• Dr. Kyleanne Hunter
• LTG Anthony R. "Tony" Ierardi, USA (Ret.)
• Akira Igata
• Elsa B. Kania
• Robert D. Kaplan
• Dr. Duyeon Kim
• Christopher D. Kolenda
• Margarita "Rita" Konaev
• Steven Kosiak
• Dr. Andrew F. Krepinevich, Jr.
• Thomas Krueger
• Peter L. Levin
• Jennifer McArdle
• Brendan McCord
• Dr. ED McGrady
• Jack Midgley
• J Travis Mosier
• Dr. Go Myong-Hyun
• Catherine A. Novelli
• Dr. John Park
• Dr. Lynne E. Parker
• Diem Salmon
• Jordan Schneider
• Peter Schroeder
• Michael Sellitto
• Vance Serchuk
• John (Jack) N.T. Shanahan
• Tom Shugart
• Daniel Silverberg
• Alexander Sullivan
• Tobias Switzer
• Rachel Tecott Metz
• Alanna C. Torres-Van Antwerp
• Jim Townsend
• Richard R. Verma
• Anthony Vinci
• Jon B. Wolfsthal
• Alex Zerden
• Rachel Ziemba
• Full-Time Staff
• Executive Team
• Board of Directors
• Board of Advisors
• Directors Emeriti
• Distinguished Senior Fellows
• Adjunct Fellows
• Senior Military Fellows
• Joseph S. Nye Interns
• CNAS Supporters
• Next Generation National Security Fellows
• Visiting Fellows
• Writer in Residence
• In Memoriam
• Internships

CNAS Programs

• The Shawn Brimley Next Generation National Security Leaders Fellowship
• Robert M. Gates Fellowship
• 1LT Andrew J. Bacevich Jr., USA Award
• Joseph S. Nye, Jr. National Security Internship and Mentoring Program
• The Pitch: A Competition of New Ideas
• Women in National Security
• Writers in Residence
• Senior Military Fellows Program
• Join the CNAS Council
• CNAS Corporate Partnership Program
• Frequently Asked Questions
• CNAS 2024 National Security Conference: High Stakes

Back to All Events

Virtual Event | Beyond China's Black Box: Trends Shaping China’s Foreign and Security Policy Decision-Making under Xi Jinping

Apr 30, 2024 9:00am to 10:00am ET

China’s foreign and security policy decision-making apparatus is often described as a metaphorical black box. A forthcoming report from CNAS argues it is possible to develop a better understanding of the people, institutions, processes, and pressures that go into making China’s policies toward the world during Xi Jinping's “new era.”

On April 30, 2024, the Center for a New American Security (CNAS) will host a virtual event featuring Jacob Stokes, Senior Fellow in the Indo-Pacific Security Program, CNAS; Jude Blanchette, Freeman Chair in China Studies, Center for Strategic and International Studies (CSIS); Amanda Hsaio, Senior Analyst, China, International Crisis Group (ICG); and Rorry Daniels, Managing Director, Asia Society Policy Institute (ASPI), and Senior Fellow, Center for China Analysis, ASPI. Panelists will discuss the report's analysis and recommendations as well as larger trends in China’s foreign and security policy decision-making under Xi.

Jacob Stokes Senior Fellow, Indo-Pacific Security Program CNAS

Jude Blanchette Freeman Chair in China Studies CSIS

Amanda Hsiao Senior Analyst, China ICG

Rorry Daniels Managing Director, ASPI, and Senior Fellow, Center for China Analysis ASPI

Sign up for event updates and reminders

Thank you for registering! You will receive a confirmation email shortly. All CNAS events are free, open to the public, and viewable from cnas.org/live .

Stay up-to-date with report releases, events, major updates, and announcements from the Center for a New American Security.

There was a problem submitting the form. Please double check your information and try again.

For registration questions, contact Jasmine Butler at [email protected] . For media inquiries, contact Alexa Whaley [email protected] .

'Sunny day flooding' increases fecal contamination of coastal waters

A new study finds that "sunny day flooding," which occurs during high tides, increases the levels of fecal bacteria in coastal waters. While the elevated bacteria levels in the coastal waters tend to dissipate quickly, the findings suggest policymakers and public health officials should be aware of potential risks associated with tidal flooding.

"Historically we see the highest levels of fecal bacteria contamination in coastal waterways after it rains, because the rain washes contaminants into the waterways," says Natalie Nelson, corresponding author of a paper on the study and an associate professor of biological and agricultural engineering at North Carolina State University. "Due to sea level rise, we're seeing an increase in flooding in coastal areas at high tide -- even when there isn't any rainfall. We wanted to see whether sunny day floods were associated with increases in fecal bacteria contamination in waterways."

For the study, researchers collected water samples every day for two summer months at three sites along a single waterway in coastal North Carolina. Two perigean spring tides occurred during the two-month sampling period. Perigean spring tides are tides characterized by especially pronounced high and low tides, caused by the moon's gravitational pull.

The researchers increased their collection of samples at each monitoring site on the days of the perigean spring tides, to capture changes in water quality throughout the tidal cycle. During the high-water levels of the perigean spring tides, water also came up out of some local storm drains and caused minor flooding. The researchers took samples of those floodwaters as well.

"We found that the floodwaters themselves had relatively high levels of fecal bacteria," Nelson says. "To be clear, these floods were inches deep; we're talking about very minor flooding as tidal waters pushed up through the storm grates. However, we've seen children playing in these sort of sunny day floodwaters, and the levels of fecal bacteria we detected were above the levels deemed safe for recreational waters."

"Our findings with regard to the coastal waters were more nuanced," says Megan Carr, lead author of the study and a Ph.D. student at NC State. "On the one hand, we did see higher concentrations of fecal bacteria in coastal waters as the floodwaters and perigean spring tides receded. On the other hand, we did not see this in every instance and in every location -- and we also found that the higher concentrations of fecal bacteria usually only lasted for a few hours."

In other words, perigean spring tides do raise some concerns about fecal bacteria and water quality in coastal waters, but they don't appear to cause concentrations of fecal bacteria at the same level as stormwater runoff caused by rainfall.

"It's important to note that these results are from samples that we took in one area along a large waterway," Nelson says. "The results are likely to vary significantly, depending on the size of the waterway. For example, post-flood contamination could last longer in waterways smaller than the waterway that we sampled. That's something that would benefit from additional research.

"Sea levels are going to continue rising for the foreseeable future," Nelson says. "So we are definitely going to see more sunny day flooding, and those floods will be getting worse. We need to continue studying the impact that these tidal floods have on our water quality, because the more we understand, the better able we will be to make informed decisions about public health and safety."

The paper, "Fecal Bacteria Contamination of Floodwaters and a Coastal Waterway from Tidally-Driven Stormwater Network Inundation," is published in the open-access journal GeoHealth . First author of the paper is Megan Carr, a Ph.D. student at NC State. The paper was co-authored by Angela Harris and Katherine Anarde, both assistant professors of civil, construction and environmental engineering at NC State; Nora Sauers, Gabe Da Silva and Catherine Gamewell, who are undergraduates at NC State; Adam Gold of the Environmental Defense Fund; and Miyuki Hino of the University of North Carolina at Chapel Hill.

The research was done with support from the National Science Foundation, under grant number 2047609; and from the U.S. Geological Survey Southeast Climate Adaptation Science Center.

• Oceanography
• Ocean Policy
• Resource Shortage
• STEM Education
• Environmental Policies
• Coastal management
• Coastal erosion
• Mid-Atlantic United States flood of 2006
• Preparations for Hurricane Katrina

Story Source:

Materials provided by North Carolina State University . Original written by Matt Shipman. Note: Content may be edited for style and length.

Journal Reference :

• M. M. Carr, A. C. Gold, A. Harris, K. Anarde, M. Hino, N. Sauers, G. Da Silva, C. Gamewell, N. G. Nelson. Fecal Bacteria Contamination of Floodwaters and a Coastal Waterway From Tidally‐Driven Stormwater Network Inundation . GeoHealth , 2024; 8 (4) DOI: 10.1029/2024GH001020

Cite This Page :

Explore More

• Fossil Frogs Share Their Skincare Secrets
• Fussy Eater? Most Parents Play Short Order Cook
• Precise Time Measurement: Superradiant Atoms
• Artificial Cells That Act Like Living Cells
• Affordable and Targeted Anticancer Agent
• This Alloy Is Kinky
• Giant Galactic Explosion: Galaxy Pollution
• Flare Erupting Around a Black Hole
• Two Species Interbreeding Created New Butterfly
• Warming Antarctic Deep-Sea and Sea Level Rise

Trending Topics

Strange & offbeat.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Health Res Policy Syst

How do we define the policy impact of public health research? A systematic review

Kristel alla.

1 School of Public Health, Faculty of Medicine, The University of Queensland, Herston Road, Herston, QLD 4006 Australia

2 Queensland Centre for Mental Health Research, The Park Centre for Mental Health, Locked Bag, Archerfield, QLD 4108 Australia

Wayne D. Hall

3 Centre for Youth Substance Abuse Research, The University of Queensland, CYSAR K Floor, Mental Health Centre, Royal Brisbane & Women’s Hospital Campus, Herston, QLD 4029 Australia

Harvey A. Whiteford

Brian w. head.

4 School of Political Science, The University of Queensland, St Lucia, QLD 4072 Australia

Carla S. Meurk

Associated data.

All data generated or analysed during this study are included in this published article and its Additional file.

In order to understand and measure the policy impact of research we need a definition of research impact that is suited to the task. This article systematically reviewed both peer-reviewed and grey literature for definitions of research impact to develop a definition of research impact that can be used to investigate how public health research influences policy.

Keyword searches of the electronic databases Web of Science, ProQuest, PubMed, EMBASE, CINAHL, Informit, PsycINFO, The Cochrane Database of Systematic Reviews and Google Scholar were conducted between August 2015 and April 2016. Keywords included ‘definition’ and ‘policy’ and ‘research impact’ or ‘research evidence’. The search terms ‘health’, public health’ or ‘mental health’ and ‘knowledge transfer’ or ‘research translation’ were used to focus the search on relevant health discipline approaches. Studies included in the review described processes, theories or frameworks associated with public health, health services or mental health policy.

We identified 108 definitions in 83 publications. The key findings were that literature on research impact is growing, but only 23% of peer-reviewed publications on the topic explicitly defined the term and that the majority (76%) of definitions were derived from research organisations and funding institutions. We identified four main types of definition, namely (1) definitions that conceptualise research impacts in terms of positive changes or effects that evidence can bring about when transferred into policies (example Research Excellence Framework definition), (2) definitions that interpret research impacts as measurable outcomes (Research Councils UK), and (3) bibliometric and (4) use-based definitions. We identified four constructs underpinning these definitions that related to concepts of contribution, change, avenues and levels of impact.

The dominance of bureaucratic definitions, the tendency to discuss but not define the concept of research impact, and the heterogeneity of definitions confirm the need for conceptual clarity in this area. We propose a working definition of research impact that can be used in a range of health policy contexts.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-017-0247-z) contains supplementary material, which is available to authorized users.

The measurement of research impact is a contested research and political agenda that poses a complex academic question. Differences in the ways in which evidence might inform policy, and its political underpinnings, highlight key challenges in understanding the policy impacts of research.

The quest to measure and investigate research impact has multiple drivers. Researchers, practitioners and policymakers continue to promote the need for, and benefits of, evidence-informed practice and policies in public health and medicine more generally [ 1 – 3 ]. Government and other funders of research increasingly demand that researchers track the impact of their research to justify research expenditure by showing economic benefits, policy uptake, improved health and community outcomes, industry application and/or positive environmental effects [ 2 , 4 – 6 ]. Accountability for research impact is typically embedded in the requirements of grant applications and project reports, in which researchers are required to anticipate the measurable outcomes arising from their proposed research [ 5 , 6 ]. Within health research, there is an expectation that evidence-based policies and practices may improve the efficiency and effectiveness of health services [ 2 , 6 ].

Definitions of research impacts are informed (explicitly or implicitly) by what we think knowledge is, what we value about it, and our understanding of the ways in which research evidence can contribute to policy [ 7 ]. There is a lack of consensus on how to define research impact [ 8 , 9 ] and on the terminology used to describe it [ 10 ]. Various terms associated with the concept of ‘research impact’, such as knowledge or research ‘translation’, ‘uptake’, ‘diffusion’, ‘utilisation’, ‘payback’, ‘valorisation’, ‘benefits’ and ‘outcomes’, are often used interchangeably [ 11 – 13 ]. Boaz et al. suggest that the “ different terms have a shared interest in change that lies beyond the research process and its primary outputs ” ([ 13 ], p. 256), as well as an “ appreciation of the complexity and diversity of research use ” ([ 13 ], p. 266). In contrast, other authors note that the lack of standard terminology reflects a deficiency in the literature and call for a clear definition of research impact [ 14 – 16 ].

Many have argued that using a ‘narrow’ approach to measuring research impact results in acknowledging only those types of impact that are easily measured and overlooking those that are hard to measure [ 2 , 17 , 18 ], for example, narrowly defining research impact through the number of citations in the literature (see for example [ 19 ]). In contrast, Milat et al. argue that “ The emerging literature on research impact highlights its complex, non-linear, unpredictable nature, and the propensity, to date, to count what can be easily measured, rather than measuring what ‘counts’ in terms of significant, enduring changes ” ([ 20 ], p. 2). Greenhalgh and Fahy argue that “ the unenhanced ‘logic model’ of impact, comprising inputs (research funding)→activities (research)→outputs (e.g. papers, guidelines)→outcomes (e.g. changed clinician behaviour, new service models)→impacts (e.g. reduced mortality), is increasingly viewed as over simplistic ” ([ 18 ], p. 3). Similarly, Haynes et al. contend that research impact constitutes “ a contested bundle of concepts subject to interpretation and tactical use ” ([ 21 ], p. 1047) (see also [ 22 – 24 ]).

This study explores different definitions of research impact, with a specific focus on the applicability of definitions to advancing an academic understanding of how evidence informs health policy. The article presents the findings of a systematic review of the literature to assess how research impact is currently defined in the health literature. On the basis of this review, we propose a working definition of research impact relevant for health policy.

This review uses the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodological review framework [ 25 ] to guide a systematic data collection and critique of the literature defining research impact. The review was conducted to answer three research questions. (1) How is research impact defined in the health literature? (2) What are some of the key constructs underpinning different definitions of research impact? (3) What are some of the implications for research, policy and theory of different ways of defining research impact in the health policy field? The review used mental health research and policy as a case study.

The research questions resulted from discussions among all authors. The search strategies were developed with input from all authors supported by the expertise of a specialist librarian. KA conducted the database searches, assessed the literature against the review criteria, and undertook data extraction, synthesis and analysis of the literature. All authors provided input into findings and conclusions and edited drafts of the article.

Search strategy

An initial scoping review was undertaken to determine the feasibility of research questions and keywords for the search strategy. The initial focus on mental health policy was expanded to include health policy broadly in the absence of a research impact literature specific to mental health. Heterogeneity of concepts in the research impact literature and the challenge in finding studies that explicitly defined research impact motivated a more expansive search strategy and broader criteria for a definition of ‘research impact’. The peer-reviewed literature was found to be too limited since the majority of definitions were generated within government, not academic, contexts. As Sibbald et al. argue, “ the exclusion of grey literature can skew the results of research syntheses ” ([ 26 ], p. 49). The review criteria were expanded to include definitions of impact in all health research found in the grey literature. Grey literature was sourced through (1) Google Scholar, (2) examining reference lists in the included articles, (3) consulting with members of the research team, and (4) contacting external experts.

Electronic databases were searched for peer-reviewed article abstracts and other literature that were evaluated against the review aims and scope. The databases Web of Science, ProQuest, PubMed, EMBASE, CINAHL, Informit, PsycINFO, The Cochrane Database of Systematic Reviews and Google Scholar were searched using a desktop research method tailored to individual databases. Filters were applied to include only articles written in English without time limits. The search of full texts was conducted between August 2015 and April 2016. Keywords included ‘definition’ and ‘policy’ and ‘research impact’ or ‘research evidence’. The search terms ‘mental health’ or ‘public health’ or ‘health’ and ‘knowledge transfer’ or ‘research translation’ were used to focus the search on relevant health discipline approaches. Studies included in the review described processes, theories or frameworks associated with public health, health services or mental health policy.

Relevant full texts were retrieved and assessed for inclusion against the review criteria. The systematic literature search and review was conducted in the stages depicted in Fig.  1 . Reference lists were explored for further relevant resources. Project team members and external experts provided recommendations on the websites of additional key organisations. Data about research impact definitions and constructs were extracted, classified into themes, discussed and synthesised.

PRISMA flow chart for the systematic review process

Inclusion criteria

The review imposed no restrictions on the study design apart from excluding unpublished dissertations (n = 3). The sample included theoretical and opinion pieces, case studies, descriptive studies, frameworks and systematic reviews describing processes, methods and conceptual models for assessing research impact. Inclusion criteria were (1) studies addressing public health, mental health, political science or health services disciplines; (2) expressly addressing policy impacts of research as focus or aim; and (3) including an explicit definition of research impact.

A definition of research impact was considered to be present when there was a statement in the article explaining the meaning of research impact (or ‘impact of research’ or ‘policy impact of research’) and there was an explicit effort made to define the term, i.e. an explanatory statement was given such as ‘definition’ or ‘term’ to indicate that research impact ‘is’, ‘denotes’ or ‘is understood as’. In some cases, ‘impact’ rather than ‘research impact’ was defined. When it could be reasonably deduced that the definition referred to research impact, then the definition was included in the study. Texts that described constructs related to research impact but did not define the term(s) were excluded from analysis. All definitions were included from sources that discussed several definitions.

Data extraction and analysis

Publications were recorded in the reference management software Endnote. Excel spreadsheets were used to record (1) information on the method used in the study; (2) research impact definition(s); (3) source reference(s) of the definition; and (4) constructs extracted from the research impact definitions.

Results were analysed and synthesised in two ways; first, definitions were ordered into types – an ordering that was based largely on the source cited. Second, underlying features of these definitions, based on keywords and constructs evident in definitions, were identified using an inductive comparative method and then categorised into definition types and domains.

As given in Fig.  1 , the search identified 866 sources. Supplemental searches, including reference list searches and expert recommendations, yielded a further 20 publications. After duplicates were removed, 661 titles were screened against the inclusion criteria; 350 articles were excluded during abstract screening because they were not on research impact, health or policy and 135 sources were excluded during full-text screening because they did not contain a definition of research impact. A total of 83 sources were included in this review, including 45 peer-reviewed journal articles, 13 books, 7 conference papers and 18 websites or online reports (Additional file 1 ). Grey literature comprised 29% of included publications.

Study characteristics

Only 23% of peer-reviewed journal articles that were on research impact (45 out of 200 that underwent full-text screening) actually defined the term.

The majority of all sources (76%) were published during or after 2011. Half (51%) of the definitions of research impact were from the United Kingdom, 22% from Australia, 16% from other European countries (Germany, Netherlands, Spain, Italy, Austria, Sweden, France and Finland), 10% from the United States of America and 2% from Canada.

A total of 108 definitions were provided. Most publications (60%) referred to a single research impact definition, while the remainder presented two to four definitions. The majority of definitions (76%) were from, or cited, research organisations and funding institutions (i.e. grey literature). The remainder provided original (i.e. unreferenced) definitions of research impact (16%) or cited other peer-reviewed literature (9%). The most highly cited definitions were provided by the United Kingdom Research Excellence Framework (REF), the Higher Education Funding Council for England (HEFCE), the Research Councils UK (RCUK), the Australian Research Quality Framework (RQF) and the Australian Research Council (ARC) (Table  1 ).

Total number of definitions referenced (n = 108)

Several commonalities were evident in the four main types of definitions identified. These were (1) research impact defined as a demonstrable contribution to society and economy (definition provided by the RCUK); (2) research impact defined as an effect, change or benefit to society and economy (REF and HEFCE); (3) bibliometric definitions; and (4) use-based definitions.

The first two types – the RCUK and the REF/HEFCE – can also be classified as research governance definitions. The research governance group also includes the RQF and ARC definitions, which were hybrids of the RCUK and REF/HEFCE definitions.

The RCUK definition

The two central definitions used in scientific journals and policy books were contributed by the RCUK and HEFCE (the REF). The RCUK defines research impact as “ the demonstrable contribution that excellent research makes to society and the economy ” [ 27 ]. The RCUK defines research impact using the adjective ‘demonstrable’, emphasising that the contribution must be provably linked to an impact (i.e. that the societal impact of research cannot be assumed) whilst the adjective ‘excellent’ equates impact with research quality. In this definition, impact is restricted to the contribution of research to the domains of ‘society’ and ‘economy’. The emphasis on contribution (input) makes this definition neutral with respect to having an expectation of a specific outcome or change.

While the RCUK does not explicitly reference policy impacts, others have expanded upon it to encompass research impacts on policy in two slightly different ways; first, is the impact of research to an area policy (as in [ 28 ]), i.e. the ‘policy benefits’ of research [ 29 ] and second is the contribution that research can make to policy and good governance, i.e. to improving the effectiveness of public services and policy [ 30 , 31 ].

The REF/HEFCE definition

The HEFCE manages the REF, which is used to assess research quality and allocate research funding in the United Kingdom [ 32 ]. HEFCE and the REF research impact definitions are equivalent and referenced interchangeably.

The HEFCE/REF guidelines define research impact as “ an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia ” [ 32 ]. These two core definitions – research impact as a ‘demonstrable contribution’ (by RCUK) versus ‘effect on, change or benefit’ to society, policy and economy – differ in the inclusivity of their concept of impact, on whether the process (contribution) or outcome (effect) is emphasised, and whether research impact can be readily measured.

The HEFCE/REF is conceptually more nuanced than that provided by the RCUK insofar as it emphasises a broader range of areas of influence. Chandler [ 33 ] adds to the core definition that research impact enables the development of new products, services and policies – in other words, research impact can be defined through its capacity to facilitate innovation. Similarly, Donovan [ 34 ] adds industry and government to the list of ‘beneficiaries’ of research impact. Pragmatic and person-centred interpretations of research impact require that research impact translates into ‘real-world outcomes’ [ 35 ] and some, such as Chandler [ 33 ], see research impact as pertaining to economic, social and cultural ‘lives’, and thus referencing (individual) human activities.

Multiple authors cite the second part of the REF/HEFCE definition, which includes a list of impact foci, namely “ activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding of an audience, beneficiary, community, constituency, organisation or individuals in any geographic location whether locally, regionally, nationally or internationally" ([ 36 ], p. 5; [ 37 ], p. 45). This definition broadens the spheres of possible impact to include psychosocial impacts and impacts at numerous organisational and geographical scales.

REF departs substantially from the RCUK definition insofar as it includes within the definition the role of research in the prevention of harms and reducing risks, costs or negative impacts [ 18 ]. The normative rendering of research impact as a benefit or a positive return (rather than the value-neutral ‘change’ and ‘effect’) is the focus of Ovseiko et al.’s [ 38 ] definition. This extends the HEFCE definition to include ‘social value’ and specify positive returns from research in terms of social cohesion, social welfare and investments, public engagement with science, and sustainable development. Reed [ 12 ] specifies that research evidence can be useful in preventing the adoption of harmful legislation and products.

The ARC and the Australian RQF definitions

The third most frequently cited definition, that of the research funding body the ARC, is a hybrid of the REF and RCUK definitions and so cannot be classified as a distinct type. This is the broadest core definition included within this review with respect to the areas of potential impact that the definition encompasses. The ARC [ 39 ] defines research impact as “ demonstrable contribution that research makes to the economy, society, culture, national security, public policy or services, health, the environment, or quality of life, beyond contributions to academia ” (for example [ 40 ], p. 158; [ 41 ], p. 32). National security is a unique feature of the ARC cited definitions. The ARC definition includes policy impacts of research within its core definition and regards a wide range of different types and levels of impact that are left open for further inclusion.

The common unifying elements between the ARC [ 39 ] and the Australian RQF [ 42 ] definitions are (1) a reference to the social, economic, cultural and environmental benefits of research and (2) extension of impact scope beyond academia. Some definitions phrased these four contributions as ‘outcomes’ instead of benefits [ 14 , 34 ]. They differed insofar as publications citing the RQF definition of research impact were more heterogeneous, narrative and interpretative than those referring to the ARC. Policy impacts were explicitly mentioned in all five publications that used the ARC definition. However, only one source that cited the RQF mentioned policy impacts.

Bibliometric definitions

Bibliometric definitions, some of which arise out of the field of economics, focus on demonstrable and measurable research impacts in the form of quantifiable data. Some authors, such as Tonta et al. [ 19 ], approach research impacts quite narrowly and define research impact quantitatively as citation frequency in literature. However, others are more inclusive and list other forms of quantifiable impacts as part of their definition. Research impact for the London School of Economics and Political Science Public Policy Group (PPG) is a “ recorded or otherwise auditable occasion of influence from academic research on another actor or organization ” ([ 43 ], p. 310; [ 44 ], p. 7). The PPG website [ 45 ] adds to this definition by indicating that “ impact is usually demonstrated by pointing to a record of the active consultation, consideration, citation, discussion, referencing or use of a piece of research ”. This approach considers research impact assessment methods beyond citations by attempting to capture oral communication, but it demands a record of impacts. The Association of Commonwealth Universities [ 46 ] cites the PPG definition and states that research impact establishes the influence of research knowledge, rather than its consequences.

Similarly, Hannemann-Weber et al. [ 47 ] explain research impact through quantifiable influence and draw direct links between activities and research outputs referred to as ‘impact of publications’. While focussed on bibliometrics, this conceptualisation acknowledges broader social processes that underpin research impact as measured bibliometrically, such as the acceptability and visibility of research, the status (reputation) of research producers and the actions of researchers in the promotion of research findings. The explanation of research impact through quality, visibility and reputation of research outputs thus provides a definition that interestingly overlaps with the RCUK’s alignment of research impact with research quality. Moed et al. ([ 48 ], p. 132) formulate a definition in which they clarified the relationship between research outputs (‘the extent to which the research creates a body of scientific results’) and impact (‘the actual influence of the research output on surrounding research activities’).

Qin [ 49 ] agrees with these ideas in defining research impact by the extent to which outputs are diffused across disciplinary and geographical boundaries (measured by citations), the extent to which these have been adopted (measured by intellectual property purchases and licences), and benefits established (measured quantitatively and qualitatively). Harland [ 50 ], citing Korhonen et al. [ 51 ], expands on the list of research outputs that constitute evidence of impact by adding the concept of ‘pathways’, notably international and cross-national platforms, that can improve impact, albeit still defining research impact narrowly, in terms of dissemination in academic circles. Nightingale and Marshall [ 52 ] expressed the idea that citations exhibit the extent of academic significance, noting, however that this is not the same thing as research impact.

The Australian National Health and Medical Research Council [ 53 ] defines citation tracking as one expression of research impact in terms of the impact of ideas and methods within academia. However, the National Health and Medical Research Council definition acknowledges that there are also less easily measurable forms of research impact such as research that improves patient care, guides policymakers to adopt health prevention strategies or translates into systems level change. Hartwell et al. [ 54 ] suggested that only research that affects practice has impact regardless of how highly cited it is. Cohen et al. [ 55 ] agree that policy impacts of research have broad effects, and can result from pro-health campaigns and from organisational and funding changes. For Cohen et al. [ 55 ] policy impacts must be tangible, measurable and manifest in a specific time frame, namely after research had been produced without feeding back into research production.

Use-based definitions

Many academic articles define research impact by distinguishing between research impact, research use and research outputs. Unlike the instrumentalist definitions found in the grey literature, these definitions tend to be more theoretical, policy and practice oriented, and focussed on the influence of research findings on the activities and knowledge of researchers and policymakers.

Walter et al. [ 56 ] defined research impacts in terms of the uses to which it is put, namely, conceptual use versus instrumental use. An extended form of this definition is provided by Nutley et al.:

“ Broadly, instrumental use refers to the direct impact of research on policy and practice decisions. It identifies the influence of a specific piece of research in making a specific decision or in defining the solution to a specific problem, and represents a widely held view of what research use means. Conceptual use is a much more wide-ranging definition of research use, comprising the complex and often indirect ways in which research can have an impact on the knowledge, understanding and attitudes of policy makers and practitioners. It happens where research changes ways of thinking, alerting policy makers and practitioners to an issue or playing a more general ‘consciousness-raising’ role ” ([ 24 ], p. 36).

Meagher et al. [ 57 ] emphasise that instrumental research impact deals with the attribution of particular policy decisions to specific research whereas conceptual impact embodies the significance of diffusion of research impacts. Their definition is different to the outcomes/benefits-based definitions. Instrumental use is understood in terms of the metaphor of a ‘hammer’. Research ‘hits’ policy and practice to cause a decision or directive. Impact, here, is causal but not necessarily linked to outcomes (beneficial or otherwise).

Jones and Cleere [ 30 ] reference the European Science Foundation in defining research impacts in terms of both their contributions to specific fields and in terms of how they are enacted. This included health impacts (‘contribution to public health, life expectancy, prevention of illnesses and quality of life’) and policy impacts of research (‘contribution to how policymakers act and how policies are constructed and to political stability’). Thus, research impacts on policy can be manifested through contributions to the political culture, the policy development process and the stability of the political regime.

Brewer [ 8 ] argues that policy-specific impacts are demonstrated in research use by policymakers, research uptake into policies, and by improved effectiveness of policies and health services. Wilkinson et al. [ 58 ] also stress that the policy impacts of research extend to private and non-governmental sectors. Their broad definition encompasses the processes of knowledge exchange and relationships that facilitate research impact.

Comparison of domains found in definitions

Definitions each varied on one of four domains of meaning, namely contribution, change, avenues and levels of impacts (Table  2 ).

Domains of research impact definitions

a Where keywords were repeated in a definition, they were only counted once

Research impact was most often defined in terms of the contribution that research made to different areas of influence, including among others the economy, society, environment, culture, policies and health. Just over half (52%) of definitions explicitly mentioned policy as an object of research impact.

Research impact definitions also varied concerning the types of avenues of impact, i.e. the mechanisms or processes by which research could be said to have impact. This was the second predominant construct found in definitions. Effects on knowledge, understanding, awareness and/or attitudes (for example, of practitioners and policymakers) were included in 59% of research impact definitions. References to ‘activities’ (22% of definitions) and ‘processes’ (16%) were also frequent. One-third (33%) of definitions distinguished research impacts as being those evident beyond academia. Many publications defined impact in terms of ‘outcomes’ achieved (14%) and ‘outputs’ (13%). Research impact was defined in terms of ‘demonstrable’ or ‘measurable’ outcomes in 28% of definitions. Two main aspects emerged from definitions, namely (1) research has impact by changing knowledge, understanding, awareness and attitudes, or through creating products (effects on different avenues of impact); and (2) research has impact through scholarly activities or excellent research (through effects on quality and behaviour).

Another important element was the variety of synonyms for impact that existed, i.e. as an effect, change or benefit to areas of influence including any positive and negative impacts that research may have. A clear ‘positivity bias’ was evident in these definitions, indicating their origins in bureaucratic documents. Importantly, over half (58%) of all definitions interpreted research impact as leading to positive gains or the reduction in societal harms; no definitions mentioned that research use may also lead to negative outcomes.

Finally, the research impact construct was also commonly defined through a range of levels of impacts that research evidence can have (i.e. international, national, local and individual impacts). References to the individual and national levels of research impact received most attention and were respectively mentioned in 19% and 18% of research impact definitions. Global and regional research impacts were mentioned in less than 10% of definitions.

The evolution, diffusion and use of research impact definitions

This review confirmed the heterogeneous and recombinant nature of research impact definitions indicating, perhaps, struggles to find an acceptable definition for this complex term [ 8 , 9 , 59 ]. However, our review also highlighted that most of the research impact literature discusses this concept without explicitly defining it, with only one-fifth of peer-reviewed journal articles doing so. Attempts to define research impact were more common in the grey than in the peer-reviewed literature, confirming the extent to which impact is a bureaucratic rather than academic term [ 60 , 61 ]. Our findings confirm previous research showing that research impact definitions, and the research impact ‘agenda’, have emanated from research funding bodies in the United Kingdom and been diffused to other countries, such as Australia [ 60 , 62 ].

The dominance of research governance definitions indicated by our review, reflects the political history of the impact agenda [ 62 , 63 ]. However, their continued dominance may limit the pursuit of academic understandings of research impact by restricting it to demonstrable returns from research investments. Our review found evidence of this type of focus in nearly a third of research impact definitions. While some authors, like Kenyon [ 15 ], express scepticism as to whether it is in fact possible to define such a complex term in an adequate manner, we echo the views of authors such as Tinkler [ 64 ] and Bornmann [ 63 ], who advocate for the need to include ‘the diversity and richness’ of research impacts into definitions; we argue that this entails the need for researchers to exercise reflexivity in relation to research impact definitions, being mindful of the origins of the different definitions, their purposes and limitations.

Debates over definitions

Most definitions of research impact emphasised positive returns. While some definitions used more neutral language, negative impacts of research were rarely mentioned. The issue from a research governance standpoint is whether a focus on impacts defined in terms of non-academic benefits creates an incentive to skew results to demonstrate benefit, even where there is none. This may create perverse incentives to implement ideas before they have been properly tested or their implications fully thought through. Most definitions interpreted research impact as leading to positive gains or the reduction in societal harms. However, there are several examples of research that has had negative or, at least, contested impacts (e.g. drugs such as thalidomide or weapons of mass destruction). Researchers may be encouraged to conduct research in favour of short-term ‘impacts’ with the result that research that is critical of prevailing governing paradigms is not pursued and not funded, resulting in longer term negative effects on innovation and advancement through research. Furthermore, a definition that encompasses a clear ‘positivity bias’, as is evident in these definitions, may be limited in pursuing academic understanding of how evidence impacts policy.

A related concept to that of ‘research impact’ is that of ‘knowledge valorisation’. Knowledge valorisation is gaining significant traction in the European Union research funding and dissemination discourse. Valorisation is a process by which academic knowledge is transformed into social and economic value [ 65 , 66 ]. Valorisation focusses on the process of value creation from academic research through commercial activities and industry associations with academia; in other words, it is closely associated with the commercialisation of academic research [ 67 ].

Valorisation is a concept that is linked to, but not the same as, a definition of research impact. Valorisation and impact are linked through their combined focus on the usefulness of research, and the ability to produce commercial and/or social returns from academic knowledge. Perhaps due to its focus on commercialisation, the literature on valorisation has paid less attention to policy impacts of research. Furthermore, Benneworth [ 67 ] has critiqued the conceptualisation of knowledge valorisation for being more applicable to the physical and life sciences than to the humanities and social sciences. In contrast, ‘research impact’ definitions provide a broader and more abstract conceptualisation concerned with the longer term application of knowledge to more complex societal problems.

Conceptualisations of ‘knowledge valorisation’ and ‘research impact’ both face the same issue in terms of some lack of conceptual clarity and approach [ 66 , 68 ].

How policy features in research impact definitions

Around half of the research impact definitions included a consideration of how research impacts on policy, mostly by mentioning policy as one of several impact foci. The complexities involved in the conceptualisation of research impact on policy have been acknowledged by many authors [ 7 , 10 , 13 ]. There is a recursive issue here, among the main challenges of defining research impact on policy are the uncertainties regarding how exactly research evidence brings about policy changes, and also how those policy changes link to ‘real-world’ outcomes [ 55 , 63 ]. However, these uncertainties exist, in part, due to a lack of agreed upon definitions of research impact that can facilitate a research agenda.

There are several recognised difficulties in attributing a policy impact to a specific piece of research [ 5 , 69 , 70 ]. The original piece of research may be re-interpreted in the policy process in ways that are incorrect or not consistent with its intent, or it may be adapted to particular contexts and transformed in the process. Multiple influences at different stages of research and policy translation may also function to diffuse knowledge. Policy change, as suggested by Thomas [ 71 ], is dynamic and the product of a web of decisions that may reflect competing values that result in political compromises. A policy relevant definition of research impact should take account of the fact that there is not always a direct pathway from evidence to policy and that impact can be more, or less, directly identifiable – depending on whether that impact is conceptual or instrumental [ 22 , 24 ].

Consequently, for a research impact definition to adequately capture the complexities of policy impacts, it must include elements that relate to two different phenomena – policy content and policy processes. We need a definition that is clear about the different, both direct and indirect, ways in which research can impact on policy and thus help us investigate it for academic purposes.

Proposed definition of research impact for (mental) health policy

Based on this review, we propose the following definition for research impact on health policy that can be tailored for use in health disciplines, including public and mental health. A definition specific to mental health is given:

Research impact is a direct or indirect contribution of research processes or outputs that have informed (or resulted in) the development of new (mental) health policy/practices, or revisions of existing (mental) health policy/practices, at various levels of governance (international, national, state, local, organisational, health unit).

This definition tailors core constructs that were identified in the literature to the field of health policy. It includes the constructs of contribution (but not demonstrable), change, research outputs, policies, practices, various avenues and levels of impacts and encompasses impacts that may be said to occur at different time points. For example, immediate impact might be the use of research processes and outcomes to increase policymakers’ knowledge and inform attitudes, medium-term impact may be an impact of research on the development and revision of policy, and a long-term outcome may be the multilevel impact of research through the implementation and evaluation of policy and practice.

The definition overcomes some of the limitations of existing definitions. It does not restrict research impact to its measurable qualities and includes both desirable and undesirable impacts, allowing for its use in different contexts to capture the full range of possible research impacts. Of the definitions available, the proposed definition is perhaps most similar to that of the ARC [ 39 ] definition.

Strengths, limitations and future research

The key strength of this review is its comprehensiveness and wide coverage of both peer-reviewed and grey literature, the latter having been neglected in previous reviews. The use of a systematic search methodology allowed us to identify the prevalence and reach of different types of definition and research impact definitions overall. The review confirmed that the two most common definitions in both peer-reviewed and grey literature originated from the grey literature, supporting the need to include the grey literature in future reviews of research impact studies.

This review is limited by its conservative search term selection, as only publications that explicitly used the term ‘research impact’ or its close derivatives were included. It is possible that relevant literature that failed to use this terminology was excluded, for example, economic literature on payback models [ 72 , 73 ]. That said, economic models such as the payback model arguably represent operational definitions rather than conceptual definitions.

The study focus is limited to literature that was in English. Thus, it may not have captured relevant discourses from European or non-English speaking literature.

Additionally, five unique definitions were identified in this review that fell outside the typologies constructed [ 61 , 74 – 77 ]. These definitions all drew distinctions between the ideas of impacts, outputs and/or outcomes, and shared some of the features of the aforementioned definitions. Future research will take account of feedback from relevant stakeholders (e.g. researchers and policymakers) on different ways of defining research impact and on the definition proposed here; academics, policymakers, bureaucrats, clinicians, patients and the general community are likely to hold different views on this topic.

Facilitating the effective translation of health research to policy and practice requires a dedicated research agenda. The dominance of bureaucratic definitions, the tendency to discuss but not define the concept of research impact, and the heterogeneity of definitions confirm the need for conceptual clarity in this area. Without wanting to impose a reductive imperative within debates around research impact definitions, we pose a definition of research impact that is primarily for the purposes of academic study of the impact of research on health policy but that could be adapted for use in other specific contexts.

Acknowledgements

We thank Dr Florin Oprescu from the University of the Sunshine Coast for his input into the research impact definition and Ms Sarah Yeates from the Centre for Youth Substance Abuse Research, The University of Queensland, for her assistance with database searches.

The project was funded by the National Health and Medical Research Council (NHMRC) Centre for Research Excellence in Mental Health Systems Improvement (APP1041131). This research was supported by the Australian Government Research Training Program Scholarship from the University of Queensland to Kristel Alla.

Availability of data and materials

Abbreviations, additional file.

Definitions of research impact included in the study. Table S1 . RCUK definitions. Table S2 . REF/HEFCE definitions. Table S2a . Research impacts as benefits, effects or changes (REF/HEFCE not cited). Table S3 . ARC/RQF definitions. Table S4 . Bibliometric definitions. Table S5 . Use-based definitions. Table S6 . Original definitions. (DOCX 58 kb)

Authors’ contributions

The research questions resulted from discussions among all the authors. The search strategies were developed with input from all authors supported by the expertise of a specialist librarian. KA conducted the database searches, assessed the literature against the review criteria, and undertook data extraction, synthesis and analysis of the literature. All authors provided input into findings and conclusions and edited drafts of the article. All authors read and approved the final manuscript.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Kristel Alla, Phone: +61424129485, Email: [email protected] .

Wayne D. Hall, Email: [email protected] .

Harvey A. Whiteford, Email: [email protected] .

Brian W. Head, Email: [email protected] .

Carla S. Meurk, Email: [email protected] .