policy analysis method of research

A guide to policy analysis as a research method

Affiliations.

1 Department of Public Health, School of Psychology and Public Health, Latrobe University, Bundoora, Victoria, Australia.
2 Department of Global, Urban and Social Studies, RMIT University, 124 La Trobe Street, Melbourne, Victoria, Australia.
3 Department of Social Sciences, Faculty of Health, Arts and Design, Swinburne University, 24 Wakefield Street, Hawthorn, Victoria, Australia.
4 Department of Nutrition Dietetics and Food, Monash University, Level 1, 264 Ferntree Gully Road, Notting Hill, Victoria, Australia.
PMID: 30101276
DOI: 10.1093/heapro/day052

Policy analysis provides a way for understanding how and why governments enact certain policies, and their effects. Public health policy research is limited and lacks theoretical underpinnings. This article aims to describe and critique different approaches to policy analysis thus providing direction for undertaking policy analysis in the field of health promotion. Through the use of an illustrative example in nutrition it aims to illustrate the different approaches. Three broad orientations to policy analysis are outlined: (i) Traditional approaches aim to identify the 'best' solution, through undertaking objective analyses of possible solutions. (ii) Mainstream approaches focus on the interaction of policy actors in policymaking. (iii) Interpretive approaches examine the framing and representation of problems and how policies reflect the social construction of 'problems'. Policy analysis may assist understanding of how and why policies to improve nutrition are enacted (or rejected) and may inform practitioners in their advocacy. As such, policy analysis provides researchers with a powerful tool to understand the use of research evidence in policymaking and generate a heightened understanding of the values, interests and political contexts underpinning policy decisions. Such methods may enable more effective advocacy for policies that can lead to improvements in health.

Keywords: interpretive policy analysis; mainstream policy analysis; nutrition; public health; sugar sweetened beverage tax.

Health Policy*
Health Promotion
Policy Making*
Public Health / legislation & jurisprudence
Public Health Administration
Public Policy
Research Design

Technical Support
Find My Rep

Methods for Policy Research Taking Socially Responsible Action

Ann Majchrzak - University of Southern California, USA
M. Lynne Markus - Bentley University, USA
Description

This book about responsible and evidence-based decision making is written for those interested in improving the decisions that affect people’s lives. It describes how to define policy research questions so that evidence can be applied to them, how to find and synthesize existing evidence, how to generate new evidence if needed, how to make acceptable recommendations that can solve policy problems without negative side effects, and how to describe evidence and recommendations in a manner that changes minds.

Policies are not just the decisions made by a country’s rulers or elected officials; policies are also set by corporate executives, managers of department stores, and project leaders in non-profit organizations pursuing environmental protection. The authors’ suggestion are based on the fundamental belief that evidence-based decision making is superior to decisions based purely on opinion, intuition, and emotion. Because much has happened since 1984 when the first edition was published, this is a substantially different book with a new co-author, new and updated examples, new chapters, and new frameworks for understanding.

See what’s new to this edition by selecting the Features tab on this page. Should you need additional information or have questions regarding the HEOA information provided for this title, including what is new to this edition, please email [email protected] . Please include your name, contact information, and the name of the title for which you would like more information. For information on the HEOA, please go to http://ed.gov/policy/highered/leg/hea08/index.html .

For assistance with your order: Please email us at [email protected] or connect with your SAGE representative.

SAGE 2455 Teller Road Thousand Oaks, CA 91320 www.sagepub.com

Loved the first edition as a graduate student when it came out in the mid-80s; so happy that a new edition was developed so that I can share it with my graduate students.

still under review for consideration.

NEW TO THIS EDITION:

Each chapter’s phase in the policy research voyage (depicted by artwork with a nautical theme) includes clearly defined activities, deliverables, criteria for successful performance, and workflow diagrams.
Policy Change Wheel and STORM Context Conditions frameworks make it easier for readers to remember what needs to be done.
New chapters on synthesizing available evidence (Chapter 3) and reflecting on policy research experiences (Chapter 7) broaden the book’s coverage.
Updated examples drawn from a variety of contexts, including international and business policy, as well as domestic issues, illustrate applications of evidence-based decision making in the real world.
Chapter 1, Making a Difference with Policy Research , now reflects an action-orientation toward not just doing policy research, but also toward fostering change and doing policy research responsibly.

KEY FEATURES:

A how-to orientation encourages readers to consider the evidence systematically and responsibly before making a decision and to communicate evidence and recommendations in a way that facilitates real change.
Real world examples throughout the text show readers the everyday applications of policy decision making.
Exercises at the end of each chapter give students an opportunity to apply what they’ve learned.

This is a substantially revised edition of Methods for Policy Research, originally published in 1984. This book reframes policy research as responsible and evidence-based decision making. It describes how to define policy research questions so that evidence can be applied to them, how to find and synthesize existing evidence, how to generate new evidence if needed, how to make acceptable recommendations that can solve policy problems without harmful side effects, how to describe evidence and recommendations in a manner that changes minds. This book is meant to help individuals who want to improve the policy decisions that affect people's lives.

Responsible and evidence-based decision making is needed not just in government and social service agencies. It is also needed in businesses and in nongovernmental organizations such as charities, foundations, and non-profits. In this book, we state our values clearly: We believe that evidence-based decision making is superior to decisions based purely on opinion, intuition, and emotion. We also believe that responsible decision-making requires taking into account the possibility of harmful consequences from policy change, no matter how well intentioned those changes may be.

Each chapter now has clearly defined activities and deliverables, supported by workflow diagrams, along with tracking indicators that policy researchers can use to assess how well they are performing the activities. New frameworks are presented such as the M2 test (meaningfulness and manageability), the Policy Change Wheel, and STORM (Social, Technical, Organizational, Regulatory, and Market) context conditions to make it easier for readers to remember what needs to be done. All examples are updated, they are drawn from a variety of contexts, including international and business policy, as well as domestic policy and social service.

Each chapter was substantially revised to make the activities and outcomes of policy research clear. We've introduced new content, including an entirely new chapter on synthesizing existing evidence. We've exposed the reader to useful websites, to new ways of involving stakeholders in the Case for Change, and to ways of ensuring that recommendations derived from evidence-gathering are meaningful and manageable. A nautical theme, a conversational style, and humor are used throughout to make the reading enjoyable. (Look out for puns!)

Sample Materials & Chapters

For instructors, select a purchasing option.

This title is also available on SAGE Research Methods , the ultimate digital methods library. If your library doesn’t have access, ask your librarian to start a trial .

Help & FAQ

A guide to policy analysis as a research method

Nutrition Dietetics & Food

Research output : Contribution to journal › Article › Research › peer-review

interpretive policy analysis
mainstream policy analysis
public health
sugar sweetened beverage tax

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1093/heapro/day052

Community Engagement Program

Supporting bi-directional community engagement to improve the relevance, quality, and impact of research.

Getting Started
Resources for Equity in Research
Community-Engaged Student Practice Placement
Maternal Health Equity
Youth Mental Health
Leadership and Membership
Past Members
Study Review Rubric
Community Ambassador Initiative
Implementation Science Working Group
Past Webinars & Podcasts
Policy Atlas
Community Advisory Board

For more information:

Policy research.

Health Policy research aims to understand how policies, regulations, and practices may influence population health. Translating research into evidence-based policies is an important approach to improve population health and address health disparities.

The policy process, although complex and dynamic, provides an opportunity to ask different types of research questions and apply various methodologies – from public health law to health services research to cost-effectiveness to policy implementation and dissemination.

Simplified version of the policy life cycle.

View image description .

Harvard Catalyst Policy Atlas

Policy Atlas is a free, web-based, curated research platform that catalogues downloadable policy-relevant data, use cases, and instructional materials and tools to facilitate health policy research. The Policy Atlas includes data on various health topics and policies, and may be used for research, evaluation, or a quick summary of state rankings or health trends. The topics vary from health laws and bills, to social and environmental determinants of health, to health disparities, and other topics.

Examples of Policy Research

Policy adoption study: Examination of Trends and Evidence-Based Elements in State Physical Education Legislation: A Content Analysis

This study comprehensively reviewed existing state legislation on school physical education (PE) requirements to identify evidence-based policies. The authors found that despite frequent PE bill introduction, the number of evidence-based bills was relatively low.

Policy implementation study: Political Analysis for Health Policy Implementation

To better understand factors that may affect the process and the success of policy implementation, a political analysis was conducted to identify stakeholder groups that are likely to play a critical role in the process. The results revealed that six groups impact the implementation process: interest groups, bureaucratic, budget, leadership, beneficiary, and external actors.

Policy impact evaluation: Reducing Disparities in Tobacco Retailer Density by Banning Tobacco Product Sales Near Schools

This study examined whether a policy ban on tobacco product sales near schools could reduce existing socioeconomic and racial/ethnic disparities in tobacco retailer density in Missouri and New York. The findings suggested that the policy ban would reduce or eliminate existing disparities in tobacco retailer density by income level and by proportion of African American residents.

Other Resources

Health Policy Analysis and Evidence : Centers for Disease Control and Prevention (CDC) resource on health policy analysis and evidence-driven policy to improve population health.

Health in All Policies (American Public Health Association) : a policy approach to address social and other factors that influence health and equity.

Four-Part Webinar Series on Policy Evaluation (National Collaboration on Childhood Obesity Research) : This seminar series aims to increase skills of researchers and practitioners in policy evaluation effectiveness.

Research Tools (Center for Health Economics and Policy, Washington University in St. Louis) : health economics and policy research tools, including cost effectiveness, policy analysis toolkit, and policy analysis web series.

Theory & Methods (Center for Public Health Law Research) : public law/ legal epidemiology methods for conducting research on the impact of laws and legislation on public health.

Introduction to Legal Mapping (ChangeLab Solutions) : an introduction to legal mapping, a method to determine what laws exist on a certain topic, collect and summarize policy data, and ultimately estimate the effects of these policies on health outcomes.

The Methods Centers at Pardee RAND Graduate School : a resource on research methods for conducting policy research, including qualitative and mixed methods, decision making, causal inference, decision making and data science and gaming approaches.

View PDF of the above information.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
My Account Login
Explore content
About the journal
Publish with us
Sign up for alerts
Open access
Published: 01 May 2023

What are the core concerns of policy analysis? A multidisciplinary investigation based on in-depth bibliometric analysis

Yuxue Yang ORCID: orcid.org/0000-0002-8772-1024 1 , 2 ,
Xuejiao Tan 1 ,
Yafei Shi 1 &
Jun Deng 1 , 2

Humanities and Social Sciences Communications volume 10 , Article number: 190 ( 2023 ) Cite this article

7946 Accesses

2 Citations

9 Altmetric

Metrics details

Environmental studies
Medical humanities
Social policy

Policy analysis provides multiple methods and tools for generating and transforming policy-relevant information and supporting policy evolution to address emerging social problems. In this study, a bibliometric analysis of a large number of studies on historical policy analysis was performed to provide a comprehensive understanding of the distribution and evolution of policy problems in different fields among countries. The analysis indicates that policy analysis has been a great concern for scholars in recent two decades, and is involved in multiple disciplines, among which the dominant ones are medicine, environment, energy and economy. The major concerns of policy analysts and scholars are human health needs, environmental pressures, energy consumption caused by economic growth and urbanization, and the resulting demand for sustainable development. The multidisciplinary dialog implies the complicated real-world social problems that calls for more endeavors to develop a harmonious society. A global profiling for policy analysis demonstrates that the central policy problems and the corresponding options align with national development, for example, developing countries represented by China are faced with greater environmental pressures after experiencing extensive economic growth, while developed countries such as the USA and the UK pay more attention to the social issues of health and economic transformation. Exploring the differences in policy priorities among countries can provide a new inspiration for further dialog and cooperation on the development of the international community in the future.

Scientific evidence on the political impact of the Sustainable Development Goals

Transforming power of research and development on inequality and well-being: a European Union perspective within the circular economy framework

Mapping the community: use of research evidence in policy and practice

Introduction.

Social problems are evolving with the rapid development of economy, and the problems mankind is facing and options they choose reflect the developmental demand. Policy is a political action with specific subjects, targets, and strategies in a certain period of time, which primarily aims to create a healthy environment for the development of society (Porter, 1998 ; Lasswell and Kaplan, 1950 ; Yang et al., 2020 ). As for policy analysis, the definition varies a lot. According to William Dunn ( 2015 ), policy analysis is ‘an applied social science discipline, which uses multiple methods of inquiry and argument to produce and transform policy-relevant information that may be utilized in political settings to resolve policy problems.’ Jabal et al. ( 2019 ) defined that policy analysis provides methods and tools for assessing whether a policy is ‘correct and fit for their use’ and supporting policy evolution. Manski ( 2019 ) regarded policy analysis as a shorthand term that describes the process of scientific evaluation for the impact of past public policies and prediction of the potential outcomes of future policies. More generically, policy analysis is aimed to understand who develops and implements certain policies, for whom, by what, with what effects, and what techniques and tools can be used, and so on (Blackmore and Lauder, 2005 ; Collins, 2005 ).

Accordingly, regarding the typology of policy analysis, three categories can be established based on ontology and epistemology (Fig. 1 ) (Bacchi, 1999 ; Colebatch, 2006 ; Jennifer et al., 2018 ): (1) Positivism paradigm. Focusing on policy facts, this orientation of policy analysis aims to identify policy problems and weighting the optimal solution guided by the theory of economic frameworks, basic scientific models, and behavioral psychology through objective analysis. Economic analysis, cost-benefit analysis, quantitative modeling and nudge politics are the most commonly used methods in this orientation (Althaus et al., 2013 ; Jennifer et al., 2018 ); (2) Constructivism paradigm. In this orientation, policy is conceptualized as ‘the interaction of values, interests and resources guided through institutions and mediated through politics’ (Davis et al., 1993 ) rather than a comprehensively rational and linear process in which analysis involves policy agenda setting, policy processes, policy networks and governance, mainly focusing on values, actors and political rationality of policy. Theoretical frameworks, such as multiple stream theory, behavioral psychology and advocacy coalition framework, etc. are typically used in such orientation (Kingdon, 1984 ; Browne et al., 2019 ; Sabatier and Weible, 2014 ); (3) Interpretivism paradigm. This orientation is focused on interpreting how policy problems can be defined or constructed and how the problem framing shapes the possible policy responses (Bardach, 2000 ). A substantial body of research has discussed the theory underlying the problem, framing and governmentality using narrative analysis, discourse analysis, ethnographic methods, etc. (Hajer, 1995 ; Hajer, 2006 ; Martson and Mcdonald, 2006 ). Therefore, a systematic review of policy analysis can present the past and present policy problems of concern and the relevant possible options from an evolutionary perspective.

The framework was organized according to Jennifer et al. ( 2018 ).

The profoundly complex and diversified realistic demands such as equity and sustainability (Akadiri et al., 2020 ), the changes of energy planning (Banerjee et al., 2000 ; Pandey et al., 2000 ; Pandey, 2002 ) and transition of modern markets (Blackman and Wu, 1999 ) have important implication on policy decisions (Munda, 2004 ). A multidisciplinary investigation on policy analysis can provide more reflections on how to develop a harmonious society. Studies have shown that the priority of policy agenda is determined by three key factors: the nature of the issue (Shiffman and Smith, 2007 ), the policy environment (Adams and Judd, 2016 ; Sweileh, 2021 ) and the capabilities of proponents (Shawar and Shiffman, 2017 ). Due to differences in geography, economics, politics and many other aspects, social concerns and policy priorities vary enormously in different countries. In the global context, how countries set policy priorities in different stages of development, and how policy priorities align with the national development remain unknown. So, developing a global profiling for policy analysis can present the differences in core concerns of polices among countries, thus promoting further dialog and cooperation on the development of the international community in the future.

Bibliometric analysis has long been used as a statistical tool to systematically review scientific literature (Hood and Concepcion, 2001 ). A rigorous bibliometric analysis can provide systematic insights into previous publications, which can not only delve into the academic research community of active and influential researchers, but also identify the current research topics, and further explore potential directions for future research (Fahimnia et al., 2015 ). Bibliometrics has been widely applied in a wide range of sectors and specific domains, for example, mapping and visualizing the knowledge progress avenues and research collaboration patterns of cultural heritage (Vlase and Lähdesmäki, 2023 ), analyzing the sub-areas and core aspects of disease (Baskaran et al., 2021 ), visualizing and graphing the evolution of research related to sustainable development goals (Belmonte-Ureña et al., 2021 ), and studying policies, such as agricultural policy (Fusco, 2021 ), medical information policy (Yuxi et al., 2018 ), and science, technology and innovation policy (Zhang et al., 2016 ). However, the research trajectory and focus of policy analysis around the world remain a black box. In the present paper, a bibliometric analysis was performed from three dimensions: time, intensity, and scope, which referred to hot point changes over time, the quantity of research and the core concerns of policy, respectively.

In the present paper, a bibliometric analysis of a large number of studies on historical policy analysis was performed to answer the questions: (1) What core concerns are reflected in the policy analysis and how does these core concerns reflect real-world social problems? (2) How do these core concerns change over time? (3) What are the differences in core concerns among countries and what drives those differences? From an evolutionary perspective, this paper aims to uncover the past and present policy problems of concern and the relevant possible options, thus providing a clue for future policy analysis. The analysis of the evolution and differences in policy problems among countries may provide a view of the development context of different countries and put forward new inspiration and hope for further dialog and cooperation on the development of the international community in the future. Furthermore, another possible key sustainability implication with respect to the core concerns of policy analysis is to provide a reference for exploring the gaps between academic research and policy agenda.

Literature research

In the present study, Web of Science (WOS) Core Collection database was used for data retrieval (Vlase and Lähdesmäki, 2023 ). This research was conducted in four steps. Firstly, articles related to policy analysis were searched to select the most cited ones, which reflect the most influential research and the cutting-edge knowledge over time. MerigÓ et al. ( 2016 ) and Markard et al. ( 2012 ) weighted the most citation in an absolute term that means the total citations of all time. According to Fusco ( 2021 ) and Essential Science Indicators, the most citation was weighted in a relative term, which means the citation number in the publication year. The top 1% papers, compared to other articles in the academic field published in the same publication year, were included in this study following the refining principle of Essential Science Indicators, ensuring that the impact of these articles does not fade with time. Secondly, the selected papers were further screened, and narrowed down to different collected datasets for in-depth analysis according to the results of screening. Thirdly, statistical analysis and network visualization of authorship, organization and geographical distribution, topics and their chronological trends in each dataset were performed using VOSviewer software, which is freely available to construct and visualize bibliometric network (see www.vosviewer.com ) (Van-Eck and Waltman, 2010 ). Lastly, the association between policy analysis and academic articles was explored in different fields.

Dataset construction

Originally, a total of 118,535 articles related to policy analysis were retrieved using the strategy “TS = (policy analysis)”. For further discipline analysis, the most cited articles were selected with the quick filtering toolbar of WOS. Consequently, 1287 most cited papers of policy analysis were included in dataset 1. Then co-citation analysis of journals was performed to provide clues for discipline research (Supplementary Table 2 ). Accordingly, policy analysis-related articles from journals in the medicine field were selected for dataset 2, and 7963 articles were finally included. Similarly, 15,705 articles from journals in the field of environment were included in dataset 3; 6253 articles from journals in the field of energy in dataset 4; 1268 articles from journals in the field of economy in dataset 5; and 2243 articles from multidisciplinary journals in dataset 6. According to Journal Citation Reports of WOS, multidisciplinary journals refer to those journals in which articles involve at least two disciplines, such as Ecological Economics that involves ecology and economics. The search strategy of each database is shown in Table 1 .

Network visualization

Publication information of policy analysis was presented, including publication number, countries and organizations of key players, which reflects the value of and actual needs for policy analysis. Then, VOSviewer was used for network visualization of co-authorship, co-occurrence and citation. Co-authorship analysis for organizations and countries, which met the thresholds identified more than 5 articles for further investigation of the key players’ geographical distributions and their collaboration patterns. Co-occurrence analysis for all keywords based on the frequency of keywords used in the same article was carried out for topic mining (Kern et al., 2019 ). Citation analysis was performed to investigate the citation attributes received by other items. Meaningless or common terms were removed (Zhang and Porter, 2021 ). The research framework is shown in Fig. 2 .

The research framework for multidisciplinary investigation in policy analysis.

Publication information of policy analysis

Firstly, the publication number of policy analysis was determined. A total of 118,535 policy analysis articles were published between 2003 and 2021 (Fig. 3 ), showing a surge in the development of policy analysis with an exponential growth rate of 53.98 and 84.03% in the last 5 years (2017–2021) and 10 years (2012–2021), respectively.

Source : Data was collected from Web of Science (WOS) Core Collection database on the topic (TS) “policy analysis”.

For network construction, 1287 most cited papers were screened. The collaboration network of countries was visualized and illustrated, showing that 112 countries have published the most cited policy analysis articles. As for the co-authorship of countries and organizations, 2286 universities were identified, and 193 of them from 59 countries met the criteria of network analysis, among which the universities from the USA (University of Washington, Harvard University), the UK (University of Oxford, University of Cambridge) and China (University of Chinese Academy of Sciences) had the largest number of links and the strongest willingness to cooperate with other organizations (Fig. 4A, B and Supplementary Table 1 ). The willingness of cooperation not only meets the needs of academic research, but also conforms to the general expectations of the international community. Citation analysis for sources identified 51 journals from five different fields (Fig. 4C and Supplementary Table 2 ), in which environment-related journals accounted for the largest number (e.g., Journal of Cleaner Production, Science of The Total Environment , Global Environmental Change-Human and Policy Dimensions , Transportation Research Part D: Transport and Environment and Environmental Modeling & Software) , followed by medicine-related journals ( The Lancet , JAMA , The Lancet Infectious Diseases , PLOS One and The Lancet Global Health) , the journals of energy science ( Sustainable Cities and Society , Energy Policy , Applied Energy , Renewable Energy and Energy ), the journals of economy ( International Journal of Production Economics and Transportation Research Part A: Policy and Practice ), and then several multidisciplinary journals ( Ecological Economics , Nature , PNAS, Nature Communications and European Journal of Operational Research ).

A Co-authorship analysis for countries; B Co-authorship analysis for organizations; C Citation network; D Co-occurrence network.

In the co-word network of policy analysis, four main clusters were displayed: the blue cluster concerned with environmental policy problems; the green cluster related to medicine (e.g., public health, prevalence and mortality of disease); the red cluster centering policy, such as policy framework, policy systems, and policy implementation; and the yellow cluster mainly concerned with energy (e.g., energy consumption, energy efficiency and electricity generation) (Fig. 4D and Table 2 ). Simultaneously, more details related to real-world social issues were also found, such as the common and core concerns about carbon emission, economic growth, prevalence and mortality of disease. Additionally, management is in the spotlight (e.g., system, framework, efficiency and challenge).

Publication information of policy analysis in different fields

Policy analysis-related articles mainly involved the fields of medicine, environment, energy, economy and multidiscipline. The publication information in different fields was investigated. First, the volume growth trend over time was traced. Generally, a growing number of articles were published annually. The most obvious growth was found in policy analysis in environment, followed by medicine and energy, and the growth in economy and multidiscipline was relatively stable (Fig. 5 ). Specifically, the first increase in the publication number of policy analysis in medicine was seen in 2009, and then a steady growth was maintained, followed by a second acceleration after 2019, which may relate to the pandemic of H1N1 influenza and COVID-19, respectively (WHO, 2012 ; Wouters et al., 2021 ). A great growth in environmental policy analysis was observed after 2015, and a linear growth after 2017. In energy policy analysis, the first increase occurred in 2009, reaching a peak in 2013, followed by a second increase in 2016, reaching another peak in 2020. Then the publication information about organizations and countries was explored. The top five countries and institutions with the largest number of policy analysis articles in different fields are presented in Supplementary Table 3 . The results showed that the USA, the UK and China attached great importance to policy analysis in all of these fields.

Publication dynamics of policy analysis-related articles in the fields of medicine, environment, energy, economy and multidiscipline between 2003 and 2021.

Policy analysis in the field of medicine

A total of 8381 organizations from 177 countries contributed to medical policy analysis. Further investigation showed that universities from the UK (e.g., University of London, London School of Hygiene & Tropical Medicine and University College London), the USA (e.g., Harvard University and University of California San Francisco), Canada (e.g., University of Toronto) and Australia (e.g., University of Melbourne, University of Sydney) contributed the most to medical policy analysis with the greatest willingness to collaborate both domestically and internationally. By contrast, Chinese universities, such as Peking University, University of Chinese Academy of Sciences and Zhejiang University, were more prone to domestic collaboration (Fig. 6A, B ).

A Co-authorship analysis for countries; B Co-authorship analysis for organizations; C Co-occurrence network; D Overlay network.

Co-occurrence analysis of keywords showed that of the 16,719 keywords identified from 7963 retrieved items, 1778 keywords met the threshold. In addition to the three core topics “medicine”, “policy” and “health” (e.g. health policy, public health), the mortality, prevalence, risk factors as well as prevention of diseases have been the key focus of medical policies. Additionally, the issues of children and adolescents, such as physical activity, overweight and childhood obesity, have also attracted medical scientists and policy analysts. Figure 6D shows the average annual overlay network of keywords. The most recent concerns are the prevalence of COVID-19 and relevant topics associated with SARS-CoV-2 and coronavirus. Moreover, sex-specific mortality, life satisfaction and affordable care act are also the hot topics in recent years (Fig. 6C, D ).

Policy analysis in the field of environment

Co-authorship analysis showed that 9060 organizations from 160 countries contributed to environmental policy analysis, among which universities from China played a key role, especially University of Chinese Academy of Sciences, Tsinghua University, Beijing Normal University, North China Electric Power University and Beijing Institute of Technology (Fig. 7A, B and Supplementary Table 3 ). Of the 44,213 keywords in retrieved 1 5705 articles related to environmental policy analysis, 3638 met the threshold of keyword co-occurrence analysis. The co-word network showed that apart from the words with vague meanings such as “policy”, “impact” and “management”, “carbon emission”, “climate change” and “sustainability” were the most visible in the network. Note that the terms like “energy”, “economic growth” and “urbanization” were also easy to notice (Fig. 7C ). The analysis for the average annual overlay showed that “kyoto protocol”, “acid deposition” and “policy development”, etc. were earlier terms, while “plastic pollution”, “Cross-Sectionally Augmented Autoregressive Distributed Lag” and “population structure”, though lightly weighted, were the most recent ones. The color of overlay network visualization of environmental policy analysis appeared to be yellow, indicating that environmental problems have attracted researchers all over the world in past decades (Fig. 7D ). The abovementioned results demonstrated the positive attitude of policy analysts and indicated a shift of their attention over time, possibly due to the evolution of environmental problems.

Policy analysis in the field of energy

The collaboration network showed that 3668 organizations from 117 countries performed policy analysis in energy. The top five organizations were Tsinghua University, University of Chinese Academy of Sciences, Xiamen University, North China Electric Power University and Beijing Institute of Technology, all of which showed strong willingness to collaborate both domestically and internationally. The network showed that there was complex knowledge interaction and flow in the citation of energy policy analysis (Fig. 8A, B ). Of the 15,027 keywords in retrieved 6253 articles, 1225 met the threshold. Co-occurrence network (Fig. 8C ) revealed that policy analysis in energy was primarily focused on the demand for renewable energy (such as “wind power”, “solar power”, “bioenergy”) due to emission (e.g. “carbon emission”, “greenhouse gas emission”) and energy consumption. The terms “restructuring”, “discount rates” and “kyoto protocol” were early noticed by researchers, and the analysis of kyoto protocol was performed earlier in energy than that in ecology. Then, “green power”, “green certificates” and “energy policy analysis” gradually came into the eyes of analysts. Similarly, the prevalence of COVID-19 was the greatest concern of energy policy analysts, followed by “energy communities” and “renewable energy consumption” (Fig. 8D ).

Policy analysis in the field of economy

1144 organizations from 67 countries were found to contribute almost the same to policy analysis in economy. Hong Kong Polytechnic University, Delft University of Technology, University of Leeds, Rensselaer Polytechnic Institute and University of Sydney had the largest number of publications. Hong Kong Polytechnic University, Delft University of Technology, University of British Columbia, University of Sydney and Rensselaer Polytechnic Institute had the highest collaboration (Fig. 9A, B ). Of the 5970 keywords in retrieved 1268 papers, 395 met the threshold. The co-word network showed that in addition to the general words frequently used in articles (e.g. “policy”, “impact”, “system”), the specific words reflecting the most common topics for policy problem of economy were “transport” (associated with vehicles, public transport, travel behavior, etc.), “supply chain” (related to supply chain management, supply chain coordination, green supply chain, etc.), and “inventory” (related to the model, control and system of inventory, etc.) (Fig. 9C ). The overlay network analysis showed that economic policy analysts had an early interest in inventory-related topics and the issue of supply chain management, but has been concerned with the sustainability of supply chain management only in recent years. Additionally, topics like “circular economy”, “life-cycle assessment”, “industry 4.0” and “automated vehicles” also attracted scholars’ attention. (Fig. 9D ).

Policy analysis in multidiscipline

In the co-authorship network, universities such as Stanford University, University of Chinese Academy of Sciences, University of Maryland, University of California, Berkeley and University of Cambridge had the most publications and a high collaboration. University of California Irvine had fewer publications but relatively higher link, showing that this university was strongly willing to cooperate with other organizations (Fig. 10A, B ). Of the 9467 keywords in retrieved 2243 articles, 648 met the threshold. This multidisciplinary research revealed the relationship between economy, environment and energy. However, there were obstacles to extend the relationship between them. Co-word network demonstrated that the policy analysis articles published on the multidisciplinary journals were mainly focused on the topics of “climate change”, “sustainability” and “inventory”. The term “climate change” is mainly related to issues of environmental resources (e.g., land use, deforestation, biodiversity), greenhouse gas emission (especially carbon emission) and energy consumption. The term “sustainability” is mainly connected with the relationship between environmental resources and economic growth. In addition to COVID-19, the terms “big data” and “circular economics” were on the cut edge (Fig. 10C, D ).

Policy analysis aims to understand what is the governments’ focal point, investigate why and how governments issue policies, evaluate the effects of certain policies (Browne et al., 2019 ), and reflect political agenda driven by social concerns or international trends (Kennedy et al., 2019 ). In this study, a bibliometric analysis of a large number of publications on historical policy analysis was carried out to explore the policy problems of concern and the relevant possible options from an evolutionary perspective, and provide a guide for future research. From 2003 to 2021, the number of publications on policy analysis grew exponentially. Before 2011, little attention was paid to policy analysis, but in recent decades, more importance has been attached to policy analysis around the world due to increasingly prominent social problems, especially the human health needs, degradation of environment, energy consumption and the relationship between economy, energy and environment.

From the perspective of global visibility, the policy analysis in medicine has received increasing attention from scholars from 8381 organizations of 177 countries, indicating that health problems, though not numerically dominant, have the widest coverage. Among these countries, the USA, the UK, Australia, Canada and China are the major contributors. The developed countries, such as the USA, the UK, Canada and Australia, have strongly supported addressing complex public health issues by developing effective policy responses (Moore et al., 2011 ; Atkinson et al., 2015 ). Typically, they spend the most on health, with 12318, 5387, 5905 and 5627 dollars per capital, respectively, while the developing countries spend relatively less, such as 894 dollars per capital in China and 231 dollars per capital in India (OECD, 2022 ). Great attempts have been made to analyze the burden of prevalence and mortality of diseases such as cancer, cardiovascular diseases and diabetes both globally and regionally (Yusuf et al., 2020 ; Rudd et al., 2020 ; Kearney et al., 2005 ). Other health issues of women, children and adolescents have been monitored and measured for years in many countries that respond to the Countdown to 2030 (Countdown to 2030 Collaboration, 2018 ). In addition, the worldwide outbreak of epidemics such as H1N1 influenza and COVID-19 pandemic has caused excess mortality and enormous social and economic costs all over the world, which greatly affect social policy and reveal the fragility of health systems to shocks (Wouters et al., 2021 ; Chu et al., 2020 ). By analyzing the global burden of disease, scholars have recommended policy-makers to give priority to the prevention and management of relevant diseases (Kearney et al., 2005 ).

Environmental policy analysis involving 15,705 articles has attracted largest attention from policy analysts and scientists. Greenhouse gas emission (mainly carbon emission) resulting in climate change and environmental degradation remains to be the most threatening and urgent issue, and has attracted attention of governments and the society (Tang et al., 2021 ; Ahmad et al., 2019 ). Different countries issued different climate policies aiming to reduce greenhouse gas emissions. The Kyoto protocol, ratified by 180 countries, committed to reduce the GHG emissions by 5% by 2012, compared with the 1990 emission levels (Kuosmanen et al., 2009 ). In the EU climate policy framework in 2014, the carbon emissions were projected to reduce by 40% by 2030, and by 80% by 2050 (European Council, 2014 ). The relationship between urbanization and environmental pressure was observed in the present research. During urbanization, the consumption of resources such as land, water and fuel has increased significantly, causing serious ecological pressure such as climate change, loss of biodiversity, land erosion and pollution. With the acceleration of economic growth and social commercialization, urbanization further increases the demands for housing, food, transportation, electricity and so on, which in turn aggravates the ecological pressure because of natural resource consumption, climate change, over-extraction and pollution (Ahmed et al., 2019 ; Wang et al., 2019 ). Hence, urbanization policies with restrictions on unplanned urban sprawl are under the way (Ahmed et al., 2020 ).

Energy is another big agenda for policy analysis. The close connection between energy and emission has been presented noticeably in this study. Governments have come to a consensus that there should be greater balance between ecological purity, energy supply and economic well-being if a country strives for healthy and sustainable economic development (Alola and Joshua, 2021 ). New environmental policies should be designed to control environmental pollution through reducing pollutant emissions and sustaining economic growth, and should be incorporated into governments’ macro policies (Halicioglu, 2009 ). Transformation of energy sector was on agenda to meet the ambitious goals (Cong, 2013 ). The UK, the USA and China are the global leaders in reducing actual emissions and increasing energy supply. In the USA, the shale revolution brought global attention to energy supply and remains to be a driving force for energy policies. Low-cost shale gas combined with the policy support for renewables have notably reduced CO 2 emissions over the past decades. Environmental deregulation is another central focus, which may affect the trajectory of greenhouse gas emission (International Energy Agency, IEA, 2019a , 2019b ). In the UK, the policy objectives of actual emission reduction, carbon budgets setting and investment in energy technology and innovation reflect the ambition for decarbonization (IEA, 2019a , 2019b ). As is known, China’s GDP grows rapidly, which has multiplied more than 170 times since the founding of the People’s Republic of China 73 years ago. However, the extensive economic growth mode depending on the primary and secondary industries has put high pressure on environment, such as large amounts of consumption and pollution (He et al., 2016 ; Yue et al., 2021 ; Yu and Liu, 2020 ). Data showed that the greenhouse gas emission (OECD, 2020 ) and air pollution exposure (OECD, 2022 ) in China have been far higher than those in other countries for a long time, posing great challenges to both the government and scholars. A specific policy package, such as the “Atmosphere Ten Articles”, “Soil Ten Plan” and “Water Ten Plan” from 2013 to 2016, and the “Regulation on the Implementation of the Environmental Protection Tax Law of the People’s Republic of China” in 2017, has been issued by Chinese government, aiming to improve the ecological environment. Furthermore, goals for renewable energy production were also set by scholars. Jacobson suggested that wind, water and sunlight energy should be produced by 2030, and then replace the existing energy by 2050 (Jacobson and Delucchi, 2011 ), while Lund proposed that renewable energy (the combination of biomass with wind, wave and solar) should account for 50% by 2030, and 100% by 2050 (Lund and Mathiesen, 2009 ). However, it remains unclear how many countries can achieve their stated goals. Numerous studies have shown the efforts of governments and scholars to transform the resource and energy usage-driven economic expansion to sustainable development.

From the economics perspective, the environmental Kuznets curve (EKC) hypothesis demonstrates the relationship between environmental quality and economic output, which has been proved by empirical studies (Fodha and Zaghdoud, 2010 ; Saboori et al., 2012 ). Additionally, the relationship between economic growth and energy consumption has also been confirmed (Shahbaz et al., 2015 ). In recent years, countries have been facing the challenge of economic structural transformation. The mode of economic growth that relies on the consumption of natural resource and waste disposal seems increasingly outdated (McDowall et al., 2017 ). Circular economy, a new mode for reconciling environmental and economic imperatives, has come into the public eye and appears to meet the common vision of sustainable development. With the increase of requirements of sustainable development and circular economy, greening of supply chain management also faces challenges, including inventory management, mode of transportation, life-cycle assessment and coordination with other areas (Ghosh and Shah, 2012 ; Ghosh and Shah, 2015 ). Thus, providing support for green supply chain supplier deserves the attention from policy-makers and practitioners.

Key findings

(1) Policy analysis has been a great concern of scholars for many years and has attracted increasing attention year by year, which reflects the value of and actual needs for policy analysis. (2) The world is facing common problems, which requires attention and efforts of the whole world, and a more harmonious social development such as the management of epidemics and complex disease, environmental-friendly development, green energy production and transformation from resource and energy usage-driven economic expansion to sustainable development is on the way. (3) Global profiling for policy analysis demonstrates that the central policy problems align with national development, which inspires further dialog and cooperation on the development of the international community in the future.

Limitations

This study has limitations. First, keywords cannot fully reflect the essential intent of an article although they are the key points of a study. Therefore, using keywords as an element for bibliometric analysis is far from enough. Second, this paper deals with academic research of policy analysis, but whether it is fully consistent with the policy agenda is unexplored. Moreover, we have shown the correlations between different phenomena, but the underlying mechanism remains indefinable.

Data availability

The datasets analyzed during the current study are available in the Dataverse repository ( https://doi.org/10.7910/DVN/XZMVMN ).

Adams B, Judd K (2016) 2030 agenda and the SDGs: indicator framework, monitoring and reporting. March 18. https://www.cadtm.org/2030-Agenda-and-the-SDGs-indicator

Ahmad M, Khan Z, Rahman ZU et al. (2019) Can innovation shocks determine CO 2 emissions (CO 2 e) in the OECD economies? a new perspective. Econ Innov New Technol 1:1–21

Google Scholar

Ahmed Z, Wang Z, Ali S (2019) Investigating the non-linear relationship between urbanization and CO2 emissions: an empirical analysis. Air Qual Atmos Health 12(8):945–953

Article CAS Google Scholar

Ahmed Z, Zafar MW, Ali S (2020) Linking urbanization, human capital, and the ecological footprint in G7 countries: an empirical analysis. Sustain Citi Soc 55:1–11

Akadiri SS, Alola AA, Bekun FV et al. (2020) Does electricity consumption and globalization increase pollutant emissions? Implications for environmental sustainability target for China. Environ Sci Pollut Res 27(20):25450–25460

Alola AA, Joshua U (2021) Carbon emission effect of energy transition and globalization: inference from the low-, lower middle-, upper middle-, and high-income economies. Environ Sci Poll Res 30:38276–38286

Althaus C, Bridgman P, David G (2013) The Australian policy handbook. Allen & Unwin, Sydney, p 137–153

Atkinson JA, Page A, Wells R et al. (2015) A modelling tool for policy analysis to support the design of efficient and effective policy responses for complex public health problems. Implement Sci 10(1):26

Article PubMed PubMed Central Google Scholar

Bacchi C (1999) Women, policy and politics: the construction of policy problems. Sage, London

Book Google Scholar

Bardach E (2000) A practical guide for policy analysis, 2nd ed. Chatham House, New York, pp. 2–46

Baskaran S, Agarwal A, Leisegang K et al. (2021) An in-depth bibliometric analysis and current perspective on male infertility research. World J Mens Health 39(2):302–314

Article PubMed Google Scholar

Banerjee R, Pandey R, Bhure A et al. (2000) Electricity demand estimation for village electrification. National Renewable Energy Conference, Indian Institute of Technology, Powai, Mumbai, India

Belmonte-Ureña LJ, Plaza-Úbeda JA, Vazquez-Brust D et al. (2021) Circular economy, degrowth and green growth as pathways for research on sustainable development goals: a global analysis and future agenda. Ecol Econ 185(19):107050

Article Google Scholar

Blackman A, Wu X (1999) Foreign direct investment in China’s power sector: trends, benefits and barriers. Energy Policy 27(12):695–711

Blackmore J, Lauder H (2005) Researching policy. In: Somekh B, Lewin C (eds) Res methods in the Soc Sci. Sage, London

Browne J, Coffey B, Cook K et al. (2019) A guide to policy analysis as a research method. Health Promot Int 34(5):1032–1044

Chu DK, Akl EA, Duda S et al. (2020) Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis. Lancet 395(10242):1973–1987

Article CAS PubMed PubMed Central Google Scholar

Colebatch H (2006) Accounting for policy in Australia. Public Policy 1(1):37–51

Collins T (2005) Health policy analysis: a simple tool for policy makers. Public Health 119(3):192–196

Article MathSciNet CAS PubMed Google Scholar

Cong RG (2013) An optimization model for renewable energy generation and its application in China: A perspective of maximum utilization. Renew Sustain Energy Rev 17:94–103

Countdown to 2030 Collaboration (2018) tracking progress towards universal coverage for reproductive, maternal, newborn, and child health. Lancet 389(10129):1538–1548

Davis G, Wanna J, Warhurst J et al. (1993) Public policy in Australia. Allen & Unwin, St Leonards, p 15

Dunn WN (2015) Public policy analysis. Routledge, Evanston, IL, USA

European Council (2014) European Council 23/24 October 2014—Conclusions (EUCO 169/14). European Council, Brussels

Fahimnia B, Sarkis J, Davarzani H (2015) Green supply chain management: A review and bibliometric analysis. Int J Prod Econ 162:101–114

Fodha M, Zaghdoud O (2010) Economic growth and pollutant emissions in Tunisia: An empirical analysis of the environmental Kuznets curve. Energy Policy 38(2):1150–1156

Fusco G (2021) Twenty years of common agricultural policy in Europe: a bibliometric analysis. Sustainability 13(19):1–15

Ghosh D, Shah J (2012) A comparative analysis of greening policies across supply chain structures. Int J Prod Econ 135(2):568–583

Ghosh D, Shah J (2015) Supply chain analysis under green sensitive consumer demand and cost sharing contract. Int J Prod Econ 164:319–329

Hajer M (1995) The politics of environmental discourse: ecological modernization and the policy process. Clarendon Press, Oxford

Hajer M (2006) Doing discourse analysis: Coalitions, Practices, Meaning. In: Brink van den, M. and Metze, T. (eds) Words matter in policy and planning: discourse theory and method in the social sciences.Koninklijk Nederlands Aardrijkskundig Genootschap, Utrecht

Halicioglu F (2009) An econometric study of CO 2 emissions, energy consumption, income and foreign trade in Turkey. Energy Policy 37(3):1156–1164

Article MathSciNet Google Scholar

He C, Han L, Zhang RQ (2016) More than 500 million Chinese urban residents (14% of the global urban population) are imperiled by fine particulate hazard. Environ Pollut 218:558–562

Article CAS PubMed Google Scholar

Hood WW, Concepcion WS (2001) The literature of bibliometrics, cientometrics, and informetrics. Scientometrics 52(2):291–314

International Energy Agency (2019a) Energy policies of IEA countries, The United States

International Energy Agency (2019b) Energy policies of IEA countries, The United Kingdom

Jabal AA, Davari M, Bertino E et al. (2019) Methods and tools for policy analysis. ACM Comput Surv 51(6):1–35

Jacobson MZ, Delucchi MA (2011) Providing all global energy with wind, water, and solar power, Part I: Technologies, energy resources, quantities and areas of infrastructure, and materials. Energy Policy 39(3):1154–1169

Jennifer B, Brian C, Kay C et al. (2018) A guide to policy analysis as a research method. Health Promot Int5:5

Kearney PM, Whelton M, Reynolds K et al. (2005) Global burden of hypertension: analysis of worldwide data. Lancet 365(9455):217–223

Kennedy C, O’Reilly P, O’Connell R et al. (2019) Integrative review: identifying the evidence base for policy making and analysis in health care. J Adv Nurs 75(2):1–15

Kern F, Rogge KS, Howlett M (2019) Policy mixes for sustainability transitions: New approaches and insights through bridging innovation and policy studies. Res Policy 48(10):103832

Kingdon J (1984) Agendas, alternatives, and public policies. Little, Brown, Boston

Kuosmanen T, Bijsterbosch N, Dellink R (2009) Environmental cost–benefit analysis of alternative timing strategies in greenhouse gas abatement: a data envelopment analysis approach. Ecol Econ 68(6):1633–1642

Lasswell HD, Kaplan A (1950) Power and society: a framework for political inquiry. Yale University Press, New Haven, p 71

Lund H, Mathiesen BV (2009) Energy system analysis of 100% renewable energy systems-the case of Denmark in years 2030 and 2050. Energy 34(5):524–531

Manski CF (2019) Communicating uncertainty in policy analysis. Proc Natl Acad Sci USA 116(16):7634–7641

Article ADS CAS PubMed Google Scholar

Markard J, Raven R, Truffer B (2012) Sustainability transitions: an emerging field of research and its prospects Research Policy 4:955–967

Martson G, Mcdonald C (2006) Analyzing social policy: a governmental approach. Edward Elgar, Cheltenham, pp. 205–222

McDowall W, Geng Y, Huang B et al. (2017) Circular economy policies in China and Europe. J Ind Ecol 21(3):651–661

MerigÓ JM, Rocafort A, Aznar-AlarcÓN JP (2016) Bibliometric overview of business & economics research. J Bus Econo Manag 17(3):397–413

Munda G (2004) Social multi-criteria evaluation: methodological foundations and operational consequences. Eur J Operat Res 158(3):662–677

Article MathSciNet MATH Google Scholar

Moore G, Redman S, Haines M, Todd A (2011) What works to increase the use of research in population health policy and programs: a review. Evid Policy 7(3):277–305

OECD (2020) Air and GHG emissions (indicator) (accessed on 24 Nov 2022)

OECD (2022) Air pollution exposure (indicator) (accessed on 24 Nov 2022)

OECD (2022) Health spending (indicator) (accessed on 30 Nov 2022)

Pandey R, Banerjee R, Phulluke S et al. (2000) Framework for design and evaluation of electric supply options for a rural area. National Renewable Energy Conference, Indian Institute of Technology, Powai, Mumbai, India

Pandey R (2002) Energy policy modelling: agenda for developing countries. Energy Policy 30(2):97–106

Porter ME (1998) The Competitive Advantage of Nations, Free Press

Rudd KE, Johnson SC, Agesa KM et al. (2020) Global, regional, and national sepsis incidence and mortality, 1990–2017: analysis for the Global Burden of Disease Study. Lancet 395(10219):200–211

Sabatier P, Weible C (2014) Theories of the policy process. Westview Press, Philadelphia

Saboori B, Sulaiman J, Mohd S (2012) Economic growth and CO 2 emissions in Malaysia: a cointegration analysis of the Environmental Kuznets Curve. Energy Policy 51:184–191

Shahbaz M, Loganathan N, Zeshan M et al. (2015) Does renewable energy consumption add in economic growth? An application of auto-regressive distributed lag model in Pakistan. Renew Sustain Energy Rev 44:576–585

Shawar YR, Shiffman J (2017) Generation of global political priority for early childhood development: the challenges of framing and governance. Lancet 389(10064):119–124

Shiffman J, Smith S (2007) Generation of political priority for global health initiatives: a framework and case study of maternal mortality. Lancet 370(9595):1370–1379

Sweileh WM (2021) Global research activity on elder abuse: a bibliometric analysis (1950-2017). J Immigr Minor Health 23(1):79–87

Tang K, Wang M, Zhou D (2021) Abatement potential and cost of agricultural greenhouse gases in australian dryland farming system. Environ Sci Pollut Res 28(17):21862–21873

Van-Eck NJ, Waltman L (2010) Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84(2):523–538

Vlase I, Lähdesmäki T (2023) A bibliometric analysis of cultural heritage research in the humanities: the web of science as a tool of knowledge management. Humanit Soc Sci Commun 10:84

Wang ZH, Ahmed Z, Bin Z (2019) The nexus between urbanization, road infrastructure, and transport energy demand: empirical evidence from Pakistan. Environ Sci Pollut Res Int 26(34):34884–34895

WHO (2012) Report of the WHO Pandemic Influenza A(H1N1) Vaccine Deployment initiative. https://www.who.int/influenza_vaccines_plan/resources/h1n1_deployment_report.pdf (accessed Dec 19, 2020)

Wouters OJ, Shadlen KC, Salcher-Konrad M et al. (2021) Challenges in ensuring global access to COVID-19 vaccines: production, affordability, allocation, and deployment. Lancet 397(10278):1023–1034

Yang C, Huang C, Su J (2020) A bibliometrics-based research framework for exploring policy evolution: a case study of China’s information technology policies. Technol Forecast Soc Change 157:120116

Yu Y, Liu HR (2020) Economic growth, industrial structure and nitrogen oxide emissions reduction and prediction in China. Atmos Pollut Res 11(7):1042–1050

Yue H, He C, Huang Q et al. (2021) Stronger policy required to substantially reduce deaths from PM2.5 pollution in China. Nat Commun 11(1):1462

Article ADS Google Scholar

Yusuf S, Joseph P, Rangarajan S et al. (2020) Modifiable risk factors, cardiovascular disease, and mortality in 155 722 individuals from 21 high-income, middle-income, and low-income countries (PURE): a prospective cohort study. Lancet 395(10226):795–808

Yuxi J, Wei W, Jun L et al. (2018) Trends and characteristics of global medical informatics conferences from 2007 to 2017: a bibliometric comparison of conference publications from Chinese, American, European and the global conferences. Comput Method Prog Biomed 166:19–32

Zhang YPorter AL, Cunningham S, Chiavetta D, Newman N (2021) Parallel or intersecting lines? Intelligent bibliometrics for investigating the involvement of data science in policy analysis. IEEE Trans Eng Manag 68(5):1259–1271

Zhang Y, Zhang G, Chen H et al. (2016) Topic analysis and forecasting for science, technology and innovation: methodology with a case study focusing on big data research. Technol Forecast Soc Change 105(2016):179–191

Download references

Acknowledgements

This work was financially supported by Special Project on Innovation and Generation of Medical Support Capacity (NO. 20WQ008) and Chongqing Special Project on Technological Foresight and Institution Innovation (NO. cstc2019jsyj-zzysbAX0037). We are also deeply grateful to prof. Ying Li and prof. Xia Zhang for their constructive suggestions to improve the manuscript.

Author information

Authors and affiliations.

Army Medical University, Chongqing, China

Yuxue Yang, Xuejiao Tan, Yafei Shi & Jun Deng

General Hospital of Xinjiang Military Command, Urumqi, China

Yuxue Yang & Jun Deng

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Deng .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent

Additional information.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Yang, Y., Tan, X., Shi, Y. et al. What are the core concerns of policy analysis? A multidisciplinary investigation based on in-depth bibliometric analysis. Humanit Soc Sci Commun 10 , 190 (2023). https://doi.org/10.1057/s41599-023-01703-0

Download citation

Received : 01 September 2022

Accepted : 18 April 2023

Published : 01 May 2023

DOI : https://doi.org/10.1057/s41599-023-01703-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

Explore articles by subject
Guide to authors
Editorial policies

Utility Menu

Harvard Ph.D. Program in Health Policy

Methods for Policy Research

Concentration Chairs: Mary Beth Landrum and Michael McWilliams (on sabbatical for the 2023-2024 academic year)

Training in the MPR concentration ( formerly called Evaluative Science & Statistics ) will position students to conduct rigorous, policy-relevant research. The concentration includes intensive methodological training that draws from several quantitative fields, including statistics, econometrics, and epidemiology, and also offers opportunities to learn qualitative methods. Students in the MPR concentration will also develop proficiency in experimental and quasi-experimental research designs. The MPR concentration does not require training in a specific discipline of social science, but students are allowed to use two of their distributional requirements to acquire some disciplinary training (e.g., in economics or sociology), and many students pursue additional coursework in a substantive area of interest or advanced levels of statistics. By virtue of this training, MPR students develop a multi-disciplinary toolkit that borrows from the strengths of different quantitative sciences and supports novel applications to health-related questions. Graduates are able to communicate and collaborate effectively with statisticians, economists, epidemiologists, and clinician-investigators, as they lead their independent work. The concentration’s broad methodological training flexibly supports scholarship on a wide range of topics, including common areas of interest in health services research related to health care access, quality, costs, and disparities, but also questions focused on clinical decision-making, behavior (e.g., of patients, physicians, or organizations), social determinants of health, and social programs with implications for health.

Methods for Policy Research Curriculum Guide

Curriculum Overview
Decision Sciences
Political Analysis
Core Course in Health Policy
Policy Areas
Joint Degrees

The institute

The International Institute for Applied Systems Analysis (IIASA) is an international research institute that advances systems analysis and applies its research methods to identify policy solutions to reduce human footprints, enhance the resilience of natural and socioeconomic systems, and help achieve the Sustainable Development Goals.

Sooyoung Kim,
Tyler Y. Headley,
Yesim Tozan

Published: May 10, 2024
https://doi.org/10.1371/journal.pgph.0003205
Reader Comments

Universal Health Coverage (UHC) and Global Health Security (GHS) are two high-priority global health agendas that seek to foster health system resilience against health emergencies. Many countries have had to prioritize one agenda over the other due to scarce resources and political pressures. To aid policymakers’ decision-making, this study investigated the individual and synergistic effects of countries’ UHC and GHS capacities in safeguarding essential health service delivery during the COVID-19 pandemic. We used a quasi-experimental difference-in-difference methodology to quantify the relationship between 192 countries’ progress towards UHC and GHS and those countries’ abilities to provide 12 essential childhood immunization services between 2015 and 2021. We used the 2019 UHC Service Coverage Index (SCI) to divide countries into a “high UHC group” (UHC SCI≥75) and the rest (UHC SCI 75), and similarly used the 2019 GHS Index (GHSI) to divide countries into a “high GHS group” (GHSI≥65) and the rest (GHSI<65). All analyses were adjusted for potential confounders. Countries with high UHC scores prevented a 1.14% (95% CI: 0.39%, 1.90%) reduction in immunization coverage across 2020 and 2021 whereas countries with high GHSI scores prevented a 1.10% (95% CI: 0.57%, 1.63%) reduction in immunization coverage over the same time period. The stratified DiD models showed that across both years, high UHC capacity needed to be augmented with high GHS capacity to prevent a decline in immunization coverage while high GHS alone was able to safeguard immunization coverage. This study found that greater progress towards both UHC and GHS capacities safeguarded essential health service delivery during the pandemic but only progress towards GHS capacity was both a necessary and likely sufficient element for yielding this protective effect. Our results call for strategic investments into both health agendas and future research into possible synergistic effects of the two health agendas.

Citation: Kim S, Headley TY, Tozan Y (2024) The synergistic impact of Universal Health Coverage and Global Health Security on health service delivery during the Coronavirus Disease-19 pandemic: A difference-in-difference study of childhood immunization coverage from 192 countries. PLOS Glob Public Health 4(5): e0003205. https://doi.org/10.1371/journal.pgph.0003205

Editor: Julia Robinson, PLOS: Public Library of Science, UNITED STATES

Received: July 3, 2023; Accepted: April 12, 2024; Published: May 10, 2024

Copyright: © 2024 Kim et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data used in this analysis is publicly available and citations to the dataset are available through the reference list.

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The ongoing Coronavirus Disease 2019 (COVID-19) pandemic has underscored the urgent need to strengthen the global health architecture for pandemic prevention, preparedness, and response [ 1 ]. Within the context of increasing health systems resilience, Universal Health Coverage (UHC) and Global Health Security (GHS) are two key global health agendas that are highly prioritized by global and national policy bodies, including the World Health Organization, and seek to foster strong health systems for a healthier and safer world [ 2 ]. There is a clear link between the two agendas: progress towards UHC promotes GHS by ensuring widespread access to comprehensive and inclusive health services before, during, and after public health emergencies [ 3 – 5 ]. Despite the interplay between UHC and GHS, there has been a long-standing disconnect between the two agendas at the policy level, and country-level efforts to align and integrate policies and funding activities have been limited due to scarce resources and domestic and global political pressures [ 2 , 5 ]. Due to these impediments, few countries have strategically prioritized policies that are aligned with both frameworks or have invested in integrated health systems, which simultaneously support UHC and GHS capacities [ 5 ]. The dearth of concurrent investment into both agendas may have led to worse health outcomes during the pandemic; some scholars have argued that countries which prioritized UHC and GHS integration better mitigated the pandemic’s impacts and might recover from COVID-19 more quickly [ 5 ]. The COVID-19 pandemic’s varied impacts on health systems have highlighted the importance of UHC as a crucial foundation of pandemic preparedness and response [ 1 ]. Countries’ progress towards UHC requires not only overall health systems strengthening but also sustainable pre-pooled funding mechanisms [ 6 ]; countries with strong UHC systems should therefore be more resilient to external shocks and more agile when responding to public health emergencies [ 2 ]. Consistent with this argument, studies investigating public health capacity showed that the lack of adequate primary health care compromised countries’ ability to implement an equitable response to COVID-19 or safeguard health services delivery [ 7 , 8 ]. In a recent study, using childhood immunization coverage data from 192 countries, we similarly showed that countries with greater progress towards UHC were associated with a significantly smaller decline in vaccine coverage during the first year of the COVID-19 pandemic [ 9 ].

An ongoing debate has been about whether investments into or the prioritization of one health framework alone, whether UHC or GHS, may result in optimal health system preparedness or response capacity, or whether a combination of investments towards both agendas is necessary to yield maximal health system resilience against pandemics and other health emergencies. To this end, some scholars have argued that investments in core GHS capacities (i.e., surveillance, risk communication, and coordination) alone are insufficient for comprehensive pandemic preparedness and response; these scholars instead advocate for concurrent investments in GHS and UHC [ 1 ]. This argument appears to be borne out by initial studies on the relationship between GHS and health system resilience, which found that GHS had null [ 10 – 12 ], mixed [ 13 ], or even negative [ 14 ] associations with countries’ ability to counter and withstand the COVID-19 pandemic. Given the suboptimal nature of investing into just one framework rather than both simultaneously, several international calls have been made for integrating GHS into UHC objectives to protect both individual and population health and to prevent health system failures during health emergencies [ 5 ]. In response to these calls, multiple international initiatives including proposals for a pandemic treaty and a pandemic fund have been launched to better prepare for and respond to the current pandemic and future health emergencies [ 1 , 15 ]. Despite growing calls for the alignment of the two health frameworks, there is no empirical evidence on the potential synergistic effects of UHC and GHS on safeguarding population health before, during, or after a public health crisis. While the ongoing COVID-19 pandemic provides an unprecedented opportunity to improve our understanding of the effects of UHC and GHS on health systems strengthening, extant studies examining their impact are limited in number, assessed the potential role of UHC and GHS individually, or generally focused on COVID-19 outcomes during the first wave of the pandemic in early 2020 [ 10 – 14 , 16 – 18 ].

Building and expanding on our previous study [ 9 ], this study examined the individual and synergistic protective effects of UHC and GHS on childhood vaccination coverage during the first two years of the COVID-19 pandemic (2020–2021). Our main hypothesis was that countries’ greater progress towards UHC and/or GHS capacities enable countries’ health systems to be more resilient to external shocks, such as the COVID-19 pandemic. This resilience will positively affect countries’ abilities to provision basic and essential public health services, including routine childhood immunizaitons. Childhood vaccination coverage is used as the outcome measure because immunization is considered an essential health service across all health care settings and all countries report annual vaccine coverage data [ 19 ]. To assess the protective effects of UHC and GHS on vaccination coverage individually, we employed a difference-in-difference (DiD) design following the methods we used in our previous study. We operationalized the UHC Service Coverage Index (SCI) 2019 as a measure of countries’ progress towards UHC and the Global Health Security Index (GHSI) as a measure of countries’ capacity for pandemic preparedness and response. To quantify the synergistic protective effects of UHC and GHS capacities on vaccination coverage during the same time period, we employed a stratified DiD approach. We hypothesized that countries with greater progress toward UHC, as represented by higher UHC SCI 2019 values, and stronger GHS capacity, as represented by higher GHSI scores, would better safeguard countries’ ability to provide essential health services during the COVID-19 pandemic. The findings of this study have the potential to inform local, national, and global policymakers when setting priorities and making investment decisions to strengthen countries’ health system readiness and resilience against future public health emergencies.

We employed a quasi-experimental difference-in-difference (DiD) design to quantify the independent and synergistic protective effects of UHC and GHS capacities on countries’ abilities to safeguard their provision of essential childhood immunization services during the COVID-19 pandemic. The DiD design has also been used to measure the pandemic’s effects on neonatal outcomes [ 20 , 21 ] and healthcare utilization rates [ 22 ]. The benefits of using childhood immunization coverage as a proxy to measure the effect of COVID-19 on health system resilience are two-fold: first, immunization is considered an essential health service across all healthcare settings [ 19 ]. While most studies have looked at the COVID-19 cases and mortality rates to elucidate overall health system resilience, this approach cannot parse apart the significant confounding effects of countries’ surveillance and laboratory test capacities, socio-behavioral factors influencing population testing behaviors, and population age structures and underlying morbidity factors [ 23 ]. Analytical frameworks measuring the impact of the pandemic on overall health system resilience through other routine but essential health services like childhood immunization programs can, therefore, better illustrate the direct impact of COVID-19 on countries’ health systems without severe concerns about the role of confounding variables. Second, the data availability and generalizability of other indicators of essential health services, especially over time, have either varied during the pandemic or did not exist prior to the pandemic; this makes the application of causal inference methods difficult. Childhood immunization rates, however, are a robust indicator both over time and at the global level because all countries have national immunization programs, and this data has been annually aggregated, reported, and standardized for decades [ 19 ].

Our dependent variable on national childhood immunization rates was derived from the WHO/UNICEF Joint Estimates of National Immunization Coverage [ 24 ]. The data includes the annual vaccination coverage rates of 14 vaccines for 195 countries spanning the years 1997 to 2021. We excluded the yellow fever vaccine (YFV) from our analysis because it is not administered widely across all countries, and thus leads to a data imbalance. We also excluded data on the first dose of the inactivated polio vaccine (IPV-1) because IPV-1 data was not available across all years under observation. After these exclusions, our analysis included 12 different childhood vaccines: Bacille Calmette-Guérin (BCG); the first and third dose of diphtheria, tetanus toxoid, and pertussis containing vaccine (DTP1, DTP3); the birth dose of hepatitis B vaccine (HEPB-3); the third dose of hepatitis B containing vaccine (HEPBB); the third dose of Haemophilus influenzae type B containing vaccine (HIB3); the first and second doses of measles containing vaccine (MCV1, MCV2); the third dose of pneumococcal conjugate vaccine (PCV3); the third dose of polio containing vaccine (POL3); the second or third dose of rotavirus vaccine (ROTAC); and the first dose of rubella containing vaccine (RCV1). Our primary independent variables were GHSI 2019 [ 25 ], which was developed by a partnership between the Nuclear Threat Initiative (NTI), the Johns Hopkins Center for Health Security at the Bloomberg School of Public Health, and the Economist Impact, and the UHC SCI 2019, which was obtained from the Institute for Health Metrics and Evaluation (IHME) [ 26 ]. GHSI 2019 is an assessment of countries’ health security and related capabilities necessary to prepare for future outbreaks including epidemics and pandemics [ 25 ]. GHSI 2019 consists of 37 indicators across 6 categories—prevention, detection and reporting, rapid response, health system, compliance with international norms, and risk environment. UHC SCI 2019 is a robust and widely used index that measures countries’ effective service coverage [ 26 ]. The UHC index is a weighted aggregate of 23 indicators measuring service coverage across the full spectrum of essential health service coverage—promotion, prevention, treatment, rehabilitation, and palliation—and across five age groups—newborn, children under 5 years, children and adolescents between 5–19 years, adults between 20–64 years, and older adults age 65 years or more—across the life course. Both indexes range from 0 to 100, with 100 indicating the highest preparedness capacity or a higher effective health service coverage. We also used the World Bank’s income level classification to assign countries to high, upper-middle, lower-middle, or low income groups [ 27 ]. Countries which lacked data on income classification or UHC/GHS scores were dropped from this analysis (Cook Island, Niue, Palestine), resulting in 192 included countries.

Statistical analysis

We first tested for the individual effects of countries’ progress toward UHC and GHS capacities on vaccination coverage using a quasi-experimental DiD design. We divided countries into treatment and control groups based on their progress toward UHC and GHS capacities using their respective index scores (UHC SCI and GHSI 2019). While both indexes are designed to summarize countries’ progress towards certain policy agendas on a standardized scale of 0 to 100, there is a dearth of prior studies identifying a threshold index value to define sufficient progress. Therefore, we first tested different UHC SCI and GHSI 2019 cutoff values to determine a meaningful delineator between the treatment and control groups. For GHSI 2019 (Fig A in S1 Text for its distribution), we performed DiD analyses using a sliding scale of cutoff values between 40 (53rd percentile) and 80 (99th percentile) with step increments of 5 to evaluate the lowest threshold value of GHSI 2019 where significant resilience occurred, as demonstrated by a statistically significant DiD estimator. Similarly, for UHC SCI 2019 (Fig B in S1 Text for its distribution), we used a sliding scale of values between 60 (53 rd percentile) and 90 (92 nd percentile) with step increments of 5 to observe the lowest threshold value when significance occurred. We then used these threshold values to define the treatment and control groups in subsequent analyses given the policy importance of understanding when and to what extent a significant positive effect is observed.

To perform DiD analyses, we used the WHO/UNICEF Joint Estimates of National Immunization Coverage data of 12 vaccines per given year and country as our dependent variable. We leveraged the COVID-19 pandemic to define a pre-post period, wherein the years prior to 2020 were defined as pre-pandemic, and the pandemic years of 2020–2021 were defined as post years (i.e., during the pandemic). We used doubly-robust DiD estimation methods proposed by Sant’Anna and Zhao [ 28 ] to obtain the average treatment effect on the treated (ATT) of higher levels of UHC or GHS capacities in safeguarding immunization coverage during the COVID-19 pandemic. The binary Treatment variable represents countries’ assigned group based on their progress towards UHC or GHS. We included time fixed effects using calendar years, as represented by γ t , and controlled for the group fixed effects using country and vaccine type, denoted by α i .

In all analyses, we included as covariates countries’ income group as per the World Bank classification and countries’ geographical region as per the WHO regional classification. For all analyses, we checked whether the parallel pre-trend assumption was satisfied after controlling for pre-trend covariates for each model. As a result of this process, our final analytical sample only included the data collected from 2015 onwards.

We next examined whether there was a synergistic impact of a country’s progress towards UHC or GHS capacities on immunization service provisioning during the pandemic by cross-tabulating the two treatment variables to further divide the countries into four separate groups: “High UHC/High GHS,” “High UHC/Low GHS,” “Low UHC/High GHS,” and “Low UHC/Low GHS.” We then performed additional DiD analyses to compare the first three groups to countries with low progress towards UHC and GHS. Acknowledging that DTP3 and MCV1 are part of the 23 indicators constituting UHC SCI 2019, we computed the correlation between UHC SCI 2019 and the overall vaccination coverage rate, DTP3 rate, and MCV1 rate, respectively, and observed no significant correlation or colinearity. We also repeated our analyses by excluding DTP3 and MCV1 from our independent variables and observed no significant impact in the coefficients of interest and their statistical significance (Figs E and F and Tables I~M in S1 Text ).

All analyses were conducted using R software (Version 4.0.3; S1 Text ). A Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist [ 29 ] is included in the Supplementary Materials (Table A in S1 Text ). Replication data and code are available at https://github.com/sk9076/UHC_GHS_2021 .

Our dataset included 16,205 observations spanning 192 countries from 2015 to 2021 that satisfied the parallel pre-trend assumption. A total of 4,630 observations (28.6%) took place during each of the pandemic years of 2020 and 2021, whereas 11,575 (71.4%) took place from 2015 to 2019. When we tested the different cutoff values to divide the countries into treatment and control groups, the lowest cutoff value with a significant effect in safeguarding a decline in immunization coverage rates during the pandemic was 75 for UHC and 60 for GHSI (Figs C and D and Tables B and C in S1 Text ). A complete list of the countries included in the analysis stratified by UHC and GHSI scores is presented in Table 1 .

PPT PowerPoint slide
PNG larger image
TIFF original image

https://doi.org/10.1371/journal.pgph.0003205.t001

Table 2 summarizes the descriptive statistics of the model variables stratified by the two treatment variables. Prior to the pandemic (2015–2019), the overall average immunization coverage rate was 87.1% (SD = 15.3); for countries with high UHC and GHSI scores, the mean coverage rate was 92.5% (SD = 8.86). This was similar to the countries with high UHC and low GHSI scores (92.6%; SD 9.36; t = -0.11249, p-value = 0.910) and countries with low UHC and high GHSI scores (95.7%; SD 6.12; t = -5.278, p-value<0.001), but higher than countries with low UHC and low GHSI scores (85.5%; SD 16.3; t = 1442.4, p-value<0.001).

https://doi.org/10.1371/journal.pgph.0003205.t002

The results of the DiD models are presented in Table 3 (Models 1 and 2) and Fig 1 . The full model results (Tables C and D in S1 Text ) are included in the Supplementary Materials. We found that countries with high GHS capacity (GHSI≥60) experienced a 1.100% (95% CI: 0.570%, 1.630%) reduced decline in vaccination coverage across 2021 and 2022 when controlled for the covariates. When disaggregated by year, the reduction in coverage decline was 0.939% during 2020 (95% CI: 0.474%, 1.404%) and 1.261% during 2021 (95% CI: 0.180%, 2.342%). Similarly, we found that countries’ progress towards UHC also safeguarded immunization service provision during the pandemic. Across 2020 and 2021, countries with high UHC (UHC SCI 2019≥75) prevented a 1.14% (95% CI: 0.39%, 1.90%) decline in vaccination coverage when controlled for the covariates; this effect was statistically significant in both years. The reduction in coverage decline was more pronounced in 2021 (DiD coefficient = 1.37%; 95% CI: 0.37%, 2.37%) compared to 2020 (DiD coefficient = 0.92%; 95% CI: 0.02%, 1.81%).

https://doi.org/10.1371/journal.pgph.0003205.g001

https://doi.org/10.1371/journal.pgph.0003205.t003

As summarized in Table 3 (Models 3~5) and Fig 2 (with full results available in Tables F~H in S1 Text ), the results overall suggest a synergy between UHC and GHS capacities. While countries with high GHS appeared to safeguard their vaccination coverage during the pandemic regardless of countries’ UHC capacities, countries with high UHC capacities did not show such protective effects during the pandemic when not supported by high GHS capacities. In summary, these findings empirically demonstrated the complementing and synergistic effects of GHS and UHC capacities, and the importance of GHS capacity for augmenting the protective effect of UHC capacity in safeguarding essential health service delivery during a public health emergency.

https://doi.org/10.1371/journal.pgph.0003205.g002

The correlation between UHC SCI 2019 and overall vaccination coverage across all vaccine types was 0.37, while the correlation between UHC SCI 2019 and DTP3 and MCV1 was 0.51 and 0.50, respectively. All of the findings were consistent when the analyses were repeated by excluding DTP3 and MCV1 coverage rates from the independent variables (Figs E and F and Tables I~M in S1 Text ).

Understanding the determinants of health system resilience against the COVID-19 pandemic is important to bolster our current and future resilience against public health emergencies. The two key global health agendas that have received particular attention from national and global policymakers are UHC and GHS. Building on our previous research [ 9 ], this study examined the individual and synergistic effects of UHC and GHS on childhood vaccination rates during the first two years of the COVID-19 pandemic. In line with previous research [ 7 – 9 ], we not only confirmed that progress towards UHC prevented a significant reduction in immunization coverage in 2020, but also showed that this preventive effect was even larger in 2021. Similar to UHC, we found that progress towards GHS also resulted in protective effects, both in 2020 and 2021. This is particularly interesting because previous studies have reported null or mixed associations between GHS and countries’ pandemic resilience [ 10 – 13 ], making it difficult to ascertain the role of GHS during public health emergencies. Our results also indicate that the protective effects of UHC and GHS became more pronounced as time elapsed—specifically, the protective effect sizes were larger in 2021 compared to 2020—which could be explained by UHC and GHS capacities enabling health system recoveries or their protective effects becoming more apparent after emergency measures and funding gradually were eased in 2021.

This study was one of the first studies to examine the potential protective effects associated with the synergy between UHC and GHS. When taken together, our results showed that progress towards UHC alone may not be sufficient to protect countries’ essential immunization service delivery against disruptions due to the COVID-19 pandemic. Contrariliy, progress towards GHS, even in the absence of sufficient UHC capacity, likely prevented declines in immunization coverage rates during the pandemic. Our results also suggest there was a synergistic protective effect resulting from progress towards both health agendas.

In addition to being one of the first studies to examine the synergistic effects of UHC and GHS on population health, this study is novel in that it quantifies the possible effects of countries’ progress towards UHC or GHS on their health system performance during the first two years of the COVID-19 pandemic. Our finding that the effects of both UHC and GHS on health system resilience became more pronounced in 2021 indicates these health agendas and indicators may become more impactful as time elapses, which could be significant for countries’ post-pandemic prioritization of recovery policies.

An important consideration in interpreting the results is the inherent limitations associated with country-level estimates. Although our data was strictly derived from commonly used and publicly available data sources, country-level estimates do not account for subnational variations, differences in data quality, or other biases inherent in such estimates [ 30 , 31 ]. There are, however, few alternative global health data sources that allow researchers to make cross-country comparisons, especially during the pandemic. Another limitation is the timeframe of our study. While we were able to extend our previous analysis of UHC’s protective effects on vaccination coverage [ 9 ] by including data from the second year of the pandemic, more data on post-pandemic years can facilitate an assessment of the long-term effects of UHC and GHS capacities in safeguarding essential health service delivery across countries.

As national and global policymakers set priorities, develop policies, and make investment decisions to combat the ongoing COVID-19 pandemic and bolster the resilience of health systems against future epidemics or pandemics, it is imperative to have empirical evidence on which health policies and agendas enhance health systems resilience. This study found that greater progress towards both UHC and GHS safeguarded essential health service delivery during the first two years of the pandemic, and progress towards GHS alone may be sufficient for yielding a protective effect. Our results emphasized the importance of sustaining strategic investments into both health agendas and call for future research into the possible synergistic effects of the two health agendas on health systems resilience.

Future analyses can be conducted by employing our methodology to assess how the effects of UHC and GHS capacities on health systems resilience change over time and across different regions across the globe. Such research with a longer time horizon has the potential to assist policymakers in identifying policies that are likely to safeguard population health during health emergencies, including pandemics. Finally, as also proposed by other studies [ 9 , 32 ], other health indicators, such as maternal and neonatal mortality rates, can be used as proxies for essential health service delivery to build a more robust body of evidence given the dearth of research in this area.

Supporting information

https://doi.org/10.1371/journal.pgph.0003205.s001

View Article
PubMed/NCBI
Google Scholar
15. 10 proposals to build a safer world together–Strengthening the Global Architecture for Health Emergency Preparedness, Response and Resilience. Geneva: World Health Organization; 2022.
25. Cameron E.JB; Bell JAGlobal Health Security Index: building collective action and accountability. USA: Johns Hopkins University. 2019.

Skip to main content
Skip to FDA Search
Skip to in this section menu
Skip to footer links

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

U.S. Food and Drug Administration

Search
Menu
Science & Research
Science and Research Special Topics
Advancing Regulatory Science

Principal stratification methods and software for intercurrent events in clinical trials

CERSI Collaborators: Triangle CERSI, Duke University: Fan Li, PhD; Laine Thomas, PhD; Anqi Zhao, PhD; Susan Halabi, PhD

FDA Collaborators: Yuan-Li Shen, Dr.P.H; Pallavi Mishra-Kalyani, PhD; Shu Wang, PhD.; Xiaoxue Li, PhD.; Joyce Cheng, PhD

CERSI Subcontractors: Flying Buttress Associates- Jeph Herrin, PhD

CERSI In-Kind Collaborators: OptumLabs - William Crown, PhD; University of San Francisco - Sanket Dhruva, MD

Non-Federal Entity Collaborators: Johnson and Johnson- Karla Childers, MSJ, Paul Coplan, ScD, MBA, Stephen Johnston, MSc

Project Start Date: September 8, 2023

Regulatory Science Challenge

Events that occur post randomization in randomized control trials, known as intercurrent events, can alter the course of the randomized clinical trials and jeopardize comparative effectiveness evaluation and consequently decision making in regulatory science. The standard approach of intention-to-treat analysis ignores intercurrent events and thus preserves the trial validity based on randomization, but it fails to capture treatment effect heterogeneity and the complex causal mechanism. The 2018 ICH E9(R1) addendum suggests principal stratification as an alternative approach to handle intercurrent events, but significant gaps exist between the theory and practice of principal stratification in regulatory science. In particular, there is a lack of transparent and accessible analytical methods, practical guidelines, and software of principal stratification in the context of regulatory science.

Project Description and Goals

This project aims to develop a suite of transparent and accessible analysis tools, software and educational material for applying the principal stratification method to analyze intercurrent events in clinical trials. Investigators will focus on two prevalent types of intercurrent events: (1) nonadherence to assigned treatment, including treatment switching and discontinuation and (2) truncation of the target outcome by a terminal event. For each type, investigators will develop estimand, computational, visualization, and sensitivity analysis tools, with a special emphasis on time-to-event outcomes. They will also develop a companion R package and tutorials with illustrations of clinical trials in oncology and other diseases. The results of this study will impact clinical trials in two ways: (1) produce new methodological tools for addressing a pressing and prevalent complication in clinical trials, (2) provide associated open-source software and educational material to disseminate the methodology to regulatory agencies, health researchers, and industry. Investigators also plan to develop scientific publications describing the outcomes of this research and discuss it at public forums.

Research Outcomes/Results

Two hundred and twenty-three patients with a mean age of 65 years completed the survey. These patients preferred a higher chance of good biopsy outcomes, and a lower chance of erectile dysfunction caused by the treatment and urinary incontinence after treatment. The patients stated in the survey that they are willing to accept:

a 15.1%-point increase in erectile dysfunction caused by the treatment to achieve a 10%-point increase in a good biopsy outcome after HIFU ablation, and
an 8.5%-point increase in urinary incontinence for a 10%-point increase in a good biopsy.

Also, further analysis revealed that patients who thought their cancer was more aggressive were more willing to tolerate urinary incontinence. Younger men were willing to tolerate less erectile dysfunction risk than older men. Respondents with a greater than college level of education were less willing to tolerate erectile dysfunction or urinary incontinence.

Research Impacts

Incorporating patient preference information into decisions that FDA makes about regulating devices is one of the major goals of FDA’s Center for Devices and Radiological Health (CDRH). Study findings show that patients prefer specific outcomes related to prostate ablation therapies like HIFU. The study results may help inform the design and regulation of current and future prostate tissue ablation devices by providing information about outcomes that patients most desire.

Publications

PMID: 34677594; Citation: Wallach JD, Deng Y, McCoy RG, Dhruva SS, Herrin J, Berkowitz A, Polley EC, Quinto K, Gandotra C, Crown W, Noseworthy P, Yao X, Shah ND, Ross JS, Lyon TD. Real-world Cardiovascular Outcomes Associated With Degarelix vs Leuprolide for Prostate Cancer Treatment.  JAMA Netw Open. 2021;4(10):e2130587. doi:10.1001/jamanetworkopen.2021.30587 .
PMID: 36191949; Citation: Deng Y, Polley EC, Wallach JD, Dhruva SS, Herrin J, Quinto K, Gandotra C, Crown W, Noseworthy P, Yao X, Lyon TD, Shah ND, Ross JS, McCoy RG. Emulating the GRADE trial using real world data: retrospective comparative effectiveness study. BMJ . 2022 Oct 3;379:e070717. doi: 10.1136/bmj-2022-070717 .

New Content From Advances in Methods and Practices in Psychological Science

Advances in Methods and Practices in Psychological Science
Cognitive Dissonance
Meta-Analysis
Methodology
Preregistration
Reproducibility

A Practical Guide to Conversation Research: How to Study What People Say to Each Other Michael Yeomans, F. Katelynn Boland, Hanne Collins, Nicole Abi-Esber, and Alison Wood Brooks

Conversation—a verbal interaction between two or more people—is a complex, pervasive, and consequential human behavior. Conversations have been studied across many academic disciplines. However, advances in recording and analysis techniques over the last decade have allowed researchers to more directly and precisely examine conversations in natural contexts and at a larger scale than ever before, and these advances open new paths to understand humanity and the social world. Existing reviews of text analysis and conversation research have focused on text generated by a single author (e.g., product reviews, news articles, and public speeches) and thus leave open questions about the unique challenges presented by interactive conversation data (i.e., dialogue). In this article, we suggest approaches to overcome common challenges in the workflow of conversation science, including recording and transcribing conversations, structuring data (to merge turn-level and speaker-level data sets), extracting and aggregating linguistic features, estimating effects, and sharing data. This practical guide is meant to shed light on current best practices and empower more researchers to study conversations more directly—to expand the community of conversation scholars and contribute to a greater cumulative scientific understanding of the social world.

Open-Science Guidance for Qualitative Research: An Empirically Validated Approach for De-Identifying Sensitive Narrative Data Rebecca Campbell, McKenzie Javorka, Jasmine Engleton, Kathryn Fishwick, Katie Gregory, and Rachael Goodman-Williams

The open-science movement seeks to make research more transparent and accessible. To that end, researchers are increasingly expected to share de-identified data with other scholars for review, reanalysis, and reuse. In psychology, open-science practices have been explored primarily within the context of quantitative data, but demands to share qualitative data are becoming more prevalent. Narrative data are far more challenging to de-identify fully, and because qualitative methods are often used in studies with marginalized, minoritized, and/or traumatized populations, data sharing may pose substantial risks for participants if their information can be later reidentified. To date, there has been little guidance in the literature on how to de-identify qualitative data. To address this gap, we developed a methodological framework for remediating sensitive narrative data. This multiphase process is modeled on common qualitative-coding strategies. The first phase includes consultations with diverse stakeholders and sources to understand reidentifiability risks and data-sharing concerns. The second phase outlines an iterative process for recognizing potentially identifiable information and constructing individualized remediation strategies through group review and consensus. The third phase includes multiple strategies for assessing the validity of the de-identification analyses (i.e., whether the remediated transcripts adequately protect participants’ privacy). We applied this framework to a set of 32 qualitative interviews with sexual-assault survivors. We provide case examples of how blurring and redaction techniques can be used to protect names, dates, locations, trauma histories, help-seeking experiences, and other information about dyadic interactions.

Impossible Hypotheses and Effect-Size Limits Wijnand van Tilburg and Lennert van Tilburg

Psychological science is moving toward further specification of effect sizes when formulating hypotheses, performing power analyses, and considering the relevance of findings. This development has sparked an appreciation for the wider context in which such effect sizes are found because the importance assigned to specific sizes may vary from situation to situation. We add to this development a crucial but in psychology hitherto underappreciated contingency: There are mathematical limits to the magnitudes that population effect sizes can take within the common multivariate context in which psychology is situated, and these limits can be far more restrictive than typically assumed. The implication is that some hypothesized or preregistered effect sizes may be impossible. At the same time, these restrictions offer a way of statistically triangulating the plausible range of unknown effect sizes. We explain the reason for the existence of these limits, illustrate how to identify them, and offer recommendations and tools for improving hypothesized effect sizes by exploiting the broader multivariate context in which they occur.

It’s All About Timing: Exploring Different Temporal Resolutions for Analyzing Digital-Phenotyping Data Anna Langener, Gert Stulp, Nicholas Jacobson, Andrea Costanzo, Raj Jagesar, Martien Kas, and Laura Bringmann

The use of smartphones and wearable sensors to passively collect data on behavior has great potential for better understanding psychological well-being and mental disorders with minimal burden. However, there are important methodological challenges that may hinder the widespread adoption of these passive measures. A crucial one is the issue of timescale: The chosen temporal resolution for summarizing and analyzing the data may affect how results are interpreted. Despite its importance, the choice of temporal resolution is rarely justified. In this study, we aim to improve current standards for analyzing digital-phenotyping data by addressing the time-related decisions faced by researchers. For illustrative purposes, we use data from 10 students whose behavior (e.g., GPS, app usage) was recorded for 28 days through the Behapp application on their mobile phones. In parallel, the participants actively answered questionnaires on their phones about their mood several times a day. We provide a walk-through on how to study different timescales by doing individualized correlation analyses and random-forest prediction models. By doing so, we demonstrate how choosing different resolutions can lead to different conclusions. Therefore, we propose conducting a multiverse analysis to investigate the consequences of choosing different temporal resolutions. This will improve current standards for analyzing digital-phenotyping data and may help combat the replications crisis caused in part by researchers making implicit decisions.

Calculating Repeated-Measures Meta-Analytic Effects for Continuous Outcomes: A Tutorial on Pretest–Posttest-Controlled Designs David R. Skvarc, Matthew Fuller-Tyszkiewicz

Meta-analysis is a statistical technique that combines the results of multiple studies to arrive at a more robust and reliable estimate of an overall effect or estimate of the true effect. Within the context of experimental study designs, standard meta-analyses generally use between-groups differences at a single time point. This approach fails to adequately account for preexisting differences that are likely to threaten causal inference. Meta-analyses that take into account the repeated-measures nature of these data are uncommon, and so this article serves as an instructive methodology for increasing the precision of meta-analyses by attempting to estimate the repeated-measures effect sizes, with particular focus on contexts with two time points and two groups (a between-groups pretest–posttest design)—a common scenario for clinical trials and experiments. In this article, we summarize the concept of a between-groups pretest–posttest meta-analysis and its applications. We then explain the basic steps involved in conducting this meta-analysis, including the extraction of data and several alternative approaches for the calculation of effect sizes. We also highlight the importance of considering the presence of within-subjects correlations when conducting this form of meta-analysis.

Reliability and Feasibility of Linear Mixed Models in Fully Crossed Experimental Designs Michele Scandola, Emmanuele Tidoni

The use of linear mixed models (LMMs) is increasing in psychology and neuroscience research In this article, we focus on the implementation of LMMs in fully crossed experimental designs. A key aspect of LMMs is choosing a random-effects structure according to the experimental needs. To date, opposite suggestions are present in the literature, spanning from keeping all random effects (maximal models), which produces several singularity and convergence issues, to removing random effects until the best fit is found, with the risk of inflating Type I error (reduced models). However, defining the random structure to fit a nonsingular and convergent model is not straightforward. Moreover, the lack of a standard approach may lead the researcher to make decisions that potentially inflate Type I errors. After reviewing LMMs, we introduce a step-by-step approach to avoid convergence and singularity issues and control for Type I error inflation during model reduction of fully crossed experimental designs. Specifically, we propose the use of complex random intercepts (CRIs) when maximal models are overparametrized. CRIs are multiple random intercepts that represent the residual variance of categorical fixed effects within a given grouping factor. We validated CRIs and the proposed procedure by extensive simulations and a real-case application. We demonstrate that CRIs can produce reliable results and require less computational resources. Moreover, we outline a few criteria and recommendations on how and when scholars should reduce overparametrized models. Overall, the proposed procedure provides clear solutions to avoid overinflated results using LMMs in psychology and neuroscience.

Understanding Meta-Analysis Through Data Simulation With Applications to Power Analysis Filippo Gambarota, Gianmarco Altoè

Meta-analysis is a powerful tool to combine evidence from existing literature. Despite several introductory and advanced materials about organizing, conducting, and reporting a meta-analysis, to our knowledge, there are no introductive materials about simulating the most common meta-analysis models. Data simulation is essential for developing and validating new statistical models and procedures. Furthermore, data simulation is a powerful educational tool for understanding a statistical method. In this tutorial, we show how to simulate equal-effects, random-effects, and metaregression models and illustrate how to estimate statistical power. Simulations for multilevel and multivariate models are available in the Supplemental Material available online. All materials associated with this article can be accessed on OSF ( https://osf.io/54djn/ ).

Feedback on this article? Email [email protected] or login to comment.

APS regularly opens certain online articles for discussion on our website. Effective February 2021, you must be a logged-in APS member to post comments. By posting a comment, you agree to our Community Guidelines and the display of your profile information, including your name and affiliation. Any opinions, findings, conclusions, or recommendations present in article comments are those of the writers and do not necessarily reflect the views of APS or the article’s author. For more information, please see our Community Guidelines .

Please login with your APS account to comment.

Privacy Overview

Search Menu
Advance Access
Collections
Author Guidelines
Submission Site
Open Access Policy
Self-Archiving Policy
Why Submit?
About Horticulture Research
About Nanjing Agricultural University
Editorial Board
Advertising & Corporate Services
Journals on Oxford Academic
Books on Oxford Academic

Article Contents

Introduction, conclusions, materials and methods, acknowledgements, author contributions, data availability, conflict of interest statement.

< Previous

Multi-omics analysis reveals key regulatory defense pathways and genes involved in salt tolerance of rose plants

These authors contributed equally to this work.

Article contents
Figures & tables
Supplementary Data

Haoran Ren, Wenjing Yang, Weikun Jing, Muhammad Owais Shahid, Yuming Liu, Xianhan Qiu, Patrick Choisy, Tao Xu, Nan Ma, Junping Gao, Xiaofeng Zhou, Multi-omics analysis reveals key regulatory defense pathways and genes involved in salt tolerance of rose plants, Horticulture Research , Volume 11, Issue 5, May 2024, uhae068, https://doi.org/10.1093/hr/uhae068

Permissions Icon Permissions

Salinity stress causes serious damage to crops worldwide, limiting plant production. However, the metabolic and molecular mechanisms underlying the response to salt stress in rose ( Rosa spp.) remain poorly studied. We therefore performed a multi-omics investigation of Rosa hybrida cv. Jardin de Granville (JDG) and Rosa damascena Mill. (DMS) under salt stress to determine the mechanisms underlying rose adaptability to salinity stress. Salt treatment of both JDG and DMS led to the buildup of reactive oxygen species (H 2 O 2 ). Palisade tissue was more severely damaged in DMS than in JDG, while the relative electrolyte permeability was lower and the soluble protein content was higher in JDG than in DMS. Metabolome profiling revealed significant alterations in phenolic acid, lipids, and flavonoid metabolite levels in JDG and DMS under salt stress. Proteome analysis identified enrichment of flavone and flavonol pathways in JDG under salt stress. RNA sequencing showed that salt stress influenced primary metabolism in DMS, whereas it substantially affected secondary metabolism in JDG. Integrating these datasets revealed that the phenylpropane pathway, especially the flavonoid pathway, is strongly enhanced in rose under salt stress. Consistent with this, weighted gene coexpression network analysis (WGCNA) identified the key regulatory gene chalcone synthase 1 ( CHS1 ), which is important in the phenylpropane pathway. Moreover, luciferase assays indicated that the bHLH74 transcription factor binds to the CHS1 promoter to block its transcription. These results clarify the role of the phenylpropane pathway, especially flavonoid and flavonol metabolism, in the response to salt stress in rose.

Rose ( Rosa spp.) is a popular ornamental crop that is also used in the cosmetics, perfume and medicine. Rose plants contains various bioactive substances, including flavonoids, fragrant components, and hydrolysable and condensed tannins, which have high value and market potential [ 1 ]. However, soil salinization is common in many rose-growing regions, and high salt concentrations in soil can severely inhibit rose plant growth, reduce flower quality, and cause significant economic losses [ 2 ]. Additionally, salt stress can enhance the secondary metabolites of roses such as citronellol, geraniol, and phenyl ethyl alcohol [ 3 , 4 ]. Such alterations in secondary metabolites may help to regulate the salt tolerance of rose. Research on roses has focused mainly on flower quality, petal development, and flower bloom [ 5–7 ], and there are limited data available regarding signaling pathways linking plant development and secondary metabolites associated with salt stress.

In plants, salt stress induces osmotic imbalances, which lead to the closure of leaf stomata, limit photosynthesis, and affect plant growth and metabolism [ 8 ]. To alleviate osmotic stress and protect themselves from its adverse effects, plants accumulate numerous compatible solutes (such as soluble proteins, soluble sugars, and proline), known collectively as osmoprotectants [ 9 ]. Moreover, plants generate reactive oxygen species (ROS) to cope with salt stress [ 10 ]. Nevertheless, excessive ROS accumulation can lead to oxidative DNA damage, affect protein biosynthesis, and ultimately result in cell damage and death [ 11 , 12 ]. Plant cells utilize both enzymatic and nonenzymatic antioxidant mechanisms to diminish ROS levels and prevent oxidative damage. Superoxide dismutase (SOD), peroxidase (POD), ascorbate peroxidase (APX), catalase (CAT), and glutathione peroxidase (GPX) are antioxidant enzymes that work as O 2− and H 2 O 2 scavengers [ 13 , 14 ]. Nonenzymatic antioxidants, such as ascorbate, glutathione, phenols, and flavonoids, also play vital roles in ROS scavenging [ 15 , 16 ].

Flavonoids are naturally occurring bioactive substances found in fruits, vegetables, tea, and medicinal plants [ 17 ]. Flavonoids comprise more than 9000 compounds and constitute a substantial category of plant secondary metabolites [ 18 ]. They have diverse biological functions in the growth and development of plants, including improving pollen fertility, imparting color, and influencing seed dormancy and germination [ 19 , 20 ]. In addition, flavonoids have protective roles against biotic and abiotic stresses, such as pathogen infections, ultraviolet (UV)-B, cold, drought, and salinity [ 21–23 ]. Flavonoids have also received widespread attention due to their possible benefits for human health [ 24 ].

The molecular mechanism of flavonoid biosynthesis has been elucidated in many plants [ 25 ]. Chalcone synthase (CHS) mediates the first step in flavonoid production, catalyzing the formation of naringenin chalcone from three molecules of malonyl CoA and one molecule of 4-coumaroyl CoA. Chalcone isomerase (CHI) then quickly converts naringenin chalcone into naringenin (flavanone), which is further biosynthesized into different flavonoids by the subsequent enzymes in this pathway [ 26 ]. Although the biosynthesis of flavonoids has attracted increasing attention from scholars, current research does not fully explain the effects of regulatory factors on the transcription and activity of the major enzymes in flavonoid metabolism. Therefore, further research on the signaling molecules and regulatory pathways associated with flavonoids, as well as their regulatory mechanisms, is needed to elucidate the physiological activity of flavonoids.

Rosa hybrida cv. Jardin de Granville (JDG) is a new hybrid rose developed by 'Les Roses Anciennes André Eve' for the Prestige range of Christian Dior skin care products. JDG possesses twice the vitality of a traditional rose and grows and blooms vigorously in the salty air and harsh winds of coastal climates. JDG is also rich in beneficial bioactive substances that are mainly used in cosmetics and anti-aging skin care creams [ 27 , 28 ]. Rosa damascena Mill. (DMS) is one of the most common fragrant roses in the Rosaceae family. Its essential oils and aromatic compounds are used extensively in the cosmetic and food industries worldwide [ 29 ]. DMS is considered an excellent rose throughout the world due to its high resistance to abiotic stress and abundance of beneficial secondary metabolites [ 30 ].

Here, we conducted an integrated analysis on the transcriptomes, proteomes, and metabolomes of JDG and DMS to explore the relationship between plant development and secondary metabolites of rose under salt stress. We used WGCNA and Cytoscape software to decipher the similarities and differences in the complex metabolic pathways and regulatory genes of JDG and DMS under salt stress. These results provide comprehensive information on the metabolic and molecular mechanisms of the response to salt stress in rose, promoting the cultivation of excellent new rose varieties that are both salt tolerant and rich in beneficial secondary metabolites.

JDG is more tolerant than DMS to salt stress

To explore the salt tolerance of rose, plants of JDG and DMS were treated with 400 mM NaCl for 2 weeks. DMS plants showed typical damage with yellowing and death of leaves, while JDG leaves only exhibited slight wilting ( Fig. 1A ). Additionally, detached rose leaves were treated with salt for 4 days; DMS leaves showed significantly more necrosis than JDG leaves ( Fig. 1B ). In order to quickly observe the response of rose cultivars to salt stress and convenience sampling, subsequent experiments mainly used detached rose leaves. To examine the overall anatomy and morphology of leaves treated for 2 days with NaCl, we stained treated and control leaves with toluidine blue and prepared thin sections. Palisade tissue damage in response to salt treatment was more severe in DMS than in JDG (indicated by red arrowheads in Fig. 1C ). To investigate ROS accumulation in response to salt stress, we performed 3, 3'-diaminobenzidine (DAB) staining. DMS leaves accumulated substantially more ROS (deeper staining) than JDG plants after salt stress, whereas there was no difference in ROS content between these two cultivars under normal conditions ( Fig. 1D, E ). Soluble protein content was higher in JDG leaves after 4 days of salt stress than after 2 days of salt stress, while the soluble protein content of DMS leaves was much higher than that of before treatment leaves after 2 days and decreased by 4 days of salt treatment ( Fig. 1F ). The relative electrolyte permeability of JDG leaves was increased slightly after 2 days of salt treatment and more substantially after 4 days of treatment, while relative electrolyte permeability was much higher in DMS than in JDG on both days after salt treatment ( Fig. 1G ). Phenotypic and physiological analyses indicated that JDG is more salt tolerant than DMS.

Phenotypes of JDG and DMS under salt stress. (A) Phenotypes of JDG and DMS plants after 2 weeks of treatment with 400 mM NaCl. Left, phenotype of the whole plant; right, enlarged image of the protruding part indicated by the red circle. Bars, 3 cm. (B) Detached leaves of rose on different days after onset of salt stress (400 mM NaCl). (C) Anatomical analysis of leaves in (B). Red arrowheads represent the palisade tissue. Mock (0 mM NaCl); NaCl (400 mM NaCl). Bars, 50 μm. (D) Tissue staining of rose leaves under salt stress using DAB. (E) Quantitative statistics of the relative staining intensity in (D). Brown staining area and total leaf area were measured using ImageJ software, their ratio is the relative staining intensity. (F) Soluble protein content of rose leaves at different days under salt treatment. (G) Relative electrolyte permeability of rose leaves at different days under salt treatment. Data are based on the mean ± SE of at least three repeated biological experiments.

Flavonoid metabolites play an important role in the salinity tolerance of rose

To better understand how salt stress affects rose metabolites, we performed a comprehensive untargeted analysis of metabolites using ultra-performance liquid chromatography/mass spectrometry (UPLC/MS). Fig. S1A shows the different metabolites detected, and Fig. S1B shows the curves of the quality control samples, indicating that the mass spectral data were highly reproducible and reliable. Principal component analysis (PCA) was used to reduce the data dimensions and clarify the relationships among the samples. The two principal components PC1, and PC2 could explain 50.07% and 23.36% of the variance, respectively. Moreover, PC1 revealed variance in genotypes, while PC2 revealed differences in time of exposure to salt stress. Thus, the metabolite-based PCA revealed obvious differences in salt tolerance between the two cultivars ( Fig. S2A ).

Our screening for differentially accumulated metabolites (DAMs) identified hundreds of metabolites with significantly altered accumulation under salt stress ( Fig. 2A , Table S1 ). Preliminary analysis indicated that DAMs included amino acids and their derivatives, nucleotides and their derivatives, phenolic acids, flavonoids, lipids, tannins, lignans and coumarins, organic acids, alkaloids, and terpenoids, and most of the DAMs were upregulated under salt stress ( Fig. 2B ). Phenolic acids, lipids, and flavonoid metabolites showed significantly altered accumulation under salt stress in both JDG and DMS. Compared with their levels in DMS, flavonoid metabolites, phenolic acid metabolites, and lipids were differentially accumulated in JDG leaves under both control conditions and salt stress ( Table S1 ). These results indicate that flavonoid metabolites, phenolic acid metabolites, and lipids may play important roles in the salt tolerance of rose.

Metabolomic analysis of JDG and DMS under salt stress. (A) Number of DAMs in different comparison groups. (B) Classification of DAMs in each comparison. (C) Classification of DAMs upregulated in both JDG and DMS under salt treatment. (D) Classification of DAMs upregulated in JDG compared with DMS under both control and salt treatments. (E, F) KEGG pathway enrichment of DAMs under salt stress: (E) JDG-NaCl vs JDG-Mock and (F) DMS-NaCl vs DMS-Mock.

To determine how metabolites differ between JDG and DMS, we summarized the differences in metabolite accumulation in the different comparison groups using Venn diagrams. Groups JDG-NaCl vs JDG-Mock and DMS-NaCl vs DMS-Mock shared 109 of the same metabolite changes, of which 79 were increases and 15 were decreases. Among the upregulated metabolites, phenolic acids and flavonoids accounted for 21.52% and 7.59%, respectively. These metabolites included ferulic acid, coniferaldehyde, pinocembrin (dihydrochrysin), naringin, eucalyptin (5-hydroxy-7,4'-dimethoxy-6,8-dimethylflavone), patuletin (quercetagetin-6-methyl ether), naringenin-7- O -rutinoside-4'- O -glucoside, naringin (naringenin-7- O -neohesperidoside), and sudachitin ( Fig. 2C , Fig. S2B–D , Table S1 ). Notably, 5,7,8,4'-tetramethoxyflavone, vanillic acid-4- O -glucoside, and 3',4',5',5,7-pentamethoxyflavone were upregulated in JDG and downregulated in DMS under salt stress, while kaempferol-3- O -arabinoside-7- O -rhamnoside was upregulated in DMS and downregulated in JDG. Groups JDG-Mock vs DMS-Mock and JDG-NaCl vs DMS-NaCl shared 408 metabolites showing the same tendency in alteration, of which accumulation of 188 was increased and 202 was decreased. Among the upregulated metabolites, phenolic acids and flavonoids accounted for 29.26% and 33.51%, respectively ( Fig. 2D ). Notably, the genkwanin (apigenin 7-methyl ether) content was 12.74-fold higher, the 5,7-dihydroxy-6,3′,4′,5′-tetramethoxyflavone (arteanoflavone) content was 15.64-fold higher, the naringenin-4′,7-dimethyl ether content was 13-fold higher, and the naringin dihydrochalcone content was 13.30-fold in JDG compared with DMS under control conditions; all of these are flavonoid metabolites. Venn analysis also showed that many metabolites displaying changes under salt stress were genotype specific, indicating that the cultivars have different mechanisms of response to salinity. There were 77 metabolites that specifically accumulated in JDG under salt stress, which may represent the major metabolites in the salt stress response of JDG. Notably, four metabolites—ethylsalicylate (a phenolic acid), salidroside (a phenolic acid), L-ornithine (amino acids and derivatives), and epiafzelechin (a flavonoid)—accumulated specifically in JDG after salt treatment and were also highly accumulated under control conditions in JDG compared with DMS ( Fig. S2B–D , Table S1 ).

All DAMs were analyzed using Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment ( Fig. 2E, F , Fig. S3A, B ). In JDG (JDG-NaCl vs JDG-Mock group), salt stress induced changes in metabolites mainly involved 'purine metabolism,' 'phenylpropanoid biosynthesis,' 'linoleic acid metabolism,' and 'alpha-linolenic acid metabolism' ( Fig. 2E ). In DMS (DMS-NaCl vs DMS-Mock group), the DAMs in leaves under salt stress were mainly associated with 'phenylpropanoid biosynthesis,' 'alpha-linolenic acid metabolism,' 'linoleic acid metabolism,' and 'pentose and glucuronate interconversions' ( Fig. 2F ). In the JDG-Mock vs DMS-Mock group, DAMs between leaves of DMS and JDG were mostly associated with 'flavonoid biosynthesis,' 'flavone and flavonol biosynthesis,' and 'phenylpropanoid biosynthesis' ( Fig. S3A ). Meanwhile, in the JDG-NaCl vs DMS-NaCl group, DAMs were largely involved in 'flavonoid biosynthesis,' 'flavone and flavonol biosynthesis,' and 'linoleic acid metabolism' ( Fig. S3B ). KEGG enrichment analysis showed that 'linolenic acid/α-linolenic acid metabolism' and 'phenylpropanoid biosynthesis' were significantly enriched under salt stress in both cultivars, indicating that these two pathways play important roles under salt stress in rose. Regardless of the presence of salt stress, DAMs between DMS and JDG were concentrated in the flavone, flavonoid, and flavonol biosynthetic pathways, indicating that differential accumulation of these metabolites may be the main reason for different salt sensitivities among rose cultivars. Notably, 'caffeine metabolism' was enriched in JDG, while 'starch and sucrose metabolism' was significantly increased in DMS.

Salt stress causes dynamic changes in distinct sets of proteins

To delve deeper into the molecular mechanisms of the salt stress response in rose plants, we performed a proteome profiling analysis under the same salt treatment and control conditions as the metabolome analysis and characterized proteins on the basis of fold changes in their accumulation level. We identified 119 (87 upregulated and 32 downregulated) and 163 (83 downregulated and 80 upregulated) proteins with significantly differential accumulation under salt stress in JDG and DMS, respectively ( Fig. 3A, B ). Only 18 differentially accumulated proteins (DAPs) overlapped between the two cultivars, of which 13 were upregulated and 4 were downregulated in both JDG and DMS, while one DUF1279 domain–containing protein was upregulated in JDG and downregulated in DMS. Moreover, 101 DAPs were unique to JDG, whereas 145 DAPs were unique to DMS ( Table S2 ).

Proteomic analysis of rose under salt stress. (A) Number of DAPs in JDG and DMS. (B) Venn diagram of the DAPs in JDG and DMS. (C) Localizations of DAPs identified in JDG. (D) Functional categorization of DAPs unique to JDG. (E, F) KEGG enrichment analysis of DAPs in JDG (upregulated, E) and DMS (upregulated, F).

We predicted that most of the DAPs are located in chloroplasts in rose, according to the WoLFPSORT database ( Fig. 3C , Fig. S4A ). Gene Ontology (GO) and KEGG analyses were performed to analyze and annotate protein functions. The 20 most highly enriched GO terms associated with the DAPs are depicted in a circle diagram ( Fig. S5A, B , Table S2 ). Among them, GO:0046658 (anchored component of plasma membrane), GO:0051554 (flavonol metabolic process), GO:0047893 (flavonol 3- O -glucosyltransferase activity), and GO:0051555 (flavonol biosynthetic process) were highly enriched in JDG under salt stress. In DMS, GO:0006720 (isoprenoid catabolic process), GO:0005764 (lysosome), and GO:0004602 (glutathione peroxidase activity) were the most enriched among all GO terms. In addition, the GO data indicated that the DAPs specific to JDG were highly involved in the 'icosanoid metabolic process,' 'diterpenoid metabolic process,' and 'diterpenoid biosynthetic process' ( Fig. 3D ), whereas the DAPs specific to DMS were enriched in 'cellular hyperosmotic salinity response,' 'monocarboxylic acid catabolic process,' 'terpenoid catabolic process,' 'sesquiterpenoid catabolic process,' and 'apocarotenoid catabolic process' functions ( Fig. S4B ). DAPs shared by JDG and DMS included Q2VA35 (xyloglucan endotransglucosylase/hydrolase) and A0A2P6P708 (glutathione peroxidase), which are present only in extracellular regions ( Table S2 ). The DAPs in different comparison groups were classified and then clustered according to enrichment of their associated GO terms ( Fig. S4C ). We determined that salinity mainly influences flavone and flavonol metabolism pathways in JDG. Flavones and flavonols are antioxidants and bioactive reagents [ 24 ]. In DMS, salt mainly influences the osmotic response, water stimulus response, and salt stress response pathways, most of which are stress related [ 31 ]. We used KEGG enrichment to determine the metabolic pathways associated with the DAPs in JDG and DMS under salt stress ( Fig. 3E, F ). Many DAPs in JDG were associated with phenylpropanoid biosynthesis and alpha-linolenic acid metabolism, with examples including lipoxygenase (A0A2P6S713), 12-oxophytodienoate reductase (A0A2P6PFD8), peroxidase (A0A2P6R8H8), and flavone 3′- O -methyltransferase (A0A2P6RK21). The DAPs upregulated in DMS under salt stress were frequently associated with alpha-linolenic acid metabolism and glutathione metabolism, whereas the DAPs that were downregulated were associated with ribosomes ( Table S2 ). Notably, alpha-linolenic acid metabolism was significantly upregulated in both JDG and DMS under salt stress. Collectively, the GO and KEGG enrichment results show that salt stress causes dynamic changes in distinct sets of proteins in rose.

Salt stress differentially alters the transcriptomes of JDG and DMS

To identify the genes involved in salt stress and explore the molecular mechanisms of salt tolerance in DMS and JDG, we sequenced the transcriptomes of JDG and DMS leaves by RNA sequencing (RNA-seq). We obtained high-quality reads for transcriptome analysis ( Table S3 ). PCA showed a distinct difference between the two cultivars along PC1, and PC2 separated the treatment from the control. The three biological replicates in the ordination space were mostly clustered together, suggesting an acceptable correlation between replicates ( Fig. 4A ).

Transcriptomic analysis of JDG and DMS under salt stress. (A) PCA score plot of transcriptomic profiles from different cultivars. (B) Number of DEGs in JDG and DMS. (C–E) Venn diagrams of DEGs in JDG and DMS: (C) total DEGs, (D) upregulated DEGs, and (E) downregulated DEGs. (F, G) KEGG enrichment analysis of DEGs in JDG (F) and DMS (G).

Correlation analysis of transcriptome, proteome, and metabolomics data. (A, B) KEGG enrichment analysis of combined transcriptome, proteome, and metabolome data: (A) JDG-NaCl vs JDG-Mock, and (B) DMS-NaCl vs DMS-Mock. The x-axis shows the enrichment factor of the pathway in different omics, and the y-axis shows the name of the KEGG pathway; the color from red to green represents the significance of enrichment from high to low (indicated by the P value). The size of bubbles indicates the number of DEGs, DAPs, or DAMs; the larger the number, the larger the symbol. The shape of bubbles illustrates the various omics: circles represent genes omics, triangles represent metabolites omics, and squares represent proteins omics. (C) Co-expression network of major genes, proteins, and metabolites in the phenylpropanoid pathway. Different colors indicate the value of log 2 Fold Change (NaCl/Mock), with red for upregulated and blue for downregulated genes, proteins, or metabolites.

We analyzed differentially expressed genes (DEGs) in JDG and DMS under control and salt stress conditions. We detected 10,662 DEGs in DMS under salt stress, of which 4651 were upregulated and 6011 were downregulated. However, only 1990 genes were differentially expressed in JDG: 1102 upregulated and 888 downregulated ( Fig. 4B ). The smaller number of DEGs in JDG than in DMS under salt stress implies that JDG is less affected by salt stress. We used a Venn diagram to display the differences between various genes in DMS and JDG under salt stress. Group DMS-NaCl vs DMS-Mock and group JDG-NaCl vs JDG-Mock shared 1120 DEGs under salt stress, with 577 upregulated genes and 433 downregulated genes ( Fig. 4C–E ).

Next, we performed GO analysis of DEGs in the categories cellular component (CC), biological process (BP), and molecular function (MF). The top 21 most enriched GO terms associated with DEGs of JDG-NaCl vs JDG-Mock and DMS-NaCl vs DMS-Mock are presented in circle diagrams ( Fig. S6 , Table S4 ). Seven GO terms associated with the JDG-NaCl vs JDG-Mock group were highly involved in the BP category, among which GO:0016052 (carbohydrate catabolic process), GO:0009813 (flavonoid biosynthetic process), and GO:0009812 (flavonoid metabolic process) contained the most DEGs (43, 26, and 27, respectively), and most of these enriched genes were upregulated. Thirteen GO terms were highly involved in the MF category, among which GO:0010427 (abscisic acid binding), GO:0016832 (aldehyde-lyase activity), and GO:0019840 (isoprenoid binding) were highly significant. One GO term was highly involved in the CC category: GO:0031226 (intrinsic component of plasma membrane). Moreover, 19 GO terms associated with the DMS-NaCl vs DMS-Mock group were enriched in the BP category, among which GO:0036294 (cellular response to decreased oxygen levels), GO:0048511 (rhythmic process), and GO:0048585 (negative regulation of response to stimulus) contained the most DEGs (85, 95, and 146, respectively), and most of these enriched genes were downregulated. One GO term was enriched in the MF category: GO:0016854 (racemase and epimerase activity). Similarly, one GO term was enriched in the CC category: GO:0009501 (amyloplast). KEGG pathway enrichment analysis for JDG-NaCl vs JDG-Mock revealed that the DEGs were mainly involved in metabolic pathways, plant hormone signal transduction, biosynthesis of secondary metabolites, and glycolysis/gluconeogenesis ( Fig. 4F , Table S4 ). In the DMS-NaCl vs DMS-Mock group, the DEGs were chiefly enriched in metabolic pathways, plant hormone signal transduction, the MAPK signaling pathway, biosynthesis of cofactors, and ubiquitin-mediated proteolysis ( Fig. 4G , Table S4 ). These findings indicate that the biosynthesis of secondary metabolites is substantially enhanced under salt stress in JDG, but not in DMS. However, the biosynthesis of cofactors associated with primary metabolism is enhanced under salt stress in DMS. Therefore, we speculate that salinity results in large changes in primary metabolism in DMS, while it influences secondary metabolism in JDG.

Transcription factors (TFs) are essential for regulating the expression of stress response genes. Among the DEGs, we identified 114 TFs in JDG and 491 TFs in DMS, covering 39 TF families ( Table S4 ). The most abundant genes belonged to the AP2/ERF-ERF, MYB, NAC, bHLH, and C2C2 families ( Fig. S7A, B ). Moreover, 64 TFs were differentially expressed in both cultivars in response to salinity. We speculate that these TFs form a highly complex transcriptional regulatory network and could perform critical functions in the mechanism of salt tolerance in rose.

Expression of phenylpropanoid-related genes is correlated with proteins and metabolites affected by salt stress

Integrated analysis of multi-omics data provides a powerful tool for identifying significantly different pathways and crucial metabolites in biological processes. Here, we integrated our transcriptome, proteome, and metabolome data to determine the performance of the two rose cultivars under salt stress. Pathways associated with alpha-linolenic acid metabolism, phenylpropanoid biosynthesis, and starch and sucrose metabolism were significantly enriched in JDG under salt stress ( Fig. 5A ), while the pathways enriched in DMS were involved in starch and sucrose metabolism, cyanoamino acid metabolism, and phenylpropanoid biosynthesis ( Fig. 5B ). Starch and sucrose metabolism represent primary metabolic functions common to different cultivars [ 32 ], while alpha-linolenic acid metabolism is related to the biosynthesis of jasmonic acid, which is a phytohormone involved in fungal invasion and senescence [ 7 ]. The phenylpropanoid biosynthesis pathway comprises multiple secondary metabolites, which confer a range of colors, flavors, nutritional components, and bioactivities in plants. Flavonoids are an important type of phenylpropanoid that play key roles in resistance against biotic and abiotic stresses [ 24 ]. Thus, we focused on the phenylpropanoid pathway.

Gene–protein–metabolite correlation networks can be used to elucidate functional relationships and identify regulatory factors. Therefore, we analyzed the regulatory networks of the DEGs, DAPs, and DAMs related to phenylpropanoid metabolism. We identified 14 DEGs that were strongly correlated with one DAP and six DAMs in JDG under salt stress. Similarly, 25 DEGs were strongly correlated with one DAP and eight DAMs in DMS under salt stress ( Table S5 ). For example, in JDG, there was a strong correlation between the expression of one gene (RchiOBHmChr4g0430951) and the abundance of one protein (A0A2P6PM56) and two metabolites [coniferyl alcohol (mws0093) and sinapyl alcohol (mws0853)]. Epiafzelechin (mws1422) was also significantly associated with the expression of the gene RchiOBHmChr2g0092641. In DMS, there was a close association between the expression of three genes (RchiOBHmChr2g0092671, RchiOBHmChr3g0480401, and RchiOBHmChr5g0041231) and the abundance of one protein (A0A2P6QM41) and one metabolite [L-tyrosine (mws0250)]. The strong association of particular genes with phenylpropanoid proteins or metabolites suggests that these genes play a major role in phenylpropanoid biosynthesis under salt stress.

We selected 20 important genes in the biosynthetic pathway of phenylpropanoid and compared their expression between rose cultivars ( Table S6 ). The transcript levels of many genes ( 4CL1 , CCR1 , HCT1 , HCT2 , HCT3 , HCT4 , CHS1 , CHS2 , CHI , DFR , F3H , and ANR ) were higher in JDG than in DMS, which may be valuable for salt tolerance by stimulating JDG to produce more flavonoids. Our multi-omics analysis revealed that ferulic acid, sinapic acid, and coniferaldehyde accumulated to high levels in JDG under salt stress ( Fig. 5C , Table S1 ). We also compared the flavonoid compounds in the two cultivars. Quercetin-3,3′-dimethyl ether, 5,7-dihydroxy-6,3′,4′,5′-tetramethoxyflavone (arteanoflavone), naringenin-4′,7-dimethyl ether, naringin dihydrochalcone, genkwanin (apigenin 7-methyl ether), and mearnsetin accumulated to greater levels in JDG than in DMS under control conditions. Correspondingly, the flavonoids brickellin, 3- O -methylquercetin, 5,2′,5′-trihydroxy-3,7,4′-trimethoxyflavone-2′- O -glucoside, and kaempferol-3- O -(6′′-acetyl)glucosyl-(1→3)-galactoside were more abundant in JDG than in DMS under salt stress. By contrast, naringenin-4′,7-dimethyl ether, aromadendrin (dihydrokaempferol), pinocembrin-7- O -(6′′- O -malonyl)glucoside, Quercetin-3- O -(2”- O -glucosyl)glucuronide, were specifically accumulated in DMS. Moreover, 3′,4′,5′,5,7-pentamethoxyflavone, 3,5,7,3′4′-pentamethoxyflavone, and 5,7,8,4′-tetramethoxyflavone were abundant in JDG under salt stress but were decreased in DMS ( Table S7 ). Overall, the integration of the three omics datasets indicated that the phenylpropane pathway, especially the flavonoid pathway, is strongly enhanced under salinity conditions and that this contributes to salt tolerance in roses, especially in the JDG genotype.

Networks of co-expressed genes associated with phenylpropanoid biosynthesis are involved in the salt stress response

To identify candidate genes associated with phenylpropanoid biosynthesis, we constructed co-expression gene network modules via weighted gene correlation network analysis (WGCNA). We constructed a cluster tree based on correlation between expression levels (indicated by fragments per kilobase of script per million fragments mapped, FPKM), which partitioned the genes into 11 different gene modules ( Fig. 6A, B ). To identify candidate genes that play significant roles within the gene networks, we extracted annotation information for all these genes from the Rosa chinensis 'Old Blush' reference genome annotation database. We selected 16 genes contributing to phenylpropanoid biosynthesis and four genes associated with flavonoid biosynthesis. Table S8 lists the annotated genes participating in flavonoid-related pathways in JDG. Among the 11 modules, the green module contained 10 of these genes: CHS1 , CHS2 , CCR1 , HCT3 , HCT4 , CCoAOMT , F3H , DFR , ANR , and CHI . The turquoise module contained three genes: CCR2 , HCT1 , and CAD2 . The blue module contained three genes: PRDX1 , 4CL1 , and ANS . The red, yellow, brown, and black modules each contained one gene: CAD1 , PRDX2 , HCT2 , and 4CL2 , respectively ( Table S8 ). After combining certain genes in modules and comparing them with the DEGs, we checked and confirmed these results using reverse-transcription quantitative PCR (RT-qPCR). The expression trends of eight DEGs from phenylpropanoid and flavonoid biosynthesis pathways matched the results of RNA-seq ( Fig. S8 ).

Co-expression network related to flavonoid biosynthesis. (A) Clustering tree based on the correlation between gene expression levels. (B) Module–sample relationships. Each row represents a gene module, with the same color in as (A); each column represents a sample; the boxes within the chart contain corresponding correlations and P values. (C–E) Networks built from correlations among structural genes and TFs. Circles represent genes, and the size of the circle represents the number of relationships between genes in the network and surrounding genes. Lines represent regulatory relationships between genes, and different colored lines represent different connection strengths: red, strong connections; green, weak connections. (F) Heat map depicting the expression profiles of 15 TF genes. The scale bar denotes the Fold change/(mean expression levels across the three treatment groups). The color indicates relative levels of gene expression, horizontal rows represent the different treatments in JDG, and vertical columns show the TFs. (G) Representative images of transient expression of bHLH74 and LUC driven by the CHS1 promoter in Nicotiana benthamiana leaves. The color scale represents the signal level. High represents a strong signal, and low represents a weak signal. (H) Relative value of LUC/REN. Data are based on the mean ± SE of at least three repeated biological experiments. Significance determined using Student’s t -test ( ** P < 0.01).

To determine the regulatory genes involved in phenylpropanoid biosynthesis in JDG, we constructed three subnetworks from the different modules using the 20 phenylpropanoid biosynthesis–related DEGs as the nodes ( Table S9 ). In the regulatory networks of phenylpropanoid biosynthesis, we identified 15 TF genes from seven TF families: AP2/ERF-ERF (5 unigenes), bHLH (3 unigenes), MYB (3 unigenes), Alfin-like (1 unigene), SBP (1 unigene), C2C2-GATA (1 unigene), and TCP (1 unigene). bHLH62 and bHLH74 were strongly associated with CHS1 , CHS2 , CHI , CCR1 , and F3H ; ERF81 was strongly associated with 4CL1 ; and ERF110 and MYB-related were strongly associated with 4CL2 ( Fig. 6C–E ), indicating that CHS and 4CL are the major target genes in phenylpropanoid biosynthesis. Therefore, we speculated that the abundance of flavonoids is increased by enhancing the expression of upstream flavonoid biosynthesis genes. Fig. 6F shows a heat map of expression of the 15 TF genes after NaCl treatment. The green module contained a substantial number of phenylpropanoid biosynthesis genes, among which CHS1 was closely related to the TFs bHLH74 and bHLH62. Therefore, dual-luciferase reporter assays were conducted to determine their regulatory relationship ( Fig. 6G, H ). We used bHLH74 and bHLH62 driven by the CaMV35S promoter as effectors in a transient expression system, with the CHS1 promoter fused with LUC as a reporter. When we cotransformed Nicotiana benthamiana leaves with the effectors and the reporter, the LUC/REN ratio of CHS1 was 0.3/1, which was drastically lower than those of the controls ( Fig. 6G, H , Fig. S9A, B ). These results indicate that bHLH74, but not bHLH62, inhibits the expression of CHS1 .

Salt stress damages the structure and osmotic potential of rose leaves

Roses belong to the Rosaceae family and are one of the most important commercial flower crops. Extracts from various parts of the rose plant have also been shown to have excellent biological activity and are used in industries such as cosmetics, perfume and medicine [ 1 ]. Meanwhile, an increasing number of wild rose varieties with significant health benefits are being domesticated and brought into mainstream cultivation [ 33 ]. Salt stress is one of the most widespread abiotic constraints for rose cultivation. Salt stress threatens plant survival and growth but can stimulate an increase in the biosynthesis of secondary metabolites [ 34 ]. Previous studies have shown that optimal coordination between leaf structure and photosynthetic processes is essential for enabling plants to tolerate salt stress [ 35 ]. When exposed to salt treatment, leaves become thicker and smaller while the palisade tissue and spongy tissue become loose and jumbled and the intercellular space of the mesophyll becomes thinner [ 36–39 ]. We observed that the palisade tissue of DMS was loose, disordered, and severely damaged compared with that in JDG under salt stress ( Fig. 1C ). This indicates that DMS is more sensitive to salt stress than JDG. Typically, excessive ROS accumulate under stress conditions, which can lead to membrane oxidative damage (lipid peroxidation) [ 40 ]. Silencing of the gene GmNAC06 in soybean ( Glycine max ) leads to accumulation of ROS under salt stress, which in turn leads to significant losses in soybean production [ 41 ]. In Arabidopsis , the sibp1 mutant accumulates more ROS than wild-type plants or AtSIBP1-overexpressing plants, resulting in a lower survival rate under salt treatment [ 42 ]. In this study, salinity led to a greater accumulation of ROS in DMS compared with JDG, as detected by DAB staining ( Fig. 1D, E ). This indicates that DMS suffers greater damage under salinity stress. Excessive accumulation of ROS in cells can lead to membrane oxidative damage and trigger the production of enzyme systems or non-enzyme free radical scavengers to cope with oxidative damage [ 10 ]. Here, antioxidant enzyme activities such as peroxidase (A0A2P6R8H8) and glutathione peroxidase (A0A2P6P708) were upregulated in roses under salt treatment ( Table S2 ). This suggests that rose plants maintain lower ROS levels by upregulating the activity of antioxidant enzymes, thereby protecting photosynthetic mechanisms and maintaining plant growth under salt stress. Among the nonenzymatic antioxidants, phenols and flavonoids accumulate in various tissues and contribute to free radical scavenging that enhances plant salt tolerance [ 43 ]. Indeed, we identified significant differences in the contents of phenolic acids, lipids, and flavonoid metabolites in JDG and DMS under control and salt stress conditions ( Table S1 ). Moreover, our transcriptomic and proteomic analysis revealed the activation of genes and proteins within the phenylpropanoid and flavonol pathways. This activation results in the accumulation of various phenolic compounds, potentially enhancing their capacity for scavenging ROS.

Flavonoids are beneficial for improving salt stress in rose

Phenolic compounds, such as flavonoids, are among the most widespread secondary metabolites observed throughout the plant kingdom [ 44 ]. These compounds fulfill various biochemical and molecular functions within plants, encompassing roles in plant defense, signal transduction, antioxidant action, and the scavenging of free radicals [ 45 ]. Environmental changes commonly trigger the flavonoid pathway, which aids in shielding plants from the harmful effects of ultraviolet radiation, salt, heat, and drought [ 23 , 46 , 47 ]. Moreover, flavonoids demonstrate potent biological activity and serve as significant antioxidants [ 48 ]. Recently, researchers and consumers have been interested in plant-based polyphenols and flavonoids for their antioxidant potential, their dietary accessibility, and their role in preventing fatal diseases such as cardiovascular disease and cancer [ 49 ]. Our transcriptomics analysis showed that salinity causes significant alterations in the secondary metabolism of JDG, while affecting the primary metabolism of DMS. Proteomics showed that phenylpropanoid biosynthesis is significantly enhanced in JDG under salt stress, especially through the flavonoid pathway. In DMS, glutathione metabolism is significantly enhanced under salt stress, indicating differences in salt tolerance pathways between the two cultivars. Our metabolome data indicated that the abundance of phenolic acid and flavonoid metabolites was significantly altered in both JDG and DMS under salt stress. Furthermore, by comparing their contents in leaves under salt stress and control conditions, we found that more flavonoids accumulated in DMS than in JDG under salt stress. This evidence suggests that DMS requires an increased presence of flavones to withstand the damage caused by salinity. By contrast, salinity stress did not trigger a substantial buildup of flavonoids in JDG, possibly due to the adequate levels of flavonoids already present under normal conditions, which provided ample tolerance to salt-induced stress. This observation could also explain the higher tolerance of JDG to salt stress ( Table S1 ). When we compared the flavonoid metabolites of the phenylpropanoid pathway to identify flavonoid metabolites associated with salt tolerance, we found that 17 phenolic acid metabolites and 6 flavonoid metabolites were significantly differentially accumulated in both genotypes. Of these compounds, ferulic acid serves as a free radical scavenger, while simultaneously serving as an inhibitor for enzymes engaged in generating free radicals and boosting the activity of scavenger enzymes [ 49 ]. Sinapic acid is a bioactive phenolic acid with anti-inflammatory and anti-anxiety effects [ 50 ]. Pinocembrin, a naturally occurring flavonoid found in fruits, vegetables, nuts, seeds, flowers, and tea, is an anti-inflammatory, antimicrobial, and antioxidant agent [ 51 ]. This indicates that these two rose cultivars contain beneficial metabolites with some economic value. We investigated the possible effects of these metabolites in conferring salt tolerance in rose by comparing specific DAMs between JDG and DMS. Among these DAMs, eight metabolites were upregulated and six metabolites were downregulated under salt treatment in JDG compared to DMS. Among these eight upregulated DAMs, the contents of 3- O -methylquercetin, brickellin, 5,2′,5′-trihydroxy-3,7,4′-trimethoxyflavone-2′- O -glucoside, and kaempferol-3- O -(6′′-acetyl)glucosyl-(1→3)-galactoside accumulated significantly with salinity ( Table S7 ). These metabolites have important functions. For example, 3- O -methylquercetin has potent anticancer, antioxidant, antiallergy, and antimicrobial activities and shows strong antiviral activity against tomato ringspot virus [ 52 ]. Kaempferol, a biologically active compound found in numerous fruits, vegetables, and herbs, demonstrates various pharmacological benefits, such as antimicrobial, antioxidant, and anticancer properties [ 53 ]. This indicates that JDG is an excellent rose cultivar that is both salt tolerant and rich in beneficial bioactive substances.

bHLHL74 regulates flavonoid biosynthesis

The biosynthesis of flavonoids is initiated from the amino acid phenylalanine, giving rise to phenylpropanoids that subsequently enter the flavonoid-anthocyanin pathway [ 25 ]. The CHS enzyme is situated at a crucial regulatory position preceding the flavonoid biosynthetic pathway, directing the flow of the phenylpropanoid pathway towards flavonoid production, which has been extensively documented in many plant species [ 54 , 55 ]. In rice ( Oryza sativa ), defects in the flavonoid biosynthesis gene CHS can alter the distribution of flavonoids and lignin [ 56 ]. In eggplant ( Solanum melongena L.), CHS regulates the content of anthocyanins in eggplant skin under heat stress [ 57 ]. In apple ( Malus domestica ), overexpression of CHS increases the accumulation of flavonoids and enhances nitrogen absorption [ 58 ]. We identified a positive correlation between flavonoid accumulation and the expression of CHS genes, in agreement with previous reports. The bHLH TFs involved in regulating flavonoid biosynthesis work in a MYB-dependent or -independent manner. For example, DvIVS, a bHLH transcription factor in dahlia ( Dahlia variabilis ), activates flavonoid biosynthesis by regulating the expression of Chalcone synthase 1 ( CHS1 ) [ 59 ]. The Arabidopsis bHLH proteins TRANSPARENT TESTA 8 (AtTT8) and ENHANCER OF GLABRA 3 (AtEGL3) are all involved in the biosynthesis of various flavonoids [ 60–62 ]. In Chrysanthemum ( Chrysanthemum morifolium ), CmbHLH2 significantly activates CmDFR transcription, leading to anthocyanin accumulation, especially when in coordination with CmMYB6 [ 63 ]. In blueberry ( Vaccinium sect. Cyanococcus ), the bHLH25 and bHLH74 TFs potentially engage with MYB or directly hinder the expression of genes responsible for flavonoid biosynthesis, thereby regulating flavonoid accumulation [ 64 ]. In apple ( Malus domestica ), expression of bHLH62, bHLH74, and bHLH162 is significantly negatively correlated with anthocyanin content and has been shown to inhibit anthocyanin biosynthesis [ 65 ]. In apple fruit skin, hypermethylation of bHLH74 in the mCG context leads to transcriptional inhibition of downstream anthocyanin biosynthesis genes [ 66 ]. In rose, our co-expression network revealed a strong correlation between CHS and genes encoding TFs such as bHLH74 and bHLH62 in the key gene network. bHLH proteins can bind to the promoter regions of pivotal genes encoding enzymes, playing important roles in regulating DAMs under salt stress. Dual-luciferase reporter assays showed that LUC bioluminescence was suppressed well below background levels in Nicotiana benthamiana leaves infiltrated with pCHS1:LUC plus 35S:bHLH74, but not 35S:bHLH62 ( Fig. 6G, H , Fig. S9A, B ). Thus, we conclude that bHLHL74 TFs negatively regulate flavonoid biosynthesis by directly inhibiting the expression of CHS1 , which is involved in the flavonoid biosynthetic pathway.

We examined the morphological phenotypes, transcriptomes, proteomes, and widely targeted metabolomes of JDG and DMS under salt stress. Multi-omics analysis revealed that the phenylpropane pathway, especially the flavonoid pathway, contributes strongly to salt tolerance in rose, particularly JDG. Meanwhile, the bHLHL74 TF negatively regulates flavonoid biosynthesis by repressing the expression of the CHS1 gene involved in the flavonoid biosynthetic pathway. This research facilitates our understanding of the regulatory mechanisms of plant development and secondary metabolites underlying salt stress responses in rose, offering valuable insights that could be used to develop new strategies for improving plant tolerance to salinity.

Plant materials and growth conditions

Rosa hybrida cv. Jardin de Granville (JDG) and Rosa damascena Mill. (DMS) were planted in the Science and Technology Park of China Agricultural University (40°03′N, 116°29′E). Rose plants were propagated by cutting culture. Rose shoots with at least two nodes and approximately 6 cm in length were used as cuttings and inserted into square flowerpots (diameter 8 cm) containing a mixture of vermiculite and peat soil [1:1 (v/v)]. Cuttings were soaked in 0.15% (v/v) indole-3-butytric acid (IBA) before insertion into pots and then grown in a growth chamber at 25°C with 50% relative humidity and a cycle of 8 hours of darkness/16 hours of light for 1 month until rooting [ 67 ].

Nicotiana benthamiana plants were used for measurement of transient expression. Seeds were sown in square flowerpots (diameter 8 cm); after 1 week, seedlings were transplanted into different pots. The soil and cultivation conditions for N. benthamiana cultivation were the same as those for roses.

Salt treatment

Twenty JDG and 20 DMS rose cuttings displaying good rooting and uniform appearance were selected for salt treatment experiments. JDG or DMS plants were randomly divided into two groups watered with either 0 or 400 mM NaCl. Phenotypes were recorded after 2 weeks. This process was repeated three times [ 68 ].

Salt treatment of rose leaves was described previously [ 68 ]. Thirty JDG and 30 DMS rose cuttings with good rooting and uniform appearance were selected, and mature leaves of similar size were collected. The leaves were divided into two treatment groups, each containing 30 leaves: group A, immersed in deionized water treatment, and group B, immersed in 400 mM NaCl treatment. Phenotypes were observed after 0, 2, and 4 days. On the second day of treatment, leaves showed obvious differences. By the fourth day of treatment, the leaves had become soft or had died. Therefore, sequencing data from the second day were used. Three independent biological replicates were assayed.

Relative electrolyte permeability

Determination of relative electrolyte permeability was as previously reported [ 69 ] with the following modifications. Salt-treated leaves (0.1 g) were weighed, placed in a 50-ml centrifuge tube, and covered with 20 ml deionized water. The conductivity of the distilled water was measured and defined as EC0. After shaking for 20 minutes at 60 rpm on an orbital shaker, the conductivity at room temperature was measured and defined as EC1. The centrifuge tube was then placed in boiling water for 10 minutes and cooled to room temperature, and the conductivity of the solution was measured as EC2. The relative permeability of the electrolytes (as a percentage) was determined as (EC1-EC0) / (EC2-EC0) × 100%.

Soluble protein content

Soluble protein content was determined following the method of Bradford (1976) [ 70 ]. Leaf samples (0.5 g) were placed in a mortar with 8 ml distilled water and a small amount of quartz sand, crushed thoroughly, and incubated at room temperature for 0.5 hours. After centrifugation at 3,000 g for 20 minutes at 4 °C, the supernatant was transferred to a 10-ml volumetric flask and the volume was adjusted to 10 ml with distilled water. Two 1.0-ml aliquots of this sample extraction solution (or distilled water as a control) were transferred to clean test tubes, 5 ml of Coomassie Brilliant Blue reagent was added, and the tubes were shaken well. After 2 minutes, when the reaction was complete, the absorbance and chromaticity at 595 nm were measured, and the protein content was determined using a standard curve.

Leaf anatomical structure

Paraffin sections were prepared as described previously with some modifications [ 71 ]. Leaves from the control and NaCl treatments were collected, washed slowly with deionized water at normal room temperature, and stored at 4°C until further use. A 3-mm × 5-mm sample was cut from the same part of each leaf, and these leaf samples were fixed in 2.5% (v/v) glutaraldehyde. Samples were dehydrated using acetone through a concentration gradient of 30%, 50%, 70%, 80%, 95%, and 100% (v/v) and then embedded in paraffin. The embedded tissues (3-μm sections) were sectioned using a Leica RM2265 rotary slicer (Leica Microsystems, Wetzlar, Germany). Slides were stained with 0.02% (v/v) toluidine blue for 5 minutes, and the residual toluidine blue was removed using distilled water. Slides were allowed to dry and then observed under a microscope (OLYMPUS BH-2, Tokyo, Japan). Three independent biological replicates were examined.

DAB (3,3′-diaminobenzidine) staining for H 2 O 2

H 2 O 2 content was detected using the DAB staining method [ 72 ]. Leaves treated with NaCl or control leaves were rinsed clean with distilled water, immersed in DAB solution (1 mg/ml, pH 3.8), and placed under vacuum at approximately 0.8 Mpa for 5 minutes; this process was repeated three to six times until the leaves were completely infiltrated. Leaves were then incubated in a box in the dark for 8 hours until a brown sediment was observed. Chlorophyll was removed by repeatedly washing with eluent (ethanol:lactic acid:glycerol, 3:1:1, v/v/v). Decolorized leaves were photographed to record their phenotypes. ImageJ was used to quantify the stained areas.

UPLC-QQQ-based widely targeted metabolome analysis

Metabolomics analysis was performed on four groups of samples: JDG-Mock, JDG-NaCl, DMS-Mock, and DMS-NaCl. Extraction and determination of metabolites were performed with the assistance of Wuhan Metware Biotechnology Co., Ltd. Samples were crushed using a stirrer containing zirconia beads (MM 400, Retsch). Freeze-dried samples (0.1 g) were incubated overnight with 1.2 ml 70% (v/v) methanol solution at 4 °C, then centrifuged at 13,400 g for 10 minutes. The extracts were filtered and subjected to LC-MS/MS analysis [ 73 ]. A previously described procedure [ 74 ] was followed for analyzing the conditions and quantifying metabolites using an LC-ESI-Q TRAP-MS/MS in multi-reaction monitoring (MRM) mode. The prcomp function was used for PCA, significantly different metabolites were determined by |log 2 Fold Change| ≥ 1, and annotated metabolites were mapped to the KEGG pathway database ( http://www.kegg.jp/kegg/pathway.html ). Comparisons are described as follows: e.g., JDG-NaCl vs JDG-Mock, indicating that the treated sample is being compared with the untreated sample and that metabolites are upregulated or downregulated in the NaCl sample compared with the Mock sample.

Tandem mass tag-based proteomic analysis

Experiments were carried out with the assistance of Hangzhou Jingjie Biotechnology Co., Ltd. Samples were thoroughly ground into powder using liquid nitrogen, and protein extraction was performed using the phenol extraction method. The protein was added to trypsin for enzymolysis overnight, and then the peptide segments were labeled with TMT tags. LC-MS/MS analysis was performed using an EASY-nLC 1200 UPLC system (ThermoFisher Scientific) and a Q Active TM HF-X (ThermoFisher Scientific) [ 75 ]. An absolute value of 1.3 was used as the threshold for significant changes. GO ( http://www.ebi.ac.uk/GOA/ ) and KEGG categories were used to annotate DAPs; WoLFPSORT software was used to predict subcellular localization ( https://wolfpsort.hgc.jp/ ).

Transcriptome sequencing

We constructed 12 cDNA libraries (three biological replicates for each of JDG and DMS under each treatment) for RNA-seq. Transcriptome sequencing was completed at Wuhan Metware Biotechnology Co., Ltd. RNA purity and RNA integrity were determined using a nanophotometer spectrophotometer and an Agilent 2100 bioanalyzer, respectively. The RNA library was then sequenced on the Illumina Hiseq platform. Raw data were filtered using fastp v 0.19.3 and compared with the reference genome ( https://lipm-browsers.toulouse.inra.fr/pub/RchiOBHm-V2/ ). FPKM (fragments per kilobase of script per million fragments mapped) was used as an indicator to measure gene expression levels, with the threshold for significant differential expression being an absolute |log 2 Fold Change| ≥ 1 and False Discovery Rate < 0.05. GO and KEGG categories were used to annotate DEGs [ 76 ].

To identify modules with high gene correlation, co-expression network analysis was performed using the R-based WGCNA package (v.1.69) with default parameters [ 77 ]. The varFilter function of the R language genefilter package was used to remove genes with low or stable expression levels in all samples. Modules based on the correlation between gene expression levels were identified, and a correlation matrix between each module and the sample was calculated using the R-based WGCNA software package. The module network was visualized using Cytoscape software (v.3.7.2).

RT-qPCR was performed on eight DEGs in the phenylpropanoid pathway to verify the accuracy of the data obtained from high-throughput sequencing. Total RNA was extracted using the hot borate method [ 72 ] and reverse transcribed using HiScript III All-in-one RT SuperMix (R333-01, Vazyme Biotech Co., Ltd., Nanjing, China). Subsequently, 2 × ChamQ SYBR qPCR Master Mix (Q331, Vazyme Biotech Co., Ltd., Nanjing, China) was used for quantitative detection of gene expression. The relative expression of genes was calculated using the 2 −ΔΔCt method [ 76 ]. GAPDH was used as an endogenous control, and primers for RT-qPCR are listed in Table S10 .

Dual-LUC reporter assay

A transactivation assay was designed to evaluate the effect of BHLH74/BHLH62 on the CHS1 promoter using methods described previously [ 78 ]. Initially, a 2000-bp segment of the CHS1 promoter was cloned into the pGreenII 0800-LUC vector, generating the ProCHS1:LUC reporter plasmid. Concurrently, the coding sequences of BHLH74/BHLH62 were inserted into the pGreenII0029 62-SK vector, resulting in the construction of Pro35S: BHLH74/BHLH62 effector plasmids. pGreenII 0800-LUC vector containing REN under control of the 35S promoter was used as a positive control.

Following plasmid construction, these constructs were introduced into Agrobacterium tumefaciens strain GV3101, which harbored the pSoup plasmid. Subsequently, A. tumefaciens containing different combinations of effector and reporter plasmids was infiltrated into N. benthamiana plants with six to eight young leaves. After a 3-day incubation period, the ratios of LUC to REN were quantified using the Bio-Lite Luciferase Assay System (DD1201, Vazyme Biotech Co., Ltd., Nanjing, China). Images capturing LUC signals were acquired using a CCD camera (Night Shade LB 985, Germany). Primer sequences are listed in Table S10 .

Statistical analysis

Statistical analyses of data were conducted using IBM SPSS Statistics, while graphical representations were created using GraphPad Prism 8.0.1. Paired data comparisons were assessed through Student's t -tests ( * P < 0.05, ** P < 0.01, *** P < 0.001). Each experiment was performed using a minimum of three biological replicates, and error bars depicted on graphs denote the standard error (SE) of the mean value. The NetWare Cloud platform ( https://cloud.metware.cn ) and OmicShare tools ( https://www.chiplot.online/ ) were used for bioinformatics analyses and mapping.

This work was supported by the Consult of Flower Industry of Jinning District (202204BI090022), General Project of Shenzhen Science and Technology and Innovation Commission (Grant No. 6020330006K0).

ZX, MN conceived and designed the experiments. RH and YW conducted the experiments. RH, YW, ZX analyzed the data. LY, JW, QX, CP, XT, GJ and MN performed the research. RH, SM and ZX wrote the manuscript. All authors read and approved the manuscript. RH and YW contributed equally to this work.

The datasets generated and analyzed during the current study are available in the Biological Research Project Data (BioProject), National Center for Biotechnology Information (NCBI) repository, accession: PRJNA1030783.

The authors declare that they have no competing interests.

Mileva M , Ilieva Y , Jovtchev G . et al. Rose flowers—a delicate perfume or a natural healer? Biomol Ther . 2021 ; 11 : 127

Google Scholar

Katsoulas N , Kittas C , Dimokas G . et al. Effect of irrigation frequency on rose flower production and quality . Biosyst Eng . 2006 ; 93 : 237 – 44

Isah T . Stress and defense responses in plant secondary metabolites production . Biol Res . 2019 ; 52 : 39

Feng D , Zhang H , Qiu X . et al. Comparative transcriptomic and metabonomic analysis revealed the relationships between biosynthesis of volatiles and flavonoid metabolites in Rosa rugosa . Ornam Plant Res . 2021 ; 1 : 1 – 10

Wang X , Zhao F , Wu Q . et al. Physiological and transcriptome analyses to infer regulatory networks in flowering transition of Rosa rugosa . Ornam Plant Res . 2023 ; 3 : 1 – 12

Jia Y , Chen C , Gong F . et al. An aux/IAA family member, RhIAA14 , involved in ethylene-inhibited petal expansion in rose ( Rosa hybrida ) . Genes . 2022 ; 13 : 1041

Ren H , Bai M , Sun J . et al. RcMYB84 and RcMYB123 mediate jasmonate-induced defense responses against Botrytis cinerea in rose ( Rosa chinensis ) . Plant J . 2020 ; 103 : 1839 – 49

Chaves MM , Flexas J , Pinheiro C . Photosynthesis under drought and salt stress: regulation mechanisms from whole plant to cell . Ann Bot . 2009 ; 103 : 551 – 60

Askari Kelestani A , Ramezanpour S , Borzouei A . et al. Application of gamma rays on salinity tolerance of wheat ( Triticum aestivum L.) and expression of genes related to biosynthesis of proline, glycine betaine and antioxidant enzymes . Physiol Mol Biol Plants . 2021 ; 27 : 2533 – 47

Qi S , Wang X , Wu Q . et al. Morphological, physiological and transcriptomic analyses reveal potential candidate genes responsible for salt stress in Rosa rugosa . Ornam Plant Res . 2023 ; 3 :21

Gill SS , Tuteja N . Reactive oxygen species and antioxidant machinery in abiotic stress tolerance in crop plants . Plant Physiol Biochem . 2010 ; 48 : 909 – 30

Ye C , Zheng S , Jiang D . et al. Initiation and execution of programmed cell death and regulation of reactive oxygen species in plants . Int J Mol Sci . 2021 ; 22 : 12942

He L , He T , Farrar S . et al. Antioxidants maintain cellular redox homeostasis by elimination of reactive oxygen species . Cell Physiol Biochem . 2017 ; 44 : 532 – 53

Challabathula D , Analin B , Mohanan A . et al. Differential modulation of photosynthesis, ROS and antioxidant enzyme activities in stress-sensitive and -tolerant rice cultivars during salinity and drought upon restriction of COX and AOX pathways of mitochondrial oxidative electron transport . J Plant Physiol . 2022 ; 268 :153583

Li C , Mur LAJ , Wang Q . et al. ROS scavenging and ion homeostasis is required for the adaptation of halophyte Karelinia caspia to high salinity . Front Plant Sci . 2022 ; 13 :

Ren G , Yang P , Cui J . et al. Multiomics analyses of two sorghum cultivars reveal the molecular mechanism of salt tolerance . Front Plant Sci . 2022 ; 13 :

Petrussa E , Braidot E , Zancani M . et al. Plant Flavonoids--Biosynthesis, Transport and Involvement in Stress Responses . Int J Mol Sci . 2013 ; 14 : 14950 – 73

Das S , Rosazza JPN . Microbial and enzymatic transformations of flavonoids . J Nat Prod . 2006 ; 69 : 499 – 508

Gao Y , Liu J , Chen Y . et al. Tomato SlAN11 regulates flavonoid biosynthesis and seed dormancy by interaction with bHLH proteins but not with MYB proteins . Hortic Res . 2018 ; 5 :

Zhang Z , Liu Y , Yuan Q . et al. The bHLH1-DTX35/DFR module regulates pollen fertility by promoting flavonoid biosynthesis in Capsicum annuum L . Hortic Res . 2022 ; 9 :

Ramaroson M , Koutouan C , Helesbeux JJ . et al. Role of Phenylpropanoids and flavonoids in plant resistance to pests and diseases . Molecules . 2022 ; 27 : 8371

Schulz E , Tohge T , Winkler JB . et al. Natural variation among Arabidopsis accessions in the regulation of flavonoid metabolism and stress gene expression by combined UV radiation and cold . Plant Cell Physiol . 2021 ; 62 : 502 – 14

Wang F , Zhu H , Kong W . et al. The antirrhinum AmDEL gene enhances flavonoids accumulation and salt and drought tolerance in transgenic Arabidopsis . Planta . 2016 ; 244 : 59 – 73

Shen N , Wang T , Gan Q . et al. Plant flavonoids: classification, distribution, biosynthesis, and antioxidant activity . Food Chem . 2022 ; 383 :132531

Liu W , Feng Y , Yu S . et al. The flavonoid biosynthesis network in plants . Int J Mol Sci . 2021 ; 22 : 12824

Zhang X , Abrahan C , Colquhoun TA . et al. A proteolytic regulator controlling chalcone synthase stability and flavonoid biosynthesis in Arabidopsis . Plant Cell . 2017 ; 29 : 1157 – 74

Riffault-Valois L , Blanchot L , Colas C . et al. Molecular fingerprint comparison of closely related rose varieties based on UHPLC-HRMS analysis and chemometrics . Phytochem Anal . 2017 ; 28 : 42 – 9

Riffault L , Destandau E , Pasquier L . et al. Phytochemical analysis of Rosa hybrida cv. ‘Jardin de Granville' by HPTLC, HPLC-DAD and HPLC-ESI-HRMS: polyphenolic fingerprints of six plant organs . Phytochemistry . 2014 ; 99 : 127 – 34

Omidi M , Khandan-Mirkohi A , Kafi M . et al. Biochemical and molecular responses of Rosa damascena mill. cv. Kashan to salicylic acid under salinity stress . BMC Plant Biol . 2022 ; 22 : 373

Azizi S , Seyed Hajizadeh H , Aghaee A . et al. In vitro assessment of physiological traits and ROS detoxification pathways involved in tolerance of damask rose genotypes under salt stress . Sci Rep . 2023 ; 13 : 17795

Zhao S , Zhang Q , Liu M . et al. Regulation of plant responses to salt stress . Int J Mol Sci . 2021 ; 22 : 4609

Zhang C , Zhang H , Zhan Z . et al. Transcriptome analysis of sucrose metabolism during bulb swelling and development in onion ( Allium cepa L.) . Front Plant Sci . 2016 ; 7 :1425

Kumari P , Raju DVS , Prasad KV . et al. Characterization of anthocyanins and their antioxidant activities in Indian rose varieties ( Rosa × hybrida ) using HPLC . Antioxidants . 2022 ; 11 : 2032

Akula R , Ravishankar GA . Influence of abiotic stress signals on secondary metabolites in plants . Plant Signal Behav . 2011 ; 6 : 1720 – 31

Barhoumi Z , Djebali W , Chaïbi W . et al. Salt impact on photosynthesis and leaf ultrastructure of Aeluropus littoralis . J Plant Res . 2007 ; 120 : 529 – 37

Jiang D , Lu B , Liu L . et al. Exogenous melatonin improves the salt tolerance of cotton by removing active oxygen and protecting photosynthetic organs . BMC Plant Biol . 2021 ; 21 : 331

Liu D , Dong S , Miao H . et al. A large-scale genomic association analysis identifies the candidate genes regulating salt tolerance in cucumber ( Cucumis sativus L.) seedlings . Int J Mol Sci . 2022 ; 23 : 8260

Garrido Y , Tudela JA , Marín A . et al. Physiological, phytochemical and structural changes of multi-leaf lettuce caused by salt stress . J Sci Food Agric . 2014 ; 94 : 1592 – 9

Yao X , Meng L , Zhao W . et al. Changes in the morphology traits, anatomical structure of the leaves and transcriptome in Lycium barbarum L. under salt stress . Front Plant Sci . 2023 ; 14 :1090366

Tan Y , Duan Y , Chi Q . et al. The role of reactive oxygen species in plant response to radiation . Int J Mol Sci . 2023 ; 24 : 3346

Li M , Chen R , Jiang Q . et al. GmNAC06 , a NAC domain transcription factor enhances salt stress tolerance in soybean . Plant Mol Biol . 2021 ; 105 : 333 – 45

Wan X , Peng L , Xiong J . et al. AtSIBP1 , a novel BTB domain-containing protein, positively regulates salt signaling in Arabidopsis thaliana . Plan Theory . 2019 ; 8 : 573

Rezayian M , Niknam V , Ebrahimzadeh H . Oxidative damage and antioxidative system in algae . Toxicol Rep . 2019 ; 6 : 1309 – 13

Liu X , Cheng X , Cao J . et al. GOLDEN 2-LIKE transcription factors regulate chlorophyll biosynthesis and flavonoid accumulation in response to UV-B in tea plants . Hortic Plant J . 2023 ; 9 : 1055 – 66

Barreca D , Gattuso G , Bellocco E . et al. Flavanones: citrus phytochemical with health-promoting properties . Biofactors . 2017 ; 43 : 495 – 506

Zhang F , Huang J , Guo H . et al. OsRLCK160 contributes to flavonoid accumulation and UV-B tolerance by regulating OsbZIP48 in rice . Sci China Life Sci . 2022 ; 65 : 1380 – 94

Cui M , Liang Z , Liu Y . et al. Flavonoid profile of Anoectochilus roxburghii (wall.) Lindl. Under short-term heat stress revealed by integrated metabolome, transcriptome, and biochemical analyses . Plant Physiol Biochem . 2023 ; 201 :107896

Dias MC , Pinto DCGA , Silva AMS . Plant flavonoids: chemical characteristics and biological activity . Molecules . 2021 ; 26 : 5377

Kumar S , Pandey AK . Chemistry and biological activities of flavonoids: an overview . Sci World J . 2013 ; 2013 : 1 – 16

Chen C . Sinapic acid and its derivatives as medicine in oxidative stress-induced diseases and aging . Oxidative Med Cell Longev . 2016 ; 2016 : 1 – 10

Rasul A , Millimouno FM , Ali Eltayb W . et al. Pinocembrin: a novel natural compound with versatile pharmacological and biological activities . Biomed Res Int . 2013 ; 2013 : 1 – 9

Doneda E , Bianchi SE , Pittol V . et al. 3-O-methylquercetin from Achyrocline satureioides -cytotoxic activity against A375-derived human melanoma cell lines and its incorporation into cyclodextrins-hydrogels for topical administration . Drug Deliv Transl Res . 2021 ; 11 : 2151 – 68

Alam W , Khan H , Shah MA . et al. Kaempferol as a dietary anti-inflammatory agent: current therapeutic standing . Molecules . 2020 ; 25 : 4073

Chen Y , Mao Y , Liu H . et al. Transcriptome analysis of differentially expressed genes relevant to variegation in peach flowers . PLoS One . 2014 ; 9 :e90842

Duan B , Tan X , Long J . et al. Integrated transcriptomic-metabolomic analysis reveals that cinnamaldehyde exposure positively regulates the phenylpropanoid pathway in postharvest Satsuma mandarin ( Citrus unshiu ) . Pestic Biochem Physiol . 2023 ; 189 :105312

Lam PY , Wang L , Lui ACW . et al. Deficiency in flavonoid biosynthesis genes CHS , CHI , and CHIL alters rice flavonoid and lignin profiles . Plant Physiol . 2022 ; 188 : 1993 – 2011

Wu X , Zhang S , Liu X . et al. Chalcone synthase (CHS) family members analysis from eggplant ( Solanum melongena L.) in the flavonoid biosynthetic pathway and expression patterns in response to heat stress . PLoS One . 2020 ; 15 :e0226537

Wang X , Chai X , Gao B . et al. Multi-omics analysis reveals the mechanism of bHLH130 responding to low-nitrogen stress of apple rootstock . Plant Physiol . 2023 ; 191 : 1305 – 23

Ohno S , Hosokawa M , Hoshino A . et al. A bHLH transcription factor, DvIVS , is involved in regulation of anthocyanin synthesis in dahlia ( Dahlia variabilis ) . J Exp Bot . 2011 ; 62 : 5105 – 16

Baudry A , Caboche M , Lepiniec L . TT8 controls its own expression in a feedback regulation involving TTG1 and homologous MYB and bHLH factors, allowing a strong and cell-specific accumulation of flavonoids in Arabidopsis thaliana . Plant J . 2006 ; 46 : 768 – 79

Gao C , Guo Y , Wang J . et al. Brassica napus GLABRA3-1 promotes anthocyanin biosynthesis and trichome formation in true leaves when expressed in Arabidopsis thaliana . Plant Biol (Stuttg) . 2018 ; 20 : 3 – 9

Feyissa DN , Løvdal T , Olsen KM . et al. The endogenous GL3 , but not EGL3 , gene is necessary for anthocyanin accumulation as induced by nitrogen depletion in Arabidopsis rosette stage leaves . Planta . 2009 ; 230 : 747 – 54

Lim S , Kim D , Jung J . et al. Alternative splicing of the basic helix-loop-helix transcription factor gene CmbHLH2 affects anthocyanin biosynthesis in ray florets of chrysanthemum ( Chrysanthemum morifolium ) . Front Plant Sci . 2021 ; 12 :

Song Y , Ma B , Guo Q . et al. UV-B induces the expression of flavonoid biosynthetic pathways in blueberry ( Vaccinium corymbosum ) calli . Front Plant Sci . 2022 ; 13 :

Li W , Mao J , Yang SJ . et al. Anthocyanin accumulation correlates with hormones in the fruit skin of 'Red Delicious' and its four generation bud sport mutants . BMC Plant Biol . 2018 ; 18 : 363

Li W , Ning GX , Mao J . et al. Whole-genome DNA methylation patterns and complex associations with gene expression associated with anthocyanin biosynthesis in apple fruit skin . Planta . 2019 ; 250 : 1833 – 47

Sun J , Lu J , Bai M . et al. Phytochrome-interacting factors interact with transcription factor CONSTANS to suppress flowering in rose . Plant Physiol . 2021 ; 186 : 1186 – 201

Su L , Zhang Y , Yu S . et al. RcbHLH59-RcPRs module enhances salinity stress tolerance by balancing Na+/K+ through callose deposition in rose ( Rosa chinensis ) . Hortic Res . 2023 ; 10 :

Liu W , Zhang R , Xiang C . et al. Transcriptomic and physiological analysis reveal that α-linolenic acid biosynthesis responds to early chilling tolerance in pumpkin rootstock varieties . Front Plant Sci . 2021 ; 12 :

Bradford MM . A rapid and sensitive method for the quantitation of microgram quantities of protein utilizing the principle of protein-dye binding . Anal Biochem . 1976 ; 72 : 248 – 54

Cheng C , Yu Q , Wang Y . et al. Ethylene-regulated asymmetric growth of the petal base promotes flower opening in rose ( Rosa hybrida ) . Plant Cell . 2021 ; 33 : 1229 – 51

Zhang Y , Wu Z , Feng M . et al. The circadian-controlled PIF8-BBX28 module regulates petal senescence in rose flowers by governing mitochondrial ROS homeostasis at night . Plant Cell . 2021 ; 33 : 2716 – 35

Meng Y , Zhang H , Fan Y . et al. Anthocyanins accumulation analysis of correlated genes by metabolome and transcriptome in green and purple peppers ( Capsicum annuum ) . BMC Plant Biol . 2022 ; 22 : 358

Deng H , Wu G , Zhang R . et al. Comparative nutritional and metabolic analysis reveals the taste variations during yellow rambutan fruit maturation . Food Chem X . 2023 ; 17 :100580

Liu D , Pan Y , Li K . et al. Proteomics reveals the mechanism underlying the inhibition of Phytophthora sojae by propyl gallate . J Agric Food Chem . 2020 ; 68 : 8151 – 62

Yang B , He S , Liu Y . et al. Transcriptomics integrated with metabolomics reveals the effect of regulated deficit irrigation on anthocyanin biosynthesis in cabernet sauvignon grape berries . Food Chem . 2020 ; 314 :126170

Umer MJ , Bin Safdar L , Gebremeskel H . et al. Identification of key gene networks controlling organic acid and sugar metabolism during watermelon fruit development by integrating metabolic phenotypes and gene expression profiles . Hortic Res . 2020 ; 7 : 193

Liang Y , Jiang C , Liu Y . et al. Auxin regulates sucrose transport to repress petal abscission in rose ( Rosa hybrida ) . Plant Cell . 2020 ; 32 : 3485 – 99

Author notes

Supplementary data, email alerts, citing articles via.

International Horticulture Research Conference
Advertising & Corporate Services

Affiliations

Online ISSN 2052-7276
Print ISSN 2662-6810
About Oxford Academic
Publish journals with us
University press partners
What we publish
New features
Open access
Institutional account management
Rights and permissions
Get help with access
Accessibility
Advertising
Media enquiries
Oxford University Press
Oxford Languages
University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

Cookie settings
Cookie policy
Privacy policy
Legal notice

This Feature Is Available To Subscribers Only

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Qualitative Methods for Policy Analysis: Case Study Research Strategy

First Online: 10 April 2022

Cite this chapter

Sarath S. Kodithuwakku 3

1388 Accesses

Many policy researchers are predisposed to use either quantitative or qualitative research methods regardless of the research questions at hand, leading to varying degrees of gaps in their findings and policy recommendations. Qualitative approaches effectively address why and how types of research questions to complement the answers for who , what , where , how many , and how much research questions, obtained using quantitative research methods, enabling researchers to make policy outcomes meaningful and contextually relevant. This chapter introduces the case study as an appropriate research strategy for accommodating qualitative and quantitative methods, followed by a brief account of qualitative research methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Available as PDF
Read on any device
Instant download
Own it forever
Available as EPUB and PDF
Compact, lightweight edition
Dispatched in 3 to 5 business days
Free shipping worldwide - see info
Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

For a lucid example for selecting a geographical boundary of a case study, see Kodithuwakku ( 1997 ) and Kodithuwakku and Rosa ( 2002 ) in which the authors selected a Sri Lankan village to explore entrepreneurial behaviour of rural farmers. Similarly, Leach ( 1967 ) drew conclusions on his fieldwork in just one village to counter some interpretations from an extensive survey conducted by Sakar and Tambiah ( 1957 ) in 57 villages on land ownership in Sri Lanka.

Bogdan R, Biklen SK (1982) Qualitative research for education. Allyn & Bacon, Boston

Google Scholar

Bonoma TV (1985) Case research in marketing: opportunities, problems, and a process. J Mark Res XXII:199–208

Article Google Scholar

Brunåker S (1993) Farm diversification-new enterprises on competitive markets. Swedish University of Agricultural Sciences, Uppsala

Bryman A, Burgess RG (1994) Analyzing qualitative data. Routledge, London

Burgess RG (1982) Field research: a source book and a field manual. George Allan & Unwin, London

Cambell DT (1955) The informant in quantitative research. Am J Sociol 60

Cassell C, Symon G (2004) Promoting new research practices in organisational research. Essential guide to qualitative methods in organisational research. Sage, London

Cochrane A (1987) What a difference the place makes: the new structuralism of locality. Antipode 19:354–363

De Vries WM (1993) Farming with other gainful activities in the Netherlands. Sociol Rural XXXIII:190–202

Downey HK, Ireland RD (1979) Quantitative versus qualitative: environmental assessment in organizational studies. Adm Sci Q 24

Eboli M, Turri E (1988) Towards a behavioural model of multiple-job holding farm families. Agric Econ 2:247–258

Eisenhardt KM (1989) Building theories from case study research. Acad Manag Rev 14(4):532–550

Fletcher AJ (2017) Applying critical realism in qualitative research: methodology meets method. Int J Soc Res Methodol 20(2):181–194

Freeman LC, Romney AK (1987) Cognitive structure and informant accuracy. Am Anthropol 89:310–325

Gartner WB, Birley S (2002) Introduction to the special issue on qualitative methods in entrepreneurship research. J Bus Ventur 17(5):431–465

Geertz CA (1973) The interpretation of culture. Basic Books, New York

Glaser BG, Strauss AL (1967) The discovery of grounded theory. Aldine, Chicago

Gummesson E (1991) Qualitative research in management research. Sage Publications Inc., California, London, and New Delhi

Gummesson E (1992) Case study research. Stockholm University, Sweden

Hamel J, Dufour S, Fortin D (1993) Case study methods (qualitative research methods series 32). Sage Publications Inc., California, London, and New Delhi

Hammersley M (1989) The dilemma of qualitative method: Herbert Blumer and the Chicago tradition. Routledge, London

Hartley JF (1994) Case studies in organizational research. In: Cassell C, Symon G (eds) Qualitative methods in organizational research. Sage Publications Inc., California, London, and New Delhi

Hartley J (2004) Case study research. In: Cassell C, Symon G (eds) Essential guide to qualitative methods in organisational research. Sage, London

Herrmann V, Uttitz P (1990) If only i didn't enjoy being a farmer: attitudes and options of monoactive and pluriactive farmers. Socilogia Ruralis 30(1):62–75

Idries S (2014) The exploits of the incomparable Mulla Nasrudin. ISF Publishing, London, p 9

Jick TD (1979) Mixing qualitative and quantitative methods: triangulation in action. Adm Sci Q 24(4):602–611

Johnson CJ (1990) Selecting ethnographic informants, qualitative research methods series 22. Sage Publications Inc., California, London, and New Delhi

Jones R (1995) Why do qualitative research? It should begin to close the gap between the sciences of discovery and implementation. Br Med J 311:2

Article CAS Google Scholar

King N (2004) Using interviews in qualitative research. In: Cassell C, Symon G (eds) Essential guide to qualitative methods in organisational research. Sage, London

Kjellen B, Soderman S (1980) Praktikfallsmetodik. SIAR/Liber, Malmo

Kodithuwakku SS (1997) Entrepreneurial processes in an apparently uniform context: a study of rural farmers in Sri Lanka. Unpublished PhD Thesis, University of Stirling

Kodithuwakku SS, Rosa P (2002) The entrepreneurial process and economic success in a constrained environment. J Bus Ventur 17(5):431–465

Kuzel AJ (1992) Sampling in qualitative inquiry. In: Crabtree BF, Miller WL (eds) Doing qualitative research: research methods for primary case series 3. Sage Publications, California

Leach ER (1967) An anthropologist’s reflections on a social survey. In: Jongmans DG, Gutkind PCW (eds) Anthropologists in the field. Van Gorcum-HJ Prakke & HMG Prakke, Assen

McClintock C (1985) Process sampling: a method for case study research on administrative behaviour. Educ Adm Q 21(3):205–222

McClintock CC, Brannon D, Moody ST (1979) Applying the logic of sample surveys to qualitative case studies: the case cluster method. Adm Sci Q 24(4):612–629

Miles MB (1979) Qualitative data as an attractive nuisance. Adm Sci Q 24(4):590–601

Miles MB, Huberman AM (1994) Qualitative data analysis: an expanded source book, 2nd edn. Sage Publications, Thousand Oaks, London, New Delhi

Miles MB, Huberman M, Saldaña J (2014) Qualitative data analysis: a methods sourcebook, 3rd edn. Sage Publications, California

Moore M (1989) The ideological history of the Sri Lankan ‘peasantry’. Mod Asian Stud 23:1

Ostrander SA (1980) Upper class women: class consciousness as conduct and meaning. In: Domhoff GW (ed) Power structure research. Sage Publications, California

Pope C, Mays N (1995) Researching the parts other methods cannot reach: an introduction to qualitative methods in health and health service research. Br Med J 311:42–45

Redclift N, Whatmore S (1990) Household consumption and livelihood: ideologies and issues in rural research. In: Marsden T, Lowe P, Whatmore S (eds) Rural restructuring: global processes and their response. David Fulton, London

Ritchie J, Spencer L (1994) Qualitative data analysis for applied policy research. In: Bryman A, Burgess RD (eds) Analysing qualitative data. Routledge

Rosa P, Bowes A (1990) Entrepreneurship: some lessons of social anthropology. EC SB 4th Workshop on Research in Entrepreneurship, University of Cologne, 29th November

Sakar NK, Tambiah S (1957) The disintegrating village. Report of a socio-economic survey conducted. University Press Board, University of Ceylon, Peradeniya

Schram W (1971) Notes on case studies of instructional media projects. Working paper, The Academy for Educational Development, Washington DC

Smith MK, Thorpe R, and Lowe A (1992) Management research: an introduction, Sage Publications Inc., London, Newbury Park, and New Delhi

Stoecker R (1991) Evaluating and rethinking the case study. Sociol Rev 39:88–112

Strauss AL, Schatzman L, Bucher R, Ehrlich D, Sabshin M (1964) Psychiatric ideologies and institutions. The Free Press, New York

Tremblay M (1957) The key informant technique: a non-ethnographical application. Am Anthropol 59:239

Van Maanen J (1979) Reclaiming qualitative methods for organizational research: a practice. Adm Sci Q 24:520–526

Weiss RS (1968) Issues in holistic research. In: Becker HS, Geer B, Riesman D, Weiss RS (eds) Institutions and the person. Aldine, Chicago

Werner O (1989) Keeping track of your interviews. CAM News Letter 1:1

Yin RK (1981) The case study crisis: some answers. Adm Sci Q 26(1):58–65

Yin RK (1994) Case study research: design and methods, 2nd edn. Sage Publications, London

Zonabend F (1992) The monograph in European ethnology. Curr Sociol 40(1):242

Yin RK (2003) Case study research: design and methods. In: Applied social research methods series volume 5. Sage Publications, London

Yin RK (2018) Case study research and applications: design and methods, 6th edn. Sage Publications, London

Author information

Authors and affiliations.

Department of Agricultural Economics and Business Management, Faculty of Agriculture, University of Peradeniya, Peradeniya, Sri Lanka

Sarath S. Kodithuwakku

You can also search for this author in PubMed Google Scholar

Editor information

Editors and affiliations.

Professor, Department of Agricultural Economics and Business Management, University of Peradeniya, Peradeniya, Sri Lanka

Jeevika Weerahewa

Policy Analyst, Technical Assistance to the Modernisation of Agriculture Programme Sri Lanka (TAMAP), Colombo, Sri Lanka

Andrew Jacque

Rights and permissions

Reprints and permissions

Copyright information

About this chapter

Kodithuwakku, S.S. (2022). Qualitative Methods for Policy Analysis: Case Study Research Strategy. In: Weerahewa, J., Jacque, A. (eds) Agricultural Policy Analysis. Springer, Singapore. https://doi.org/10.1007/978-981-16-3284-6_7

Download citation

DOI : https://doi.org/10.1007/978-981-16-3284-6_7

Published : 10 April 2022

Publisher Name : Springer, Singapore

Print ISBN : 978-981-16-3283-9

Online ISBN : 978-981-16-3284-6

eBook Packages : Biomedical and Life Sciences Biomedical and Life Sciences (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Publish with us

Policies and ethics

Find a journal
Track your research

Log in using your username and password

Search More Search for this keyword Advanced search
Latest content
Current issue
BMJ Journals More You are viewing from: Google Indexer

http://orcid.org/0000-0001-9475-1363 Sara K Tedeschi 1 ,
Keigo Hayashi 1 ,
http://orcid.org/0000-0001-7638-0888 Yuqing Zhang 2 ,
Hyon Choi 2 ,
http://orcid.org/0000-0001-8202-5428 Daniel H Solomon 1
1 Brigham and Women's Hospital , Boston , Massachusetts , USA
2 Massachusetts General Hospital , Boston , Massachusetts , USA
Correspondence to Dr Sara K Tedeschi, Brigham and Women's Hospital, Boston, Massachusetts, USA; stedeschi1{at}bwh.harvard.edu

Objective To investigate gout flare rates based on repeated serum urate (SU) measurements in a randomised controlled trial of urate-lowering therapy (ULT), accounting for dropout and death.

Methods We performed a secondary analysis using data from Cardiovascular Safety of Febuxostat or Allopurinol in Patients with Gout, which randomised participants to febuxostat or allopurinol, titrated to target SU <6 mg/dL with flare prophylaxis for 6 months. SU was categorised as ≤3.9, 4.0–5.9, 6.0–7.9, 8.0–9.9 or ≥ 10 mg/dL at each 3–6 month follow-up. The primary outcome was gout flare. Poisson regression models, adjusted for covariates and factors related to participant retention versus dropout, estimated gout flare incidence rate ratios by time-varying SU category.

Results Among 6183 participants, the median age was 65 years and 84% were male. Peak gout flare rates for all SU categories were observed in months 0–6, coinciding with the initiation of ULT and months 6–12 after stopping prophylaxis. Flare rates were similar across SU groups in the initial year of ULT. During months 36–72, a dose–response relationship was observed between the SU category and flare rate. Lower flare rates were observed when SU ≤3.9 mg/dL and greater rates when SU ≥10 mg/dL, compared with SU 4.0–5.9 mg/dL (p for trend <0.01).

Conclusion Gout flare rates were persistently higher when SU ≥6 mg/dL after the first year of ULT after accounting for censoring. The spike in flares in all categories after stopping prophylaxis suggests a longer duration of prophylaxis may be warranted.

Crystal arthropathies

Data availability statement

Data are available upon reasonable request. This publication is based on research using data from data contributors Takeda that has been made available through Vivli.

https://doi.org/10.1136/ard-2024-225761

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

A previous analysis of the Cardiovascular Safety of Febuxostat or Allopurinol in Patients with Gout (CARES) dataset reported on the relationship between time-varying serum urate (SU) and gout flares starting at month 12 after randomisation and identified that gout flare rates were lowest when SU<4.0 mg/dL and highest when SU ≥6 mg/dL. A recent population-based study reported that individuals with baseline SU ≥10 mg/dL at enrollment in the UK Biobank had a 15-fold greater gout flare rate than those with baseline SU <6 mg/dL after 5 years of follow-up.

WHAT THIS STUDY ADDS

The present analysis of CARES data reports on gout flare rates in the 12 months after urate-lowering therapy (ULT) initiation (the highest-risk period for flares), takes censoring into account, evaluates smaller SU increments and reports flare incidence rate ratios in each of four time periods among participants that continued in follow-up. This analysis provides a more nuanced assessment of gout flare rates by SU measurement at standardised time points after initiation of ULT, whereas the UK Biobank study did not focus on ULT initiators.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

The spike in flares in all SU categories after stopping prophylaxis suggests a prophylaxis for longer than 6 months may be warranted. Aiming for lower levels of SU has the potential to lower gout flare rates.

Introduction

Gout flares negatively impact the quality of life, and patients with gout identify flare reduction as an important outcome. 1–3 Urate-lowering therapy (ULT) is a cornerstone of gout management and achieving serum urate (SU) <6 mg/dL is recommended to reduce the risk of gout flare recurrence. 1 However, the precise target for SU to minimise gout flares has not been empirically determined. The American College of Radiology (ACR) 2020 Gout Treatment Guidelines recommend titrating ULT to achieve a target SU <6 mg/dL, the target used in three large trials testing strategies to implement ULT. 4–6 Urate dissolves in serum at a concentration below 6.8 mg/dL, which provides rationale for achieving SU <6 mg/dL. The utility of achieving and maintaining SU below 6 mg/dL has been tested in just one randomised controlled trial (RCT) among patients with erosive gout; the frequency of gout flares was not significantly different at months 12 or 24 in those randomised to achieve target SU<0.30 mmol/L (<5 mg/dL) compared with SU<0.20 mmol/L (<3.4 mg/dL). 7 Whether maintaining SU <6 mg/dL long term (eg, more than 2 years after initiating ULT) provides benefit to reducing gout flare rates has not been tested.

It is well recognised that gout flares are common in the months after initiating ULT. 8 The ACR 2020 Gout Treatment Guidelines recommend using flare prophylaxis for at least 3–6 months after starting ULT and to continue or resume prophylaxis on a case-by-case basis in patients with recurrent flares. 1 The benefit of a longer duration of gout flare prophylaxis for all patients starting ULT has not been tested.

The Cardiovascular Safety of Febuxostat or Allopurinol in Patients with Gout (CARES) randomised controlled trial tested the effect of ULT with febuxostat or allopurinol on cardiovascular (CV) endpoints. 9 SU was measured at regular intervals and gout flares were assessed by self-report at each visit. We investigated gout flare rates based on repeated measurements of SU in the CARES trial, accounting for dropout and death.

CARES trial overview

CARES randomised 6190 participants with gout to febuxostat or allopurinol; the primary outcome was a composite of CV events. 9 The median follow-up was 3.2 years. SU was measured at months 0, 3, 6 and then every 6 months, and ULT was titrated to achieve SU <6 mg/dL. Participants received gout flare prophylaxis for 6 months with colchicine 0.6 mg daily or naproxen 250 mg two times per day if colchicine was not tolerated. The trial had a high dropout rate, with 45% of participants missing at least one visit.

Observational study design

We performed a secondary analysis using CARES trial data. Participants were followed prospectively from randomization (month 0) through censoring (dropout or death) or the end of the trial, whichever came first. For this analysis, randomisation was broken and ULT was considered a covariate rather than exposure. Follow-up intervals were 3 or 6 months, as per the trial study visit schedule. Participants with baseline SU <4 mg/dL were excluded.

Exposure and covariates

SU at the start of each interval was the exposure for the next interval. For example, SU at month 0 was the exposure for outcomes reported in the month 0–3 interval. Time-varying SU exposure categories were ≤3.9, 4.0–5.9, 6.0–7.9, 8.0–9.9 and ≥10 mg/dL.

Baseline covariates included age, sex, race, body mass index (kg/m 2 ), gout duration in years, tophus (present or absent), SU, randomised ULT assignment (febuxostat or allopurinol), CV risk factors (diabetes, hypertension, hyperlipidaemia, myocardial infarction, unstable angina, coronary revascularisation, cerebral revascularisation, stroke and peripheral vascular disease) and creatinine clearance (mL/min).

Time-varying covariates included absolute change in SU since the prior interval (mg/dL), direction of change in SU since the prior interval (increase, decrease or no change), flare prophylaxis (colchicine, naproxen or none), ULT use (febuxostat, allopurinol or none) and gout flare in the prior interval (present if ≥1 flare or absent).

Self-reported gout flares were recorded at each visit. Multiple flares per interval were counted if the dates of flare onset were at least 14 days apart.

Statistical analysis

Baseline characteristics were summarised with descriptive statistics. We derived the inverse probability of censoring weights (IPCW) during each follow-up interval using the baseline covariates, time-varying covariates and interval duration (3 or 6 months). IPCWs were used to account for potential differential dropouts between treatment groups (ie, informative censoring) by up-weighting participants who remained in the study and who had similar traits to those who dropped out. Dropout and death were both treated as censoring. Very few deaths occurred in the trial, providing a rationale for this approach, and the Fine and Grey models to evaluate the competing risk of death were not compatible for estimating gout flare rates over time. Gout flare incidence rates (IR) and incidence rate ratios (IRRs) were estimated using Poisson models adjusted for IPCW, baseline covariates and time-varying covariates. Poisson models were stratified into four periods of time: 0–6 months, 6–12 months, 12–36 months and 36–72 months postrandomisation. We performed a test for linear trend across SU categories within each time period to calculate p for trend. We also assessed whether the velocity of SU lowering impacted the number of gout flares. To do so, we identified individuals with a 2.5–3.5 mg/dL decrease in SU over 3 months or over 6 months and used a t-test to compare the mean number of flares during that SU-lowering period (3 or 6 months). Missing SU values were imputed using multiple imputations. Analyses were performed using R V.4.1. A two-sided p-value <0.05 was considered significant. This study was exempt from Institutional Review Board approval.

Among the 6183 participants in this analysis, the median age was 65 (IQR 58–71) years and 84% were male. Baseline characteristics are provided in table 1 . The median follow-up was 32 months, and 442 participants (7.1%) died after randomisation. Median SU was 8.6 mg/dL (IQR 7.6–9.7) at randomisation. 71% achieved SU <6 mg/dL by month 3, and this percentage increased slightly over time among retained participants ( figure 1A,B ).

Download figure
Open in new tab
Download powerpoint

Serum urate levels over time in the Cardiovascular Safety of Febuxostat or Allopurinol in Patients with Gout trial dataset. (A) Serum urate level over time by serum urate category at randomisation. (B) The proportion of subjects in each serum urate category over time.

View inline

Characteristics of CARES trial participants included in this secondary analysis

Gout flare rates were highest during nearly all intervals when SU ≥10 mg/dL and lowest when SU ≤3.9 mg/dL ( table 2 and figure 2 ). Peak gout flare rates for all SU categories were observed between months 0 and 3, coinciding with the initiation of ULT and the greatest change in SU. A second spike in gout flares occurred in all SU groups between months 6 and 12, coinciding with the discontinuation of prophylaxis ( figure 2 ).

Age-adjusted and sex-adjusted gout flare rates per person-year by serum urate category in the Cardiovascular Safety of Febuxostat or Allopurinol in Patients with Gout trial dataset.

Time-varying serum urate category and adjusted gout flare IRR in the CARES trial dataset

In the initial year of ULT, flare rates did not significantly differ between SU groups, but flare rates were consistently highest when SU ≥10 mg/dL ( table 2 , p for trend >0.5 during months 0–6 and 6–12). A dose–response relationship was observed between the SU category and flare rate after the first year of ULT after adjustment for IPCW weights, baseline covariates and time-varying covariates with a relatively greater flare rate. Gout flare rates were significantly lower when SU ≤3.9 mg/dL was compared with SU 4.0–5.9 mg/dL after month 12 (IRR during months 12–36: 0.80, 95% CI 0.66 to 0.98; IRR during months 36–72: 0.85, 95% CI 0.62 to 1.18) ( table 2 ). By contrast, flare rates were nearly 50% greater when SU ≥10 mg/dL was compared with SU 4.0–5.9 mg/dL during months 12–36 (IRR 1.44, 95% CI 0.96 to 2.16; p for trend 0.01). The relative risk of gout flare was more than four times higher when SU ≥10 mg/dL was compared with SU 4.0–5.9 mg/dL during months 36–72 (IRR 4.47, 95% CI 2.42 to 8.26; p for trend<0.01).

There were 607 participants with a 2.5–3.5 mg/dL decrease in SU over 3 months and 116 with the same decrease in SU over 6 months. The median number of gout flares was zero in both groups. The mean number of gout flares was 0.24 (SD 0.56) over 3 months if the SU decrease occurred over 3 months, and 0.38 flares (SD 0.71) over 6 months if the SU decrease occurred over 6 months (p=0.02).

A significant dose–response relationship was present between SU and gout flare rate after the first year of ULT after accounting for censoring in this observational study of CARES trial data. Gout flare rates remained significantly higher after the first year of ULT when SU ≥6 mg/dL compared with<6 mg/dL (at target) and were significantly lower when SU≤3.9 mg/dL. Peak flare rates coincided with two study protocol elements: first, after the initiation of ULT, and next, after discontinuing prophylaxis. These peaks occurred in all SU categories, suggesting that other factors aside from SU also contribute to flare risk. Gout flare rates were four times as high when SU was ≥10 mg/dL three or more years after starting ULT, as compared with SU 4.0–5.9 mg/dL.

Nearly three-quarters of participants achieved target SU within 3 months of starting febuxostat or allopurinol, demonstrating the efficacy of xanthine oxidase inhibitors when taken in a trial setting. However, approximately 10% of SU levels were ≥10 mg/dL in each interval after the first year of follow-up, and gout flare rates were significantly higher in this SU category than all other categories. It is possible that participants whose SU remained ≥10 mg/dL differed from other participants in a number of ways, including genetic differences in SU metabolism, adherence to the study drug or reaching the protocol’s maximum dose of allopurinol (600 mg or 400 mg if renal impairment). Identifying which patients are at the greatest risk of persistent hyperuricaemia despite escalated ULT dosing might allow for different therapeutic approaches to reduce the risk of flares.

A previous analysis of the CARES dataset reported on the relationship between time-varying SU and gout flares starting at month 12. 10 Unadjusted gout flare rates were lowest when SU <4.0 mg/dL (0.27 flares/person-year, 95% CI 0.24 to 0.30) and highest when SU ≥6 mg/dL (0.50 flares/person-year, 95% CI 0.48 to 0.53). The present analysis takes censoring into account, meaning that the flare rates are more reflective of ‘real-world’ patients (although in a clinical trial population). Our analysis included the 12 months after randomisation, which is recognised as a high-risk period for gout flares. We identified gout flare rates for SU categories above the target of 6 mg/dL, which is pertinent to patient care as most gout patients, unfortunately, have SU above target and reported flare IRR in each of four time periods among participants that continued in follow-up. A recent population-based study reported that individuals with baseline SU ≥10 mg/dL at enrollment in the UK Biobank had a 15-fold greater gout flare rate than those with baseline SU <6 mg/dL after 5 years of follow-up—much higher than the fourfold greater gout flare rate when SU ≥10 mg/dL in the present analysis of clinical trial data. 11 A key reason underlying this difference is that the UK Biobank data reflects stable SU levels, which more accurately represent the total body urate pool than SU levels from participants in a treat-to-target trial of ULT. The present analysis of CARES trial data provides a more nuanced assessment of gout flare rates for up to 5 years after initiation of ULT and used time-varying SU measurements at standardised time points. Gout flares are common in the months after initiating ULT, and the present analysis additionally suggests that the rate of SU lowering may be relevant for gout flare risk. 8

Results from the present study raise questions about the optimal duration of gout flare prophylaxis after initiating ULT. In addition to causing joint pain and functional limitation, gout flares have been temporally associated with increased risk for CV events—providing yet another reason to reduce gout flare risk. 12 Low-dose colchicine, which is commonly used for gout flare prophylaxis, reduced the risk of recurrent CV events in patients with prior CV events. 13 We observed a peak in gout flares during months 6–12, temporally coinciding with the protocol-specified cessation of prophylaxis at month 6. A recent RCT testing the effect of colchicine versus placebo on gout flares in the first 6 months of ULT similarly reported an increase in gout flares after stopping colchicine. 14 Taken together, these suggest a potential benefit of a longer duration of prophylaxis in all patients starting ULT.

The ultimate goal of ULT is to dissolve all monosodium urate crystals by normalising the enlarged total body urate pool, which would prevent future gout flares and might also mitigate the risk of CV events, venous thromboembolism and other complications that have been associated with gout flares. 12 15 16 During periods of very low SU (≤3.9 mg/dL), gout flare rates were lower than periods of SU 4.0–5.9 mg/dL. Both of these SU categories are at target, defined as <6 mg/dL. The ACR gout treatment guidelines do not recommend any alternative targets due to ‘a lack of supporting evidence for additional specific thresholds’. 1 The EULAR gout treatment guidelines recommend against maintaining SU <3 mg/dL long term due to concerns about associations between very low SU and neurodegenerative disease. 17 A recent RCT testing the effect of different ULT targets on bone erosions in gout reported that the frequency of gout flares was not significantly different at year 2 in those randomised to achieve target SU <0.30 mmol/L (<5 mg/dL) compared with SU <0.20 mmol/L (<3.4 mg/dL). 7 Achievement of SU <0.20 mmol/L was also noted to be challenging using oral ULT. 7 Results from the present analysis, showing a lower flare rate when SU is maintained below 4 mg/dL beyond year 2 of ULT, raise the question of whether shared decision-making in gout has a role in determining the target SU. Patients who express a strong preference to avoid flares for the rest of their lives might opt to aim for a lower SU well below 6 mg/dL, weighing the risk of flares against the possible risk of neurocognitive decline.

This secondary data analysis leveraged a large dataset generated from an RCT of ULT, in which frequent SU monitoring was performed. Data were available for more than 5 years, providing information about longitudinal SU values and flares. Limitations of this analysis include a decreased sample size after the first year of follow-up, though we accounted for this by including IPCW in our IRR estimates. Given the high amount of dropout in later years, the IRR in later years may be prone to selection bias and should be interpreted with caution. Gout flares were self-reported and did not require physician evaluation, though a self-reported gout flare measure (not employed in this trial) showed excellent accuracy. 18

In conclusion, gout flares were lower when SU was <6 mg/dL compared with ≥6 mg/dL after the first year of ULT. The potential benefit of maintaining very low long-term SU levels (≤3.9 mg/dL) for longer than 2 years to reduce flare rates deserves further study. It may be prudent to continue gout flare prophylaxis, especially with colchicine, beyond the first 6 months of ULT given the observed spike in gout flare rates after stopping prophylaxis, CV risks associated with gout flare and CV benefits of colchicine.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

This was a secondary analysis of previously published clinical trial data, thus this study was exempt from Mass General Brigham Institutional Review Board approval.

Acknowledgments

This publication is based on research using data from data contributor Takeda that has been made available through Vivli. Vivli has not contributed to or approved and is not in any way responsible for the contents of this publication.

FitzGerald JD ,
Dalbeth N ,
Mikuls T , et al
Topless R ,
Noorbaloochi S ,
Merriman TR , et al
Doherty M ,
Jenkins W ,
Richardson H , et al
Goldfien R ,
Pressman A ,
Jacobson A , et al
Mikuls TR ,
Cheetham TC ,
Levy GD , et al
Billington K , et al
Yamanaka H ,
Ide Y , et al
Becker MA , et al
Becker MA ,
White WB , et al
McCormick N ,
Challener GJ , et al
Cipolletta E ,
Nakafero G , et al
Nidorf SM ,
Fiolet ATL ,
Mosterd A , et al
Mihov B , et al
Reginato AM , et al
Richette P ,
Pascual E , et al
Saag KG , et al

Handling editor Josef S Smolen

Contributors SKT, DHS, YZ and HC conceived the study and contributed to study design. KH performed data analyses. All authors contributed to data interpretation. SKT drafted the manuscript and all authors provided critical feedback and approved the final version of the manuscript. SKT is the guarantor and accepts full responsibility for the work, had access to the data and controlled the decision to publish.

Funding National Institutes of Health (K23 AR075070 (Tedeschi), L30 AR070514 (Tedeschi), R03 AR081309 (Tedeschi), P30 AR072577 (Solomon).

Competing interests SKT: consulting fees for Novartis and Avalo Therapeutics. HC: research grants from Horizon; service on a board or committee for LG Chem, Shanton and ANI Pharmaceuticals. DHS: research grants from CorEvitas, Janssen, Moderna and Novartis. Royalties from UpToDate. KH and YZ: no competing interests reported.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

IMAGES

(PDF) A guide to policy analysis as a research method
(PDF) A guide to policy analysis as a research method
Steps in a policy analysis study.
15 Research Methodology Examples (2023)
Policy Analysis Template
Introduction to Policy Analysis

VIDEO

Narrative Analysis
Why meta-analysis is policy-relevant
What are the components of research methodology?
What is Policy Research?
List of 5 Common Types of Policy Evaluation Quick Video
Data Preparation

COMMENTS

A guide to policy analysis as a research method
Policy analysis provides a way for understanding how and why governments enact certain policies, and their effects. Public health policy research is limited and lacks theoretical underpinnings. This article aims to describe and critique different approaches to policy analysis thus providing directio …
(PDF) A guide to policy analysis as a research method
Three broad orientations to policy analysis. are outlined: (i) Traditional approaches aim to identify the 'best' solution, through undertaking objec-. tive analyses of possible solutions. (ii ...
A guide to policy analysis as a research method
This article aimed to explain policy analysis as a valuable research method in health promotion. It summarized three key orientations to policy analysis—traditional, mainstream and interpretive, and, using nutrition as an example, detailed some of the major approaches that can be used. The authors suggest that applying mainstream and/or ...
Research Methods for Public Policy
It examines a variety of research methods and their use in public policy engagements and analysis for evidence-informed policymaking. It explains qualitative methods, quantitative methods, multiple and mixed-method research. Other issues addressed include causal research in public policy, report writing and communication and related issues in ...
Policy Analytics: Definitions, Components, Methods, and Illustrative
A traditional view of policy analysis emphasizes the use of models and empirical research to better understand causes and effects of policy choices, which has been dominated in the last 40 years by an economics orientation. ... we identified the main policy analytics methods and techniques that currently aid and shape public action in different ...
Methods for Policy Research
Second Edition. This book about responsible and evidence-based decision making is written for those interested in improving the decisions that affect people's lives. It describes how to define policy research questions so that evidence can be applied to them, how to find and synthesize existing evidence, how to generate new evidence if needed ...
40 The Unique Methodology of Policy Research
The scope of analysis/factors of policy research is shown to encompass all the major facets of the social phenomenon it is trying to deal with. ... and when there is an inclination to use the product in the first place. However, when these methods are applied to changing attitudes about matters as different as condom use, 8 the United Nations ...
A guide to policy analysis as a research method
As such, policy analysis provides researchers with a powerful tool to understand the use of research evidence in policymaking and generate a heightened understanding of the values, interests and political contexts underpinning policy decisions. Such methods may enable more effective advocacy for policies that can lead to improvements in health.
PDF Analysing Policy as Discourse: Methodological Advances in Policy Analysis
sis for policy and policy analysis about policy is rarely clear cut, as policy analysts often seek also to contribute alternative 'ways of doing' policy. Here we see one of the distinctive aspects of policy analysis as a research method: it can be understood asbothaninformation-collection activityandapolitical activity.Unlikemany other
PDF Methods of Analysis Policy Analysis
nineteenth century, policy analysis was a subset of the "science of administration" (p. 197). Simultaneously, policy analysis developed as a critical method as it was evident that administrators and policymakers could be overly generous or self-interested in explaining the need and characteristics of their policies.
(PDF) Perspectives on Policy Analysis: A Framework for ...
Policy analysis is a broad and versatile field of applied policy research and advice, where a multitude of perspectives and methods have developed. In this paper, we attempt to (re)structure the ...
(PDF) Policy Analysis: A Systematic Approach to ...
The paper provides a brief history of policy analysis, describes the most important elements of the policy analysis process, provides an illustrative example of the use of the approach and ...
PDF GUIDELINES FOR SUCCESSFUL POLICY ANALYSES
GUIDELINES FOR SUCCESSFUL POLICY ANALYSES • Start with an overview of recommendations, methodology, and a roadmap, not with background material • Create chapter breakdowns according to findings and recommendations, not according to the steps in your research journey • Specify your criteria and assumptions and justify them when necessary ...
PDF A Practical Guide for Policy Analysis
Conducting a Policy Research Interview 94 CONTENTS. viii contents Using Language to Characterize and Calibrate 103 Protecting Credibility 104 Strategic Dilemmas of Policy Research 106 ... his handbook serves as a guide to concepts and methods applied in the analysis of policy. I have developed the general approach and
Handbook of Public Policy Analysis
The Handbook of Public Policy Analysis: Theory, Politics, and Methods describes the theoretical debates that have recently defined the field, including the work of postpositivist, interpretivist, and social constructionist scholars. This book also explores the interplay between empirical and normative analysis, a crucial issue running through ...
A guide to policy analysis as a research method
icy analysis as a valuable research method in health pro-motion ( Breton and De Leeuw, 2011 ). It describes and critique different orientations and approaches to policy analysis thus providing direction for undertaking policy analysis in public health research. Through the use of a specific, illustrative policy example, it provides direction
Policy Research
Research Tools (Center for Health Economics and Policy, Washington University in St. Louis): health economics and policy research tools, including cost effectiveness, policy analysis toolkit, and policy analysis web series. Theory & Methods (Center for Public Health Law Research): public law/ legal epidemiology methods for conducting research ...
What are the core concerns of policy analysis? A ...
Policy analysis provides multiple methods and tools for generating and transforming policy-relevant information and supporting policy evolution to address emerging social problems. In this study ...
Methods for Policy Research
Methods for Policy Research. Concentration Chairs: Mary Beth Landrum and Michael McWilliams (on sabbatical for the 2023-2024 academic year) Training in the MPR concentration (formerly called Evaluative Science & Statistics) will position students to conduct rigorous, policy-relevant research. The concentration includes intensive methodological ...
The institute
The International Institute for Applied Systems Analysis (IIASA) is an international research institute that advances systems analysis and applies its research methods to identify policy solutions to reduce human footprints, enhance the resilience of natural and socioeconomic systems, and help achieve the Sustainable Development Goals.
The synergistic impact of Universal Health Coverage and Global Health
Health Research Policy and Systems. 2022;20(1):1-9. View Article Google Scholar 24. Burton A, Monasch R, Lautenbach B, Gacic-Dobo M, Neill M, Karimov R, et al. WHO and UNICEF estimates of national infant immunization coverage: methods and processes. Bulletin of the World Health Organization. 2009;87:535-41.
Qualitative Methods for Policy Analysis: Case Study Research Strategy
Qualitative approaches effectively address why and how types of research questions to complement the answers for who, what, where, how many, and how much research questions, obtained using quantitative research methods, enabling researchers to make policy outcomes meaningful and contextually relevant. This chapter introduces the case study as ...
Principal stratification methods and software for intercurrent events
Project Description and Goals. This project aims to develop a suite of transparent and accessible analysis tools, software and educational material for applying the principal stratification method ...
New Content From Advances in Methods and Practices in Psychological
Participants included 103 research-methods instructors, academics, students, and nonacademic psychologists. Of 78 items included in the consensus process, 34 reached consensus. We coupled these results with a qualitative analysis of 707 open-ended text responses to develop nine recommendations for organizations that accredit undergraduate ...
Multi-omics analysis reveals key regulatory defense pathways and genes
This research facilitates our understanding of the regulatory mechanisms of plant development and secondary metabolites underlying salt stress responses in rose, offering valuable insights that could be used to develop new strategies for improving plant tolerance to salinity. Materials and methods Plant materials and growth conditions
To Ease Energy Transmission Gridlock, States Look to Grid-Enhancing
One analysis showed that DLRs yield average increases in transmission capacity of more than 33% in summer and 19% in winter. These boosts mean that the technologies can often pay for themselves within six months of becoming fully operational because they reduce congestion and allow for uptake of lower-cost, renewable energy.
Agriculture
(1) Background: This challenge is exacerbated by the aging of the rural population, leading to a scarcity of available manpower. To address this issue, the automation and mechanization of outdoor vegetable cultivation are imperative. Therefore, developing an automated cultivation platform that reduces labor requirements and improves yield by efficiently performing all the cultivation ...
PDF Qualitative Methods for Policy Analysis: Case Study Research ...
7.1 Introduction. The main objective of policy analysis is to assess whether a given policy or set of policies has achieved its intended goals/objectives and, if not, why and how they can be ne-tuned for increased effectiveness. Policy analysis can be carried out using. fi. both quantitative and qualitative approaches.
Identifying the association between serum urate levels and gout flares
Methods We performed a secondary analysis using data from Cardiovascular Safety of Febuxostat or Allopurinol in Patients with Gout, which randomised participants to febuxostat or allopurinol, titrated to target SU <6 mg/dL with flare prophylaxis for 6 months. SU was categorised as ≤3.9, 4.0-5.9, 6.0-7.9, 8.0-9.9 or ≥ 10 mg/dL at each 3-6 month follow-up.

A guide to policy analysis as a research method

You are here

Methods for Policy Research Taking Socially Responsible Action

Sample Materials & Chapters

A guide to policy analysis as a research method

Access to Document

Other files and links

Community Engagement Program

For more information:

Harvard Catalyst Policy Atlas

Examples of Policy Research

Other Resources

What are the core concerns of policy analysis? A multidisciplinary investigation based on in-depth bibliometric analysis

Similar content being viewed by others

Scientific evidence on the political impact of the Sustainable Development Goals

Transforming power of research and development on inequality and well-being: a European Union perspective within the circular economy framework

Mapping the community: use of research evidence in policy and practice

Literature research

Dataset construction

Network visualization

Publication information of policy analysis

Publication information of policy analysis in different fields

Policy analysis in the field of medicine

Policy analysis in the field of environment

Policy analysis in the field of energy

Policy analysis in the field of economy

Policy analysis in multidiscipline

Key findings

Limitations

Data availability

Acknowledgements

Author information

Corresponding author

Ethics declarations

Ethical approval

Informed consent

Supplementary information

About this article

Share this article

Quick links

Harvard Ph.D. Program in Health Policy

The institute

Introduction

Statistical analysis

Supporting information

U.S. Food and Drug Administration

Principal stratification methods and software for intercurrent events in clinical trials

Regulatory Science Challenge

Project Description and Goals

Research Outcomes/Results

Research Impacts

Publications

New Content From Advances in Methods and Practices in Psychological Science

Privacy Overview

Article Contents

Multi-omics analysis reveals key regulatory defense pathways and genes involved in salt tolerance of rose plants

JDG is more tolerant than DMS to salt stress

Flavonoid metabolites play an important role in the salinity tolerance of rose

Salt stress causes dynamic changes in distinct sets of proteins

Salt stress differentially alters the transcriptomes of JDG and DMS

Expression of phenylpropanoid-related genes is correlated with proteins and metabolites affected by salt stress

Networks of co-expressed genes associated with phenylpropanoid biosynthesis are involved in the salt stress response

Salt stress damages the structure and osmotic potential of rose leaves

Flavonoids are beneficial for improving salt stress in rose

bHLHL74 regulates flavonoid biosynthesis

Plant materials and growth conditions

Salt treatment

Relative electrolyte permeability

Soluble protein content

Leaf anatomical structure

DAB (3,3′-diaminobenzidine) staining for H 2 O 2

UPLC-QQQ-based widely targeted metabolome analysis

Tandem mass tag-based proteomic analysis

Transcriptome sequencing

Dual-LUC reporter assay

Statistical analysis

Author notes

Affiliations

This Feature Is Available To Subscribers Only

Qualitative Methods for Policy Analysis: Case Study Research Strategy