scholarly articles research questions

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

A Practical Guide to Writing Quantitative and Qualitative Research Questions and Hypotheses in Scholarly Articles

Affiliations.

• 1 Department of General Education, Graduate School of Nursing Science, St. Luke's International University, Tokyo, Japan. [email protected].
• 2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.
• PMID: 35470596
• PMCID: PMC9039193
• DOI: 10.3346/jkms.2022.37.e121

The development of research questions and the subsequent hypotheses are prerequisites to defining the main research purpose and specific objectives of a study. Consequently, these objectives determine the study design and research outcome. The development of research questions is a process based on knowledge of current trends, cutting-edge studies, and technological advances in the research field. Excellent research questions are focused and require a comprehensive literature search and in-depth understanding of the problem being investigated. Initially, research questions may be written as descriptive questions which could be developed into inferential questions. These questions must be specific and concise to provide a clear foundation for developing hypotheses. Hypotheses are more formal predictions about the research outcomes. These specify the possible results that may or may not be expected regarding the relationship between groups. Thus, research questions and hypotheses clarify the main purpose and specific objectives of the study, which in turn dictate the design of the study, its direction, and outcome. Studies developed from good research questions and hypotheses will have trustworthy outcomes with wide-ranging social and health implications.

Keywords: Hypotheses; Qualitative Research; Quantitative Research; Research Questions.

© 2022 The Korean Academy of Medical Sciences.

PubMed Disclaimer

Conflict of interest statement

The authors have no potential conflicts of interest to disclose.

Fig. 1. General flow for constructing effective…

Fig. 1. General flow for constructing effective research questions and hypotheses prior to conducting research.

Fig. 2. Algorithm for building research question…

Fig. 2. Algorithm for building research question and hypothesis in quantitative research, and illustrative example…

Fig. 3. Algorithm for building research question…

Fig. 3. Algorithm for building research question and hypothesis in qualitative research, and illustrative example…

Similar articles

• Conducting and Writing Quantitative and Qualitative Research. Barroga E, Matanguihan GJ, Furuta A, Arima M, Tsuchiya S, Kawahara C, Takamiya Y, Izumi M. Barroga E, et al. J Korean Med Sci. 2023 Sep 18;38(37):e291. doi: 10.3346/jkms.2023.38.e291. J Korean Med Sci. 2023. PMID: 37724495 Free PMC article. Review.
• A Guide to Writing a Qualitative Systematic Review Protocol to Enhance Evidence-Based Practice in Nursing and Health Care. Butler A, Hall H, Copnell B. Butler A, et al. Worldviews Evid Based Nurs. 2016 Jun;13(3):241-9. doi: 10.1111/wvn.12134. Epub 2016 Jan 20. Worldviews Evid Based Nurs. 2016. PMID: 26790142
• The qualitative research proposal. Klopper H. Klopper H. Curationis. 2008 Dec;31(4):62-72. doi: 10.4102/curationis.v31i4.1062. Curationis. 2008. PMID: 19653539 Review.
• Research proposal writing: breaking the myth. Nte AR, Awi DD. Nte AR, et al. Niger J Med. 2006 Oct-Dec;15(4):373-81. doi: 10.4314/njm.v15i4.37249. Niger J Med. 2006. PMID: 17111720
• Research in adolescent healthcare: The value of qualitative methods. Lefèvre H, Moro MR, Lachal J. Lefèvre H, et al. Arch Pediatr. 2019 Oct;26(7):426-430. doi: 10.1016/j.arcped.2019.09.012. Epub 2019 Oct 12. Arch Pediatr. 2019. PMID: 31611145 Review.
• Daily experiences of non-psychiatric nurses in acute psychiatric wards. Rangwaneni ME, Raliphaswa NS, Maluleke M, Masutha TC. Rangwaneni ME, et al. Nurs Open. 2024 May;11(5):e2174. doi: 10.1002/nop2.2174. Nurs Open. 2024. PMID: 38728530 Free PMC article.
• Lessons From the Pandemic for Hand Surgery in Wales. Lawrence OJ, Shanbhag V. Lawrence OJ, et al. Cureus. 2024 Mar 20;16(3):e56577. doi: 10.7759/cureus.56577. eCollection 2024 Mar. Cureus. 2024. PMID: 38646319 Free PMC article.
• Exploring barriers and facilitators of implementing an at-home SARS-CoV-2 antigen self-testing intervention: The Rapid Acceleration of Diagnostics-Underserved Populations (RADx-UP) initiatives. Cross LM, DeFosset A, Yusuf B, Conserve D, Anderson R, Carilli C, Kibbe W, Cohen-Wolkowiez M, Richmond A, Corbie G, Dave G. Cross LM, et al. PLoS One. 2023 Nov 16;18(11):e0294458. doi: 10.1371/journal.pone.0294458. eCollection 2023. PLoS One. 2023. PMID: 37971996 Free PMC article.
• Research.com . How to write a research question: Types, steps, and examples. [Updated 2021]. [Accessed January 2, 2022]. https://research.com/research/how-to-write-a-research-question .
• Chigbu UE. Visually hypothesising in scientific paper writing: confirming and refuting qualitative research hypotheses using diagrams. Publications. 2019;7(1):22.
• International Institute of Health Sciences. Developing hypothesis and research question. [Updated 2022]. [Accessed January 3, 2022]. https://www.iihs.edu.lk/mod/resource/view.php?id=34513&forceview=1 .
• Excelsior College. Research hypotheses. [Updated 2022]. [Accessed February 3, 2022]. https://owl.excelsior.edu/research/research-hypotheses/
• Wordvice. How to write a hypothesis or research question. [Updated 2021]. [Accessed January 4, 2022]. https://blog.wordvice.com/how-to-write-a-hypothesis-or-research-question/

Publication types

• Search in MeSH

Related information

Linkout - more resources, full text sources.

• Europe PubMed Central
• Korean Academy of Medical Sciences
• PubMed Central

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Starting the research process
• Writing Strong Research Questions | Criteria & Examples

Writing Strong Research Questions | Criteria & Examples

Published on October 26, 2022 by Shona McCombes . Revised on November 21, 2023.

A research question pinpoints exactly what you want to find out in your work. A good research question is essential to guide your research paper , dissertation , or thesis .

All research questions should be:

• Focused on a single problem or issue
• Researchable using primary and/or secondary sources
• Feasible to answer within the timeframe and practical constraints
• Specific enough to answer thoroughly
• Complex enough to develop the answer over the space of a paper or thesis
• Relevant to your field of study and/or society more broadly

Table of contents

How to write a research question, what makes a strong research question, using sub-questions to strengthen your main research question, research questions quiz, other interesting articles, frequently asked questions about research questions.

You can follow these steps to develop a strong research question:

• Choose your topic
• Do some preliminary reading about the current state of the field
• Narrow your focus to a specific niche
• Identify the research problem that you will address

The way you frame your question depends on what your research aims to achieve. The table below shows some examples of how you might formulate questions for different purposes.

Using your research problem to develop your research question

Note that while most research questions can be answered with various types of research , the way you frame your question should help determine your choices.

Prevent plagiarism. Run a free check.

Research questions anchor your whole project, so it’s important to spend some time refining them. The criteria below can help you evaluate the strength of your research question.

Focused and researchable

Feasible and specific

Complex and arguable

Relevant and original

Chances are that your main research question likely can’t be answered all at once. That’s why sub-questions are important: they allow you to answer your main question in a step-by-step manner.

Good sub-questions should be:

• Less complex than the main question
• Focused only on 1 type of research
• Presented in a logical order

Here are a few examples of descriptive and framing questions:

• Descriptive: According to current government arguments, how should a European bank tax be implemented?
• Descriptive: Which countries have a bank tax/levy on financial transactions?
• Framing: How should a bank tax/levy on financial transactions look at a European level?

Keep in mind that sub-questions are by no means mandatory. They should only be asked if you need the findings to answer your main question. If your main question is simple enough to stand on its own, it’s okay to skip the sub-question part. As a rule of thumb, the more complex your subject, the more sub-questions you’ll need.

Try to limit yourself to 4 or 5 sub-questions, maximum. If you feel you need more than this, it may be indication that your main research question is not sufficiently specific. In this case, it’s is better to revisit your problem statement and try to tighten your main question up.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

Methodology

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

The way you present your research problem in your introduction varies depending on the nature of your research paper . A research paper that presents a sustained argument will usually encapsulate this argument in a thesis statement .

A research paper designed to present the results of empirical research tends to present a research question that it seeks to answer. It may also include a hypothesis —a prediction that will be confirmed or disproved by your research.

As you cannot possibly read every source related to your topic, it’s important to evaluate sources to assess their relevance. Use preliminary evaluation to determine whether a source is worth examining in more depth.

This involves:

• Reading abstracts , prefaces, introductions , and conclusions
• Looking at the table of contents to determine the scope of the work
• Consulting the index for key terms or the names of important scholars

A research hypothesis is your proposed answer to your research question. The research hypothesis usually includes an explanation (“ x affects y because …”).

A statistical hypothesis, on the other hand, is a mathematical statement about a population parameter. Statistical hypotheses always come in pairs: the null and alternative hypotheses . In a well-designed study , the statistical hypotheses correspond logically to the research hypothesis.

Formulating a main research question can be a difficult task. Overall, your question should contribute to solving the problem that you have defined in your problem statement .

However, it should also fulfill criteria in three main areas:

• Researchability
• Feasibility and specificity
• Relevance and originality

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, November 21). Writing Strong Research Questions | Criteria & Examples. Scribbr. Retrieved September 3, 2024, from https://www.scribbr.com/research-process/research-questions/

Is this article helpful?

Shona McCombes

Other students also liked, how to define a research problem | ideas & examples, how to write a problem statement | guide & examples, 10 research question examples to guide your research project, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Research Question Examples 🧑🏻‍🏫

25+ Practical Examples & Ideas To Help You Get Started 

By: Derek Jansen (MBA) | October 2023

A well-crafted research question (or set of questions) sets the stage for a robust study and meaningful insights.  But, if you’re new to research, it’s not always clear what exactly constitutes a good research question. In this post, we’ll provide you with clear examples of quality research questions across various disciplines, so that you can approach your research project with confidence!

Research Question Examples

• Psychology research questions
• Business research questions
• Education research questions
• Healthcare research questions
• Computer science research questions

Examples: Psychology

Let’s start by looking at some examples of research questions that you might encounter within the discipline of psychology.

How does sleep quality affect academic performance in university students?

This question is specific to a population (university students) and looks at a direct relationship between sleep and academic performance, both of which are quantifiable and measurable variables.

What factors contribute to the onset of anxiety disorders in adolescents?

The question narrows down the age group and focuses on identifying multiple contributing factors. There are various ways in which it could be approached from a methodological standpoint, including both qualitatively and quantitatively.

Do mindfulness techniques improve emotional well-being?

This is a focused research question aiming to evaluate the effectiveness of a specific intervention.

How does early childhood trauma impact adult relationships?

This research question targets a clear cause-and-effect relationship over a long timescale, making it focused but comprehensive.

Is there a correlation between screen time and depression in teenagers?

This research question focuses on an in-demand current issue and a specific demographic, allowing for a focused investigation. The key variables are clearly stated within the question and can be measured and analysed (i.e., high feasibility).

Examples: Business/Management

Next, let’s look at some examples of well-articulated research questions within the business and management realm.

How do leadership styles impact employee retention?

This is an example of a strong research question because it directly looks at the effect of one variable (leadership styles) on another (employee retention), allowing from a strongly aligned methodological approach.

What role does corporate social responsibility play in consumer choice?

Current and precise, this research question can reveal how social concerns are influencing buying behaviour by way of a qualitative exploration.

Does remote work increase or decrease productivity in tech companies?

Focused on a particular industry and a hot topic, this research question could yield timely, actionable insights that would have high practical value in the real world.

How do economic downturns affect small businesses in the homebuilding industry?

Vital for policy-making, this highly specific research question aims to uncover the challenges faced by small businesses within a certain industry.

Which employee benefits have the greatest impact on job satisfaction?

By being straightforward and specific, answering this research question could provide tangible insights to employers.

Examples: Education

Next, let’s look at some potential research questions within the education, training and development domain.

How does class size affect students’ academic performance in primary schools?

This example research question targets two clearly defined variables, which can be measured and analysed relatively easily.

Do online courses result in better retention of material than traditional courses?

Timely, specific and focused, answering this research question can help inform educational policy and personal choices about learning formats.

What impact do US public school lunches have on student health?

Targeting a specific, well-defined context, the research could lead to direct changes in public health policies.

To what degree does parental involvement improve academic outcomes in secondary education in the Midwest?

This research question focuses on a specific context (secondary education in the Midwest) and has clearly defined constructs.

What are the negative effects of standardised tests on student learning within Oklahoma primary schools?

This research question has a clear focus (negative outcomes) and is narrowed into a very specific context.

Need a helping hand?

Examples: Healthcare

Shifting to a different field, let’s look at some examples of research questions within the healthcare space.

What are the most effective treatments for chronic back pain amongst UK senior males?

Specific and solution-oriented, this research question focuses on clear variables and a well-defined context (senior males within the UK).

How do different healthcare policies affect patient satisfaction in public hospitals in South Africa?

This question is has clearly defined variables and is narrowly focused in terms of context.

Which factors contribute to obesity rates in urban areas within California?

This question is focused yet broad, aiming to reveal several contributing factors for targeted interventions.

Does telemedicine provide the same perceived quality of care as in-person visits for diabetes patients?

Ideal for a qualitative study, this research question explores a single construct (perceived quality of care) within a well-defined sample (diabetes patients).

Which lifestyle factors have the greatest affect on the risk of heart disease?

This research question aims to uncover modifiable factors, offering preventive health recommendations.

Examples: Computer Science

Last but certainly not least, let’s look at a few examples of research questions within the computer science world.

What are the perceived risks of cloud-based storage systems?

Highly relevant in our digital age, this research question would align well with a qualitative interview approach to better understand what users feel the key risks of cloud storage are.

Which factors affect the energy efficiency of data centres in Ohio?

With a clear focus, this research question lays a firm foundation for a quantitative study.

How do TikTok algorithms impact user behaviour amongst new graduates?

While this research question is more open-ended, it could form the basis for a qualitative investigation.

What are the perceived risk and benefits of open-source software software within the web design industry?

Practical and straightforward, the results could guide both developers and end-users in their choices.

Remember, these are just examples…

In this post, we’ve tried to provide a wide range of research question examples to help you get a feel for what research questions look like in practice. That said, it’s important to remember that these are just examples and don’t necessarily equate to good research topics . If you’re still trying to find a topic, check out our topic megalist for inspiration.

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

Research ideas on Political Science

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Educational resources and simple solutions for your research journey

How to Craft a Strong Research Question (With Research Question Examples)

A sound and effective research question is a key element that must be identified and pinned down before researchers can even begin their research study or work. A strong research question lays the foundation for your entire study, guiding your investigation and shaping your findings. Hence, it is critical that researchers spend considerable time assessing and refining the research question based on in-depth reading and comprehensive literature review. In this article, we will discuss how to write a strong research question and provide you with some good examples of research questions across various disciplines.

Table of Contents

The importance of a research question

A research question plays a crucial role in driving scientific inquiry, setting the direction and purpose of your study, and guiding your entire research process. By formulating a clear and focused research question, you lay the foundation for your investigation, ensuring that your research remains on track and aligned with your objectives so you can make meaningful contribution to the existing body of knowledge. A well-crafted research question also helps you define the scope of your study and identify the appropriate methodologies and data collection techniques to employ.

Key components of a strong research question

A good research question possesses several key components that contribute to the quality and impact of your study. Apart from providing a clear framework to generate meaningful results, a well-defined research question allows other researchers to understand the purpose and significance of your work. So, when working on your research question, incorporate the following elements:

• Specificity : A strong research question should be specific about the main focus of your study, enabling you to gather precise data and draw accurate conclusions. It clearly defines the variables, participants, and context involved, leaving no room for ambiguity.
• Clarity : A good research question is clear and easily understood, so articulate the purpose and intent of your study concisely without being generic or vague. Ensuring clarity in your research question helps both you and your readers grasp the research objective.
• Feasibility : While crafting a research question, consider the practicality of conducting the research and availability of necessary data or access to participants. Think whether your study is realistic and achievable within the constraints of time, resources, and ethical considerations.

How to craft a well-defined research question

A first step that will help save time and effort is knowing what your aims are and thinking about a few problem statements on the area or aspect one wants to study or do research on. Contemplating these statements as one undertakes more progressive reading can help the researcher in reassessing and fine-tuning the research question. This can be done over time as they read and learn more about the research topic, along with a broad literature review and parallel discussions with peer researchers and supervisors. In some cases, a researcher can have more than one research question if the research being undertaken is a PhD thesis or dissertation, but try not to cover multiple concerns on a topic.

A strong research question must be researchable, original, complex, and relevant. Here are five simple steps that can make the entire process easier.

• Identify a broad topic from your areas of interest, something that is relevant, and you are passionate about since you’ll be spending a lot of time conducting your research.
• Do a thorough literature review to weed out potential gaps in research and stay updated on what’s currently being done in your chosen topic and subject area.
• Shortlist possible research questions based on the research gaps or see how you can build on or refute previously published ideas and concepts.
• Assess your chosen research question using the FINER criteria that helps you evaluate whether the research is Feasible, Interesting, Novel, Ethical, and Relevant. 1
• Formulate the final research question, while ensuring it is clear, well-written, and addresses all the key elements of a strong research question.

Examples of research questions

Remember to adapt your research question to suit your purpose, whether it’s exploratory, descriptive, comparative, experimental, qualitative, or quantitative. Embrace the iterative nature of the research process, continually evaluating and refining your question as you progress. Here are some good examples of research questions across various disciplines.

Exploratory research question examples

• How does social media impact interpersonal relationships among teenagers?
• What are the potential benefits of incorporating mindfulness practices in the workplace?

Descriptive research question examples

• What factors influence customer loyalty in the e-commerce industry?
• Is there a relationship between socioeconomic status and academic performance among elementary school students?

Comparative research question examples

• How does the effectiveness of traditional teaching methods compare to online learning platforms in mathematics education?
• What is the impact of different healthcare policies on patient outcomes in various countries?

Experimental research question examples

• What are the effects of a new drug on reducing symptoms of a specific medical condition?
• Does a dietary intervention have an impact on weight loss among individuals with obesity?

Qualitative research question examples

• What are the lived experiences of immigrants adapting to a new culture?
• What factors influence job satisfaction among healthcare professionals?

Quantitative research question examples

• Is there a relationship between sleep duration and academic performance among college students?
• How effective is a specific intervention in reducing anxiety levels among individuals with phobias?

With these simple guidelines and inspiring examples of research questions, you are equipped to embark on your research journey with confidence and purpose. Here’s wishing you all the best for your future endeavors!

References:

• How to write a research question: Steps and examples. Indeed Career Guide. Available online at https://www.indeed.com/career-advice/career-development/how-to-write-research-questions

R Discovery is a literature search and research reading platform that accelerates your research discovery journey by keeping you updated on the latest, most relevant scholarly content. With 250M+ research articles sourced from trusted aggregators like CrossRef, Unpaywall, PubMed, PubMed Central, Open Alex and top publishing houses like Springer Nature, JAMA, IOP, Taylor & Francis, NEJM, BMJ, Karger, SAGE, Emerald Publishing and more, R Discovery puts a world of research at your fingertips.  

Try R Discovery Prime FREE for 1 week or upgrade at just US$72 a year to access premium features that let you listen to research on the go, read in your language, collaborate with peers, auto sync with reference managers, and much more. Choose a simpler, smarter way to find and read research – Download the app and start your free 7-day trial today !  

Related Posts

Research in Shorts: R Discovery’s New Feature Helps Academics Assess Relevant Papers in 2mins 

How Does R Discovery’s Interplatform Capability Enhance Research Accessibility 

Evaluating Information Sources

• Evaluate Your Sources
• Publication Types and Bias

Structure of Scientific Papers

Reading a scholarly article, additional reading tips, for more information.

• Reading Scholarly Articles
• Impact Factors and Citation Counts
• Predatory Publishing

Research papers generally follow a specific format. Here are the different parts of the scholarly article.

Abstract (Summary)

The abstract, generally written by the author(s) of the article, provides a concise summary of the whole article. Usually it highlights the focus, study results and conclusion(s) of the article. 

Introduction (Why)

In this section, the authors introduce their topic, explain the purpose of the study, and present why it is important, unique or how it adds to existing knowledge in their field. Look for the author's hypothesis or thesis here. 

Introduction - Literature Review (Who else)

Many scholarly articles include a summary of previous research or discussions published on this topic, called a "Literature Review".  This section outlines what others have found and what questions still remain.

Methodology  / Materials and Methods (How) 

Find the details of how the study was performed in this section. There should be enough specifics so that you could repeat the study if you wanted. 

Results   (What happened)

This section includes the findings from the study. Look for the data and statistical results in the form of tables, charts, and graphs. Some papers include an analysis here.

Discussion  / Analysis  (What it means)

This section should tell you what the authors felt was significant about their results. The authors analyze their data and describe what they believe it means.

Conclusion (What was learned)

Here the authors offer their final thoughts and conclusions and may include: how the study addressed their hypothesis, how it contributes to the field, the strengths and weaknesses of the study, and recommendations for future research. Some papers combine the discussion and conclusion.

A scholarly paper can be difficult to read. Instead of reading straight through, try focusing on the different sections and asking specific questions at each point.

What is your research question? 

When you select an article to read for a project or class, focus on your topic. Look for information in the article that is relevant to your research question. 

Read the abstract first  as it covers basics of the article. Questions to consider: 

• What is this article about? What is the working hypothesis or thesis?
• Is this related to my question or area of research?

Second: Read the introduction and discussion/conclusion.  These sections offer the main argument and hypothesis of the article. Questions to consider for the introduction: 

• What do we already know about this topic and what is left to discover?
• What have other people done in regards to this topic?
• How is this research unique?
• Will this tell me anything new related to my research question?

Questions for the discussion and conclusion: 

• What does the study mean and why is it important?
• What are the weaknesses in their argument?
• Is the conclusion valid?

Next: Read about the Methods/Methodology.  If what you've read addresses your research question, this should be your next section. Questions to consider:

• How did the author do the research? Is it a qualitative or quantitative project?
• What data are the study based on?
• Could I repeat their work? Is all the information present in order to repeat it?

Finally: Read the Results and Analysis.  Now read the details of this research. What did the researchers learn? If graphs and statistics are confusing, focus on the explanations around them. Questions to consider: 

• What did the author find and how did they find it?
• Are the results presented in a factual and unbiased way?
• Does their analysis agree with the data presented?
• Is all the data present?
• What conclusions do you formulate from this data? (And does it match with the Author's conclusions?)

Review the References (anytime): These give credit to other scientists and researchers and show you the basis the authors used to develop their research.  The list of references, or works cited, should include all of the materials the authors used in the article. The references list can be a good way to identify additional sources of information on the topic. Questions to ask:

• What other articles should I read?
• What other authors are respected in this field?
• What other research should I explore?

When you read these scholarly articles, remember that you will be writing based on what you read.

While you are Reading:

• Keep in mind your research question
• Focus on the information in the article relevant to your question (feel free to skim over other parts)
• Question everything you read - not everything is 100% true or performed effectively
• Think critically about what you read and seek to build your own arguments
• Read out of order! This isn't a mystery novel or movie, you want to start with the spoiler
• Use any keywords printed by the journals as further clues about the article
• Look up words you don't know

How to Take Notes on the Article

Try different ways, but use the one that fits you best. Below are some suggestions:

• Print the article and highlight, circle and otherwise mark while you read (for a PDF, you can use the highlight text  feature in Adobe Reader)
• Take notes on the sections, for example in the margins (Adobe Reader offers pop-up  sticky notes )
• Highlight only very important quotes or terms - or highlight potential quotes in a different color
• Summarize the main or key points

Reflect on what you have read - draw your own conclusions . As you read jot down questions that come to mind. These may be answered later on in the article or you may have found something that the authors did not consider. Here are a few questions that might be helpful:

• Have I taken time to understand all the terminology?
• Am I spending too much time on the less important parts of this article?
• Do I have any reason to question the credibility of this research?
• What specific problem does the research address and why is it important?
• How do these results relate to my research interests or to other works which I have read?
• Anatomy of a Scholarly Article (Interactive tutorial) Andreas Orphanides, North Carolina State University Libraries, 2009
• How to Read an Article in a Scholarly Journal (Research Guide) Cayuga Community College Library, 2016
• How To Read a Scholarly Journal Article (YouTube Video) Tim Lockman, Kishwaukee College Library, 2012.
• How To Read a Scientific Paper (Interactive tutorial) Michael Fosmire, Purdue University Libraries, 2013. PDF
• How to Read a Scientific Paper (Online article) Science Buddies, 2012
• How to Read a Scientific Research Paper (Article) Durbin Jr., C. G. Respiratory Care, 2009
• The Illusion of Certainty and the Certainty of Illusion: A Caution when Reading Scientific Articles (Article) T. A. Lang, International Journal of Occupational and Environmental Medicine, 2011,
• Infographic: How to Read Scientific Papers Natalia Rodriguez, Elsevier, 2015
• Library Research Methods: Read & Evaluate Culinary Institute of America Library, 2016
• << Previous: Publication Types and Bias
• Next: Impact Factors and Citation Counts >>
• Last Updated: Mar 8, 2024 1:17 PM
• URL: https://libguides.usc.edu/evaluate

Choose Your Test

• Search Blogs By Category
• College Admissions
• AP and IB Exams
• GPA and Coursework

113 Great Research Paper Topics

General Education

One of the hardest parts of writing a research paper can be just finding a good topic to write about. Fortunately we've done the hard work for you and have compiled a list of 113 interesting research paper topics. They've been organized into ten categories and cover a wide range of subjects so you can easily find the best topic for you.

In addition to the list of good research topics, we've included advice on what makes a good research paper topic and how you can use your topic to start writing a great paper.

What Makes a Good Research Paper Topic?

Not all research paper topics are created equal, and you want to make sure you choose a great topic before you start writing. Below are the three most important factors to consider to make sure you choose the best research paper topics.

#1: It's Something You're Interested In

A paper is always easier to write if you're interested in the topic, and you'll be more motivated to do in-depth research and write a paper that really covers the entire subject. Even if a certain research paper topic is getting a lot of buzz right now or other people seem interested in writing about it, don't feel tempted to make it your topic unless you genuinely have some sort of interest in it as well.

#2: There's Enough Information to Write a Paper

Even if you come up with the absolute best research paper topic and you're so excited to write about it, you won't be able to produce a good paper if there isn't enough research about the topic. This can happen for very specific or specialized topics, as well as topics that are too new to have enough research done on them at the moment. Easy research paper topics will always be topics with enough information to write a full-length paper.

Trying to write a research paper on a topic that doesn't have much research on it is incredibly hard, so before you decide on a topic, do a bit of preliminary searching and make sure you'll have all the information you need to write your paper.

#3: It Fits Your Teacher's Guidelines

Don't get so carried away looking at lists of research paper topics that you forget any requirements or restrictions your teacher may have put on research topic ideas. If you're writing a research paper on a health-related topic, deciding to write about the impact of rap on the music scene probably won't be allowed, but there may be some sort of leeway. For example, if you're really interested in current events but your teacher wants you to write a research paper on a history topic, you may be able to choose a topic that fits both categories, like exploring the relationship between the US and North Korea. No matter what, always get your research paper topic approved by your teacher first before you begin writing.

113 Good Research Paper Topics

Below are 113 good research topics to help you get you started on your paper. We've organized them into ten categories to make it easier to find the type of research paper topics you're looking for.

Arts/Culture

• Discuss the main differences in art from the Italian Renaissance and the Northern Renaissance .
• Analyze the impact a famous artist had on the world.
• How is sexism portrayed in different types of media (music, film, video games, etc.)? Has the amount/type of sexism changed over the years?
• How has the music of slaves brought over from Africa shaped modern American music?
• How has rap music evolved in the past decade?
• How has the portrayal of minorities in the media changed?

Current Events

• What have been the impacts of China's one child policy?
• How have the goals of feminists changed over the decades?
• How has the Trump presidency changed international relations?
• Analyze the history of the relationship between the United States and North Korea.
• What factors contributed to the current decline in the rate of unemployment?
• What have been the impacts of states which have increased their minimum wage?
• How do US immigration laws compare to immigration laws of other countries?
• How have the US's immigration laws changed in the past few years/decades?
• How has the Black Lives Matter movement affected discussions and view about racism in the US?
• What impact has the Affordable Care Act had on healthcare in the US?
• What factors contributed to the UK deciding to leave the EU (Brexit)?
• What factors contributed to China becoming an economic power?
• Discuss the history of Bitcoin or other cryptocurrencies  (some of which tokenize the S&P 500 Index on the blockchain) .
• Do students in schools that eliminate grades do better in college and their careers?
• Do students from wealthier backgrounds score higher on standardized tests?
• Do students who receive free meals at school get higher grades compared to when they weren't receiving a free meal?
• Do students who attend charter schools score higher on standardized tests than students in public schools?
• Do students learn better in same-sex classrooms?
• How does giving each student access to an iPad or laptop affect their studies?
• What are the benefits and drawbacks of the Montessori Method ?
• Do children who attend preschool do better in school later on?
• What was the impact of the No Child Left Behind act?
• How does the US education system compare to education systems in other countries?
• What impact does mandatory physical education classes have on students' health?
• Which methods are most effective at reducing bullying in schools?
• Do homeschoolers who attend college do as well as students who attended traditional schools?
• Does offering tenure increase or decrease quality of teaching?
• How does college debt affect future life choices of students?
• Should graduate students be able to form unions?

• What are different ways to lower gun-related deaths in the US?
• How and why have divorce rates changed over time?
• Is affirmative action still necessary in education and/or the workplace?
• Should physician-assisted suicide be legal?
• How has stem cell research impacted the medical field?
• How can human trafficking be reduced in the United States/world?
• Should people be able to donate organs in exchange for money?
• Which types of juvenile punishment have proven most effective at preventing future crimes?
• Has the increase in US airport security made passengers safer?
• Analyze the immigration policies of certain countries and how they are similar and different from one another.
• Several states have legalized recreational marijuana. What positive and negative impacts have they experienced as a result?
• Do tariffs increase the number of domestic jobs?
• Which prison reforms have proven most effective?
• Should governments be able to censor certain information on the internet?
• Which methods/programs have been most effective at reducing teen pregnancy?
• What are the benefits and drawbacks of the Keto diet?
• How effective are different exercise regimes for losing weight and maintaining weight loss?
• How do the healthcare plans of various countries differ from each other?
• What are the most effective ways to treat depression ?
• What are the pros and cons of genetically modified foods?
• Which methods are most effective for improving memory?
• What can be done to lower healthcare costs in the US?
• What factors contributed to the current opioid crisis?
• Analyze the history and impact of the HIV/AIDS epidemic .
• Are low-carbohydrate or low-fat diets more effective for weight loss?
• How much exercise should the average adult be getting each week?
• Which methods are most effective to get parents to vaccinate their children?
• What are the pros and cons of clean needle programs?
• How does stress affect the body?
• Discuss the history of the conflict between Israel and the Palestinians.
• What were the causes and effects of the Salem Witch Trials?
• Who was responsible for the Iran-Contra situation?
• How has New Orleans and the government's response to natural disasters changed since Hurricane Katrina?
• What events led to the fall of the Roman Empire?
• What were the impacts of British rule in India ?
• Was the atomic bombing of Hiroshima and Nagasaki necessary?
• What were the successes and failures of the women's suffrage movement in the United States?
• What were the causes of the Civil War?
• How did Abraham Lincoln's assassination impact the country and reconstruction after the Civil War?
• Which factors contributed to the colonies winning the American Revolution?
• What caused Hitler's rise to power?
• Discuss how a specific invention impacted history.
• What led to Cleopatra's fall as ruler of Egypt?
• How has Japan changed and evolved over the centuries?
• What were the causes of the Rwandan genocide ?

• Why did Martin Luther decide to split with the Catholic Church?
• Analyze the history and impact of a well-known cult (Jonestown, Manson family, etc.)
• How did the sexual abuse scandal impact how people view the Catholic Church?
• How has the Catholic church's power changed over the past decades/centuries?
• What are the causes behind the rise in atheism/ agnosticism in the United States?
• What were the influences in Siddhartha's life resulted in him becoming the Buddha?
• How has media portrayal of Islam/Muslims changed since September 11th?

Science/Environment

• How has the earth's climate changed in the past few decades?
• How has the use and elimination of DDT affected bird populations in the US?
• Analyze how the number and severity of natural disasters have increased in the past few decades.
• Analyze deforestation rates in a certain area or globally over a period of time.
• How have past oil spills changed regulations and cleanup methods?
• How has the Flint water crisis changed water regulation safety?
• What are the pros and cons of fracking?
• What impact has the Paris Climate Agreement had so far?
• What have NASA's biggest successes and failures been?
• How can we improve access to clean water around the world?
• Does ecotourism actually have a positive impact on the environment?
• Should the US rely on nuclear energy more?
• What can be done to save amphibian species currently at risk of extinction?
• What impact has climate change had on coral reefs?
• How are black holes created?
• Are teens who spend more time on social media more likely to suffer anxiety and/or depression?
• How will the loss of net neutrality affect internet users?
• Analyze the history and progress of self-driving vehicles.
• How has the use of drones changed surveillance and warfare methods?
• Has social media made people more or less connected?
• What progress has currently been made with artificial intelligence ?
• Do smartphones increase or decrease workplace productivity?
• What are the most effective ways to use technology in the classroom?
• How is Google search affecting our intelligence?
• When is the best age for a child to begin owning a smartphone?
• Has frequent texting reduced teen literacy rates?

How to Write a Great Research Paper

Even great research paper topics won't give you a great research paper if you don't hone your topic before and during the writing process. Follow these three tips to turn good research paper topics into great papers.

#1: Figure Out Your Thesis Early

Before you start writing a single word of your paper, you first need to know what your thesis will be. Your thesis is a statement that explains what you intend to prove/show in your paper. Every sentence in your research paper will relate back to your thesis, so you don't want to start writing without it!

As some examples, if you're writing a research paper on if students learn better in same-sex classrooms, your thesis might be "Research has shown that elementary-age students in same-sex classrooms score higher on standardized tests and report feeling more comfortable in the classroom."

If you're writing a paper on the causes of the Civil War, your thesis might be "While the dispute between the North and South over slavery is the most well-known cause of the Civil War, other key causes include differences in the economies of the North and South, states' rights, and territorial expansion."

#2: Back Every Statement Up With Research

Remember, this is a research paper you're writing, so you'll need to use lots of research to make your points. Every statement you give must be backed up with research, properly cited the way your teacher requested. You're allowed to include opinions of your own, but they must also be supported by the research you give.

#3: Do Your Research Before You Begin Writing

You don't want to start writing your research paper and then learn that there isn't enough research to back up the points you're making, or, even worse, that the research contradicts the points you're trying to make!

Get most of your research on your good research topics done before you begin writing. Then use the research you've collected to create a rough outline of what your paper will cover and the key points you're going to make. This will help keep your paper clear and organized, and it'll ensure you have enough research to produce a strong paper.

What's Next?

Are you also learning about dynamic equilibrium in your science class? We break this sometimes tricky concept down so it's easy to understand in our complete guide to dynamic equilibrium .

Thinking about becoming a nurse practitioner? Nurse practitioners have one of the fastest growing careers in the country, and we have all the information you need to know about what to expect from nurse practitioner school .

Want to know the fastest and easiest ways to convert between Fahrenheit and Celsius? We've got you covered! Check out our guide to the best ways to convert Celsius to Fahrenheit (or vice versa).

These recommendations are based solely on our knowledge and experience. If you purchase an item through one of our links, PrepScholar may receive a commission.

Trending Now

How to Get Into Harvard and the Ivy League

How to Get a Perfect 4.0 GPA

How to Write an Amazing College Essay

What Exactly Are Colleges Looking For?

ACT vs. SAT: Which Test Should You Take?

When should you take the SAT or ACT?

Get Your Free

Find Your Target SAT Score

Free Complete Official SAT Practice Tests

How to Get a Perfect SAT Score, by an Expert Full Scorer

Score 800 on SAT Math

Score 800 on SAT Reading and Writing

How to Improve Your Low SAT Score

Score 600 on SAT Math

Score 600 on SAT Reading and Writing

Find Your Target ACT Score

Complete Official Free ACT Practice Tests

How to Get a Perfect ACT Score, by a 36 Full Scorer

Get a 36 on ACT English

Get a 36 on ACT Math

Get a 36 on ACT Reading

Get a 36 on ACT Science

How to Improve Your Low ACT Score

Get a 24 on ACT English

Get a 24 on ACT Math

Get a 24 on ACT Reading

Get a 24 on ACT Science

Stay Informed

Get the latest articles and test prep tips!

Christine graduated from Michigan State University with degrees in Environmental Biology and Geography and received her Master's from Duke University. In high school she scored in the 99th percentile on the SAT and was named a National Merit Finalist. She has taught English and biology in several countries.

Ask a Question Below

Have any questions about this article or other topics? Ask below and we'll reply!

Library Data Resources for Social Data Analytics

• Introduction
• Plan your research project
• Find social science data
• Analyze data

Introduction: Find peer-reviewed articles

Key databases for sociology, annual reviews, business and entrepreneurship, political/ policy related, public health, general social sciences databases, google scholar.

• Scholarly resources: books and e-books
• Writing & Citing

Social Sciences Librarian

A key concept in conducting library research is  searching subject databases that collect peer-reviewed articles in your discipline.  You can find many sociology-related articles using general article search strategies, like BruKnow or Google Scholar. But if you want to target a specific topic area, find a more diverse range of sources and perspectives that may be difficult to spot in general-interest searches, or learn about the most up-to-date ideas in your field, you'll want to dive into the specialized world of subject databases.

Below, you will find some of the most useful sociology databases that will lead you to peer-reviewed articles.

Provides access to more than 12 million academic journal articles, books, and primary sources in 75 disciplines.

Provides image and full text online access to back issues of selected scholarly journals in history, economics, political science, demography, mathematics and other fields of the humanities and social sciences. Consult the online tables of contents for holdings, as coverage varies for each title.  If you are experiencing printing problems from JSTOR while using a Macintosh computer, please download the most recent version of Adobe Reader. http://www.adobe.com/products/acrobat/readstep2.html.  If you need assistance, please contact the CIS Help Desk or [email protected].

Here is a broader collection of social science holdings with a strong Sociology focus

An Annual Review provides a roundup of recent key research in a field

• Annual Review of Sociology
• Annual Review of Political Science
• Annual Review of Public Health
• Annual Review of Financial Economics
• Annual Review of Anthropology

Business, Entrepreneurship & Related Sources

See the Business and Entrepreneurship Resources guide for secondary resources related to organizations, business, entrepreneurship, and finance.

  Encycopedia of Social Measurement    The Encyclopedia of Social Measurement captures the data, techniques, theories, designs, applications, histories, and implications of assigning numerical values to social phenomena. Provides information on transdisciplinary descriptions of quantitative and qualitative techniques, measurement, sampling, and statistical methods. Covering all core social science disciplines, the 300+ articles of the Encyclopedia of Social Measurement not only present a comprehensive summary of observational frameworks and mathematical models, but also offer tools, background information, qualitative methods, and guidelines for structuring the research process.

• << Previous: Analyze data
• Next: Scholarly resources: books and e-books >>
• Last Updated: Sep 4, 2024 11:13 AM
• URL: https://libguides.brown.edu/sda

Brown University Library  |  Providence, RI 02912  |  (401) 863-2165  |  Contact  |  Comments  |  Library Feedback  |  Site Map

Library Intranet

Journal Article Reporting Standards (JARS)

APA Style Journal Article Reporting Standards offer guidance on what information should be included in all manuscript sections for quantitative, qualitative, and mixed methods research and include how to best discuss race, ethnicity, and culture.

Introducing Journal Article Reporting Standards for Race, Ethnicity, and Culture (JARS–REC)

JARS–REC were created to develop best practices related to the manner in which race, ethnicity, and culture are discussed within scientific manuscripts in psychological science.

Quantitative research

Use JARS–Quant when you collect your study data in numerical form or report them through statistical analyses.

Qualitative research

Use JARS–Qual when you collect your study data in the form of natural language and expression.

Mixed methods research

Use JARS–Mixed when your study combines both quantitative and qualitative methods.

Race, ethnicity, culture

Use JARS–REC for all studies for guidance on how to discuss race, ethnicity, and culture.

What are APA Style JARS?

APA Style Journal Article Reporting Standards (APA Style Jars ) are a set of standards designed for journal authors, reviewers, and editors to enhance scientific rigor in peer-reviewed journal articles. Educators and students can use APA Style JARS as teaching and learning tools for conducting high quality research and determining what information to report in scholarly papers.

The standards include information on what should be included in all manuscript sections for:

• Quantitative research ( Jars –Quant)
• Qualitative research ( Jars –Qual)
• Mixed methods research ( Jars –Mixed)

Additionally, the APA Style Journal Article Reporting Standards for Race, Ethnicity, and Culture ( Jars – Rec ) provide guidance on how to discuss race, ethnicity, and culture in scientific manuscripts. Jars – Rec should be applied to all research, whether it is quantitative, qualitative, or mixed methods.

• Race, Ethnicity, and Culture ( Jars – Rec )

Using these standards will make your research clearer and more accurate as well as more transparent for readers. For quantitative research, using the standards will increase the reproducibility of science. For qualitative research, using the standards will increase the methodological integrity of research.

Jars –Quant should be used in research where findings are reported numerically (quantitative research). Jars –Qual should be used in research where findings are reported using nonnumerical descriptive data (qualitative research). Jars –Mixed should be applied to research that includes both quantitative and qualitative research (mixed methods research). JARS–REC should be applied to all research, whether it is quantitative, qualitative, or mixed methods.

For more information on APA Style JARS:

• Read Editorial: Journal Article Reporting Standards
• View an infographic (PDF, 453KB) to learn about the benefits of JARS and how they are relevant to you
• Listen to a podcast with Drs. Harris Cooper and David Frost discussing JARS and implications for research in psychology

Many aspects of research methodology warrant a close look, and journal editors can promote better methods if we encourage authors to take responsibility to report their work in clear, understandable ways. —Nelson Cowan, Editor, Journal of Experimental Psychology: General

Read more testimonials

Watch a video about JARS

This content is disabled due to your privacy settings. To re-enable, please adjust your cookie preferences.

This video describes and discusses the updated APA Style Journal Article Reporting Standards.

Related products

Reporting Qualitative Research in Psychology

Journal article reporting standards for qualitative research

Reporting Quantitative Research in Psychology

Journal article reporting standards for quantitative research

Publication Manual, 7th Edition

The official source for writing papers and creating references in seventh edition APA Style

Jars resources

• History of APA’s journal article reporting standards
• APA Style JARS supplemental glossary
• Supplemental resource on the ethic of transparency in JARS
• Frequently asked questions
• JARS-Quant Decision Flowchart (PDF, 98KB)
• JARS-Quant Participant Flowchart (PDF, 98KB)

Jars articles

• Jars –Quant article
• Jars –Qual / Mixed article
• Jars – rec executive summary

Questions / feedback

Email an APA Style Expert if you have questions, feedback, or suggestions for modules to be included in future JARS updates.

APA resources

• APA Databases and Electronic Resources
• APA Journals
• Journal Author Resource Center
• Education and Career
• Psychological Science
• Open Science at APA
• How to Review a Manuscript

From the APA Style blog

Introducing APA Style Journal Article Reporting Standards for Race, Ethnicity, and Culture

These standards are for all authors, reviewers, and editors seeking to improve manuscript quality by encouraging more racially and ethnically conscious and culturally responsive journal reporting standards for empirical studies in psychological science.

APA Style JARS for high school students

In this post, we provide an overview of APA Style JARS and resources that can be shared with high school students who want to learn more about effective communication in scholarly research.

Happy 2022, APA Stylers!

This blog post is dedicated to our awesome APA Style users. You can use the many resources on our website to help you master APA Style and improve your scholarly writing.

APA Style JARS on the EQUATOR Network

The APA Style Journal Article Reporting Standards (APA Style JARS) have been added to the EQUATOR Network. The network aims to promote accuracy and quality in reporting of research.

APA Style JARS: Resources for instructors and students

APA Style Journal Article Reporting Standards (APA Style JARS) are a set of guidelines for papers reporting quantitative, qualitative, and mixed methods research that can be used by instructors, students, and all others reading and writing research papers.

Commonwealth Honors College: Getting Started With Library Research

• Discovery Search

Why Databases?

Peer reviewed/refereed/scholarly articles, best databases for starting education research, find databases by subject and format: databases a-z list, find databases by subject or topic: research guides, what if the article i want isn't available full-text, google scholar, know the journal name of the article you want try publication finder.

• Print Materials
• More Formats (Videos, Data, Etc.)
• Interdisciplinary Topics
• Borrowing from Other Libraries
• Citation Management
• Reservable Library Spaces
• Library Workshops and Events
• Getting Help!

Databases are collections of information. We purchase access to several databases that contain journals and magazines where you can find articles for your research.

There are two types of databases for articles:

Subject-specific: These databases gather articles from journals about specific disciplines or topics, such as Education or Art or Psychology.

• Good for: Finding scholarly articles on very specific topics

Multidisciplinary: These databases gather articles from across multiple disciplines. It could be a database that covers a wide variety of social sciences or it could be a database that covers a wide variety across the arts, humanities, social sciences and sciences. Using a subject-specific database often means you can search for very specific topics and find materials.

• Good for: Finding scholarly articles on your topic from a variety of perspectives from different disciplines

Articles that are peer-reviewed can also be referred to as  peer-reviewed, refereed or scholarly articles.

Scholarly articles are written by researchers or experts in a field to share the results of their original research or analysis with other researchers, experts and students. These articles go through a process known as "peer review" where the article is reviewed by a group of experts in the field and revised based on peer feedback before being accepted and published by a journal.

This short video further explains what peer review is and why it's important.

• Video: Peer Review

These databases are examples of good subject-specific databases for researching the disciplines of Art, Education, and Psychology:

Education journal articles (EJ references) and ERIC documents (ED references), 1967-present. EDs before 1997 are requestable using the Microforms Request page and usable in the Microforms Vewing Room in the LC.

A free version of ERIC is available for all to use at this link: https://eric.ed.gov/ .

Available on campus to all, or off-campus to UMass Amherst students, staff and faculty with an UMass Amherst IT NetID (user name) and password.

These are examples of multidisciplinary databases that also have a broader focus. Social Science Premium Collection  covers multiple disciplines in the social sciences and Scopus has coverage in the arts, humanities, social sciences and sciences. With Scopus, you can sort by citation to see highly cited articles.

• Scopus This link opens in a new window Scopus is an indexing and abstracting database of peer-reviewed scholarly content covering the sciences, social sciences, and arts & humanities, comparable to the Web of Science. Scopus allows for the discovery, tracking, and analysis of scholarship that includes: journal articles, conference proceedings, trade magazines, book series, books and book chapters, and patents. Use Scopus to: • Search for documents by topic, title, author, or institutional affiliation • Perform citation searches and establish citation alerts • Export citations to reference management systems • View impact metrics for authors and journals • Integrate Scopus content with ORCID profiles more... less... Available on campus to all, or off-campus to UMass Amherst students, staff and faculty with an UMass Amherst IT NetID (user name) and password.

We have more than 600 databases on a wide variety of topics. The spectrum ranges from databases that have a very specific topic to databases that are multidisciplinary.

The easiest way to find databases with articles on your research topic is to use the Databases A-Z List. Use the link below to go the list.

You can use the following filters to find databases based on subject and format:

• Click on the Subjects filter to narrow down to a specific subject. If you select Multidisciplinary , you will get databases that cover a wide variety of publications.
• Click on the Types filter and select the Articles  filter. This narrows down the list to databases with articles (abstract only and full-text).
• Finally, click Search .

• You can select multiple subjects. Once you've picked one subject, you can go back and select another to add.
• If you use the filters, make sure to click on Clear Filters  before switching to another subject and/or format.
• Try exploring different subjects to find databases that have other discipline perspective on your topic. For instance, you might want to explore psychology databases if you're researching the effects of a specific learning theory.
• If there's a database you want to bookmark, make sure to bookmark the link from the Databases A-Z list.
• Databases A-Z List of databases by subject and type.

Library staff at the UMass Libraries have developed research guides by subjects, topics and collections. You can look at various guides and see what resources librarians recommend for those subjects, which includes databases where you can find articles.

• UMass Amherst Libraries Research Guides

If the article that you want doesn't have full-text available, look for this icon in the result for the article and click on it:

This will search our other databases to see if it's available full-text. You'll go to a page that may list several of the options if they are available:

Google Scholar searches scholarly literature across many topics. However, we don't know what it searches - you can't tell if it's a comprehensive search of the literature. The benefit of using library databases is that you can see where the information in the database is from, such as a list of publications.

Use Our Google Scholar Link!

You want to use the Google Scholar link from the Databases A-Z list or use the link below (and use that link if you want a bookmark!)

This will allow you to search Google Scholar and if the article is in one our databases, you'll either see a link to the article on the right and/or you will see UMass Check for Full Text . The check for full text will do the same as the UMass icon described above.

Google Scholar Search Tips

• You can then limit the search by exact phrases, exclude specific words, or select where the words searched occur (anywhere or just the title). You can also search by the author, journal and/or specific date ranges.
• Most of the article search tips below will work for Google Scholar!
• Google Scholar This link opens in a new window Use to access many UMass online journal subscriptions. more... less... Available on campus to all, or off-campus to UMass Amherst students, staff and faculty with an UMass Amherst IT NetID (user name) and password. You can access Google Scholar with UMLinks buttons from outside the UMass Amherst IP range ("off campus") by two methods: 1. Access Google Scholar through the Library web site by using this link. 2. Go to generic Google Scholar. a. Click on "Settings." b. Click on Library links. c. Type in "University of Massachusetts" or "UMass Amherst" (or a few other variations). d. Check "University of Massachusetts Amherst - UMass Check for Full Text" and Save. e. You will be asked to authenticate somewhere along the way to full text.

If you know the name of the journal of the article that you want, you can use Publication Finder to see if we have electronic access to the journal. You can search for the name of the publication and limit by publication type.

How To Search

• If you are getting too many results, you may want to change Contains to Exact Match or Begins With to narrow the results down.
• You can also switch from Title to ISSN and search by the ISSN for the journal if you have it. You can often find the ISSN on the publisher's page for the journal. This is helpful for journals with frequently used words in the title, such as Journal or Education .
• If you see Full Text Delay , this means that there are only abstracts available for the specified number of years.

• Once you've determined that the date is available, click on the name of the database. This will bring you to the details for the publication.
• Usually there is some way to browse by the year (often on the right or in a drop-down field in a bar under the publication's name).
• There is often a link to click to search within that publication or sometimes a search bar to immediately search within the publication.
• Publication Finder Search PubFinder to see if we have electronic access to a publication by name or ISSN.
• << Previous: Discovery Search
• Next: Print Materials >>
• Last Updated: Sep 4, 2024 12:05 PM
• URL: https://guides.library.umass.edu/getstartedchc

© 2022 University of Massachusetts Amherst • Site Policies • Accessibility

Research Paper Topics for 2024: Explore Ideas Across Various Fields

When you start writing a research paper, it’s like diving into a big pool of exploration and analysis. A good research paper goes beyond just gathering facts. It’s more about exploring a topic, asking the right questions, and coming up with thoughtful answers. Whether you're looking at historical events, scientific discoveries, or cultural trends, the trick is to find interesting research topics that catch your interest and keep you motivated throughout the process.

This article is here to help with that sometimes tricky job of picking a topic. We’ll cover a variety of interesting research topics from different areas, making it easier for you to find one that not only fits your assignment but also grabs your attention.

But let’s be honest, picking the right topic isn’t always easy. If you’re still unsure after reading this article, EssayService is a great place to turn for help, whether you need assistance choosing a topic or writing the entire paper.

How to Pick a Topic for a Research Paper

Choosing the right topic can make or break your research paper. Here's how to make it easier:

• Start with your interests: Pick a few areas or subjects that genuinely interest you. Narrow it down to the one that excites you the most. If you’re interested, it’ll show in your writing.
• Check for resources: Before committing, do a quick search to ensure there are enough references available. You’ll want a topic that’s well-discussed so you have plenty of material to work with.
• Stick to guidelines: Make sure your topic fits within any guidelines your teacher has set. Whether it's avoiding certain subjects or meeting specific requirements, this step is crucial for getting your paper off to a good start.

If you’re looking for easy research paper topics, keep these tips in mind to ensure you choose one that’s both manageable and engaging.

What Are Good Research Topics?

Choosing a successful research topic isn’t just about what sounds interesting — it’s about finding a topic that will help you produce a strong, insightful paper. Good research topic ideas should tick a few key boxes to ensure they’re both impactful and manageable.

Best Research Paper Topics for 2024

In 2024, new challenges and innovations are shaping the world around us, making it an exciting time to dive into research. Here are 15 detailed and highly relevant topics that will keep your paper ahead of the curve:

• The impact of remote work on urban development in major U.S. cities.
• Ethical implications of AI-driven decision-making in healthcare.
• The role of social media algorithms in shaping public opinion during elections.
• Effects of climate change on global food security and crop yields.
• The influence of blockchain technology on supply chain transparency.
• Mental health outcomes related to long-term social media use among teenagers.
• Renewable energy adoption in emerging economies and its impact on local communities.
• The rise of electric vehicles and its effect on traditional automotive industries.
• Privacy concerns surrounding the use of biometric data in consumer devices.
• The evolution of cybersecurity threats in the age of quantum computing.
• Gender disparities in STEM education and their long-term effects on the workforce.
• The economic impact of climate migration on coastal regions.
• Implications of CRISPR technology in human genetic modification.
• The effectiveness of universal basic income trials in reducing poverty.
• The role of telemedicine in improving access to healthcare in rural areas.

College Research Paper Topics

These topics explore some of the most relevant and intriguing issues facing college students today, offering plenty of angles to explore in your research:

• How student loan debt shapes career paths and financial stability after graduation.
• Comparing online learning to traditional classrooms: What works best for today’s college students?
• Social media’s influence on mental health and academic success among college students.
• Diversity and inclusion: How initiatives are changing campus life and student experiences.
• University sustainability efforts: How climate change is driving new campus policies.
• The rise of esports: Transforming college athletics and student engagement.
• Campus housing: How living arrangements affect academic success and student retention.
• Balancing part-time jobs with academics: The impact on college students’ grades and well-being.
• Navigating controversial topics: The importance of academic freedom in college debates.
• Digital vs. traditional libraries: How technology is reshaping student research habits.
• Study abroad programs: Enhancing global awareness and boosting future career opportunities.
• Evaluating campus mental health services: Are they meeting students’ needs?
• Fraternities and sororities: Examining their influence on college culture and student life.
• Free college tuition: Exploring the economic and social outcomes in different countries.
• Standardized testing: How it’s affecting college admissions and the diversity of student bodies.

Research Paper Topics By Subject

Choosing a good research topic that aligns with your academic focus can make your work more relevant and engaging. Below, you’ll find topics organized by subject to help you get started.

Research Paper Topics on Health

Health is a dynamic field with ongoing developments and challenges, making it a rich area for research. These topics cover a range of health-related issues, from public health policies to advancements in medical technology:

• How COVID-19 has changed the approach to mental health care.
• Adoption rates of telemedicine among different age groups.
• Antibiotic-resistant bacteria: Exploring new treatment options.
• Barriers to healthcare access in low-income neighborhoods.
• Ethical dilemmas in using genetic testing for personalized treatments.
• Success rates of mental health programs in high schools.
• Comparing dietary patterns in managing type 2 diabetes across cultures.
• Teen vaping trends and their connection to lung health issues.
• Strategies for supporting healthcare needs in rapidly aging populations.
• Tracking climate-related health issues in coastal communities.
• Innovations in vaccine development for emerging diseases.
• Social isolation during pandemics and its link to anxiety disorders.
• Recent changes in U.S. healthcare laws and their influence on patient choices.
• Exploring how traditional beliefs shape approaches to medical treatment.
• Evaluating progress in global vaccination campaigns against childhood diseases.

Research Paper Topics on Medicine

Medicine is a vast field with plenty of areas to explore. Here are some specific topics that focus on medical advancements, practices, and challenges:

• New techniques in minimally invasive surgery for heart conditions.
• Developments in gene therapy for treating inherited diseases.
• Challenges in diagnosing and treating rare diseases.
• The role of AI in improving diagnostic accuracy in radiology.
• Progress in personalized cancer treatments based on genetic profiling.
• The rise of antibiotic alternatives in treating infections.
• Stem cell research advancements for spinal cord injuries.
• Managing chronic pain: Exploring non-opioid treatment options.
• Trends in telemedicine for rural healthcare delivery.
• Breakthroughs in vaccine technology for emerging viruses.
• Long-term outcomes of organ transplants in pediatric patients.
• Advances in robotic surgery and their impact on patient recovery.
• New approaches to treating drug-resistant tuberculosis.
• Innovations in prenatal care and fetal surgery techniques.
• The future of regenerative medicine and tissue engineering.

Research Paper Topics on Media

Explore the ever-changing world of media with these fresh and relevant topics. Each one dives into the trends and challenges shaping how we consume and create content today.

• Analyze the impact of TikTok on modern marketing strategies.
• Investigate the role of influencers in shaping public opinion during elections.
• Explore the effects of streaming services on traditional cable TV viewership.
• Examine how social media platforms handle misinformation and its consequences.
• Study the rise of podcasts and their influence on news consumption.
• Compare the portrayal of mental health in TV shows across different cultures.
• Track the evolution of digital journalism and its impact on print media.
• Look into the ethics of deepfake technology in video production.
• Research the effects of binge-watching on viewer behavior and mental health.
• Explore the relationship between video game streaming and the gaming industry.
• Analyze the shift from traditional news outlets to social media for breaking news.
• Investigate how algorithms curate personalized content and influence user behavior.
• Study the changing landscape of advertising in the age of ad-blockers.
• Examine the role of memes in political discourse and cultural commentary.
• Explore the use of virtual reality in media and entertainment.

Research Paper Topics on Politics

Politics is a field that’s constantly evolving, with new issues and debates emerging all the time. Whether you're interested in global dynamics, domestic policies, or the role of technology in politics, there’s no shortage of interesting topics to explore:

• How social media is influencing voter behavior in recent elections.
• The rise and impact of grassroots movements on political change.
• Fake news and its role in shaping public perception of political events.
• The effects of immigration policies on relationships between countries.
• Populism’s growth in global politics and what it means for the future.
• How economic inequality contributes to political instability.
• The power of political lobbying in creating and shaping laws.
• Challenges faced by democracies under authoritarian regimes.
• Youth activism and its growing influence in modern politics.
• How climate change policies are impacting national security.
• The role of technology in improving election security and voter turnout.
• Government approval ratings and their connection to pandemic responses.
• Influence of international organizations on a country’s domestic policies.
• Shifts in global trade agreements and their effects on international relations.
• The impact of gerrymandering on election results and fairness.

Research Paper Ideas on Technology

Technology is rapidly transforming our world, offering endless opportunities for research. Here are some intriguing ideas to explore:

• The ethics of artificial intelligence in decision-making processes.
• How blockchain technology is revolutionizing financial transactions.
• The role of 5G networks in shaping the future of communication.
• Cybersecurity challenges in the era of smart homes and IoT devices.
• The environmental impact of cryptocurrency mining.
• Virtual reality’s influence on education and training programs.
• How autonomous vehicles are changing urban planning and infrastructure.
• The potential of quantum computing in solving complex global problems.
• Social media algorithms and their impact on public discourse.
• The digital divide: Access to technology in rural versus urban areas.
• How wearable tech is transforming personal health management.
• The implications of deepfake technology in media and politics.
• The future of remote work and its long-term effects on productivity.
• Advancements in drone technology for disaster management and rescue operations.
• The role of big data in personalizing online shopping experiences.

Research Topic Ideas on Culture

Whether you’re interested in examining specific cultural practices or looking at how modern trends reshape traditional customs, these research topics will provide you with a focused and detailed starting point:

• Adoption of traditional Japanese tea ceremonies in contemporary urban settings.
• Practices of food preservation among Inuit communities in the Arctic.
• The revival of Celtic languages in Wales and Ireland through education programs.
• Depiction of queer relationships in Netflix original series from 2015 to 2024.
• Evolution of traditional African hairstyles in Black communities across the U.S.
• Transformation of street art in Berlin post-German reunification.
• Cultural significance of Día de los Muertos celebrations in Mexican-American neighborhoods.
• Popularity of Korean skincare routines among Western beauty bloggers.
• Modern interpretations of Norse mythology in Scandinavian literature.
• Changes in wedding rituals among Indian diaspora in the UK.
• Resurgence of indigenous Australian painting techniques in contemporary art.
• Representation of disability in children’s books published in the last decade.
• Use of traditional Māori patterns in New Zealand’s fashion industry.
• Changes in burial customs in urbanized areas of Southeast Asia.
• Incorporation of First Nations symbols in Canadian public architecture.

Research Paper Topics on Math

If you're looking to explore the depth and applications of math, these research topics are both specific and engaging:

• Applications of fractal geometry in modeling natural phenomena.
• Mathematical approaches to solving complex optimization problems in logistics.
• Development of new algorithms for large-scale data encryption.
• Mathematical modeling of population dynamics in ecology.
• The use of game theory in economic decision-making processes.
• Exploring the mathematics behind machine learning algorithms.
• Advancements in numerical methods for solving partial differential equations.
• Topological data analysis and its applications in computational biology.
• Mathematical analysis of voting systems and fairness.
• The role of number theory in modern cryptography.
• Predictive models for financial markets using stochastic calculus.
• Mathematical foundations of quantum computing and quantum algorithms.
• Applications of chaos theory in weather prediction.
• Geometry of space-time in the context of general relativity.
• Mathematical techniques for analyzing big data in social networks.

Research Paper Topics on Art

Art is full of fascinating details and stories waiting to be explored. If you’re into art research, here are some research topics that might catch your interest:

• How Caravaggio used light and shadow in his religious paintings.
• The way Cubism shaped Picasso’s "Les Demoiselles d’Avignon."
• Gustav Klimt’s "The Kiss" and its ties to Viennese culture.
• Hokusai’s woodblock techniques in "The Great Wave off Kanagawa."
• Bauhaus principles that still influence graphic design today.
• Emotions and color in Mark Rothko’s abstract paintings.
• Leonora Carrington’s role in the Surrealist movement.
• Gaudí’s architectural genius in designing La Sagrada Familia.
• Industrial scenes captured in Charles Sheeler’s Precisionist art.
• Jean-Michel Basquiat’s take on graffiti and cultural identity.
• Frida Kahlo’s evolving self-portraits through her life.
• Claude Monet’s unique use of light in his Impressionist works.
• Diego Rivera’s murals as powerful political statements.
• The simplicity and impact of Donald Judd’s minimalist sculptures.
• How African art influenced Henri Matisse during his Fauvist period.

Research Topics on Sports

Sports offer a wide range of topics that are both intriguing and highly relevant. Here are some specific research ideas to consider if you're looking to explore the world of sports:

• The biomechanics behind sprinting techniques in elite athletes.
• The psychological effects of team sports on adolescent development.
• Injury prevention strategies in professional football (soccer).
• The impact of altitude training on endurance performance in marathon runners.
• Gender equity in sports: The evolution of women’s participation in the Olympics.
• The role of nutrition in recovery and performance for endurance athletes.
• How advanced analytics are changing strategies in basketball.
• The effects of early specialization in youth sports on long-term athletic development.
• The influence of sports media coverage on public perceptions of athletes.
• Technology in sports: The use of wearable devices to monitor athlete performance.
• Doping scandals and their long-term impact on athletes' careers.
• Mental health challenges faced by retired professional athletes.
• The economics of hosting major sporting events like the World Cup or Olympics.
• How climate change is affecting outdoor sports events and training schedules.
• The evolution of sports science in enhancing athlete training programs.

In 2024, some of the most popular research topics include the impact of technology on sports, the psychological aspects of team dynamics, and the evolution of gender equity in athletics. 

If you’re still unsure about which topic to choose or need help with your essay, EssayService is a great option. Our research paper writing service can assist with everything from selecting the perfect topic to crafting a well-written paper, making the whole process a lot easier.

Frequently asked questions

She was flawless! first time using a website like this, I've ordered article review and i totally adored it! grammar punctuation, content - everything was on point

This writer is my go to, because whenever I need someone who I can trust my task to - I hire Joy. She wrote almost every paper for me for the last 2 years

Term paper done up to a highest standard, no revisions, perfect communication. 10s across the board!!!!!!!

I send him instructions and that's it. my paper was done 10 hours later, no stupid questions, he nailed it.

Sometimes I wonder if Michael is secretly a professor because he literally knows everything. HE DID SO WELL THAT MY PROF SHOWED MY PAPER AS AN EXAMPLE. unbelievable, many thanks

New posts to your inbox!

Stay in touch

Developing Surveys on Questionable Research Practices: Four Challenging Design Problems

• Open access
• Published: 02 September 2024

Cite this article

You have full access to this open access article

• Christian Berggren   ORCID: orcid.org/0000-0002-4233-5138 1 ,
• Bengt Gerdin   ORCID: orcid.org/0000-0001-8360-5387 2 &
• Solmaz Filiz Karabag   ORCID: orcid.org/0000-0002-3863-1073 1 , 3  

2 Altmetric

The exposure of scientific scandals and the increase of dubious research practices have generated a stream of studies on Questionable Research Practices (QRPs), such as failure to acknowledge co-authors, selective presentation of findings, or removal of data not supporting desired outcomes. In contrast to high-profile fraud cases, QRPs can be investigated using quantitative, survey-based methods. However, several design issues remain to be solved. This paper starts with a review of four problems in the QRP research: the problem of precision and prevalence, the problem of social desirability bias, the problem of incomplete coverage, and the problem of controversiality, sensitivity and missing responses. Various ways to handle these problems are discussed based on a case study of the design of a large, cross-field QRP survey in the social and medical sciences in Sweden. The paper describes the key steps in the design process, including technical and cognitive testing and repeated test versions to arrive at reliable survey items on the prevalence of QRPs and hypothesized associated factors in the organizational and normative environments. Partial solutions to the four problems are assessed, unresolved issues are discussed, and tradeoffs that resist simple solutions are articulated. The paper ends with a call for systematic comparisons of survey designs and item quality to build a much-needed cumulative knowledge trajectory in the field of integrity studies.

Similar content being viewed by others

Lies, Damned Lies, and Crafty Questionnaire Design

Design, Run, and Interpret Survey-Based Research in the Fields of Academic Integrity and Misconduct

Explore related subjects.

• Medical Ethics

Avoid common mistakes on your manuscript.

Introduction

The public revelations of research fraud and non-replicable findings (Berggren & Karabag, 2019 ; Levelt et al., 2012 ; Nosek et al., 2022 ) have created a lively interest in studying research integrity. Most studies in this field tend to focus on questionable research practices, QRPs, rather than blatant fraud, which is less common and hard to study with rigorous methods (Butler et al., 2017 ). Despite the significant contributions of this research about the incidence of QRPs in various countries and contexts, several issues still need to be addressed regarding the challenges of designing precise and valid survey instruments and achieving satisfactory response rates in this sensitive area. While studies in management (Hinkin, 1998 ; Lietz, 2010 ), behavioral sciences, psychology (Breakwell et al., 2020 ), sociology (Brenner, 2020 ), and education (Hill et al., 2022 ) have provided guidelines to design surveys, they rarely discuss how to develop, test, and use surveys targeting sensitive and controversial issues such as organizational or individual corruption (Lin & Yu, 2020 ), fraud (Lawlor et al., 2021 ), and misconduct. The aim of this study is to contribute to a systematic discussion of challenges facing survey designers in these areas and, by way of a detailed case study, highlight alternative ways to increase participation and reliability of surveys focusing on questionable research practices, scientific norms, and organizational climate.

The following section starts with a literature-based review of four important problems:

the lack of conceptual consensus and precise measurements,

the problem of social desirability bias.

the difficulty of covering both quantitative and qualitative research fields.

the problem of controversiality and sensitivity.

Section 3 presents an in-depth case study of developing and implementing a survey on QRPs in the social and medical sciences in Sweden 2018–2021, designed to target these problems. Its first results were presented in this journal (Karabag et al., 2024 ). The section also describes the development process and the survey content and highlights the general design challenges. Section 4 returns to the four problems by discussing partial solutions, difficult tradeoffs, and remaining issues.

Four Design Problems in the Study of Questionable Research Practices

Extant QRP studies have generated an impressive body of knowledge regarding the occurrence and complexities of questionable practices, their increasing trend in several academic fields, and the difficulty of mitigating them with conventional interventions such as ethics courses and espousal of integrity policies (Gopalakrishna et al., 2022 ; Karabag et al., 2024 ; Necker, 2014 ). However, investigations on the prevalence of QRPs have so far lacked systematic problem analysis. Below, four main problems are discussed.

The Problem of Conceptual Clarity and Measurement Precision

Studies of QRP prevalence in the literature exhibit high levels of questionable behaviors but also considerable variation in their estimates. This is illustrated in the examples below:

“42% hade collected more data after inspecting whether results were statistically significant… and 51% had reported an unexpected finding as though it had been hypothesized from the start (HARKing)”( Fraser et al., 2018 , p. 1) , “51 , 3% of respondents engaging frequently in at least one QRP” ( Gopalakrishna et al., 2022 , p. 1) , “…one third of the researchers stated that for the express purpose of supporting hypotheses with statistical significance they engaged in post hoc exclusion of data” ( Banks et al., 2016 , p. 10).

On a general level, QRPs constitute deviations from the responsible conduct of research, that are not severe enough to be defined as fraud and fabrication (Steneck, 2006 ). Within these borders, there is no conceptual consensus regarding specific forms of QRPs (Bruton et al., 2020 ; Xie et al., 2021 ). This has resulted in a considerable variation in prevalence estimates (Agnoli et al., 2017 ; Artino et al. Jr, 2019 ; Fiedler & Schwarz, 2016 ). Many studies emphasize the role of intentionality, implying a purpose to support a specific assertion with biased evidence (Banks et al., 2016 ). This tends to be backed by reports of malpractices in quantitative research, such as p-hacking or HARKing, where unexpected findings or results from an exploratory analysis are reported as having been predicted from the start (Andrade, 2021 ). Other QRP studies, however, build on another, often implicit conceptual definition and include practices that could instead be defined as sloppy or under-resourced research, e.g. insufficient attention to equipment, deficient supervision of junior co-workers, inadequate note-keeping of the research process, or use of inappropriate research designs (Gopalakrishna et al., 2022 ). Alternatively, those studies include behaviors such as “Fashion-determined choice of research topic”, “Instrumental and marketable approach”, and “Overselling methods, data or results” (Ravn & Sørensen, 2021 , p. 30; Vermeulen & Hartmann, 2015 ) which may be opportunistic or survivalist but not necessarily involve intentions to mislead.

To shed light on the prevalence of QRPs in different environments, the first step is to conceptualize and delimit the practices to be considered. The next step is to operationalize the conceptual approach into useful indicators and, if needed, to reformulate and reword the indicators into unambiguous, easily understood items (Hinkin, 1995 , 1998 ). The importance of careful item design has been demonstrated by Fiedler and Schwarz ( 2016 ). They show how the perceived QRP prevalence changes by adding specifications to well-known QRP items. Such specifications include: “ failing to report all dependent measures that are relevant for a finding ”, “ selectively reporting studies related to a specific finding that ‘’worked’ ” (Fiedler & Schwarz, 2016 , p. 46, italics in original ), or “collecting more data after seeing whether results were significant in order to render non-significant results significant ” (Fiedler & Schwarz, 2016 , p. 49, italics in original ). These specifications demonstrate the importance of precision in item design, the need for item tests before applications in a large-scale survey, and as the case study in Sect. 3 indicates, the value of statistically analyzing the selected items post-implementation.

The Problem of Social Desirability

Case studies of publicly exposed scientific misconduct have the advantage of explicitness and possible triangulation of sources (Berggren & Karabag, 2019 ; Huistra & Paul, 2022 ). Opinions may be contradictory, but researchers/investigators may often approach a variety of stakeholders and compare oral statements with documents and other sources (Berggren & Karabag, 2019 ). By contrast, quantitative studies of QRPs need to rely on non-public sources in the form of statements and appraisals of survey respondents for the dependent variables and for potentially associated factors such as publication pressure, job insecurity, or competitive climate.

Many QRP surveys use items that target the respondents’ personal attitudes and preferences regarding the dependent variables, indicating QRP prevalence, as well as the explanatory variables. This has the advantage that the respondents presumably know their own preferences and practices. A significant disadvantage, however, concerns social desirability, which in this context means the tendency of respondents to portray themselves, sometimes inadvertently, in more positive ways than justified by their behavior. The extent of this problem was indicated in a meta-study by Fanelli ( 2009 ), which demonstrated major differences between answers to sensitive survey questions that targeted the respondents’ own behavior and questions that focused on the behavior of their colleagues. In the case study below, the pros and cons of the latter indirect approaches are analyzed.

The Problem of Covering Both Quantitative and Qualitative Research

Studies of QRP prevalence are dominated by quantitative research approaches, where there exists a common understanding of the meaning of facts, proper procedures and scientific evidence. Several research fields, also in the social and medical sciences, include qualitative approaches — case studies, interpretive inquiries, or discourse analysis — where assessments of ‘truth’ and ‘evidence’ may be different or more complex to evaluate.

This does not mean that all qualitative endeavors are equal or that deceit—such as presenting fabricated interview quotes or referring to non-existent protocols —is accepted. However, while there are defined criteria for reporting qualitative research, such as the Consolidated Criteria for Reporting Qualitative Research (COREQ) (Tong et al., 2007 ) or the Standards for Reporting Qualitative Research (SRQR checklist) (O’Brien et al., 2014 ), the field of qualitative research encompasses a wide range of different approaches. This includes comparative case studies that offer detailed evidence to support their claims—such as the differences between British and Japanese factories (Dore, 1973 /2011)—as well as discourse analyses and interpretive studies, where the concept of ‘evidence’ is more fluid and hard to apply. The generative richness of the analysis is a key component of their quality (Flick, 2013 ). This intra-field variation makes it hard to pin down and agree upon general QRP items to capture such behaviors in qualitative research. Some researchers have tried to interpret and report qualitative research by means of quantified methods (Ravn & Sørensen, 2021 ), but so far, these attempts constitute a marginal phenomenon. Consequently, the challenges of measuring the prevalence of QRPs (or similar issues) in the variegated field of qualitative research remain largely unexplored.

The Problem of Institutional Controversiality and Personal Sensitivity

Science and academia depend on public trust for funding and executing research. This makes investigations of questionable behaviors a controversial issue for universities and may lead to institutional refusal/non-response. This resistance was experienced by the designers of a large-scale survey of norms and practices in the Dutch academia when several universities decided not to take part, referring to the potential danger of negative publicity (de Vrieze, 2021 ). A Flemish survey on academic careers encountered similar participation problems (Aubert Bonn & Pinxten, 2019 ). Another study on universities’ willingness to solicit whistleblowers for participation revealed that university officers, managers, and lawyers tend to feel obligated to protect their institution’s reputation (Byrn et al., 2016 ). Such institutional actors may resist participation to avoid the exposure of potentially negative information about their institutions and management practices, which might damage the university’s brand (Byrn et al., 2016 ; Downes, 2017 ).

QRP surveys involve sensitive and potentially intrusive questions also from a respondent’s personal perspective that can lead to a reluctance to participate and non-response behavior (Roberts & John, 2014 ; Tourangeau & Yan, 2007 ). Studies show that willingness to participate declines for surveys covering sensitive issues such as misconduct, crime, and corruption, compared to less sensitive ones like leisure activities (cf. Tourangeau et al., 2010 ). The method of survey administration—whether face-to-face, over the phone, via the web, or paper-based—can influence the perceived sensitivity and response rate (Siewert & Udani, 2016 ; Szolnoki & Hoffmann, 2013 ). In the case study below, the survey did not require any institutional support. Instead, the designers focused on minimizing the individual sensitivity problem by avoiding questions about the respondents’ personal practices. To manage this, they concentrated on their colleagues’ behaviors (see Sect. 4.2). Even if a respondent agrees to participate, they may not answer the QRP items due to insufficient knowledge about her colleagues’ practices or a lack of motivation to answer critical questions about their colleagues’ practices (Beatty & Herrmann, 2002 ; Yan & Curtin, 2010 ). Additionally, a significant time gap between observing specific QRPs in the respondent’s research environment and receiving the survey may make it difficult to recall and accurately respond to the questions. Such issues may also result in non-response problems.

Addressing the Problems: Case Study of a Cross-Field QRP Survey – Design Process, Survey Content, Design Challenges

This section presents a case study of the way these four problems were addressed in a cross-field survey intended to capture QRP prevalence and associated factors across the social and medical sciences in Sweden. The account is based on the authors’ intensive involvement in the design and analysis of the survey, including the technical and cognitive testing, and post-implementation analysis of item quality, missing responses, and open respondent comments. The theoretical background and the substantive results of the study are presented in a separate paper (Karabag et al., 2024 ). Method and language experts at Statistics Sweden, a government agency responsible for public statistics in Sweden, supported the testing procedures, the stratified respondent sampling and administered the survey roll-out.

The Survey Design Process – Repeated Testing and Prototyping

The design process included four steps of testing, revising, and prototyping, which allowed the researchers to iteratively improve the survey and plan the roll-out.

Step 1: Development of the Baseline Survey

This step involved searching the literature and creating a list of alternative constructs concerning the key concepts in the planned survey. Based on the study’s aim, the first and third authors compared these constructs and examined how they had been itemized in the literature. After two rounds of discussions, they agreed on construct formulations and relevant ways to measure them, rephrased items if deemed necessary, and designed new items in areas where the extant literature did not provide any guidance. In this way, Survey Version 1 was compiled.

Step 2: Pre-Testing by Means of a Large Convenience Sample

In the second step, this survey version was reviewed by two experts in organizational behavior at Linköping University. This review led to minor adjustments and the creation of Survey Version 2 , which was used for a major pretest. The aim was both to check the quality of individual items and to garner enough responses for a factor analysis that could be used to build a preliminary theoretical model. This dual aim required a larger sample than suggested in the literature on pretesting (Perneger et al., 2015 ). At the same time, it was essential to minimize the contamination of the planned target population in Sweden. To accomplish this, the authors used their access to a community of organization scholars to administer Survey Version 2 to 200 European management researchers.

This mass pre-testing yielded 163 responses. The data were used to form preliminary factor structures and test a structural equation model. Feedback from a few of the respondents highlighted conceptual issues and duplicated questions. Survey Version 3 was developed and prepared for detailed pretesting based on this feedback.

Step 3: Focused Pre-Testing and Technical Assessment

This step focused on the pre-testing and technical assessment. The participants in this step’s pretesting were ten researchers (six in the social sciences and four in the medical sciences) at five Swedish universities: Linköping, Uppsala, Gothenburg, Gävle, and Stockholm School of Economics. Five of those researchers mainly used qualitative research methods, two used both qualitative and quantitative methods, and three used quantitative methods. In addition, Statistics Sweden conducted a technical assessment of the survey items, focusing on wording, sequence, and response options. Footnote 1 Based on feedback from the ten pretest participants and the Statistics Sweden assessment, Survey Version 4 was developed, translated into Swedish, and reviewed by two researchers with expertise in research ethics and scientific misconduct.

It should be highlighted that Swedish academia is predominantly bilingual. While most researchers have Swedish as their mother tongue, many are more proficient in English, and a minority have limited or no knowledge of Swedish. During the design process, the two language versions were compared item by item and slightly adjusted by skilled bilingual researchers. This task was relatively straightforward since most items and concepts were derived from previously published literature in English. Notably, the Swedish versions of key terms and concepts have long been utilized within Swedish academia (see for example Berggren, 2016 ; Hasselberg, 2012 ). To secure translation quality, the language was controlled by a language expert at Statistics Sweden.

Step 4: Cognitive Interviews by Survey and Measurement Experts

Next, cognitive interviews (Willis, 2004 ) were organized with eight researchers from the social and medical sciences and conducted by an expert from Statistics Sweden (Wallenborg Likidis, 2019 ). The participants included four women and four men, ranging in age from 30 to 60. They were two doctoral students, two lecturers, and four professors, representing five different universities and colleges. Additionally, two participants had a non-Nordic background. To ensure confidentiality, no connections are provided between these characteristics and the individual participants.

An effort was made to achieve a distribution of gender, age, subject, employment, and institution. Four social science researchers primarily used qualitative research methods, while the remaining four employed qualitative and quantitative methods. Additionally, four respondents completed the Swedish version of the survey, and four completed the English version.

The respondents completed the survey in the presence of a methods expert from Statistics Sweden, who observed their entire response process. The expert noted spontaneous reactions and recorded instances where respondents hesitated or struggled to understand an item. After the survey, the expert conducted a structured interview with all eight participants, addressing details in each section of the survey, including the missive for recruiting respondents. Some respondents provided oral feedback while reading the cover letter and answering the questions, while others offered feedback during the subsequent interview.

During the cognitive interview process, the methods expert continuously communicated suggestions for improvements to the design team. A detailed test protocol confirmed that most items were sufficiently strong, although a few required minor modifications. The research team then finalized Survey Version 5 , which included both English and Swedish versions (for the complete survey, see Supplementary Material S1).

Although the test successfully captured a diverse range of participants, it would have been desirable to conduct additional tests of the English survey with more non-Nordic participants; as it stands, only one such test was conducted. Despite the participants’ different approaches to completing the survey, the estimated time to complete it was approximately 15–20 min. No significant time difference was observed between completing the survey in Swedish and English.

Design Challenges – the Dearth of an Item-Specific Public Quality Discussion

The design decision to employ survey items from the relevant literature as much as possible was motivated by a desire to increase comparability with previous studies of questionable research practices. However, this approach came with several challenges. Survey-based studies of QRPs rely on the respondents’ subjective assessments, with no possibility to compare the answers with other sources. Thus, an open discussion of survey problems would be highly valuable. However, although published studies usually present the items used in the surveys, there is seldom any analysis of the problems and tradeoffs involved when using a particular type of item or response format and meager information about item validity. Few studies, for example, contain any analysis that clarifies which items that measured the targeted variables with sufficient precision and which items that failed to do so.

Another challenge when using existing survey studies is the lack of information regarding the respondents’ free-text comments about the survey’s content and quality. This could be because the survey did not contain any open questions or because the authors of the report could not statistically analyze the answers. As seen below, however, open respondent feedback on a questionnaire involving sensitive or controversial aspects may provide important feedback regarding problems that did not surface during the pretest process, which by necessity targets much smaller samples.

Survey Content

The survey started with questions about the respondent’s current employment and research environment. It ended with background questions on the respondents’ positions and the extent of their research activity, plus space for open comments about the survey. The core content of the survey consisted of sections on the organizational climate (15 items), scientific norms (13 items), good and questionable research practices (16 items), perceptions of fairness in the academic system (4 items), motivation for conducting research (8 items), ethics training and policies (5 items); and questions on the quality of the research environment and the respondent’s perceived job security.

Sample and Response Rate

All researchers, teachers, and Ph.D. students employed at Swedish universities are registered by Statistics Sweden. To ensure balanced representation and perspectives from both large universities and smaller university colleges, the institutions were divided into three strata based on the number of researchers, teachers, and Ph.D. students: more than 1,000 individuals (7 universities and university colleges), 500–999 individuals (3 institutions), and fewer than 500 individuals (29 institutions). From these strata, Statistics Sweden randomly sampled 35%, 45%, and 50% of the relevant employees, resulting in a sample of 10,047 individuals. After coverage analysis and exclusion of wrongly included, 9,626 individuals remained.

The selected individuals received a personal postal letter with a missive in both English and Swedish informing them about the project and the survey and notifying them that they could respond on paper or online. The online version provided the option to answer in either English or Swedish. The paper version was available only in English to reduce the cost of production and posting. The missive provided the recipients with comprehensive information about the study and what their involvement would entail. It emphasized the voluntary character of participation and their right to withdraw from the survey at any time, adding: “If you do not want to answer the questions , we kindly ask you to contact us. Then you will not receive any reminders.” Sixty-three individuals used this decline option. In line with standard Statistics Sweden procedures, survey completion implied an agreement to participation and to the publication of anonymized results and indicated participants’ understanding of the terms provided (Duncan & Cheng, 2021 ). An email address was provided for respondents to request study outputs or for any other reason. The survey was open for data collection for two months, during which two reminders were sent to non-responders who had not opted out.

Once Statistics Sweden had collected the answers, they were anonymized and used to generate data files delivered to the authors. Statistics Sweden also provided anonymized information about age, gender, and type of employment of each respondent in the dataset delivered to the researchers. Of the targeted individuals, 3,295 responded, amounting to an overall response rate of 34.2%. An analysis of missing value patterns revealed that 290 of the respondents either lacked data for an entire factor or had too many missing values dispersed over several survey sections. After removing these 290 responses, we used SPSS algorithms (IBM-SPSS Statistics 27) to analyze the remaining missing values, which were randomly distributed and constituted less than 5% of the data. These values were replaced using the program’s imputation program (Madley-Dowd et al., 2019 ). The final dataset consisted of 3,005 individuals, evenly distributed between female and male respondents (53,5% vs. 46,5%) and medical and social scientists (51,3% vs. 48,5%). An overview of the sample and the response rate is provided in Table  1 , which can also be found in (Karabag et al., 2024 ). As shown in Table  1 , the proportion of male and female respondents, as well as the proportion of respondents from medical and social science, and the age distribution of the respondents compared well with the original selection frame from Statistics Sweden.

Revisiting the Four Problems. Partial Solutions and Remaining Issues

Managing the precision problem - the value of factor analyses.

As noted above, the lack of conceptual consensus and standard ways to measure QRPs has resulted in a huge variation in estimated prevalence. In the case studied here, the purpose was to investigate deviations from research integrity and not low-quality research in general. This conceptual focus implied that selected survey items regarding QRP should build on the core aspect of intention, as suggested by Banks et al. ( 2016 , p. 323): “design, analytic, or reporting practices that have been questioned because of the potential for the practice to be employed with the purpose of presenting biased evidence in favor of an assertion”. After scrutinizing the literature, five items were selected as general indicators of QRP, irrespective of the research approach (see Table  2 ).

An analysis of the survey responses indicated that the general QRP indicators worked well in terms of understandability and precision. Considering the sensitive nature of the items, features that typically yield very high rates of missing data (Fanelli, 2009 ; Tourangeau & Yan, 2007 ), our missing rates of 11–21% must be considered modest. In addition, there were a few critical comments on the item formulation in the open response section at the end of the survey (see below).

Regarding the explanatory (independent) variables, the survey was inspired by studies showing the importance of the organizational climate and the normative environment within academia (Anderson et al., 2010 ). Organizational climate can be measured in several ways; the studied survey focused on items related to a collegial versus a competitive climate. The analysis of the normative environment was inspired by the classical norms of science articulated by Robert Merton in his CUDOS framework: communism (communalism), universalism, disinterestedness, and organized skepticism (Merton, 1942 /1973). This framework has been extensively discussed and challenged but remains a key reference (Anderson et al., 2010 ; Chalmers & Glasziou, 2009 ; Kim & Kim, 2018 ; Macfarlane & Cheng, 2008 ). Moreover, we were inspired by the late work of Merton on the ambivalence and ambiguities of scientists (Merton, 1942 /1973), and the counter norms suggested by Mitroff ( 1974 ). Thus, the survey involved a composite set of items to capture the contradictory normative environment in academia: classical norms as well as their counter norms.

To reduce the problems of social desirability bias and personal sensitivity, the survey design avoided items about the respondent’s personal adherence to explicit ideals, which are common in many surveys (Gopalakrishna et al., 2022 ). Instead, the studied survey focused on the normative preferences and attitudes within the respondent’s environment. This necessitated the identification, selection, and refinement of 3–4 items for each potentially relevant norm/counter-norm. The selection process was used in previous studies of norm subscription in various research communities (Anderson et al., 2007 ; Braxton, 1993 ; Bray & von Storch, 2017 ). For the norm “skepticism”, we consulted studies in the accounting literature of the three key elements of professional skepticism: questioning mind, suspension of judgment and search for knowledge (Hurtt, 2010 ).

The first analytical step after receiving the completed survey set from Statistics Sweden was to conduct a set of factor analyses to assess the quality and validity of the survey items related to the normative environment and the organizational climate. These analyses suggested three clearly identifiable factors related to the normative environment: (1) a counter norm factor combining Mitroff’s particularism and dogmatism (‘Biasedness’ in the further analysis), and two Mertonian factors: (2) Skepticism and (3) Openness, a variant of Merton’s Communalism (see Table  3 ). A fourth Merton factor, Disinterestedness, could not be identified in our analysis.

The analytical process for organizational climate involved reducing the number of items from 15 to 11 (see Table 4 ). Here, the factor analysis suggested two clearly identifiable factors, one related to collegiality and the other related to competition (see Table  4 ). Overall, the factor analyses suggested that the design efforts had paid off in terms of high item quality, robust factor loadings, and a very limited need to remove any items.

In a parallel step, the open comments were assessed as an indication of how the study was perceived by the respondents (see Table  5 ). Of the 3005 respondents, 622 provided comprehensible comments, and many of them were extensive. 187 comments were related to the respondents’ own employment/role, 120 were related to the respondents’ working conditions and research environment, and 98 were related to the academic environment and atmosphere. Problems in knowing details of collegial practices were mentioned in 82 comments.

Reducing Desirability Bias - the Challenge of Nonresponse

It is well established that studies on topics where the respondent has anything embarrassing or sensitive to report suffer from more missing responses than studies on neutral subjects and that respondents may edit the information they provide on sensitive topics (Tourangeau & Yan, 2007 ). Such a social desirability bias is applicable for QRP studies which explicitly target the respondents’ personal attitudes and behaviors. To reduce this problem, the studied survey applied a non-self-format focusing on the behaviors and preferences of the respondents’ colleagues. Relevant survey items from published studies were rephrased from self-format designs to non-self-questions about practices in the respondent’s environment, using the format: “In my research environment, colleagues…” followed by a five-step incremental response format from “(1) never” to “(5) always”. In a similar way the survey avoided “should”-statements about ideal normative values: “Scientists and scholars should critically examine…”. Instead, the survey used items intended to indicate the revealed preferences in the respondent’s normative environment regarding universalism versus particularism or openness versus secrecy.

As indicated by Fanelli ( 2009 ), these redesign efforts probably reduced the social desirability bias significantly. At the same time, however, the redesign seemed to increase a problem not discussed by Fanelli ( 2009 ): an increased uncertainty problem related to the respondents’ difficulties of knowing the practices of their colleagues in questionable areas. This issue was indicated by the open comment at the end of the studied survey, where 13% of the 622 respondents pointed out that they lacked sufficient knowledge about the behavior of their colleagues to answer the QRP questions (see Table  5 ). One respondent wrote:

“It’s difficult to answer questions about ‘colleagues in my research area’ because I don’t have an insight into their research practices; I can only make informed guesses and generalizations. Therefore, I am forced to answer ‘don’t know’ to a lot of questions”.

Regarding the questions on general QRPs, the rate of missing responses varied between 11% and 21%. As for the questions targeting specific QRP practices in quantitative and qualitative research, the rate of missing responses ranged from 38 to 49%. Unfortunately, the non-response alternative to these questions (“Don’t know/not relevant”) combined the two issues: the lack of knowledge and the lack of relevance. Thus, we don’t know what part of the missing responses related to a non-presence of the specific research approach in the respondent’s environment and what part signaled a lack of knowledge about collegial practices in this environment.

Measuring QRPs in Qualitative Research - the Limited Role of Pretests

Studies of QRP prevalence focus on quantitative research approaches, where there exists a common understanding of the interpretation of scientific evidence, clearly recommended procedures, and established QRP items related to compliance with these procedures. In the heterogenous field of qualitative research, there are several established standards for reporting the research (O’Brien et al., 2014 ; Tong et al., 2007 ), but, as noted above, hardly any commonly accepted survey items that capture behaviors that fulfill the criteria for QRPs. As a result, the studied survey project designed such items from the start during the survey development process. After technical and cognitive tests, four items were selected. See Table  6 .

Despite the series of pretests, however, the first two of these items met severe criticism from a few respondents in the survey’s open commentary section. Here, qualitative researchers argued that the items were unduly influenced by the truth claims in quantitative studies, whereas their research dealt with interpretation and discourse analysis. Thus, they rejected the items regarding selective usage of respondents and of interview quotes as indicators of questionable practices:

“The alternative regarding using quotes is a bit misleading. Supporting your results by quotes is a way to strengthen credibility in a qualitative method….” “The question about dubious practices is off target for us, who work with interpretation rather than solid truths. You can present new interpretations, but normally that does not imply that previous ‘findings’ should be considered incorrect.” “The questions regarding qualitative research were somewhat irrelevant. Often this research is not guided by a given hypothesis, and researchers may use a convenient sample without this resulting in lower quality.”

One comment focused on other problems related to qualitative research:

“Several questions do not quite capture the ethical dilemmas we wrestle with. For example , is the issue of dishonesty and ‘inaccuracies’ a little misplaced for us who work with interpretation? …At the same time , we have a lot of ethical discussions , which , for example , deal with power relations between researchers and ‘researched’ , participant observation/informal contacts and informed consent (rather than patients participating in a study)”.

Unfortunately, the survey received these comments and criticism only after the full-scale rollout and not during the pretest rounds. Thus, we had no chance to replace the contested items with other formulations or contemplate a differentiation of the subsection to target specific types of qualitative research with appropriate questions. Instead, we had to limit the post-roll-out survey analysis to the last two items in Table  6 , although they captured devious behaviors rather than gray zone practices.

Why then was this criticism of QRP items related to qualitative research not exposed in the pretest phase? This is a relevant question, also for future survey designers. An intuitive answer could be that the research team only involved quantitative researchers. However, as highlighted above, the pretest participants varied in their research methods: some exclusively used qualitative methods, others employed mixed methods, and some utilized quantitative methods. This diversity suggests that the selection of test participants was appropriate. Moreover, all three members of the research team had experience of both quantitative and qualitative studies. However, as discussed above, the field of qualitative research involves several different types of research, with different goals and methods – from detailed case studies grounded in original empirical fieldwork to participant observations of complex organizational phenomena to discursive re-interpretations of previous studies. Of the 3,005 respondents who answered the survey in a satisfactory way, only 16 respondents, or 0,5%, had any critical comments about the QRP items related to qualitative research. A failure to capture the objections from such a small proportion in a pretest phase is hardly surprising. The general problem could be compared with the challenge of detecting negative side-effects in drug development. Although the pharmaceutical firms conduct large-scale tests of candidate drugs before government approval, doctors nevertheless detect new side-effects when the medicine is rolled out to significantly more people than the test populations – and report these less frequent problems in the additional drug information (Galeano et al., 2020 ; McNeil et al., 2010 ).

In the social sciences, the purpose of pre-testing is to identify problems related to ambiguities and bias in item formulation and survey format and initiate a search for relevant solutions. A pre-test on a small, selected subsample cannot guarantee that all respondent problems during the full-scale data collection will be detected. The pretest aims to reduce errors to acceptable levels and ensure that the respondents will understand the language and terminology chosen. Pretesting in survey development is also essential to help the researchers to assess the overall flow and structure of the survey, and to make necessary adjustments to enhance respondent engagement and data quality (Ikart, 2019 ; Presser & Blair, 1994 ).

In our view, more pretests would hardly solve the epistemological challenge of formulating generally acceptable QRP items for qualitative research. The open comments studied here suggest that there is no one-size-fits-all solution. If this is right, the problem should rather be reformulated to a question of identifying different strands of qualitative research with diverse views of integrity and evidence which need to be measured with different measures. To address this challenge in a comprehensive way, however, goes far beyond the current study.

Controversiality and Collegial sensitivity - the Challenge of Predicting Nonresponse

Studies of research integrity, questionable research practices, and misconduct in science tend to be organizationally controversial and personally sensitive. If university leaders are asked to support such studies, there is a considerable risk that the answer will be negative. In the case studied here, the survey roll-out was not dependent on any active organizational participation since Statistics Sweden possessed all relevant respondent information in-house. This, we assumed, would take the controversiality problem off the agenda. Our belief was supported by the non-existent complaints regarding a potential negativity bias from the pretest participants. Instead, the problem surfaced when the survey was rolled out, and all the respondents contemplated the survey. The open comment section at the end of the survey provided insights into this reception.

Many respondents provided positive feedback, reflected in 30 different comments such as:

“Thank you for doing this survey. I really hope it will lead to changes because it is needed”. “This is an important survey. However , there are conflicting norms , such as those you cite in the survey , /concerning/ for example , data protection. How are researchers supposed to be open when we cannot share data for re-analysis?” “I am glad that the problems with egoism and non-collegiality are addressed in this manner ”.

Several of them asked for more critical questions regarding power, self-interest, and leadership:

“What I lack in the survey were items regarding academic leadership. Otherwise, I am happy that someone is doing research on these issues”. “A good survey but needs to be complemented with questions regarding researchers who put their commercial interests above research and exploit academic grants for commercial purposes”.

A small minority criticized the survey for being overly negative towards academia:

“A major part of the survey feels very negative and /conveys/ the impression that you have a strong pre-understanding of academia as a horrible environments”. “Some of the questions are uncomfortable and downright suggestive. Why such a negative attitude towards research?” “The questions have a tendency to make us /the respondents/ informers. An unpleasant feeling when you are supposed to lay information against your university”. “Many questions are hard to answer, and I feel that they measure my degree of suspicion against my closest colleagues and their motivation … Several questions I did not want to answer since they contain a negative interpretation of behaviors which I don’t consider as automatically negative”.

A few of these respondents stated that they abstained from answering some of the ‘negative questions’, since they did not want to report on or slander their colleagues. The general impact is hard to assess. Only 20% of the respondents offered open survey comments, and only seven argued that questions were “negative”. The small number explains why the issue of negativity did not show up during the testing process. However, a perceived sense of negativity may have affected the willingness to answer among more respondents than those who provided free test comments.

Conclusion - The Needs for a Cumulative Knowledge Trajectory in Integrity Studies

In the broad field of research integrity studies, investigations of QRPs in different contexts and countries play an important role. The comparability of the results, however, depends on the conceptual focus of the survey design and the quality of the survey items. This paper starts with a discussion of four common problems in QRP research: the problems of precision, social desirability, incomplete coverage, and organizational controversiality and sensitivity. This is followed by a case study of how these problems were addressed in a detailed survey design process. An assessment of the solutions employed in the studied survey design reveals progress as well as unresolved issues.

Overall, the paper shows that the problem and challenges of precision could be effectively managed through explicit conceptual definitions and careful item design.

The problem of social desirability bias was probably reduced by means of a non-self-response format referring to preferences and behaviors among colleagues instead of personal behaviors. However, an investigation of open respondent comments indicated that the reduced risk of social bias came at the expense of higher uncertainty due to the respondents’ lack of insight in the concrete practices of their colleagues.

The problem of incomplete coverage of QRPs in qualitative research, the authors initially linked to “the lack of standard items” to capture QRPs in qualitative studies. Open comments at the end of the survey, however, suggested that the lack of such standards would not be easily managed by the design of new items. Rather, it seems to be an epistemological challenge related to the multifarious nature of the qualitative research field, where the understanding of ‘evidence’ is unproblematic in some qualitative sub-fields but contested in others. This conjecture and other possible explanations will hopefully be addressed in forthcoming epistemological and empirical studies.

Regarding the problem of controversiality and sensitivity, previous studies show that QRP research is a controversial and sensitive area for academic executives and university brand managers. The case study discussed here indicates that this is a sensitive subject also for rank-and-file researchers who may hesitate to answer, even when the questions do not target the respondents’ own practices but the practices and preferences of their colleagues. Future survey designers may need to engage in framing, presenting, and balancing sensitive items to reduce respondent suspicions and minimize the rate of missing responses. Reflections on the case indicate that this is doable but requires thoughtful design, as well as repeated tests, including feedback from a broad selection of prospective participants.

In conclusion, the paper suggests that more resources should be spent on the systematic evaluation of different survey designs and item formulations. In the long term, such investments in method development will yield a higher proportion of robust and comparable studies. This would mitigate the problems discussed here and contribute to the creation of a much-needed cumulative knowledge trajectory in research integrity studies.

An issue not covered here is that surveys, however finely developed, only give quantitative information about patterns, behaviors, and structures. An understanding of underlying thoughts and perspectives requires other procedures. Thus, methods that integrate and triangulate qualitative and quantitative data —known as mixed methods (Karabag & Berggren, 2016 ; Ordu & Yılmaz, 2024 ; Smajic et al., 2022 )— may give a deeper and more complete picture of the phenomenon of QRP.

Data Availability

The data supporting the findings of this study are available from the corresponding author, upon reasonable request.

Wallenborg Likidis ( 2019 ). Academic norms and scientific attitudes: Metrology Review of a survey for doctoral students , researchers and academic teachers (In Swedish: Akademiska normer och vetenskapliga förhallningssätt. Mätteknisk granskning av en enkät till doktorander , forskare och akademiska lärare) . Prod.nr. 8,942,146, Statistics Sweden, Örebro.

Agnoli, F., Wicherts, J. M., Veldkamp, C. L., Albiero, P., & Cubelli, R. (2017). Questionable research practices among Italian research psychologists. PLoS One , 12(3), e0172792.

Anderson, M. S., Ronning, E. A., De Vries, R., & Martinson, B. C. (2007). The perverse effects of competition on scientists’ work and relationships. Science and Engineering Ethics , 13 , 437–461.

Article   Google Scholar  

Anderson, M. S., Ronning, E. A., Devries, R., & Martinson, B. C. (2010). Extending the Mertonian norms: Scientists’ subscription to norms of Research. The Journal of Higher Education , 81 (3), 366–393. https://doi.org/10.1353/jhe.0.0095

Andrade, C. (2021). HARKing, cherry-picking, p-hacking, fishing expeditions, and data dredging and mining as questionable research practices. The Journal of Clinical Psychiatry , 82 (1), 25941.

ArtinoJr, A. R., Driessen, E. W., & Maggio, L. A. (2019). Ethical shades of gray: International frequency of scientific misconduct and questionable research practices in health professions education. Academic Medicine , 94 (1), 76–84.

Aubert Bonn, N., & Pinxten, W. (2019). A decade of empirical research on research integrity: What have we (not) looked at? Journal of Empirical Research on Human Research Ethics , 14 (4), 338–352.

Banks, G. C., O’Boyle Jr, E. H., Pollack, J. M., White, C. D., Batchelor, J. H., Whelpley, C. E., & Adkins, C. L. (2016). Questions about questionable research practices in the field of management: A guest commentary. Journal of Management , 42 (1), 5–20.

Beatty, P., & Herrmann, D. (2002). To answer or not to answer: Decision processes related to survey item nonresponse. Survey Nonresponse , 71 , 86.

Google Scholar  

Berggren, C. (2016). Scientific Publishing: History, practice, and ethics (in Swedish: Vetenskaplig Publicering: Historik, Praktik Och Etik) . Studentlitteratur AB.

Berggren, C., & Karabag, S. F. (2019). Scientific misconduct at an elite medical institute: The role of competing institutional logics and fragmented control. Research Policy , 48 (2), 428–443. https://doi.org/10.1016/j.respol.2018.03.020

Braxton, J. M. (1993). Deviancy from the norms of science: The effects of anomie and alienation in the academic profession. Research in Higher Education , 54 (2), 213–228. https://www.jstor.org/stable/40196105

Bray, D., & von Storch, H. (2017). The normative orientations of climate scientists. Science and Engineering Ethics , 23 (5), 1351–1367.

Breakwell, G. M., Wright, D. B., & Barnett, J. (2020). Research questions, design, strategy and choice of methods. Research Methods in Psychology , 1–30.

Brenner, P. S. (2020). Why survey methodology needs sociology and why sociology needs survey methodology: Introduction to understanding survey methodology: Sociological theory and applications. In Understanding survey methodology: Sociological theory and applications (pp. 1–11). https://doi.org/10.1007/978-3-030-47256-6_1

Bruton, S. V., Medlin, M., Brown, M., & Sacco, D. F. (2020). Personal motivations and systemic incentives: Scientists on questionable research practices. Science and Engineering Ethics , 26 (3), 1531–1547.

Butler, N., Delaney, H., & Spoelstra, S. (2017). The gray zone: Questionable research practices in the business school. Academy of Management Learning & Education , 16 (1), 94–109.

Byrn, M. J., Redman, B. K., & Merz, J. F. (2016). A pilot study of universities’ willingness to solicit whistleblowers for participation in a study. AJOB Empirical Bioethics , 7 (4), 260–264.

Chalmers, I., & Glasziou, P. (2009). Avoidable waste in the production and reporting of research evidence. The Lancet , 374 (9683), 86–89.

de Vrieze, J. (2021). Large survey finds questionable research practices are common. Science . https://doi.org/10.1126/science.373.6552.265

Dore, R. P. (1973/2011). British Factory Japanese Factory: The origins of National Diversity in Industrial Relations, with a New Afterword . University of California Press/Routledge.

Downes, M. (2017). University scandal, reputation and governance. International Journal for Educational Integrity , 13 , 1–20.

Duncan, L. J., & Cheng, K. F. (2021). Public perception of NHS general practice during the first six months of the COVID-19 pandemic in England. F1000Research , 10 .

Fanelli, D. (2009). How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One , 4(5), e5738.

Fiedler, K., & Schwarz, N. (2016). Questionable research practices revisited. Social Psychological and Personality Science , 7 (1), 45–52.

Flick, U. (2013). The SAGE Handbook of Qualitative Data Analysis . sage.

Fraser, H., Parker, T., Nakagawa, S., Barnett, A., & Fidler, F. (2018). Questionable research practices in ecology and evolution. PLoS One , 13(7), e0200303.

Galeano, D., Li, S., Gerstein, M., & Paccanaro, A. (2020). Predicting the frequencies of drug side effects. Nature Communications , 11 (1), 4575.

Gopalakrishna, G., Ter Riet, G., Vink, G., Stoop, I., Wicherts, J. M., & Bouter, L. M. (2022). Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in the Netherlands. PLoS One , 17 (2), e0263023.

Hasselberg, Y. (2012). Science as Work: Norms and Work Organization in Commodified Science (in Swedish: Vetenskap Som arbete: Normer och arbetsorganisation i den kommodifierade vetenskapen) . Gidlunds förlag.

Hill, J., Ogle, K., Gottlieb, M., Santen, S. A., & ArtinoJr, A. R. (2022). Educator’s blueprint: a how-to guide for collecting validity evidence in survey‐based research. AEM Education and Training , 6(6), e10835.

Hinkin, T. R. (1995). A review of scale development practices in the study of organizations. Journal of Management , 21 (5), 967–988.

Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods , 1 (1), 104–121.

Huistra, P., & Paul, H. (2022). Systemic explanations of scientific misconduct: Provoked by spectacular cases of norm violation? Journal of Academic Ethics , 20 (1), 51–65.

Hurtt, R. K. (2010). Development of a scale to measure professional skepticism. Auditing: A Journal of Practice & Theory , 29 (1), 149–171.

Ikart, E. M. (2019). Survey questionnaire survey pretesting method: An evaluation of survey questionnaire via expert reviews technique. Asian Journal of Social Science Studies , 4 (2), 1.

Karabag, S. F., & Berggren, C. (2016). Misconduct, marginality and editorial practices in management, business and economics journals. PLoS One , 11 (7), e0159492. https://doi.org/10.1371/journal.pone.0159492

Karabag, S. F., Berggren, C., Pielaszkiewicz, J., & Gerdin, B. (2024). Minimizing questionable research practices–the role of norms, counter norms, and micro-organizational ethics discussion. Journal of Academic Ethics , 1–27. https://doi.org/10.1007/s10805-024-09520-z

Kim, S. Y., & Kim, Y. (2018). The ethos of Science and its correlates: An empirical analysis of scientists’ endorsement of Mertonian norms. Science Technology and Society , 23 (1), 1–24. https://doi.org/10.1177/0971721817744438

Lawlor, J., Thomas, C., Guhin, A. T., Kenyon, K., Lerner, M. D., Consortium, U., & Drahota, A. (2021). Suspicious and fraudulent online survey participation: Introducing the REAL framework. Methodological Innovations , 14 (3), 20597991211050467.

Levelt, W. J., Drenth, P., & Noort, E. (2012). Flawed science: The fraudulent research practices of social psychologist Diederik Stapel (in Dutch: Falende wetenschap: De frauduleuze onderzoekspraktijken van social-psycholoog Diederik Stapel) . Commissioned by the Tilburg University, University of Amsterdam and the University of Groningen. https://doi.org/http://hdl.handle.net/11858/00-001M-0000-0010-258A-9

Lietz, P. (2010). Research into questionnaire design: A summary of the literature. International Journal of Market Research , 52 (2), 249–272.

Lin, M. W., & Yu, C. (2020). Can corruption be measured? Comparing global versus local perceptions of corruption in East and Southeast Asia. In Regional comparisons in comparative policy analysis studies (pp. 90–107). Routledge.

Macfarlane, B., & Cheng, M. (2008). Communism, universalism and disinterestedness: Re-examining contemporary support among academics for Merton’s scientific norms. Journal of Academic Ethics , 6 , 67–78.

Madley-Dowd, P., Hughes, R., Tilling, K., & Heron, J. (2019). The proportion of missing data should not be used to guide decisions on multiple imputation. Journal of Clinical Epidemiology , 110 , 63–73.

McNeil, J. J., Piccenna, L., Ronaldson, K., & Ioannides-Demos, L. L. (2010). The value of patient-centred registries in phase IV drug surveillance. Pharmaceutical Medicine , 24 , 281–288.

Merton, R. K. (1942/1973). The normative structure of science. In The sociology of science: Theoretical and empirical investigations . The University of Chicago Press.

Mitroff, I. I. (1974). Norms and counter-norms in a select group of the Apollo Moon scientists: A case study of the ambivalence of scientists. American Sociological Review , 39 (4), 579–595. https://doi.org/10.2307/2094423

Necker, S. (2014). Scientific misbehavior in economics. Research Policy , 43 (10), 1747–1759. https://doi.org/10.1016/j.respol.2014.05.002

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., & Nuijten, M. B. (2022). Replicability, robustness, and reproducibility in psychological science. Annual Review of Psychology , 73 (1), 719–748.

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine , 89 (9). https://journals.lww.com/academicmedicine/fulltext/2014/09000/standards_for_reporting_qualitative_research__a.21.aspx

Ordu, Y., & Yılmaz, S. (2024). Examining the impact of dramatization simulation on nursing students’ ethical attitudes: A mixed-method study. Journal of Academic Ethics , 1–13.

Perneger, T. V., Courvoisier, D. S., Hudelson, P. M., & Gayet-Ageron, A. (2015). Sample size for pre-tests of questionnaires. Quality of life Research , 24 , 147–151.

Presser, S., & Blair, J. (1994). Survey pretesting: Do different methods produce different results? Sociological Methodology , 73–104.

Ravn, T., & Sørensen, M. P. (2021). Exploring the gray area: Similarities and differences in questionable research practices (QRPs) across main areas of research. Science and Engineering Ethics , 27 (4), 40.

Roberts, D. L., & John, F. A. S. (2014). Estimating the prevalence of researcher misconduct: a study of UK academics within biological sciences. PeerJ , 2 , e562.

Siewert, W., & Udani, A. (2016). Missouri municipal ethics survey: Do ethics measures work at the municipal level? Public Integrity , 18 (3), 269–289.

Smajic, E., Avdic, D., Pasic, A., Prcic, A., & Stancic, M. (2022). Mixed methodology of scientific research in healthcare. Acta Informatica Medica , 30 (1), 57–60. https://doi.org/10.5455/aim.2022.30.57-60

Steneck, N. H. (2006). Fostering integrity in research: Definitions, current knowledge, and future directions. Science and Engineering Ethics , 12 , 53–74.

Szolnoki, G., & Hoffmann, D. (2013). Online, face-to-face and telephone surveys—comparing different sampling methods in wine consumer research. Wine Economics and Policy , 2 (2), 57–66.

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care , 19 (6), 349–357. https://doi.org/10.1093/intqhc/mzm042

Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin , 133 (5), 859.

Tourangeau, R., Groves, R. M., & Redline, C. D. (2010). Sensitive topics and reluctant respondents: Demonstrating a link between nonresponse bias and measurement error. Public Opinion Quarterly , 74 (3), 413–432.

Vermeulen, I., & Hartmann, T. (2015). Questionable research and publication practices in communication science. Communication Methods and Measures , 9 (4), 189–192.

Wallenborg Likidis, J. (2019). Academic norms and scientific attitudes: Metrology review of a survey for doctoral students, researchers and academic teachers (In Swedish: Akademiska normer och vetenskapliga förhallningssätt. Mätteknisk granskning av en enkät till doktorander, forskare och akademiska lärare) . Prod.nr. 8942146, Statistics Sweden, Örebro.

Willis, G. B. (2004). Cognitive interviewing: A tool for improving questionnaire design . Sage Publications.

Xie, Y., Wang, K., & Kong, Y. (2021). Prevalence of research misconduct and questionable research practices: A systematic review and meta-analysis. Science and Engineering Ethics , 27 (4), 41.

Yan, T., & Curtin, R. (2010). The relation between unit nonresponse and item nonresponse: A response continuum perspective. International Journal of Public Opinion Research , 22 (4), 535–551.

Download references

Acknowledgements

We thank Jennica Wallenborg Likidis, Statistics Sweden, for providing expert support in the survey design. We are grateful to colleagues Ingrid Johansson Mignon, Cecilia Enberg, Anna Dreber Almenberg, Andrea Fried, Sara Liin, Mariano Salazar, Lars Bengtsson, Harriet Wallberg, Karl Wennberg, and Thomas Magnusson, who joined the pretest or cognitive tests. We also thank Ksenia Onufrey, Peter Hedström, Jan-Ingvar Jönsson, Richard Öhrvall, Kerstin Sahlin, and David Ludvigsson for constructive comments or suggestions.

Open access funding provided by Linköping University. Swedish Forte: Research Council for Health, Working Life and Welfare ( https://www.vr.se/swecris?#/project/2018-00321_Forte ) Grant No. 2018-00321.

Open access funding provided by Linköping University.

Author information

Authors and affiliations.

Department of Management and Engineering [IEI], Linköping University, Linköping, SE-581 83, Sweden

Christian Berggren & Solmaz Filiz Karabag

Department of Surgical Sciences, Uppsala University, Uppsala University Hospital, entrance 70, Uppsala, SE-751 85, Sweden

Bengt Gerdin

Department of Civil and Industrial Engineering, Uppsala University, Box 169, Uppsala, SE-751 04, Sweden

Solmaz Filiz Karabag

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: CB. Survey Design: SFK, CB, Methodology: SFK, BG, CB. Visualization: SFK, BG. Funding acquisition: SFK. Project administration and management: SFK. Writing – original draft: CB. Writing – review & editing: CB, BG, SFK. Approval of the final manuscript: SFK, BG, CB.

Corresponding author

Correspondence to Solmaz Filiz Karabag .

Ethics declarations

Ethics approval and consent to participate.

The Swedish Act concerning the Ethical Review of Research Involving Humans (2003:460) defines the type of studies which requires an ethics approval. In line with the General Data Protection Regulation (EU 2016/67), the act is applicable for studies that collect personal data that reveal racial or ethnic origin, political opinions, trade union membership, religious or philosophical beliefs, or health and sexual orientation. The present study does not involve any of the above, why no formal ethical permit was required. The ethical aspects of the project and its compliance with the guidelines of the Swedish Research Council (2017) were also part of the review process at the project’s public funding agency Forte.

Competing Interests

The authors declare that they have no competing interests.

Supporting Information

The complete case study survey of social and medical science researchers in Sweden 2020.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Berggren, C., Gerdin, B. & Karabag, S.F. Developing Surveys on Questionable Research Practices: Four Challenging Design Problems. J Acad Ethics (2024). https://doi.org/10.1007/s10805-024-09565-0

Download citation

Accepted : 23 August 2024

Published : 02 September 2024

DOI : https://doi.org/10.1007/s10805-024-09565-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Questionable Research Practices
• Normative Environment
• Organizational Climate
• Survey Development
• Design Problems
• Problem of Incomplete Coverage
• Survey Design Process
• Baseline Survey
• Pre-testing
• Technical Assessment
• Cognitive Interviews
• Social Desirability
• Sensitivity
• Organizational Controversiality
• Challenge of Nonresponse
• Qualitative Research
• Quantitative Research
• Find a journal
• Publish with us
• Track your research

• About The Journalist’s Resource
• Follow us on Facebook
• Follow us on Twitter
• Criminal Justice
• Environment
• Politics & Government
• Race & Gender

Expert Commentary

Eight questions to ask when interpreting academic studies: A primer for media

Scholarly research is a great source for rigorous, unbiased information, but making judgments about its quality can be difficult. Here are some important questions to ask when reading studies.

Republish this article

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License .

by Justin Feldman and John Wihbey, The Journalist's Resource March 26, 2015

This <a target="_blank" href="https://journalistsresource.org/home/interpreting-academic-studies-primer-media/">article</a> first appeared on <a target="_blank" href="https://journalistsresource.org">The Journalist's Resource</a> and is republished here under a Creative Commons license.<img src="https://journalistsresource.org/wp-content/uploads/2020/11/cropped-jr-favicon-150x150.png" style="width:1em;height:1em;margin-left:10px;">

Reading scholarly studies can help journalists integrate rigorous, unbiased sources of information into their reporting. These studies are typically carried out by professors and professional researchers — at universities, think tanks and government institutions — and are published through a peer-review process in which those familiar with the study area ensure that there are no major flaws.

Even for people who carry out research, however, interpreting scientific (and social science) studies and making judgments about their quality can be difficult tasks. In a now-famous article, Stanford professor John Ioannidis argues that “ most published research findings are false ” due to inherent limitations in how researchers design studies. (Health and medical studies can be particularly attractive to media, but be aware that there is a long history of faulty findings .) Occasionally, too, studies can be the product of outright fraud: A 1998 study falsely linking vaccines and autism is now perhaps the canonical example, as it spurred widespread and long-lasting societal damage . Journalists should also always examine the funding sources behind the study, which are frequently declared at the study’s conclusion.

Before journalists write about research and speak with authors, they should be able to both interpret a study’s results generally and understand the appropriate degree of skepticism that a given study’s findings warrant. This requires data literacy , some familiarity with statistical terms and a basic knowledge of hypothesis testing and construction of theories .

Journalists should also be well aware that most academic research contains careful qualifications about findings. The common complaint from scientists and social scientists is that news media tend to pump up findings and hype studies through catchy headlines, distorting public understanding. But landmark studies sometimes do no more than tighten the margin of error around a given measurement — not inherently flashy, but intriguing to an audience if explained with rich context and clear presentation.

Here are some important questions to ask when reading a scientific study:

1. What are the researchers’ hypotheses?

A hypothesis is a research question that a study seeks to answer. Sometimes researchers state their hypotheses explicitly, but more often their research questions are implicit. Hypotheses are testable assertions usually involving the relationship between two variables. In a study of smoking and lung cancer, the hypothesis might be that smokers develop lung cancer at a higher rate than non-smokers over a five-year period.

It is also important to note that there are formal definitions of null and alternative hypotheses for use with statistical analysis.

2. What are the independent and dependent variables?

Independent variables are factors that influence particular outcomes. Dependent variables are measures of the outcomes themselves. In the study assessing the relationship between smoking and lung cancer, smoking is the independent variable because the researcher assumes it predicts lung cancer, the dependent variable. (Some fields use related terms such as “exposure” and “outcome.”)

Pay particular attention to how the researchers define all of the variables — there can be quite a bit of nuance in the definitions. Also look at the methods by which the researchers measure the variables. Generally speaking, a variable measured using a subject’s response to a survey question is less trustworthy than one measured through more objective means — reviewing laboratory findings in their medical records, for example.

3. What is the unit of analysis?

For most studies involving human subjects, the individual person is the unit of analysis. However, studies are sometimes interested in a different level of analysis that makes comparisons between classrooms, hospitals, schools or states, for example, rather than between individuals.

4. How well does the study design address causation?

Most studies identify correlations or associations between variables, but typically the ultimate goal is to determine causation . Certain study designs are more useful than others for the purpose of determining causation.

At the most basic level, studies can be placed into one of two categories: experimental and observational . In experimental studies, the researchers decide who is exposed to the independent variable and who is not. In observational studies, the researchers do not have any control over who is exposed to the independent variable — instead they make comparisons between groups that are already different from one another. In nearly all cases, experimental studies provide stronger evidence than observational studies.

Here are descriptions of some of the most common study designs, presented along with their respective values for inferring causation:

• Randomized controlled trials (RCTs), also known as clinical trials, are experimental studies that are considered the “gold standard” in research. Out of all study designs, they have the most value for determining causation although they do have limitations. In an RCT, researchers randomly divide subjects into at least two groups: One that receives a treatment, and the other — the control group — that receives either no treatment or a simulated version of the treatment called a placebo . The independent variable in these experiments is whether or not the subject receives the real treatment. Ideally an RCT should be double-blind — the participants should not know to which treatment group they have been assigned, nor should the study staff know. This arrangement helps to avoid bias. Researchers commonly use RCTs to meet regulatory requirements, such as evaluating pharmaceuticals for the Food and Drug Administration. Due to issues of cost, logistics and ethics, RCTs are fairly uncommon for other purposes. Example: “ Short-Term Soy Isoflavone Intervention in Patients with Localized Prostate Cancer ”
• Longitudinal studies , like RCTs, follow the same subjects over a given time period. Unlike in RCTs, they are observational. Researchers do not assign the independent variable in longitudinal studies — they instead observe what happens in the real world. A longitudinal study might compare the risk for heart disease among one group of people who are exposed to high levels of air pollution to the risk of heart disease among another group exposed to low levels of air pollution. The problem is that, because there is no random assignment, the groups may differ from one another in other important ways and, as a result, we cannot completely isolate the effects of air pollution. These differences result in confounding and other forms of bias. For that reason, longitudinal studies have less validity for inferring causation than RCTs and other experimental study designs. Longitudinal studies have more validity than other kinds of observational studies, however. Example: “ Mood after Moderate and Severe Traumatic Brain Injury: A Prospective Cohort Study ”
• Case-control studies are technically a type of longitudinal study, but they are unique enough to discuss separately. Common in public health and medical research, case-control studies begin with a group of people who have already developed a particular disease and compare them to a similar but disease-free group recruited by the researchers. These studies are more likely to suffer from bias than other longitudinal studies for two reasons. First, they are always retrospective , meaning they collect data about independent variables years after the exposures of interest occurred — sometimes even after the subject has died. Second, the group of disease-free people is very likely to differ from the group that developed the disease, creating a substantial risk for confounding. Example: “ Risk Factors for Preeclampsia in Women from Colombia ”.
• Cross-sectional studies are a kind of observational study that measure both dependent and independent variables at a single point in time. Although researchers may administer the same cross-sectional survey every few years, they do not follow the same subjects over time. An important part of determining causation is establishing that the independent variable occurred for a given subject before the dependent variable occurred. But because they do not measure the variables over time, cross-sectional studies cannot determine that a hypothesized cause precedes its effect, so the design is limited to making inferences about correlations rather than causation. Example : “ Physical Predictors of Cognitive Performance in Healthy Older Adults ”
• Ecological studies are observational studies that are similar to cross-sectional studies except that they measure at least one variable on the group-level rather that the subject-level. For example, an ecological study may look at the relationship between individuals’ meat consumption and their incidence of colon cancer. But rather than using individual-level data, the study relies on national cancer rates and national averages for meat consumption. While it might seem that higher meat consumption is linked to a higher risk of cancer, there is no way to know if the individuals eating more meat within a country are the same people who are more likely to develop cancer. This means that ecological studies are not only inadequate for inferring causation, they are also inadequate for establishing a correlation. As a consequence, they should be regarded with strong skepticism. Example: “ A Multi-country Ecological Study of Cancer Incidence Rates in 2008 with Respect to Various Risk-Modifying Factors ”
• Systematic reviews are surveys of existing studies on a given topic. Investigators specify inclusion and exclusion criteria to weed out studies that are either irrelevant to their research question or poorly designed. Using keywords, they systematically search research databases, present the findings of the studies they include and draw conclusions based on their consideration of the findings. Assuming that the review includes only well-designed studies, systematic reviews are more useful for inferring causation than any single well-designed study. Example: “ Enablers and Barriers to Large-Scale Uptake of Improved Solid Fuel Stoves. ” For a sense of how systematic reviews are interpreted and used by researchers in the field, see “How to Read a Systematic Review and Meta-analysis and Apply the Results to Patient Care,” published in the Journal of the American Medical Association (JAMA.)
• Meta-analyses are similar to systematic reviews but use the original data from all included studies to create a new analysis. As a result, a meta-analysis is able to draw conclusions that are more meaningful than a systematic review. Again, a meta-analysis is more useful for inferring causation than any single study, assuming that all studies are well-designed. Example: “ Occupational Exposure to Asbestos and Ovarian Cancer ”

5. What are the study’s results?

There are several aspects involved in understanding a study’s results:

• Understand whether or not the study found statistically significant relationships between the dependent and independent variables. If the relationship is statistically significant, it means that any difference observed between groups is unlikely to be due to random chance. P-values help researchers to decide whether observed differences are simply due to chance or represent a true difference between groups.
• If the relationship is statistically significant, it is then important to determine the effect size , which is the size of the difference observed between the groups. Subjects enrolled in a weight loss program may have experienced a statistically significant reduction in weight compared to those in a control group, but is that difference one ounce, one pound or ten pounds? There are myriad ways in which studies present effect sizes — such obscure terms as regression coefficients, odds ratios, and population attributable fractions may come into play. Unfortunately, research articles sometimes fail to interpret effect sizes in words. In these cases, it may be best to consult an expert to help develop a plain-English interpretation.
• Even if there is a statistically significant difference between comparison groups, this does not mean the effect size is meaningful. A weight loss program that leads to a total weight reduction of one ounce on average or a policy that saves one life out of a billion may not be meaningful. Again, consulting an expert in the field can help to determine how meaningful an effect size is, a determination that is ultimately a subjective judgment call.

6. How generalizable are the results?

Study results are useful because they help us make inferences about the relationship between independent and dependent variables among a larger population. The subjects enrolled in the study must be similar to those in the larger population, however, in order to generalize the findings. Even a perfectly designed study may be of limited value when its results cannot be generalized. It is important to pay attention to the composition of the study sample. If the unit of analysis is the individual, important factors to consider regarding the group’s composition include age, race/ethnicity, gender, socioeconomic status, and geographic location. While some samples are deliberately constructed to be representative of a country or region, most are not.

7. What limitations do the authors note?

Within a research article, authors often state some of the study’s limitations explicitly. This information can be very helpful in determining the strength of the evidence presented in the study.

8. What conclusions do similar studies draw?

With some notable exceptions, a single study is unlikely to fundamentally change what is already known about the research question it addresses. It is important to compare a new study’s findings to existing studies that address similar research questions, particularly systematic reviews or meta-analyses if available.

Further: One hidden form of bias that is easily missed is what’s called “selecting on the dependent variable,” which is the research practice of focusing on only those areas where there are effects and ignoring ones where there are not. This can lead to exaggerated conclusions (and thereby false media narratives). For example, it is tempting to say that “science has become polarized,” as survey data suggest significant differences in public opinion on issues such as climate change, vaccinations and nuclear power. However, on most scientific issues, there is almost no public debate or controversy . Additionally, the reality of “publication bias” — academic journals have traditionally been more interested in publishing studies that show effects, rather than no effects — can create a biased incentive structure that distorts larger truths.

For an updated overview, see a 2014 paper by Stanford’s John Ioannidis, “How to Make More Published Research True.”

Keywords: training

About the Authors

Justin Feldman

John Wihbey

• Password Reset
• Course Listings
• Leganto – faculty
• More Logins...
• Campus Maps
• Contact a librarian for assistance
• Contact us – staff directory
• Research Guides
• Bush Memorial Library
• Good Places to Start

2024 FSEM 1010-21: Images of Resistance: Visual Arts & Social Activism (Prof. Schlink): Databases & Articles

• Pre-Search Resources
• HamCAT for discovery
• Databases & Articles
• Evaluating Sources
• Generative AI Content
• Learning Outcomes

Search HamCAT for Articles

Enter the title of an article or an author to search across databases

Add LibKey Nomad to your browser!

Just click the image below (time needed : 3 minutes) :  

for Chrome, Firefox, Microsoft Edge, Vivaldi, or Brave

• Access e-texts in ONE CLICK from HamCAT results.
• Access e-texts in HamCAT from the Web or 'Request' via ILL.

Contact Mary O'Dea

Ask a Librarian 24/7

Linking Bush Library to Google Scholar

Watch this video on how to make make your Google Scholar results indicate if you can get the item through Bush Library. 

https://www.youtube.com/watch?v=xEM3f0f0UEw

Media Bias Charts

• AllSides Media Bias Chart This organization looks at media each year and tracks whether their content is politically left or right leaning or more central in it's coverage.
• Ad Fontes Media: Media Bias Chart Annual review of "news and opinion" sources, and where they skew politically.

Multi-disciplinary Databases = Good for Most Topics

You may also want to consider using Google Scholar for broad searches and JSTOR for older articles.

• Google Scholar This link opens in a new window Search broadly across many subjects and sources for articles, dissertations, books, abstracts and court opinions from academic publishers, professional societies, online repositories, universities and other web sites. Note: Google does not identify the databases it uses; nor does it indicate the quality of the sources it provides. Use Google Scholar as a supplement to Hamline database searching.

Popular Magazines vs. Scholarly Journals

  Popular Magazines vs. Scholarly Journals  

Magazines and journals are both types of periodicals. A periodical is any publication produced periodically, in regularly recurring intervals .   Periodicals are made up of articles.  They exist in both print and electronic format.   To locate articles in a periodical, use a database or index.

To determine which type of publication is most suited for your research, consult the chart of characteristics below. The nature of your information need will impact which types of periodicals are most appropriate for your research.

  Articles accepted in scholarly journals generally undergo one of two review processes before being accepted for publication.

• peer-review (also know as refereed)  journal articles have been sent off to other experts in the field for review. Those experts provide comments to the author, that the author considers in editing and re-submission for publication.
• editorial review board  is a group of scholars in a field who agree to work as editors for the journal, reviewing manuscripts submitted for consideration. 

Some databases provide filtering options to limit your results to either magazines or academic journals. You may still need to do additional filtering of your results to locate journals which are peer-reviewed.

 Andruss Library at Bloomsburg University of Pennsylvania diagrams the scholarly review processes like this:    

and popular magazines vs. scholarly journals, like this:

Two other types of periodicals you may encounter in your research are professional and trade publications . Periodicals of these types are usually produced by either the professional or trade association to which a member belongs. Examples include : American Libraries , a membership publication for professional libraries and library staff, or  Construction News , a membership publication for the building and construction trade. Professional and trade publications are useful for reading membership news and catching the pulse of people working in a field.

• << Previous: HamCAT for discovery
• Next: Websites >>
•  : Sep 5, 2024 8:37 AM
• URL: https://bushlibraryguides.hamline.edu/2024ImagesofResistance

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.53(4); 2010 Aug

Research questions, hypotheses and objectives

Patricia farrugia.

* Michael G. DeGroote School of Medicine, the

Bradley A. Petrisor

† Division of Orthopaedic Surgery and the

Forough Farrokhyar

‡ Departments of Surgery and

§ Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ont

Mohit Bhandari

There is an increasing familiarity with the principles of evidence-based medicine in the surgical community. As surgeons become more aware of the hierarchy of evidence, grades of recommendations and the principles of critical appraisal, they develop an increasing familiarity with research design. Surgeons and clinicians are looking more and more to the literature and clinical trials to guide their practice; as such, it is becoming a responsibility of the clinical research community to attempt to answer questions that are not only well thought out but also clinically relevant. The development of the research question, including a supportive hypothesis and objectives, is a necessary key step in producing clinically relevant results to be used in evidence-based practice. A well-defined and specific research question is more likely to help guide us in making decisions about study design and population and subsequently what data will be collected and analyzed. 1

Objectives of this article

In this article, we discuss important considerations in the development of a research question and hypothesis and in defining objectives for research. By the end of this article, the reader will be able to appreciate the significance of constructing a good research question and developing hypotheses and research objectives for the successful design of a research study. The following article is divided into 3 sections: research question, research hypothesis and research objectives.

Research question

Interest in a particular topic usually begins the research process, but it is the familiarity with the subject that helps define an appropriate research question for a study. 1 Questions then arise out of a perceived knowledge deficit within a subject area or field of study. 2 Indeed, Haynes suggests that it is important to know “where the boundary between current knowledge and ignorance lies.” 1 The challenge in developing an appropriate research question is in determining which clinical uncertainties could or should be studied and also rationalizing the need for their investigation.

Increasing one’s knowledge about the subject of interest can be accomplished in many ways. Appropriate methods include systematically searching the literature, in-depth interviews and focus groups with patients (and proxies) and interviews with experts in the field. In addition, awareness of current trends and technological advances can assist with the development of research questions. 2 It is imperative to understand what has been studied about a topic to date in order to further the knowledge that has been previously gathered on a topic. Indeed, some granting institutions (e.g., Canadian Institute for Health Research) encourage applicants to conduct a systematic review of the available evidence if a recent review does not already exist and preferably a pilot or feasibility study before applying for a grant for a full trial.

In-depth knowledge about a subject may generate a number of questions. It then becomes necessary to ask whether these questions can be answered through one study or if more than one study needed. 1 Additional research questions can be developed, but several basic principles should be taken into consideration. 1 All questions, primary and secondary, should be developed at the beginning and planning stages of a study. Any additional questions should never compromise the primary question because it is the primary research question that forms the basis of the hypothesis and study objectives. It must be kept in mind that within the scope of one study, the presence of a number of research questions will affect and potentially increase the complexity of both the study design and subsequent statistical analyses, not to mention the actual feasibility of answering every question. 1 A sensible strategy is to establish a single primary research question around which to focus the study plan. 3 In a study, the primary research question should be clearly stated at the end of the introduction of the grant proposal, and it usually specifies the population to be studied, the intervention to be implemented and other circumstantial factors. 4

Hulley and colleagues 2 have suggested the use of the FINER criteria in the development of a good research question ( Box 1 ). The FINER criteria highlight useful points that may increase the chances of developing a successful research project. A good research question should specify the population of interest, be of interest to the scientific community and potentially to the public, have clinical relevance and further current knowledge in the field (and of course be compliant with the standards of ethical boards and national research standards).

FINER criteria for a good research question

Adapted with permission from Wolters Kluwer Health. 2

Whereas the FINER criteria outline the important aspects of the question in general, a useful format to use in the development of a specific research question is the PICO format — consider the population (P) of interest, the intervention (I) being studied, the comparison (C) group (or to what is the intervention being compared) and the outcome of interest (O). 3 , 5 , 6 Often timing (T) is added to PICO ( Box 2 ) — that is, “Over what time frame will the study take place?” 1 The PICOT approach helps generate a question that aids in constructing the framework of the study and subsequently in protocol development by alluding to the inclusion and exclusion criteria and identifying the groups of patients to be included. Knowing the specific population of interest, intervention (and comparator) and outcome of interest may also help the researcher identify an appropriate outcome measurement tool. 7 The more defined the population of interest, and thus the more stringent the inclusion and exclusion criteria, the greater the effect on the interpretation and subsequent applicability and generalizability of the research findings. 1 , 2 A restricted study population (and exclusion criteria) may limit bias and increase the internal validity of the study; however, this approach will limit external validity of the study and, thus, the generalizability of the findings to the practical clinical setting. Conversely, a broadly defined study population and inclusion criteria may be representative of practical clinical practice but may increase bias and reduce the internal validity of the study.

PICOT criteria 1

A poorly devised research question may affect the choice of study design, potentially lead to futile situations and, thus, hamper the chance of determining anything of clinical significance, which will then affect the potential for publication. Without devoting appropriate resources to developing the research question, the quality of the study and subsequent results may be compromised. During the initial stages of any research study, it is therefore imperative to formulate a research question that is both clinically relevant and answerable.

Research hypothesis

The primary research question should be driven by the hypothesis rather than the data. 1 , 2 That is, the research question and hypothesis should be developed before the start of the study. This sounds intuitive; however, if we take, for example, a database of information, it is potentially possible to perform multiple statistical comparisons of groups within the database to find a statistically significant association. This could then lead one to work backward from the data and develop the “question.” This is counterintuitive to the process because the question is asked specifically to then find the answer, thus collecting data along the way (i.e., in a prospective manner). Multiple statistical testing of associations from data previously collected could potentially lead to spuriously positive findings of association through chance alone. 2 Therefore, a good hypothesis must be based on a good research question at the start of a trial and, indeed, drive data collection for the study.

The research or clinical hypothesis is developed from the research question and then the main elements of the study — sampling strategy, intervention (if applicable), comparison and outcome variables — are summarized in a form that establishes the basis for testing, statistical and ultimately clinical significance. 3 For example, in a research study comparing computer-assisted acetabular component insertion versus freehand acetabular component placement in patients in need of total hip arthroplasty, the experimental group would be computer-assisted insertion and the control/conventional group would be free-hand placement. The investigative team would first state a research hypothesis. This could be expressed as a single outcome (e.g., computer-assisted acetabular component placement leads to improved functional outcome) or potentially as a complex/composite outcome; that is, more than one outcome (e.g., computer-assisted acetabular component placement leads to both improved radiographic cup placement and improved functional outcome).

However, when formally testing statistical significance, the hypothesis should be stated as a “null” hypothesis. 2 The purpose of hypothesis testing is to make an inference about the population of interest on the basis of a random sample taken from that population. The null hypothesis for the preceding research hypothesis then would be that there is no difference in mean functional outcome between the computer-assisted insertion and free-hand placement techniques. After forming the null hypothesis, the researchers would form an alternate hypothesis stating the nature of the difference, if it should appear. The alternate hypothesis would be that there is a difference in mean functional outcome between these techniques. At the end of the study, the null hypothesis is then tested statistically. If the findings of the study are not statistically significant (i.e., there is no difference in functional outcome between the groups in a statistical sense), we cannot reject the null hypothesis, whereas if the findings were significant, we can reject the null hypothesis and accept the alternate hypothesis (i.e., there is a difference in mean functional outcome between the study groups), errors in testing notwithstanding. In other words, hypothesis testing confirms or refutes the statement that the observed findings did not occur by chance alone but rather occurred because there was a true difference in outcomes between these surgical procedures. The concept of statistical hypothesis testing is complex, and the details are beyond the scope of this article.

Another important concept inherent in hypothesis testing is whether the hypotheses will be 1-sided or 2-sided. A 2-sided hypothesis states that there is a difference between the experimental group and the control group, but it does not specify in advance the expected direction of the difference. For example, we asked whether there is there an improvement in outcomes with computer-assisted surgery or whether the outcomes worse with computer-assisted surgery. We presented a 2-sided test in the above example because we did not specify the direction of the difference. A 1-sided hypothesis states a specific direction (e.g., there is an improvement in outcomes with computer-assisted surgery). A 2-sided hypothesis should be used unless there is a good justification for using a 1-sided hypothesis. As Bland and Atlman 8 stated, “One-sided hypothesis testing should never be used as a device to make a conventionally nonsignificant difference significant.”

The research hypothesis should be stated at the beginning of the study to guide the objectives for research. Whereas the investigators may state the hypothesis as being 1-sided (there is an improvement with treatment), the study and investigators must adhere to the concept of clinical equipoise. According to this principle, a clinical (or surgical) trial is ethical only if the expert community is uncertain about the relative therapeutic merits of the experimental and control groups being evaluated. 9 It means there must exist an honest and professional disagreement among expert clinicians about the preferred treatment. 9

Designing a research hypothesis is supported by a good research question and will influence the type of research design for the study. Acting on the principles of appropriate hypothesis development, the study can then confidently proceed to the development of the research objective.

Research objective

The primary objective should be coupled with the hypothesis of the study. Study objectives define the specific aims of the study and should be clearly stated in the introduction of the research protocol. 7 From our previous example and using the investigative hypothesis that there is a difference in functional outcomes between computer-assisted acetabular component placement and free-hand placement, the primary objective can be stated as follows: this study will compare the functional outcomes of computer-assisted acetabular component insertion versus free-hand placement in patients undergoing total hip arthroplasty. Note that the study objective is an active statement about how the study is going to answer the specific research question. Objectives can (and often do) state exactly which outcome measures are going to be used within their statements. They are important because they not only help guide the development of the protocol and design of study but also play a role in sample size calculations and determining the power of the study. 7 These concepts will be discussed in other articles in this series.

From the surgeon’s point of view, it is important for the study objectives to be focused on outcomes that are important to patients and clinically relevant. For example, the most methodologically sound randomized controlled trial comparing 2 techniques of distal radial fixation would have little or no clinical impact if the primary objective was to determine the effect of treatment A as compared to treatment B on intraoperative fluoroscopy time. However, if the objective was to determine the effect of treatment A as compared to treatment B on patient functional outcome at 1 year, this would have a much more significant impact on clinical decision-making. Second, more meaningful surgeon–patient discussions could ensue, incorporating patient values and preferences with the results from this study. 6 , 7 It is the precise objective and what the investigator is trying to measure that is of clinical relevance in the practical setting.

The following is an example from the literature about the relation between the research question, hypothesis and study objectives:

Study: Warden SJ, Metcalf BR, Kiss ZS, et al. Low-intensity pulsed ultrasound for chronic patellar tendinopathy: a randomized, double-blind, placebo-controlled trial. Rheumatology 2008;47:467–71.

Research question: How does low-intensity pulsed ultrasound (LIPUS) compare with a placebo device in managing the symptoms of skeletally mature patients with patellar tendinopathy?

Research hypothesis: Pain levels are reduced in patients who receive daily active-LIPUS (treatment) for 12 weeks compared with individuals who receive inactive-LIPUS (placebo).

Objective: To investigate the clinical efficacy of LIPUS in the management of patellar tendinopathy symptoms.

The development of the research question is the most important aspect of a research project. A research project can fail if the objectives and hypothesis are poorly focused and underdeveloped. Useful tips for surgical researchers are provided in Box 3 . Designing and developing an appropriate and relevant research question, hypothesis and objectives can be a difficult task. The critical appraisal of the research question used in a study is vital to the application of the findings to clinical practice. Focusing resources, time and dedication to these 3 very important tasks will help to guide a successful research project, influence interpretation of the results and affect future publication efforts.

Tips for developing research questions, hypotheses and objectives for research studies

• Perform a systematic literature review (if one has not been done) to increase knowledge and familiarity with the topic and to assist with research development.
• Learn about current trends and technological advances on the topic.
• Seek careful input from experts, mentors, colleagues and collaborators to refine your research question as this will aid in developing the research question and guide the research study.
• Use the FINER criteria in the development of the research question.
• Ensure that the research question follows PICOT format.
• Develop a research hypothesis from the research question.
• Develop clear and well-defined primary and secondary (if needed) objectives.
• Ensure that the research question and objectives are answerable, feasible and clinically relevant.

FINER = feasible, interesting, novel, ethical, relevant; PICOT = population (patients), intervention (for intervention studies only), comparison group, outcome of interest, time.

Competing interests: No funding was received in preparation of this paper. Dr. Bhandari was funded, in part, by a Canada Research Chair, McMaster University.

Peer Reviewed

GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation

Article metrics.

CrossRef Citations

Altmetric Score

PDF Downloads

Academic journals, archives, and repositories are seeing an increasing number of questionable research papers clearly produced using generative AI. They are often created with widely available, general-purpose AI applications, most likely ChatGPT, and mimic scientific writing. Google Scholar easily locates and lists these questionable papers alongside reputable, quality-controlled research. Our analysis of a selection of questionable GPT-fabricated scientific papers found in Google Scholar shows that many are about applied, often controversial topics susceptible to disinformation: the environment, health, and computing. The resulting enhanced potential for malicious manipulation of society’s evidence base, particularly in politically divisive domains, is a growing concern.

Swedish School of Library and Information Science, University of Borås, Sweden

Department of Arts and Cultural Sciences, Lund University, Sweden

Division of Environmental Communication, Swedish University of Agricultural Sciences, Sweden

Research Questions

• Where are questionable publications produced with generative pre-trained transformers (GPTs) that can be found via Google Scholar published or deposited?
• What are the main characteristics of these publications in relation to predominant subject categories?
• How are these publications spread in the research infrastructure for scholarly communication?
• How is the role of the scholarly communication infrastructure challenged in maintaining public trust in science and evidence through inappropriate use of generative AI?

research note Summary

• A sample of scientific papers with signs of GPT-use found on Google Scholar was retrieved, downloaded, and analyzed using a combination of qualitative coding and descriptive statistics. All papers contained at least one of two common phrases returned by conversational agents that use large language models (LLM) like OpenAI’s ChatGPT. Google Search was then used to determine the extent to which copies of questionable, GPT-fabricated papers were available in various repositories, archives, citation databases, and social media platforms.
• Roughly two-thirds of the retrieved papers were found to have been produced, at least in part, through undisclosed, potentially deceptive use of GPT. The majority (57%) of these questionable papers dealt with policy-relevant subjects (i.e., environment, health, computing), susceptible to influence operations. Most were available in several copies on different domains (e.g., social media, archives, and repositories).
• Two main risks arise from the increasingly common use of GPT to (mass-)produce fake, scientific publications. First, the abundance of fabricated “studies” seeping into all areas of the research infrastructure threatens to overwhelm the scholarly communication system and jeopardize the integrity of the scientific record. A second risk lies in the increased possibility that convincingly scientific-looking content was in fact deceitfully created with AI tools and is also optimized to be retrieved by publicly available academic search engines, particularly Google Scholar. However small, this possibility and awareness of it risks undermining the basis for trust in scientific knowledge and poses serious societal risks.

Implications

The use of ChatGPT to generate text for academic papers has raised concerns about research integrity. Discussion of this phenomenon is ongoing in editorials, commentaries, opinion pieces, and on social media (Bom, 2023; Stokel-Walker, 2024; Thorp, 2023). There are now several lists of papers suspected of GPT misuse, and new papers are constantly being added. 1 See for example Academ-AI, https://www.academ-ai.info/ , and Retraction Watch, https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/ . While many legitimate uses of GPT for research and academic writing exist (Huang & Tan, 2023; Kitamura, 2023; Lund et al., 2023), its undeclared use—beyond proofreading—has potentially far-reaching implications for both science and society, but especially for their relationship. It, therefore, seems important to extend the discussion to one of the most accessible and well-known intermediaries between science, but also certain types of misinformation, and the public, namely Google Scholar, also in response to the legitimate concerns that the discussion of generative AI and misinformation needs to be more nuanced and empirically substantiated  (Simon et al., 2023).

Google Scholar, https://scholar.google.com , is an easy-to-use academic search engine. It is available for free, and its index is extensive (Gusenbauer & Haddaway, 2020). It is also often touted as a credible source for academic literature and even recommended in library guides, by media and information literacy initiatives, and fact checkers (Tripodi et al., 2023). However, Google Scholar lacks the transparency and adherence to standards that usually characterize citation databases. Instead, Google Scholar uses automated crawlers, like Google’s web search engine (Martín-Martín et al., 2021), and the inclusion criteria are based on primarily technical standards, allowing any individual author—with or without scientific affiliation—to upload papers to be indexed (Google Scholar Help, n.d.). It has been shown that Google Scholar is susceptible to manipulation through citation exploits (Antkare, 2020) and by providing access to fake scientific papers (Dadkhah et al., 2017). A large part of Google Scholar’s index consists of publications from established scientific journals or other forms of quality-controlled, scholarly literature. However, the index also contains a large amount of gray literature, including student papers, working papers, reports, preprint servers, and academic networking sites, as well as material from so-called “questionable” academic journals, including paper mills. The search interface does not offer the possibility to filter the results meaningfully by material type, publication status, or form of quality control, such as limiting the search to peer-reviewed material.

To understand the occurrence of ChatGPT (co-)authored work in Google Scholar’s index, we scraped it for publications, including one of two common ChatGPT responses (see Appendix A) that we encountered on social media and in media reports (DeGeurin, 2024). The results of our descriptive statistical analyses showed that around 62% did not declare the use of GPTs. Most of these GPT-fabricated papers were found in non-indexed journals and working papers, but some cases included research published in mainstream scientific journals and conference proceedings. 2 Indexed journals mean scholarly journals indexed by abstract and citation databases such as Scopus and Web of Science, where the indexation implies journals with high scientific quality. Non-indexed journals are journals that fall outside of this indexation. More than half (57%) of these GPT-fabricated papers concerned policy-relevant subject areas susceptible to influence operations. To avoid increasing the visibility of these publications, we abstained from referencing them in this research note. However, we have made the data available in the Harvard Dataverse repository.

The publications were related to three issue areas—health (14.5%), environment (19.5%) and computing (23%)—with key terms such “healthcare,” “COVID-19,” or “infection”for health-related papers, and “analysis,” “sustainable,” and “global” for environment-related papers. In several cases, the papers had titles that strung together general keywords and buzzwords, thus alluding to very broad and current research. These terms included “biology,” “telehealth,” “climate policy,” “diversity,” and “disrupting,” to name just a few.  While the study’s scope and design did not include a detailed analysis of which parts of the articles included fabricated text, our dataset did contain the surrounding sentences for each occurrence of the suspicious phrases that formed the basis for our search and subsequent selection. Based on that, we can say that the phrases occurred in most sections typically found in scientific publications, including the literature review, methods, conceptual and theoretical frameworks, background, motivation or societal relevance, and even discussion. This was confirmed during the joint coding, where we read and discussed all articles. It became clear that not just the text related to the telltale phrases was created by GPT, but that almost all articles in our sample of questionable articles likely contained traces of GPT-fabricated text everywhere.

Evidence hacking and backfiring effects

Generative pre-trained transformers (GPTs) can be used to produce texts that mimic scientific writing. These texts, when made available online—as we demonstrate—leak into the databases of academic search engines and other parts of the research infrastructure for scholarly communication. This development exacerbates problems that were already present with less sophisticated text generators (Antkare, 2020; Cabanac & Labbé, 2021). Yet, the public release of ChatGPT in 2022, together with the way Google Scholar works, has increased the likelihood of lay people (e.g., media, politicians, patients, students) coming across questionable (or even entirely GPT-fabricated) papers and other problematic research findings. Previous research has emphasized that the ability to determine the value and status of scientific publications for lay people is at stake when misleading articles are passed off as reputable (Haider & Åström, 2017) and that systematic literature reviews risk being compromised (Dadkhah et al., 2017). It has also been highlighted that Google Scholar, in particular, can be and has been exploited for manipulating the evidence base for politically charged issues and to fuel conspiracy narratives (Tripodi et al., 2023). Both concerns are likely to be magnified in the future, increasing the risk of what we suggest calling evidence hacking —the strategic and coordinated malicious manipulation of society’s evidence base.

The authority of quality-controlled research as evidence to support legislation, policy, politics, and other forms of decision-making is undermined by the presence of undeclared GPT-fabricated content in publications professing to be scientific. Due to the large number of archives, repositories, mirror sites, and shadow libraries to which they spread, there is a clear risk that GPT-fabricated, questionable papers will reach audiences even after a possible retraction. There are considerable technical difficulties involved in identifying and tracing computer-fabricated papers (Cabanac & Labbé, 2021; Dadkhah et al., 2023; Jones, 2024), not to mention preventing and curbing their spread and uptake.

However, as the rise of the so-called anti-vaxx movement during the COVID-19 pandemic and the ongoing obstruction and denial of climate change show, retracting erroneous publications often fuels conspiracies and increases the following of these movements rather than stopping them. To illustrate this mechanism, climate deniers frequently question established scientific consensus by pointing to other, supposedly scientific, studies that support their claims. Usually, these are poorly executed, not peer-reviewed, based on obsolete data, or even fraudulent (Dunlap & Brulle, 2020). A similar strategy is successful in the alternative epistemic world of the global anti-vaccination movement (Carrion, 2018) and the persistence of flawed and questionable publications in the scientific record already poses significant problems for health research, policy, and lawmakers, and thus for society as a whole (Littell et al., 2024). Considering that a person’s support for “doing your own research” is associated with increased mistrust in scientific institutions (Chinn & Hasell, 2023), it will be of utmost importance to anticipate and consider such backfiring effects already when designing a technical solution, when suggesting industry or legal regulation, and in the planning of educational measures.

Recommendations

Solutions should be based on simultaneous considerations of technical, educational, and regulatory approaches, as well as incentives, including social ones, across the entire research infrastructure. Paying attention to how these approaches and incentives relate to each other can help identify points and mechanisms for disruption. Recognizing fraudulent academic papers must happen alongside understanding how they reach their audiences and what reasons there might be for some of these papers successfully “sticking around.” A possible way to mitigate some of the risks associated with GPT-fabricated scholarly texts finding their way into academic search engine results would be to provide filtering options for facets such as indexed journals, gray literature, peer-review, and similar on the interface of publicly available academic search engines. Furthermore, evaluation tools for indexed journals 3 Such as LiU Journal CheckUp, https://ep.liu.se/JournalCheckup/default.aspx?lang=eng . could be integrated into the graphical user interfaces and the crawlers of these academic search engines. To enable accountability, it is important that the index (database) of such a search engine is populated according to criteria that are transparent, open to scrutiny, and appropriate to the workings of  science and other forms of academic research. Moreover, considering that Google Scholar has no real competitor, there is a strong case for establishing a freely accessible, non-specialized academic search engine that is not run for commercial reasons but for reasons of public interest. Such measures, together with educational initiatives aimed particularly at policymakers, science communicators, journalists, and other media workers, will be crucial to reducing the possibilities for and effects of malicious manipulation or evidence hacking. It is important not to present this as a technical problem that exists only because of AI text generators but to relate it to the wider concerns in which it is embedded. These range from a largely dysfunctional scholarly publishing system (Haider & Åström, 2017) and academia’s “publish or perish” paradigm to Google’s near-monopoly and ideological battles over the control of information and ultimately knowledge. Any intervention is likely to have systemic effects; these effects need to be considered and assessed in advance and, ideally, followed up on.

Our study focused on a selection of papers that were easily recognizable as fraudulent. We used this relatively small sample as a magnifying glass to examine, delineate, and understand a problem that goes beyond the scope of the sample itself, which however points towards larger concerns that require further investigation. The work of ongoing whistleblowing initiatives 4 Such as Academ-AI, https://www.academ-ai.info/ , and Retraction Watch, https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/ . , recent media reports of journal closures (Subbaraman, 2024), or GPT-related changes in word use and writing style (Cabanac et al., 2021; Stokel-Walker, 2024) suggest that we only see the tip of the iceberg. There are already more sophisticated cases (Dadkhah et al., 2023) as well as cases involving fabricated images (Gu et al., 2022). Our analysis shows that questionable and potentially manipulative GPT-fabricated papers permeate the research infrastructure and are likely to become a widespread phenomenon. Our findings underline that the risk of fake scientific papers being used to maliciously manipulate evidence (see Dadkhah et al., 2017) must be taken seriously. Manipulation may involve undeclared automatic summaries of texts, inclusion in literature reviews, explicit scientific claims, or the concealment of errors in studies so that they are difficult to detect in peer review. However, the mere possibility of these things happening is a significant risk in its own right that can be strategically exploited and will have ramifications for trust in and perception of science. Society’s methods of evaluating sources and the foundations of media and information literacy are under threat and public trust in science is at risk of further erosion, with far-reaching consequences for society in dealing with information disorders. To address this multifaceted problem, we first need to understand why it exists and proliferates.

Finding 1: 139 GPT-fabricated, questionable papers were found and listed as regular results on the Google Scholar results page. Non-indexed journals dominate.

Most questionable papers we found were in non-indexed journals or were working papers, but we did also find some in established journals, publications, conferences, and repositories. We found a total of 139 papers with a suspected deceptive use of ChatGPT or similar LLM applications (see Table 1). Out of these, 19 were in indexed journals, 89 were in non-indexed journals, 19 were student papers found in university databases, and 12 were working papers (mostly in preprint databases). Table 1 divides these papers into categories. Health and environment papers made up around 34% (47) of the sample. Of these, 66% were present in non-indexed journals.

Finding 2: GPT-fabricated, questionable papers are disseminated online, permeating the research infrastructure for scholarly communication, often in multiple copies. Applied topics with practical implications dominate.

The 20 papers concerning health-related issues are distributed across 20 unique domains, accounting for 46 URLs. The 27 papers dealing with environmental issues can be found across 26 unique domains, accounting for 56 URLs.  Most of the identified papers exist in multiple copies and have already spread to several archives, repositories, and social media. It would be difficult, or impossible, to remove them from the scientific record.

As apparent from Table 2, GPT-fabricated, questionable papers are seeping into most parts of the online research infrastructure for scholarly communication. Platforms on which identified papers have appeared include ResearchGate, ORCiD, Journal of Population Therapeutics and Clinical Pharmacology (JPTCP), Easychair, Frontiers, the Institute of Electrical and Electronics Engineer (IEEE), and X/Twitter. Thus, even if they are retracted from their original source, it will prove very difficult to track, remove, or even just mark them up on other platforms. Moreover, unless regulated, Google Scholar will enable their continued and most likely unlabeled discoverability.

A word rain visualization (Centre for Digital Humanities Uppsala, 2023), which combines word prominences through TF-IDF 5 Term frequency–inverse document frequency , a method for measuring the significance of a word in a document compared to its frequency across all documents in a collection. scores with semantic similarity of the full texts of our sample of GPT-generated articles that fall into the “Environment” and “Health” categories, reflects the two categories in question. However, as can be seen in Figure 1, it also reveals overlap and sub-areas. The y-axis shows word prominences through word positions and font sizes, while the x-axis indicates semantic similarity. In addition to a certain amount of overlap, this reveals sub-areas, which are best described as two distinct events within the word rain. The event on the left bundles terms related to the development and management of health and healthcare with “challenges,” “impact,” and “potential of artificial intelligence”emerging as semantically related terms. Terms related to research infrastructures, environmental, epistemic, and technological concepts are arranged further down in the same event (e.g., “system,” “climate,” “understanding,” “knowledge,” “learning,” “education,” “sustainable”). A second distinct event further to the right bundles terms associated with fish farming and aquatic medicinal plants, highlighting the presence of an aquaculture cluster.  Here, the prominence of groups of terms such as “used,” “model,” “-based,” and “traditional” suggests the presence of applied research on these topics. The two events making up the word rain visualization, are linked by a less dominant but overlapping cluster of terms related to “energy” and “water.”

The bar chart of the terms in the paper subset (see Figure 2) complements the word rain visualization by depicting the most prominent terms in the full texts along the y-axis. Here, word prominences across health and environment papers are arranged descendingly, where values outside parentheses are TF-IDF values (relative frequencies) and values inside parentheses are raw term frequencies (absolute frequencies).

Finding 3: Google Scholar presents results from quality-controlled and non-controlled citation databases on the same interface, providing unfiltered access to GPT-fabricated questionable papers.

Google Scholar’s central position in the publicly accessible scholarly communication infrastructure, as well as its lack of standards, transparency, and accountability in terms of inclusion criteria, has potentially serious implications for public trust in science. This is likely to exacerbate the already-known potential to exploit Google Scholar for evidence hacking (Tripodi et al., 2023) and will have implications for any attempts to retract or remove fraudulent papers from their original publication venues. Any solution must consider the entirety of the research infrastructure for scholarly communication and the interplay of different actors, interests, and incentives.

We searched and scraped Google Scholar using the Python library Scholarly (Cholewiak et al., 2023) for papers that included specific phrases known to be common responses from ChatGPT and similar applications with the same underlying model (GPT3.5 or GPT4): “as of my last knowledge update” and/or “I don’t have access to real-time data” (see Appendix A). This facilitated the identification of papers that likely used generative AI to produce text, resulting in 227 retrieved papers. The papers’ bibliographic information was automatically added to a spreadsheet and downloaded into Zotero. 6 An open-source reference manager, https://zotero.org .

We employed multiple coding (Barbour, 2001) to classify the papers based on their content. First, we jointly assessed whether the paper was suspected of fraudulent use of ChatGPT (or similar) based on how the text was integrated into the papers and whether the paper was presented as original research output or the AI tool’s role was acknowledged. Second, in analyzing the content of the papers, we continued the multiple coding by classifying the fraudulent papers into four categories identified during an initial round of analysis—health, environment, computing, and others—and then determining which subjects were most affected by this issue (see Table 1). Out of the 227 retrieved papers, 88 papers were written with legitimate and/or declared use of GPTs (i.e., false positives, which were excluded from further analysis), and 139 papers were written with undeclared and/or fraudulent use (i.e., true positives, which were included in further analysis). The multiple coding was conducted jointly by all authors of the present article, who collaboratively coded and cross-checked each other’s interpretation of the data simultaneously in a shared spreadsheet file. This was done to single out coding discrepancies and settle coding disagreements, which in turn ensured methodological thoroughness and analytical consensus (see Barbour, 2001). Redoing the category coding later based on our established coding schedule, we achieved an intercoder reliability (Cohen’s kappa) of 0.806 after eradicating obvious differences.

The ranking algorithm of Google Scholar prioritizes highly cited and older publications (Martín-Martín et al., 2016). Therefore, the position of the articles on the search engine results pages was not particularly informative, considering the relatively small number of results in combination with the recency of the publications. Only the query “as of my last knowledge update” had more than two search engine result pages. On those, questionable articles with undeclared use of GPTs were evenly distributed across all result pages (min: 4, max: 9, mode: 8), with the proportion of undeclared use being slightly higher on average on later search result pages.

To understand how the papers making fraudulent use of generative AI were disseminated online, we programmatically searched for the paper titles (with exact string matching) in Google Search from our local IP address (see Appendix B) using the googlesearch – python library(Vikramaditya, 2020). We manually verified each search result to filter out false positives—results that were not related to the paper—and then compiled the most prominent URLs by field. This enabled the identification of other platforms through which the papers had been spread. We did not, however, investigate whether copies had spread into SciHub or other shadow libraries, or if they were referenced in Wikipedia.

We used descriptive statistics to count the prevalence of the number of GPT-fabricated papers across topics and venues and top domains by subject. The pandas software library for the Python programming language (The pandas development team, 2024) was used for this part of the analysis. Based on the multiple coding, paper occurrences were counted in relation to their categories, divided into indexed journals, non-indexed journals, student papers, and working papers. The schemes, subdomains, and subdirectories of the URL strings were filtered out while top-level domains and second-level domains were kept, which led to normalizing domain names. This, in turn, allowed the counting of domain frequencies in the environment and health categories. To distinguish word prominences and meanings in the environment and health-related GPT-fabricated questionable papers, a semantically-aware word cloud visualization was produced through the use of a word rain (Centre for Digital Humanities Uppsala, 2023) for full-text versions of the papers. Font size and y-axis positions indicate word prominences through TF-IDF scores for the environment and health papers (also visualized in a separate bar chart with raw term frequencies in parentheses), and words are positioned along the x-axis to reflect semantic similarity (Skeppstedt et al., 2024), with an English Word2vec skip gram model space (Fares et al., 2017). An English stop word list was used, along with a manually produced list including terms such as “https,” “volume,” or “years.”

• Artificial Intelligence
• / Search engines

Cite this Essay

Haider, J., Söderström, K. R., Ekström, B., & Rödl, M. (2024). GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation. Harvard Kennedy School (HKS) Misinformation Review . https://doi.org/10.37016/mr-2020-156

• / Appendix B

Bibliography

Antkare, I. (2020). Ike Antkare, his publications, and those of his disciples. In M. Biagioli & A. Lippman (Eds.), Gaming the metrics (pp. 177–200). The MIT Press. https://doi.org/10.7551/mitpress/11087.003.0018

Barbour, R. S. (2001). Checklists for improving rigour in qualitative research: A case of the tail wagging the dog? BMJ , 322 (7294), 1115–1117. https://doi.org/10.1136/bmj.322.7294.1115

Bom, H.-S. H. (2023). Exploring the opportunities and challenges of ChatGPT in academic writing: A roundtable discussion. Nuclear Medicine and Molecular Imaging , 57 (4), 165–167. https://doi.org/10.1007/s13139-023-00809-2

Cabanac, G., & Labbé, C. (2021). Prevalence of nonsensical algorithmically generated papers in the scientific literature. Journal of the Association for Information Science and Technology , 72 (12), 1461–1476. https://doi.org/10.1002/asi.24495

Cabanac, G., Labbé, C., & Magazinov, A. (2021). Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals . arXiv. https://doi.org/10.48550/arXiv.2107.06751

Carrion, M. L. (2018). “You need to do your research”: Vaccines, contestable science, and maternal epistemology. Public Understanding of Science , 27 (3), 310–324. https://doi.org/10.1177/0963662517728024

Centre for Digital Humanities Uppsala (2023). CDHUppsala/word-rain [Computer software]. https://github.com/CDHUppsala/word-rain

Chinn, S., & Hasell, A. (2023). Support for “doing your own research” is associated with COVID-19 misperceptions and scientific mistrust. Harvard Kennedy School (HSK) Misinformation Review, 4 (3). https://doi.org/10.37016/mr-2020-117

Cholewiak, S. A., Ipeirotis, P., Silva, V., & Kannawadi, A. (2023). SCHOLARLY: Simple access to Google Scholar authors and citation using Python (1.5.0) [Computer software]. https://doi.org/10.5281/zenodo.5764801

Dadkhah, M., Lagzian, M., & Borchardt, G. (2017). Questionable papers in citation databases as an issue for literature review. Journal of Cell Communication and Signaling , 11 (2), 181–185. https://doi.org/10.1007/s12079-016-0370-6

Dadkhah, M., Oermann, M. H., Hegedüs, M., Raman, R., & Dávid, L. D. (2023). Detection of fake papers in the era of artificial intelligence. Diagnosis , 10 (4), 390–397. https://doi.org/10.1515/dx-2023-0090

DeGeurin, M. (2024, March 19). AI-generated nonsense is leaking into scientific journals. Popular Science. https://www.popsci.com/technology/ai-generated-text-scientific-journals/

Dunlap, R. E., & Brulle, R. J. (2020). Sources and amplifiers of climate change denial. In D.C. Holmes & L. M. Richardson (Eds.), Research handbook on communicating climate change (pp. 49–61). Edward Elgar Publishing. https://doi.org/10.4337/9781789900408.00013

Fares, M., Kutuzov, A., Oepen, S., & Velldal, E. (2017). Word vectors, reuse, and replicability: Towards a community repository of large-text resources. In J. Tiedemann & N. Tahmasebi (Eds.), Proceedings of the 21st Nordic Conference on Computational Linguistics (pp. 271–276). Association for Computational Linguistics. https://aclanthology.org/W17-0237

Google Scholar Help. (n.d.). Inclusion guidelines for webmasters . https://scholar.google.com/intl/en/scholar/inclusion.html

Gu, J., Wang, X., Li, C., Zhao, J., Fu, W., Liang, G., & Qiu, J. (2022). AI-enabled image fraud in scientific publications. Patterns , 3 (7), 100511. https://doi.org/10.1016/j.patter.2022.100511

Gusenbauer, M., & Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Research Synthesis Methods , 11 (2), 181–217.   https://doi.org/10.1002/jrsm.1378

Haider, J., & Åström, F. (2017). Dimensions of trust in scholarly communication: Problematizing peer review in the aftermath of John Bohannon’s “Sting” in science. Journal of the Association for Information Science and Technology , 68 (2), 450–467. https://doi.org/10.1002/asi.23669

Huang, J., & Tan, M. (2023). The role of ChatGPT in scientific communication: Writing better scientific review articles. American Journal of Cancer Research , 13 (4), 1148–1154. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10164801/

Jones, N. (2024). How journals are fighting back against a wave of questionable images. Nature , 626 (8000), 697–698. https://doi.org/10.1038/d41586-024-00372-6

Kitamura, F. C. (2023). ChatGPT is shaping the future of medical writing but still requires human judgment. Radiology , 307 (2), e230171. https://doi.org/10.1148/radiol.230171

Littell, J. H., Abel, K. M., Biggs, M. A., Blum, R. W., Foster, D. G., Haddad, L. B., Major, B., Munk-Olsen, T., Polis, C. B., Robinson, G. E., Rocca, C. H., Russo, N. F., Steinberg, J. R., Stewart, D. E., Stotland, N. L., Upadhyay, U. D., & Ditzhuijzen, J. van. (2024). Correcting the scientific record on abortion and mental health outcomes. BMJ , 384 , e076518. https://doi.org/10.1136/bmj-2023-076518

Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023). ChatGPT and a new academic reality: Artificial Intelligence-written research papers and the ethics of the large language models in scholarly publishing. Journal of the Association for Information Science and Technology, 74 (5), 570–581. https://doi.org/10.1002/asi.24750

Martín-Martín, A., Orduna-Malea, E., Ayllón, J. M., & Delgado López-Cózar, E. (2016). Back to the past: On the shoulders of an academic search engine giant. Scientometrics , 107 , 1477–1487. https://doi.org/10.1007/s11192-016-1917-2

Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2021). Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A multidisciplinary comparison of coverage via citations. Scientometrics , 126 (1), 871–906. https://doi.org/10.1007/s11192-020-03690-4

Simon, F. M., Altay, S., & Mercier, H. (2023). Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. Harvard Kennedy School (HKS) Misinformation Review, 4 (5). https://doi.org/10.37016/mr-2020-127

Skeppstedt, M., Ahltorp, M., Kucher, K., & Lindström, M. (2024). From word clouds to Word Rain: Revisiting the classic word cloud to visualize climate change texts. Information Visualization , 23 (3), 217–238. https://doi.org/10.1177/14738716241236188

Swedish Research Council. (2017). Good research practice. Vetenskapsrådet.

Stokel-Walker, C. (2024, May 1.). AI Chatbots Have Thoroughly Infiltrated Scientific Publishing . Scientific American. https://www.scientificamerican.com/article/chatbots-have-thoroughly-infiltrated-scientific-publishing/

Subbaraman, N. (2024, May 14). Flood of fake science forces multiple journal closures: Wiley to shutter 19 more journals, some tainted by fraud. The Wall Street Journal . https://www.wsj.com/science/academic-studies-research-paper-mills-journals-publishing-f5a3d4bc

The pandas development team. (2024). pandas-dev/pandas: Pandas (v2.2.2) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.10957263

Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science , 379 (6630), 313–313. https://doi.org/10.1126/science.adg7879

Tripodi, F. B., Garcia, L. C., & Marwick, A. E. (2023). ‘Do your own research’: Affordance activation and disinformation spread. Information, Communication & Society , 27 (6), 1212–1228. https://doi.org/10.1080/1369118X.2023.2245869

Vikramaditya, N. (2020). Nv7-GitHub/googlesearch [Computer software]. https://github.com/Nv7-GitHub/googlesearch

This research has been supported by Mistra, the Swedish Foundation for Strategic Environmental Research, through the research program Mistra Environmental Communication (Haider, Ekström, Rödl) and the Marcus and Amalia Wallenberg Foundation [2020.0004] (Söderström).

Competing Interests

The authors declare no competing interests.

The research described in this article was carried out under Swedish legislation. According to the relevant EU and Swedish legislation (2003:460) on the ethical review of research involving humans (“Ethical Review Act”), the research reported on here is not subject to authorization by the Swedish Ethical Review Authority (“etikprövningsmyndigheten”) (SRC, 2017).

This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided that the original author and source are properly credited.

Data Availability

All data needed to replicate this study are available at the Harvard Dataverse: https://doi.org/10.7910/DVN/WUVD8X

Acknowledgements

The authors wish to thank two anonymous reviewers for their valuable comments on the article manuscript as well as the editorial group of Harvard Kennedy School (HKS) Misinformation Review for their thoughtful feedback and input.