qualitative research bias how to minimize it

Research Gaps

Because every research has a research gap that needs to be filled

Minimizing Bias in Qualitative Research: Strategies for Ensuring Validity and Reliability

Qualitative research, with its emphasis on understanding human experiences and perspectives, plays a vital role in various fields. However, like any research methodology, it is susceptible to bias, which can threaten the validity and reliability of findings. Therefore, researchers must actively strive to minimize bias throughout the research process. This blog post delves into the importance of minimizing bias in qualitative research and explores various strategies to achieve this goal.

Recognizing and Understanding Bias 

The first step towards minimizing bias is acknowledging its existence. Bias can creep into research in various ways, often subconsciously, influencing how researchers design studies, collect data, analyze results, and interpret findings. Recognizing potential sources of bias is crucial for developing strategies to mitigate their impact.

Types of Bias in Qualitative Research

Researchers should be aware of different types of bias that can affect qualitative research. These include:

• Design bias: Occurs when the research design itself favors certain outcomes or perspectives. For instance, if a study is designed in such a way that it only includes participants from a specific demographic or uses a particular method that leans towards a certain result, it can lead to design bias. This can skew the results and make them less representative of the broader population or phenomenon being studied.
• Selection bias: Arises when the sample of participants is not representative of the population being studied. For example, if a study on dietary habits only includes participants from a health club, the results may not represent the dietary habits of the general population, as people who attend a health club may have different dietary habits compared to those who do not.
• Omission bias: Occurs when important data is excluded from the analysis. For example, if a researcher conducting a study on the effects of a new drug only includes positive results and excludes negative ones, this would be an example of omission bias. The results of the study would then be skewed, as they do not take into account all relevant data.
• Inclusive bias: Occurs when irrelevant data is included in the analysis. For instance, if a researcher is studying the impact of diet on heart disease and includes data about participants’ favorite colors, this would be an example of inclusive bias. The favorite color is likely irrelevant to the development of heart disease and its inclusion could confuse the analysis.
• Procedural bias: Occurs due to inconsistencies in data collection or analysis procedures. For example, if a researcher uses different methods to collect data from different participants, or if the criteria used to analyze data changes during the course of the study, this could introduce procedural bias. The results may then not accurately reflect the phenomenon being studied, but rather the inconsistencies in the procedures used.
• Measurement bias: Occurs when the research instruments or methods used to collect data are flawed. For example, if a researcher uses a faulty scale to measure weight in a study on obesity, the data collected would be inaccurate, leading to measurement bias. Similarly, if a survey question is poorly worded or ambiguous, it could lead to inconsistent responses, introducing measurement bias.
• Interviewer bias: Occurs when the interviewer’s own beliefs or expectations influence how they interact with participants and interpret their responses. For example, if an interviewer has strong opinions about a topic, they may unconsciously lead participants to respond in a certain way, or they may interpret responses in a way that aligns with their own beliefs. Similarly, characteristics such as the interviewer’s age, gender, or race may influence how participants respond.
• Response bias: Occurs when participants intentionally or unintentionally misrepresent their experiences or opinions. For example, participants might give socially desirable responses, or they might try to guess what the researcher is looking for and tailor their responses accordingly. They might also misunderstand the question, forget relevant information, or exaggerate their responses.
• Reporting bias: Occurs when researchers selectively report findings that support their hypotheses or preconceived notions. For example, if a researcher only reports positive outcomes of a clinical trial and neglects to mention negative or neutral outcomes, this would be an example of reporting bias. The results of the study would then not accurately reflect the true effects of the treatment being studied.

Strategies for Minimizing Bias

Several strategies can be employed to minimize bias in qualitative research. These include:

• Multiple Coders: Using multiple researchers to code the data can help identify and address individual biases. Consistency in interpretation across coders strengthens the validity of findings.
• Participant Review: Allowing participants to review and provide feedback on the research findings can help ensure that their perspectives are accurately represented.
• Reflexivity, Peer Debriefing, and Triangulation: Engaging in reflexivity, where researchers critically examine their own biases and assumptions, can help mitigate their influence on the research process. Additionally, peer debriefing with colleagues and using triangulation, where data is collected from multiple sources and methods, can further strengthen the objectivity of the research.
• Changing Experimental Design: When unavoidable omission bias is identified, researchers can consider modifying the research design to address it. This may involve expanding the scope of the study or including additional data sources.
• Respecting Participants: Ensuring that participants are treated with respect and are not coerced into participating is essential for protecting them from exploitation and minimizing response bias.

Minimizing bias in qualitative research is an ongoing process that requires researchers to be vigilant and proactive. By employing the strategies discussed above, researchers can enhance the validity and reliability of their findings, ensuring that they accurately reflect the lived experiences and perspectives of the participants they study. Remember, while complete elimination of bias may be challenging, these strategies can help ensure that research findings remain as unbiased and faithful representations of participants’ perspectives as possible.

Related Posts

Delving Deeper: A Comprehensive Review of “A Guide to the Professional Interview”

15 April، 2024 15 April، 2024

Mixed Method Overview

How to Choose Between Positivism and Constructivism for Your Research

24 December، 2023 24 December، 2023

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Pin It on Pinterest

How to Avoid Bias in Qualitative Research

You can also listen to this article as an audio recording.

Research bias occurs when researchers try to influence the results of their work, in order to get the outcome they want. Often, researchers may not be aware they are doing this. Whether they are aware or not, such behavior clearly severely affects the impartiality of a study and greatly reduces the value of the results.

The Issues in Qualitative Research

Recently, I discussed the problem of bias with a researcher friend.

“I heard that research bias is a bigger problem for qualitative research than quantitative research.”

“Why is that?”

“Qualitative research relies more on the experience and judgment of the researcher. Also, the type of data collected is subjective and unique to the person or situation. So it is much harder to avoid bias than in quantitative research.”

“Are there ways to avoid bias ?”

“A good start is to recognize that bias exists in all research. We can then try to predict what type of bias we might have in our study, and try to avoid it as much as possible.”

Types of Bias in Research

“Are there different types of bias to watch out for?”

• There’s design bias , where the researcher does not consider bias in the design of the study. Factors like sample size , the range of participants, for example – all of these can cause bias.
• Next there’s also selection or sampling bias . For example, you might omit people of certain ages or ethnicities from your study. This is called omission bias. The other type, inclusive bias, is when you select a sample just because it is convenient. For example, if the people you select for your study are all college students, they are likely to share many characteristics.”

“Are there more?”

“Yes, there are lots of different types of bias.

• There’s procedural bias , where the way you carry out a study affects the results. For example, if you give people only a short time to answer questions, their responses will be rushed.
• There’s also measurement bias that can happen if the equipment you are using is faulty, or you are not using it correctly.”

“That’s a lot to think about.”

“I can think of three more.

• There’s interviewer bias , which is very hard to avoid. This is when an interviewer subconsciously influences the responses of the interviewee. Their body language might indicate their opinion, for example.
• Furthermore, there’s response bias , where someone tries to give the answers they think are “correct.”
• Finally, there’s reporting bias . This is often outside the researcher’s control. It means that research with positive, or exciting, results is far more likely to be reported, so can seem more critical.”

How to Avoid Bias in Research

“With so many types of bias, how can it be avoided?”

“There are a number of things the researcher can do to avoid bias.

• Read the guidelines : Check the guidelines of your institution or sponsor and make sure you follow them.
• Think about our objectives : Plan your study early. Be clear about what you want to achieve, and how. This will help to avoid bias when you start collecting data.”

“And next?”

• Maintain records : Keep detailed records. This reduces the chance of making mistakes.
• Be honest when reporting : Make sure you include all your results in your report. Even the results that don’t seem important. Finally, be honest about the limitations of your study in your report.”

Avoiding Participant Bias

“That explains what researchers can do. But what about participant bias?”

“Try asking indirect questions. People might change their answers to direct questions to make a good impression. But if you ask them what a friend or colleague might think, you might get a more honest response.”

“Are open-ended questions useful?”

“Yes. They allow information to flow more freely, by not forcing a limited set of answers. But even these should be used with caution . You should try to be impartial about all parts of the study, and avoid implying that there is a right answer. It might help to ask people to rate their responses on a scale of 1-5, for example, rather than agree/disagree.”

Reducing Researcher Bias

“All researchers should try to avoid confirmation bias. This is when you interpret your data in a way that supports your hypothesis. Secondly, you should make sure to analyze all your data, even if it doesn’t seem useful. Finally, always get an independent person to check your work, ideally several times during your study.”

Identifying and avoiding research bias in qualitative research is clearly tricky, with many different factors to consider. However, it is also vital. Biased research has little value; it is a waste of researchers’ valuable time and resources.

Learn even more about bias here . How did you overcome bias in your research? Share your experiences and thoughts in the comment section below.

Can you please tell me who is the author of this article: How to Avoid Bias in Qualitative Research Last updated May 3, 2019

Hi Nuzhat, the author of this article is Enago Academy

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Trending Now
• Upcoming Webinars
• Webinar Mobile App

Mastering Research Funding: A step-by-step guide to finding and winning grants

Identifying relevant funding opportunities Importance of eligibility criteria Understanding the funder’s perspective Crafting a strong…

• Career Corner

Academic Webinars: Transforming knowledge dissemination in the digital age

Digitization has transformed several areas of our lives, including the teaching and learning process. During…

• Manuscripts & Grants
• Reporting Research

Mastering Research Grant Writing in 2024: Navigating new policies and funder demands

Entering the world of grants and government funding can leave you confused; especially when trying…

How to Create a Poster That Stands Out: Tips for a smooth poster presentation

It was the conference season. Judy was excited to present her first poster! She had…

Academic Essay Writing Made Simple: 4 types and tips

The pen is mightier than the sword, they say, and nowhere is this more evident…

Recognizing the Signs: A guide to overcoming academic burnout

Intersectionality in Academia: Dealing with diverse perspectives

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

• Industry News
• Publishing Research
• AI in Academia
• Promoting Research
• Diversity and Inclusion
• Infographics
• Expert Video Library
• Other Resources
• Enago Learn
• Upcoming & On-Demand Webinars
• Peer Review Week 2024
• Open Access Week 2023
• Conference Videos
• Enago Report
• Journal Finder
• Enago Plagiarism & AI Grammar Check
• Editing Services
• Publication Support Services
• Research Impact
• Translation Services
• Publication solutions
• AI-Based Solutions
• Thought Leadership
• Call for Articles
• Call for Speakers
• Author Training
• Edit Profile

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

Which among these features would you prefer the most in a peer review assistant?

9 types of research bias and how to avoid them

To reduce the risk of bias in qual, researchers must focus on the human elements of the research process in order to identify and avoid the nine core types of bias.

Editor’s note: Rebecca Sarniak is a moderating services specialist iModerate , a Denver research firm.

Seasoned research experts know that bias can find its way into any research program – it’s naïve to think that any research could be 100 percent free from it. But when does bias become a problem? And how do we identify and control the sources of bias to deliver the highest-quality research possible?

The goal of reducing bias isn’t to make everyone the same but to make sure that questions are thoughtfully posed and delivered in a way that allows respondents to reveal their true feelings without distortions. The risk of bias exists in all components of qualitative research and can come from the questions, the respondents and the moderator. To reduce bias – and deliver better research – let’s explore its primary sources.  

When we focus on the human elements of the research process and look at the nine core types of bias – driven from the respondent, the researcher or both – we are able to minimize the potential impact that bias has on qualitative research.

Respondent bias

1. Acquiescence bias: Also known as “yea-saying” or the friendliness bias, acquiescence bias occurs when a respondent demonstrates a tendency to agree with and be positive about whatever the moderator presents. In other words, they think every idea is a good one and can see themselves liking, buying and acting upon every situation that is proposed. Some people have acquiescent personalities, while others acquiesce because they perceive the interviewer to be an expert. Acquiescence is the easy way out, as it takes less effort than carefully weighing each option. This path escalates if fatigue sets in – some people will agree just to complete the interview. To avoid it, researchers must replace questions that imply there is a right answer with those that focus on the respondent’s true point of view.

2. Social desirability bias 1 : This bias involves respondents answering questions in a way that they think will lead to being accepted and liked. Regardless of the research format, some people will report inaccurately on sensitive or personal topics to present themselves in the best possible light. Researchers can minimize this bias by focusing on unconditional positive regard. This includes phrasing questions to show it’s okay to answer in a way that is not socially desirable. Indirect questioning – asking about what a third-party thinks, feels and how they will behave – can also be used for socially sensitive questions. This allows respondents to project their own feelings onto others and still provide honest, representative answers.

3. Habituation 2 : In cases of habituation bias, respondents provide the same answers to questions that are worded in similar ways. This is a biological response: being responsive and paying attention takes a lot of energy. To conserve energy, our brains habituate or go on autopilot. Respondents often show signs of fatigue, such as mentioning that the questions seem repetitive, or start giving similar responses across multiple questions. Moderators must keep the engagement conversational and continue to vary question wording to minimize habituation.

4. Sponsor bias 3 : When respondents know – or suspect – the sponsor of the research, their feelings and opinions about that sponsor may bias their answers. Respondents’ views on the sponsoring organization’s mission or core beliefs, for example, can influence how they answer all questions related to that brand. This is an especially important type of bias for moderators to navigate by maintaining a neutral stance, limiting moderator reinforcement to positive respondent feedback that can be construed as moderator affiliation to brand and reiterating, when possible, the moderator’s independent status.   

Researcher bias

5. Confirmation bias 4 : One of the longest-recognized and most pervasive forms of bias in research, confirmation bias occurs when a researcher forms a hypothesis or belief and uses respondents’ information to confirm that belief. This takes place in-the-moment as researchers’ judge and weight responses that confirm their hypotheses as relevant and reliable, while dismissing evidence that doesn’t support a hypothesis. Confirmation bias then extends into analysis, with researchers tending to remember points that support their hypothesis and points that disprove other hypotheses. Confirmation bias is deeply seated in the natural tendencies people use to understand and filter information, which often lead to focusing on one hypothesis at a time. To minimize confirmation bias, researchers must continually reevaluate impressions of respondents and challenge preexisting assumptions and hypotheses.

6. Culture bias 5 : Assumptions about motivations and influences that are based on our cultural lens (on the spectrum of ethnocentricity or cultural relativity) create the culture bias. Ethnocentrism is judging another culture solely by the values and standards of one's own culture. Cultural relativism is the principle that an individual's beliefs and activities should be understood by others in terms of that individual's own culture. To minimize culture bias, researchers must move toward cultural relativism by showing unconditional positive regard and being cognizant of their own cultural assumptions. Complete cultural relativism is never 100 percent achievable.

7. Question-order bias: One question can influence answers to subsequent questions, creating question-order bias . Respondents are primed by the words and ideas presented in questions that impact their thoughts, feelings and attitudes on subsequent questions. For example, if a respondent rates one product a 10 and is then asked to rate a competitive product, they will make a rating that is relative to the 10 they just provided. While question-order bias is sometimes unavoidable, asking general questions before specific, unaided before aided and positive before negative will minimize bias.

8. Leading questions and wording bias 6 : Elaborating on a respondent’s answer puts words in their mouth and, while leading questions and wording aren’t types of bias themselves, they lead to bias or are a result of bias. Researchers do this because they are trying to confirm a hypothesis, build rapport or overestimate their understanding of the respondent. To minimize this bias, ask questions that use the respondents’ language and inquire about the implications of a respondent’s thoughts and reactions. Avoid summarizing what the respondents said in your own words and do not take what they said further. Try not to assume relationships between a feeling and a behavior.

9. The halo effect 7 : Moderators and respondents have a tendency to see something or someone in a certain light because of a single, positive attribute. There are several cognitive reasons for the halo effect, so researchers must work to address it on many fronts. For example, a moderator can make assumptions about a respondent because of one positive answer they’ve provided. Moderators should reflect on their assumptions about each respondent: Why are you asking each question? What is the assumption behind it? Additionally, respondents may rate or respond to a stimulus positively overall due to one factor. Researchers should address all questions about one brand before asking for feedback on a second brand, as when respondents are required to switch back and forth rating two brands, they are likely to project their opinion on one attribute to their opinion of the brand as a whole.

Bias in qualitative research can be minimized if you know what to look for and how to manage it. By asking quality questions at the right time and remaining aware and focused on sources of bias, researchers can enable the truest respondent perspectives and ensure that the resulting research lives up to the highest qualitative standards.  

1 Dodou, D., & de Winter, J. C. F. (2014). Social desirability is the same in offline, online and paper surveys: A meta-analysis. Computers in Human Behavior, 36, 487–495. doi: 10.1016/j.chb.2014.04.005.  https://www.iser.essex.ac.uk/research/publications/working-papers/iser/2013-04.pdf  

2 Habituation of event related potentials: a tool for assessment of cognition in headache patients Neelam Vaney, Abhinav Dixit, Tandra Ghosh, Ravi Gupta, M.S. Bhatia Departments of Physiology and Psychiatry, University College of Medical Sciences & G.T.B. Hospital, Dilshad Garden, Delhi-110095,  http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2605404/ .

3 Essentials or Marketing Research, An Applied Orientation By Naresh Malhotra, John Hall, Mike Shaw, Peter Oppenheim. Pp 227.  http://www.readexresearch.com/understanding-survey-data/ .

4 http://psy2.ucsd.edu/~mckenzie/nickersonConfirmationBias.pdf;  http://www.anderson.ucla.edu/faculty/keith.chen/negot.%20papers/RabinSchrag_ConfirmBias99.pdf  UCLA

5 Pirkey, W. (2015, May 6). Personal Interview. 

6 Essentials or Marketing Research, An Applied Orientation By Naresh Malhotra, John Hall, Mike Shaw, Peter Oppenheim. Pp 227.

7 Halo effects in consumer theories, Master Thesis, Erasmus University Rotterdam, thesis.eur.nl/pub/11759/Luttin,%20L.V.%20(352879ll).pdf  

Enhancing the virtual backroom experience Related Categories: Research Industry, Qualitative Research, Qualitative-Online, Marketing Research-General Research Industry, Qualitative Research, Qualitative-Online, Marketing Research-General, Focus Group-Videoconference, One-on-One (Depth) Interviews, Online Research

How to use laddering to uncover the emotions surrounding B2B jobs to be done Related Categories: Research Industry, Qualitative Research, Marketing Research-General Research Industry, Qualitative Research, Marketing Research-General, Advertising Research, Behavioral Economics, Bus.-To-Bus. Research, Business-To-Business, Consumer Research, Consumers, Psychological/Emotion Research

How AI can actually make research more 
people-centric Related Categories: Research Industry, Data Analysis, Marketing Research-General Research Industry, Data Analysis, Marketing Research-General, Artificial Intelligence / AI, Consumer Research, Consumers, Market Segmentation Studies, Segmentation Studies, Survey Research

Researchers are experienced and satisfied with current organizations, positions Related Categories: Research Industry, Data Analysis, Marketing Research-General Research Industry, Data Analysis, Marketing Research-General, Employee Opinion Studies, Employees, Research Employment

The Ultimate Guide to Qualitative Research - Part 1: The Basics

• Introduction and overview
• What is qualitative research?
• What is qualitative data?
• Examples of qualitative data
• Qualitative vs. quantitative research
• Mixed methods
• Qualitative research preparation
• Theoretical perspective
• Theoretical framework
• Literature reviews
• Research question
• Conceptual framework
• Conceptual vs. theoretical framework
• Data collection
• Qualitative research methods
• Focus groups
• Observational research
• Case studies
• Ethnographical research
• Ethical considerations
• Confidentiality and privacy

What is research bias?

Understanding unconscious bias, how to avoid bias in research, bias and subjectivity in research.

• Power dynamics
• Reflexivity

Bias in research

In a purely objective world, research bias would not exist because knowledge would be a fixed and unmovable resource; either one knows about a particular concept or phenomenon, or they don't. However, qualitative research and the social sciences both acknowledge that subjectivity and bias exist in every aspect of the social world, which naturally includes the research process too. This bias is manifest in the many different ways that knowledge is understood, constructed, and negotiated, both in and out of research.

Understanding research bias has profound implications for data collection methods and data analysis , requiring researchers to take particular care of how to account for the insights generated from their data .

Research bias, often unavoidable, is a systematic error that can creep into any stage of the research process , skewing our understanding and interpretation of findings. From data collection to analysis, interpretation , and even publication , bias can distort the truth we seek to capture and communicate in our research.

It’s also important to distinguish between bias and subjectivity, especially when engaging in qualitative research . Most qualitative methodologies are based on epistemological and ontological assumptions that there is no such thing as a fixed or objective world that exists “out there” that can be empirically measured and understood through research. Rather, many qualitative researchers embrace the socially constructed nature of our reality and thus recognize that all data is produced within a particular context by participants with their own perspectives and interpretations. Moreover, the researcher’s own subjective experiences inevitably shape how they make sense of the data. These subjectivities are considered to be strengths, not limitations, of qualitative research approaches, because they open new avenues for knowledge generation. This is also why reflexivity is so important in qualitative research. When we refer to bias in this guide, on the other hand, we are referring to systematic errors that can negatively affect the research process but that can be mitigated through researchers’ careful efforts.

To fully grasp what research bias is, it's essential to understand the dual nature of bias. Bias is not inherently evil. It's simply a tendency, inclination, or prejudice for or against something. In our daily lives, we're subject to countless biases, many of which are unconscious. They help us navigate our world, make quick decisions, and understand complex situations. But when conducting research, these same biases can cause significant issues.

Research bias can affect the validity and credibility of research findings, leading to erroneous conclusions. It can emerge from the researcher's subconscious preferences or the methodological design of the study itself. For instance, if a researcher unconsciously favors a particular outcome of the study, this preference could affect how they interpret the results, leading to a type of bias known as confirmation bias.

Research bias can also arise due to the characteristics of study participants. If the researcher selectively recruits participants who are more likely to produce desired outcomes, this can result in selection bias.

Another form of bias can stem from data collection methods . If a survey question is phrased in a way that encourages a particular response, this can introduce response bias. Moreover, inappropriate survey questions can have a detrimental effect on future research if such studies are seen by the general population as biased toward particular outcomes depending on the preferences of the researcher.

Bias can also occur during data analysis . In qualitative research for instance, the researcher's preconceived notions and expectations can influence how they interpret and code qualitative data, a type of bias known as interpretation bias. It's also important to note that quantitative research is not free of bias either, as sampling bias and measurement bias can threaten the validity of any research findings.

Given these examples, it's clear that research bias is a complex issue that can take many forms and emerge at any stage in the research process. This section will delve deeper into specific types of research bias, provide examples, discuss why it's an issue, and provide strategies for identifying and mitigating bias in research.

What is an example of bias in research?

Bias can appear in numerous ways. One example is confirmation bias, where the researcher has a preconceived explanation for what is going on in their data, and any disconfirming evidence is (unconsciously) ignored. For instance, a researcher conducting a study on daily exercise habits might be inclined to conclude that meditation practices lead to greater engagement in exercise because that researcher has personally experienced these benefits. However, conducting rigorous research entails assessing all the data systematically and verifying one’s conclusions by checking for both supporting and refuting evidence.

What is a common bias in research?

Confirmation bias is one of the most common forms of bias in research. It happens when researchers unconsciously focus on data that supports their ideas while ignoring or undervaluing data that contradicts their ideas. This bias can lead researchers to mistakenly confirm their theories, despite having insufficient or conflicting evidence.

What are the different types of bias?

There are several types of research bias, each presenting unique challenges. Some common types include:

Confirmation bias: As already mentioned, this happens when a researcher focuses on evidence supporting their theory while overlooking contradictory evidence.

Selection bias: This occurs when the researcher's method of choosing participants skews the sample in a particular direction.

Response bias: This happens when participants in a study respond inaccurately or falsely, often due to misleading or poorly worded questions.

Observer bias (or researcher bias): This occurs when the researcher unintentionally influences the results because of their expectations or preferences.

Publication bias: This type of bias arises when studies with positive results are more likely to get published, while studies with negative or null results are often ignored.

Analysis bias: This type of bias occurs when the data is manipulated or analyzed in a way that leads to a particular result, whether intentionally or unintentionally.

What is an example of researcher bias?

Researcher bias, also known as observer bias, can occur when a researcher's expectations or personal beliefs influence the results of a study. For instance, if a researcher believes that a particular therapy is effective, they might unconsciously interpret ambiguous results in a way that supports the efficacy of the therapy, even if the evidence is not strong enough.

Even quantitative research methodologies are not immune from bias from researchers. Market research surveys or clinical trial research, for example, may encounter bias when the researcher chooses a particular population or methodology to achieve a specific research outcome. Questions in customer feedback surveys whose data is employed in quantitative analysis can be structured in such a way as to bias survey respondents toward certain desired answers.

Turn your data into findings with ATLAS.ti

Key insights are at your fingertips with our powerful interface. See how with a free trial.

Identifying and avoiding bias in research

As we will remind you throughout this chapter, bias is not a phenomenon that can be removed altogether, nor should we think of it as something that should be eliminated. In a subjective world involving humans as researchers and research participants , bias is unavoidable and almost necessary for understanding social behavior. The section on reflexivity later in this guide will highlight how different perspectives among researchers and human subjects are addressed in qualitative research. That said, bias in excess can place the credibility of a study's findings into serious question. Scholars who read your research need to know what new knowledge you are generating, how it was generated, and why the knowledge you present should be considered persuasive. With that in mind, let's look at how bias can be identified and, where it interferes with research, minimized.

How do you identify bias in research?

Identifying bias involves a critical examination of your entire research study involving the formulation of the research question and hypothesis , the selection of study participants, the methods for data collection, and the analysis and interpretation of data. Researchers need to assess whether each stage has been influenced by bias that may have skewed the results. Tools such as bias checklists or guidelines, peer review , and reflexivity (reflecting on one's own biases) can be instrumental in identifying bias.

How do you identify research bias?

Identifying research bias often involves careful scrutiny of the research methodology and the researcher's interpretations. Was the sample of participants relevant to the research question ? Were the interview or survey questions leading? Were there any conflicts of interest that could have influenced the results? It also requires an understanding of the different types of bias and how they might manifest in a research context. Does the bias occur in the data collection process or when the researcher is analyzing data?

Research transparency requires a careful accounting of how the study was designed, conducted, and analyzed. In qualitative research involving human subjects, the researcher is responsible for documenting the characteristics of the research population and research context. With respect to research methods, the procedures and instruments used to collect and analyze data are described in as much detail as possible.

While describing study methodologies and research participants in painstaking detail may sound cumbersome, a clear and detailed description of the research design is necessary for good research. Without this level of detail, it is difficult for your research audience to identify whether bias exists, where bias occurs, and to what extent it may threaten the credibility of your findings.

How to recognize bias in a study?

Recognizing bias in a study requires a critical approach. The researcher should question every step of the research process: Was the sample of participants selected with care? Did the data collection methods encourage open and sincere responses? Did personal beliefs or expectations influence the interpretation of the results? External peer reviews can also be helpful in recognizing bias, as others might spot potential issues that the original researcher missed.

The subsequent sections of this chapter will delve into the impacts of research bias and strategies to avoid it. Through these discussions, researchers will be better equipped to handle bias in their work and contribute to building more credible knowledge.

Unconscious biases, also known as implicit biases, are attitudes or stereotypes that influence our understanding, actions, and decisions in an unconscious manner. These biases can inadvertently infiltrate the research process, skewing the results and conclusions. This section aims to delve deeper into understanding unconscious bias, its impact on research, and strategies to mitigate it.

What is unconscious bias?

Unconscious bias refers to prejudices or social stereotypes about certain groups that individuals form outside their conscious awareness. Everyone holds unconscious beliefs about various social and identity groups, and these biases stem from a tendency to organize social worlds into categories.

How does unconscious bias infiltrate research?

Unconscious bias can infiltrate research in several ways. It can affect how researchers formulate their research questions or hypotheses , how they interact with participants, their data collection methods, and how they interpret their data . For instance, a researcher might unknowingly favor participants who share similar characteristics with them, which could lead to biased results.

Implications of unconscious bias

The implications of unconscious research bias are far-reaching. It can compromise the validity of research findings , influence the choice of research topics, and affect peer review processes . Unconscious bias can also lead to a lack of diversity in research, which can severely limit the value and impact of the findings.

Strategies to mitigate unconscious research bias

While it's challenging to completely eliminate unconscious bias, several strategies can help mitigate its impact. These include being aware of potential unconscious biases, practicing reflexivity , seeking diverse perspectives for your study, and engaging in regular bias-checking activities, such as bias training and peer debriefing .

By understanding and acknowledging unconscious bias, researchers can take steps to limit its impact on their work, leading to more robust findings.

Why is researcher bias an issue?

Research bias is a pervasive issue that researchers must diligently consider and address. It can significantly impact the credibility of findings. Here, we break down the ramifications of bias into two key areas.

How bias affects validity

Research validity refers to the accuracy of the study findings, or the coherence between the researcher’s findings and the participants’ actual experiences. When bias sneaks into a study, it can distort findings and move them further away from the realities that were shared by the research participants . For example, if a researcher's personal beliefs influence their interpretation of data , the resulting conclusions may not reflect what the data show or what participants experienced.

The transferability problem

Transferability is the extent to which your study's findings can be applied beyond the specific context or sample studied. Applying knowledge from one context to a different context is how we can progress and make informed decisions. In quantitative research , the generalizability of a study is a key component that shapes the potential impact of the findings. In qualitative research , all data and knowledge that is produced is understood to be embedded within a particular context, so the notion of generalizability takes on a slightly different meaning. Rather than assuming that the study participants are statistically representative of the entire population, qualitative researchers can reflect on which aspects of their research context bear the most weight on their findings and how these findings may be transferable to other contexts that share key similarities.

How does bias affect research?

Research bias, if not identified and mitigated, can significantly impact research outcomes. The ripple effects of research bias extend beyond individual studies, impacting the body of knowledge in a field and influencing policy and practice. Here, we delve into three specific ways bias can affect research.

Distortion of research results

Bias can lead to a distortion of your study's findings. For instance, confirmation bias can cause a researcher to focus on data that supports their interpretation while disregarding data that contradicts it. This can skew the results and create a misleading picture of the phenomenon under study.

Undermining scientific progress

When research is influenced by bias, it not only misrepresents participants’ realities but can also impede scientific progress. Biased studies can lead researchers down the wrong path, resulting in wasted resources and efforts. Moreover, it could contribute to a body of literature that is skewed or inaccurate, misleading future research and theories.

Influencing policy and practice based on flawed findings

Research often informs policy and practice. If the research is biased, it can lead to the creation of policies or practices that are ineffective or even harmful. For example, a study with selection bias might conclude that a certain intervention is effective, leading to its broad implementation. However, suppose the transferability of the study's findings was not carefully considered. In that case, it may be risky to assume that the intervention will work as well in different populations, which could lead to ineffective or inequitable outcomes.

While it's almost impossible to eliminate bias in research entirely, it's crucial to mitigate its impact as much as possible. By employing thoughtful strategies at every stage of research, we can strive towards rigor and transparency , enhancing the quality of our findings. This section will delve into specific strategies for avoiding bias.

How do you know if your research is biased?

Determining whether your research is biased involves a careful review of your research design, data collection , analysis , and interpretation . It might require you to reflect critically on your own biases and expectations and how these might have influenced your research. External peer reviews can also be instrumental in spotting potential bias.

Strategies to mitigate bias

Minimizing bias involves careful planning and execution at all stages of a research study. These strategies could include formulating clear, unbiased research questions , ensuring that your sample meaningfully represents the research problem you are studying, crafting unbiased data collection instruments, and employing systematic data analysis techniques. Transparency and reflexivity throughout the process can also help minimize bias.

Mitigating bias in data collection

To mitigate bias in data collection, ensure your questions are clear, neutral, and not leading. Triangulation, or using multiple methods or data sources, can also help to reduce bias and increase the credibility of your findings.

Mitigating bias in data analysis

During data analysis , maintaining a high level of rigor is crucial. This might involve using systematic coding schemes in qualitative research or appropriate statistical tests in quantitative research . Regularly questioning your interpretations and considering alternative explanations can help reduce bias. Peer debriefing , where you discuss your analysis and interpretations with colleagues, can also be a valuable strategy.

By using these strategies, researchers can significantly reduce the impact of bias on their research, enhancing the quality and credibility of their findings and contributing to a more robust and meaningful body of knowledge.

Impact of cultural bias in research

Cultural bias is the tendency to interpret and judge phenomena by standards inherent to one's own culture. Given the increasingly multicultural and global nature of research, understanding and addressing cultural bias is paramount. This section will explore the concept of cultural bias, its impacts on research, and strategies to mitigate it.

What is cultural bias in research?

Cultural bias refers to the potential for a researcher's cultural background, experiences, and values to influence the research process and findings. This can occur consciously or unconsciously and can lead to misinterpretation of data, unfair representation of cultures, and biased conclusions.

How does cultural bias infiltrate research?

Cultural bias can infiltrate research at various stages. It can affect the framing of research questions , the design of the study, the methods of data collection , and the interpretation of results . For instance, a researcher might unintentionally design a study that does not consider the cultural context of the participants, leading to a biased understanding of the phenomenon being studied.

Implications of cultural bias

The implications of cultural bias are profound. Cultural bias can skew your findings, limit the transferability of results, and contribute to cultural misunderstandings and stereotypes. This can ultimately lead to inaccurate or ethnocentric conclusions, further perpetuating cultural bias and inequities.

As a result, many social science fields like sociology and anthropology have been critiqued for cultural biases in research. Some of the earliest research inquiries in anthropology, for example, have had the potential to reduce entire cultures to simplistic stereotypes when compared to mainstream norms. A contemporary researcher respecting ethical and cultural boundaries, on the other hand, should seek to properly place their understanding of social and cultural practices in sufficient context without inappropriately characterizing them.

Strategies to mitigate cultural bias

Mitigating cultural bias requires a concerted effort throughout the research study. These efforts could include educating oneself about other cultures, being aware of one's own cultural biases, incorporating culturally diverse perspectives into the research process, and being sensitive and respectful of cultural differences. It might also involve including team members with diverse cultural backgrounds or seeking external cultural consultants to challenge assumptions and provide alternative perspectives.

By acknowledging and addressing cultural bias, researchers can contribute to more culturally competent, equitable, and valid research. This not only enriches the scientific body of knowledge but also promotes cultural understanding and respect.

Ready to jumpstart your research with ATLAS.ti?

Conceptualize your research project with our intuitive data analysis interface. Download a free trial today.

Keep in mind that bias is a force to be mitigated, not a phenomenon that can be eliminated altogether, and the subjectivities of each person are what make our world so complex and interesting. As things are continuously changing and adapting, research knowledge is also continuously being updated as we further develop our understanding of the world around us.

Ready to analyze your data with ATLAS.ti?

See how our intuitive software can draw key insights from your data with a free trial today.

Shopping Cart

No products in the cart.

Unlocking the Value of Qualitative Research: How to Avoid Quantitative Research Biases

S ubmitted by: sami kaplan.

In the realm of research, qualitative methods often take a backseat to their quantitative counterparts. However, qualitative research offers unique insights and strengths that are crucial for a well-rounded approach to evidence-based medicine. By understanding and appraising qualitative research effectively, you can significantly enhance your contribution to evidence-based medicine and increase your value in the workplace.

Join Sami Kaplan, PhD, a distinguished expert in qualitative research and instructor for the annual Evidence-Based Practice for Health Sciences Librarians workshop, as she guides you through the essentials of qualitative research. She is the Research and Education Liaison Librarian for the Doctor of Medicine program at the Duke School of Medicine, enabling and promoting evidence-based practice at Duke Health.

In this upcoming webinar on Tuesday, September 17, 2024, at 1:00 p.m., central time, you will delve into the nuances of qualitative research, learning how to:

• Appraise Qualitative Research: Understand the criteria and methods for evaluating qualitative studies, enabling you to assess their quality and relevance effectively.
• Differentiate Between Qualitative and Quantitative Research: Gain clarity on how qualitative research diverges from quantitative research in its approach, data collection, and analysis.
• Identify Hallmarks of Qualitative Research Rigor: Discover the key characteristics that define rigorous qualitative research and why traditional quantitative appraisal tools often fall short.
• Utilize Appropriate Appraisal Tools: Learn about tools specifically designed for qualitative research evaluation and how to apply them effectively.

Through a combination of theoretical insights and practical exercises, you’ll learn to critically appraise qualitative research articles, enhancing your ability to apply these findings in real-world clinical settings.

By the end of the webinar, you will be able to:

• Explain fundamental differences between qualitative research and quantitative research designs common to Evidence Based Medicine (EBM).
• Describe hallmarks of qualitative research rigor.
• Identify useful qualitative research appraisal tools.
• Demonstrate proficiency in critical appraisal of some qualitative research methodologies.

This webinar will enrich your understanding of qualitative research and expand your skill set, offering valuable insights that will bolster your contributions to evidence-based medicine and elevate your role within your organization. Don’t miss this opportunity to enhance your research capabilities and make a meaningful impact in your field.

Sciencing_Icons_Science SCIENCE

Sciencing_icons_biology biology, sciencing_icons_cells cells, sciencing_icons_molecular molecular, sciencing_icons_microorganisms microorganisms, sciencing_icons_genetics genetics, sciencing_icons_human body human body, sciencing_icons_ecology ecology, sciencing_icons_chemistry chemistry, sciencing_icons_atomic & molecular structure atomic & molecular structure, sciencing_icons_bonds bonds, sciencing_icons_reactions reactions, sciencing_icons_stoichiometry stoichiometry, sciencing_icons_solutions solutions, sciencing_icons_acids & bases acids & bases, sciencing_icons_thermodynamics thermodynamics, sciencing_icons_organic chemistry organic chemistry, sciencing_icons_physics physics, sciencing_icons_fundamentals-physics fundamentals, sciencing_icons_electronics electronics, sciencing_icons_waves waves, sciencing_icons_energy energy, sciencing_icons_fluid fluid, sciencing_icons_astronomy astronomy, sciencing_icons_geology geology, sciencing_icons_fundamentals-geology fundamentals, sciencing_icons_minerals & rocks minerals & rocks, sciencing_icons_earth scructure earth structure, sciencing_icons_fossils fossils, sciencing_icons_natural disasters natural disasters, sciencing_icons_nature nature, sciencing_icons_ecosystems ecosystems, sciencing_icons_environment environment, sciencing_icons_insects insects, sciencing_icons_plants & mushrooms plants & mushrooms, sciencing_icons_animals animals, sciencing_icons_math math, sciencing_icons_arithmetic arithmetic, sciencing_icons_addition & subtraction addition & subtraction, sciencing_icons_multiplication & division multiplication & division, sciencing_icons_decimals decimals, sciencing_icons_fractions fractions, sciencing_icons_conversions conversions, sciencing_icons_algebra algebra, sciencing_icons_working with units working with units, sciencing_icons_equations & expressions equations & expressions, sciencing_icons_ratios & proportions ratios & proportions, sciencing_icons_inequalities inequalities, sciencing_icons_exponents & logarithms exponents & logarithms, sciencing_icons_factorization factorization, sciencing_icons_functions functions, sciencing_icons_linear equations linear equations, sciencing_icons_graphs graphs, sciencing_icons_quadratics quadratics, sciencing_icons_polynomials polynomials, sciencing_icons_geometry geometry, sciencing_icons_fundamentals-geometry fundamentals, sciencing_icons_cartesian cartesian, sciencing_icons_circles circles, sciencing_icons_solids solids, sciencing_icons_trigonometry trigonometry, sciencing_icons_probability-statistics probability & statistics, sciencing_icons_mean-median-mode mean/median/mode, sciencing_icons_independent-dependent variables independent/dependent variables, sciencing_icons_deviation deviation, sciencing_icons_correlation correlation, sciencing_icons_sampling sampling, sciencing_icons_distributions distributions, sciencing_icons_probability probability, sciencing_icons_calculus calculus, sciencing_icons_differentiation-integration differentiation/integration, sciencing_icons_application application, sciencing_icons_projects projects, sciencing_icons_news news.

• Share Tweet Email Print
• Home ⋅

How to Eliminate Bias in Qualitative Research

Steps & Procedures for Conducting Scientific Research

Qualitative research is a type of scientific investigation that aims to provide answers to a question without bias. It uses predetermined procedures such as interviewing participants to collect information and produce findings. Biases occur naturally in the design of your research, but you can minimize their impact by recognizing and dealing with them. An impartial qualitative research project respects the dignity of the research participants, observes fundamental principles of ethics and takes all of the variables into account.

Avoid design problems by understanding the limitations of the sample group. For example, if you are researching the health benefits of a certain food, be aware if only females or people over a particular age are involved. Bias can occur when certain groups are left out. Account for any unavoidable omission bias by changing the experimental design.

Ensure that the research participants are independent and treated with respect so that they are protected from exploitation. This ensures that people are not selected based on a desire to prove a specific research objective. Avoid becoming focused on one viewpoint when observing participants as this endangers the impartiality of the research.

Allow the research participants enough time to complete questionnaires. Procedural bias can occur if you put too much pressure on them. For example, employees who are asked to complete a survey during a coffee break are more likely to skim through the questions without reading them properly.

Be aware of errors in data collection and measuring processes. For example, when collecting information on prejudice against people of other races, know that most people are reluctant to give answers in an interview because they fear being judged and appearing racist. Researchers often deal with measurement bias by using numerous interviews and an anonymous questionnaires. They recognize that people will tell the interviewer what they think he wants to hear instead of the truth.

Review all the variables arising from the experiment to ensure that there are no experimental errors. False positives and negatives will create biased results.

Ensure that the results of the research are accurately recorded in literature to avoid reporting bias. Show that you understand that certain biases exist and that you have made every effort to consider this in the analysis and statistics.

Seek training and certification in research ethics before beginning the preliminary work and data collection of qualitative research.

Be wary of bias in the findings of research on the Internet. Some research companies hide some research and promote others with more positive results.

Related Articles

Five characteristics of the scientific method, what is a constant in the scientific method, how to calculate significance, what is the difference between a control & a controlled..., music science fair project ideas, how to do a quantitative research questionnaire, what is the purpose of factor analysis, laboratory observation methods, how to calculate p-hat, how do i create graph results for questionnaires, the definition of an uncontrolled variable, experiments on which mouthwash kills bacteria, what is the meaning of variables in research, what is the next step if an experiment fails to confirm..., differences between conceptual independent variables..., how to calculate success rate, how to minimize a sampling error, what are the advantages & disadvantages of using ordinal....

About the Author

Carola Finch began freelancing for newspapers and magazines in 1976. She specializes in writing about people with disabilities, business, Christianity and social issues. Finch studied journalism and communications at Red River Community College.

Photo Credits

people image by cloud1971 from Fotolia.com

Find Your Next Great Science Fair Project! GO

8 Ways to Rule Out Bias in Qualitative Research

The word ‘bias’ is a highly debated topic in the qualitative industry. Due to the naturalistic approach of qualitative studies, there are many interpretations of what ‘bias’ actually means.

It could mean a ‘particular point of view.’ In this case, we could say that bias would be difficult, if not impossible, to eliminate from research. This is because humans inherently interpret the world they observe based on their own experiences and preconceptions.

On the other hand, bias can be defined as a manipulation of data that distorts understanding. This could be highly detrimental to your study because it affects the reliability and validity of your findings. However, you can take measures to prevent this issue from happening.

In this article, we’ll show you a total of eight ways on how to avoid bias in qualitative research for more accurate results.

How to Identify and Avoid Biases

To determine what preventative steps should be taken, you should first learn how to identify which type of bias you are dealing with. Broadly, biases can be divided into two categories – respondent and researcher bias.

Respondent Bias

This type of bias refers to any situation wherein your participant's responses are not an accurate reflection of their thoughts or feelings. For any reason, this may occur because respondents:

To establish credibility, a number of the following rationale are used:

• Want to align their answers regarding sensitive or controversial questions in a socially acceptable way (Social desirability bias)
• Want to please the interviewer or moderator, leading to answers that they think the researcher is looking for (Agreement bias)
• Have opinionated views about the research sponsor and are easily influenced by their reputation (Sponsor bias)

How to avoid response bias

• Ask indirect questions Framing questions in a third-person perspective (e.g., ”What would he/she do?” instead of “What would you do?”) helps participants project their own thoughts onto others – resulting in truthful answers that are representative of their true feelings.
• Ask open-ended questions Asking open-ended questions prompts participants to reflect and expound on their responses, thus, preventing them from simply agreeing or disagreeing with the moderator and giving one-worded answers such as ‘Yes’ or ‘No.’
• Don’t give away sponsor details  Clarity is important in qualitative research because it helps participants understand what is being asked of them. Regardless, some details such as sponsor names or logos should not be revealed so as to not influence respondent answers.

Researcher Bias

This type of bias in qualitative research occurs when the researcher intentionally or unintentionally influences their results in favor of a specific outcome. For instance, researchers may:

• Interpret data in a manner that supports their hypothesis while also removing any unfavorable data (Confirmation bias)
• Arrange research questions in a way that could affect how the succeeding questions are answered (Question-order bias)
• Formulate conclusions based on their own cultural lens (Cultural bias)
• Make assumptions about a respondent because of a positive or negative attribute (Halo/Horn effect)
• Ask questions that only lead respondents in one particular direction (Question-wording bias)

How to avoid researcher bias

• Use multiple coders to interpret data Using multiple people to code data is an effective strategy for knowing if your interpretation is consistent with another person’s understanding. They bring a variety of perspectives to your study – helping you determine if your data is in agreement with your hypothesis.
• Conduct an external review of your work An external peer review helps reveal questions that need modification or gaps in your argument that should be addressed. Having a fresh pair of eyes to examine your work allows you to see bias-causing details that may have been missed.
• Acknowledge your role in the study According to the Association of Qualitative Research , you should aim for impartiality in your study instead of objectivity. When analyzing data, you have to be critically self-reflective about how your identity as a whole could impact your findings. Be transparent about your methods and interpretations so readers can see the logic behind your process.
• Triangulate data sources Triangulation involves looking to secondary sources to verify if your primary data is valid and reliable. If external sources confirm your interpretations, you can be confident that the information you’ve collected is legitimate.
• Ask participants to evaluate your findings Having participants validate your results gives you a clear picture of whether or not your findings are an accurate representation of their beliefs – ultimately helping you avoid bias in qualitative research.

Secure Trustworthy Qualitative Data with Civicom® Marketing Research Services

Civicom® Marketing Research Services is the global leader in providing web-enabled solutions for your market research projects. Our comprehensive services include online IDI and focus group facilitation, a mobile insights app, global respondent recruitment, a qualitative multimedia curation tool, plus other solutions that will cater to your research needs. Get in touch with our team of experts to learn more about how we can help you achieve project success.

• Data Security
• Device Testing Research
• global marketing research
• Mobile Ethnography
• mobile research
• Mock Jury Trials Online
• online bulletin board
• Online Communities
• Product Testing Research
• qualitative research
• Respondent Recruiting
• Video Ethnography

Elevate Your Project Success with Civicom: Your Project Success Is Our Number One Priority

Explore more, related blogs.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Research bias

Observer Bias | Definition, Examples, Prevention

Published on December 8, 2021 by Pritha Bhandari . Revised on March 13, 2023.

Observer bias happens when a researcher’s expectations, opinions, or prejudices influence what they perceive or record in a study. It often affects studies where observers are aware of the research aims and hypotheses . Observer bias is also called detection bias.

Table of contents

What is observational research, subjective methods, objective methods, how to minimize observer bias, other biases, other types of research bias, frequently asked questions about observer bias.

In observational studies, you often record behaviors or take measurements from participants without trying to influence the outcomes or the situation. Observational studies are used in many research fields, including medicine, psychology, behavioral science, and ethnography .

Observer bias can occur regardless of whether you use qualitative or quantitative research methods.

Subjective research methods involve some type of interpretation before you record the observations.

In any research involving others, your own experiences, habits, or emotions can influence how you perceive and interpret others’ behaviors. They may lead you to note some observations as relevant while ignoring other equally important observations.

Your expectations about the research may lead to skewed results. There’s a risk you may be subconsciously primed to see only what you expect to observe.

Observer bias may still influence your study even when you use more objective methods (e.g., physiological devices, medical images) for measurement.

That’s because people have a tendency to interpret readings differently, so results can vary between observers in a study.

A lack of training, poor control, and inadequate procedures or protocols may lead to systematic errors from observer bias.

Observer drift

As you collect data , you become more familiar with the procedures and you might become less careful when taking or recording measurements. Observer drift happens when observers depart from the standard procedures in set ways and therefore rate the same events differently over time.

It’s important to design research in a way that minimizes observer bias. Note that, while you can try to reduce observer bias, you may not be able to fully eliminate it from your study.

Use masking

Masking , or blinding , helps you make sure that both your participants and your observers are unaware of the research aims.

This can remove some of the research expectations that come from knowing the study purpose, so observers are less likely to be biased in a particular way.

You can implement masking by involving other people in your studies as observers and giving them a cover story to mislead them about the true purpose of your study.

Use triangulation

Triangulation means using multiple observers, information sources, or research methods to make sure your findings are credible . It’s always a good idea to use triangulation to corroborate your measurements and check that they line up with each other.

To reduce observer bias, it’s especially important to involve multiple observers and to try to use multiple data collection methods for the same observations. When the data from different observers or different methods converge, you reduce the risk of bias and can feel more confident in your results.

Multiple observers

With more than one observer, you make sure that your data are consistent and unlikely to be skewed by any single observer’s biases.

When you have multiple observers , it’s important to check and maintain high interrater reliability. Interrater reliability refers to how consistently multiple observers rate the same observation.

With quantitative data , you can compare data from multiple observers, calculate interrater reliability, and set a threshold that you want to meet. Usually, you train observers in the procedures until they can consistently produce the same or similar observations for every event in training sessions.

Train observers

Before you start any study, it’s a good idea to train all observers to make sure everyone collects and records data in exactly the same way.

It’s important to calibrate your methods so that there’s very little or no variation in how different observers report the same observation. You can recalibrate your procedures between observers at various points in the study to keep interrater reliability high and minimize observer drift as well.

Standardize your procedures

It’s best to create standardized procedures or protocols that are structured and easy to understand for all observers. For example, if your study is about behaviors, make sure to specify all behaviors that observers should note.

Record these procedures (in videos or text) so you can refer back to them at any point in the research process to refresh your memory.

Observer bias is closely related to several other types of research bias.

Observer-expectancy effect

The observer-expectancy effect occurs when researchers influence the results of their own study through interactions with participants.

Researchers may unintentionally signal their own beliefs and expectations about the study and influence participants through demand characteristics .

The observer-expectancy effect also goes by other names:

• Experimenter-expectancy effect
• Rosenthal effect
• Pygmalion effect
• Group A (treatment group) receives the actual treatment with the new painkiller
• Group B ( control group ) receives no treatment, but instead takes a placebo

Actor–observer bias

The actor–observer bias is an attributional bias where you tend to attribute the cause of something differently depending on whether you’re the actor or observer in that situation.

As an actor in a situation, you may tend to attribute your own behavior to external factors. As an observer, you may instead attribute another person’s behavior, even if it’s the same as yours, to internal factors. The actor–observer bias is a social psychological topic.

• Hawthorne effect

The Hawthorne effect refers to some research participants’ tendency to work harder in order to perform better when they believe they’re being observed. It describes what participants being observed may inadvertently do in a study.

The Hawthorne effect is named after Hawthorne Works, a company where employee productivity supposedly improved, regardless of the experimental treatment , due to the presence of observers.

Experimenter bias

Experimenter bias covers all types of biases from researchers that may influence their studies. This includes observer bias, observer expectancy effects, actor–observer bias, and other biases. Experimenter bias is also called experimenter effect.

Cognitive bias

• Confirmation bias
• Baader–Meinhof phenomenon
• Availability heuristic
• Halo effect
• Hindsight bias
• Framing effect
• Ingroup bias
• Affect heuristic
• Representativeness heuristic
• Anchoring heuristic
• Primacy bias
• Optimism bias

Selection bias

• Sampling bias
• Ascertainment bias
• Attrition bias
• Self-selection bias
• Survivorship bias
• Nonresponse bias
• Undercoverage bias
• Observer bias
• Omitted variable bias
• Publication bias
• Recall bias
• Social desirability bias
• Placebo effect
• Actor-observer bias
• Ceiling effect
• Ecological fallacy
• Affinity bias

Observer bias occurs when a researcher’s expectations, opinions, or prejudices influence what they perceive or record in a study. It usually affects studies when observers are aware of the research aims or hypotheses. This type of research bias is also called detection bias or ascertainment bias .

It’s impossible to completely avoid observer bias in studies where data collection is done or recorded manually, but you can take steps to reduce this type of bias in your research .

You can use several tactics to minimize observer bias .

• Use masking (blinding) to hide the purpose of your study from all observers.
• Triangulate your data with different data collection methods or sources.
• Use multiple observers and ensure interrater reliability.
• Train your observers to make sure data is consistently recorded between them.
• Standardize your observation procedures to make sure they are structured and clear.

Researchers’ own beliefs and expectations about the study results may unintentionally influence participants through demand characteristics .

The observer-expectancy effect is often used synonymously with the Pygmalion or Rosenthal effect .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, March 13). Observer Bias | Definition, Examples, Prevention. Scribbr. Retrieved September 9, 2024, from https://www.scribbr.com/research-bias/observer-bias/

Is this article helpful?

Pritha Bhandari

Other students also liked, demand characteristics | definition, examples & control, random vs. systematic error | definition & examples, single, double, & triple blind study | definition & examples.

A Qualitative Investigation of the Relationships Between Foster Care Stakeholders and Research

• Open access
• Published: 09 September 2024

Cite this article

You have full access to this open access article

• Saralyn Ruff   ORCID: orcid.org/0000-0002-2078-5509 1 ,
• Deanna Linville 2 &
• Quanice Hawkins 3  

137 Accesses

Explore all metrics

Research on foster care from the perspective of key stakeholders with lived and professional experience is necessary to inform programs, policy and practice. Numerous barriers exist to accessing these populations and ensuring inclusion and representation in research. This study interviewed twenty-two stakeholders with lived and/or professional experience in foster care to gain their recommendations on how to understand and conduct research on foster care and specifically and how to (a) increase stakeholders’ participation in research and (b) capture a broader representation of those impacted. Findings offer observations of who does and does not participate in research and how this may affect public perception, as well as direct recommendations for future research.

Similar content being viewed by others

What are the challenges when recruiting to a trial in children’s social care? A qualitative evaluation of a trial of foster carer training

Resource workers’ relationships with foster parents.

Family Reunification Decision-Making in Dutch Family Foster Care: A Dual Perspective Approach

Explore related subjects.

• Artificial Intelligence
• Medical Ethics

Avoid common mistakes on your manuscript.

At any given time in the United States, there are approximately 500,000 youths in the foster care system (Adoption and Foster Care Analysis and Reporting System (AFCARS), 2022 ). This number is a snapshot of substantiated child welfare cases open at any one point in time and is not reflective of the number of individuals who have been involved in foster care across their childhood. Since the number of individuals who are current or former foster youth in the United States is unknown, it is difficult to capture representative research. Consequently, foster youth are noted as “one of the hardest populations to study” (Jackson et al., 2012 , p. 1212). For the present study, we recruited and interviewed twenty-two stakeholders who had lived and/or professional experiences with and in foster care to gain recommendations on how to understand and conduct research on foster care. The current study had two primary objectives: (a) to learn how to increase stakeholders’ participation in research, in order to (b) capture a more inclusive representation of those impacted personally and/or professionally by foster care.

Research on Foster Care

Conducting research on stakeholders’ experiences in child welfare can be challenging, requiring compromise alongside creative and innovative solutions (Jackson et al., 2012 ). Even considering who should participate and represent the experience of foster youth involves ethical and legal considerations. Foster youth “have often been excluded from participating in research because they are viewed as vulnerable children who lack agency and also due to an adult-centric perspective of protection” (Garcia-Quiroga & Salvo Agoglia, 2020 , p. 1). By design, the identities of children and youth involved in the foster care system are confidential and protected. If able to be accessed, it is unknown who should and could consent to the minors’ research participation and what information is developmentally appropriate to access or ask (Berrick et al., 2000 ; Greiner et al., 2018 ).

Among former foster youth, unique sets of challenges exist that hinder participation in research. While one may be of legal age to consent for one’s own self, former foster youth may no longer have ties to the foster care system, either because of limited resources to support former foster youth and/or because they may wish to distance themselves from foster care affiliation altogether (Steenbakkers et al., 2016 ). Even if interested in participation, there are limited pathways to identify and access former foster youth. These protections are critical, but create a tension when working to conduct inclusive and representative research.

A common trend in child welfare research is the reliance on third parties to represent perspectives in child welfare, without firsthand and/or lived experience. These key stakeholders may be limited as to the accuracy of what they disclose, and/or due to existing and necessary legal, ethical, and professional regulations on confidentiality (Gilbertson & Barber, 2002 ; Jackson et al., 2012 ). Researchers may alternatively seek access to extant reports and legal documentation for secondary data analysis. However, these data may not have been intended or designed for research and may not align well with established research questions (Greiner et al., 2020 ). While researchers continue to utilize these accessible alternatives, limitations remain.

The transient nature of foster care leads to continued methodological challenges in research and, in particular, with participant retention. One example involves stakeholders’ participation in longitudinal studies and both the high attrition rates in these studies and/or the reliance on cross-sectional research (Jackson et al., 2012 ). Following a sample of foster children, parents, or any other stakeholder status may prove difficult with staff turnover, foster family turnover, and both exits and entries into care (Leake et al., 2017 ). For example, longitudinal research utilizing a child welfare staff perspective may become nearly impossible when considering the high rate of staff turnover annually, attributed to high demand on their personal and professional resources (DePanfilis & Zlotnik, 2008 ; Gopalan et al., 2019 ). It would be difficult to not only access a behavioral health workforce that has turnover rates estimated between 30% annually and 100% within a four-year period, but one that while working in child welfare is under-resourced in time and support (Beidas et al., 2016 ; Substance Abuse & Mental Health Services Administration, 2014 ).

Initial and/or continued research participation may also be affected by stakeholders’ perceptions of the merits of and motivation for the research. There may be mistrust that permeates disclosures of experience in foster care, particularly when such disclosures are often associated with system-level decision making. This may influence potential participants’ perceptions of research, making them wonder whether or not to participate, how much to participate, and/or whether or not to maintain participation. Gilbertson and Barber ( 2002 ) found that in research studies with stakeholders, non-response rates ranged from 72.5 to 82% across questions, possibly reflecting participants’ discomfort in answering certain questions and necessitating further clarification on appropriate scope and approach of inquiries.

As seen in the limited literature that is available on research with stakeholders in child welfare, the challenges to participation exist on many levels and may mirror the barriers that many stakeholders in child welfare routinely navigate already. There is a need for “flexible and responsive methodology” informed by stakeholders’ experiences and recommendations to guide research-informed practice and policy (Jackson et al., 2012 , p. 1212). We focused this study on learning more about how twenty-two key stakeholders in foster care perceive and experience research and what their recommendations might be to improve representation and the integrity of research.

Study Context

This study is part of the first phase of a larger evaluation of services offered through A Home Within, a national nonprofit offering pro bono mental health services to current and former foster youth. Specifically, before conducting a randomized-controlled trial (RCT), we used community-based participatory action research (CBPAR) methods to conduct a needs assessment with key stakeholders in foster care to inform the methods and design of the RCT. CBPAR methodologies prioritize partnerships with the communities that are the focus of the research as co-investigators to ensure the relevance of research findings for those communities involved (Israel et al., 2005 ). CBPAR methods vary widely, but aim to ensure equitable decision making (Israel et al., 2005 ). We describe various CBPAR methods utilized in this study, below.

For this needs assessment, we worked with our community partner and identified qualitative methodologies as most appropriate for the research questions at hand. These included interviews and focus groups with stakeholders using semi-structured guides. We analyzed data from the twenty-two qualitative interviews as the follow-up focus groups did not collect data on the relevant research questions. While all qualitative methodologies have a similar goal of understanding a phenomenon from those that are experiencing it (Vaismoradi et al., 2013 ), we felt that using generic thematic analysis methods allowed us to provide a rich and detailed, yet complex, account of the data (Braun & Clarke, 2006 ). As is expected for any research involving human participants, but particularly important when using CBPAR methodology, we sought and obtained institutional review board approval for the study protocol. Participants were informed about limits and risks to their confidentiality prior to the provision of their consent to participate.

Research Team

The research team consisted of two principal investigators, a project coordinator, five research assistants, and a research Community Advisory Board (CAB). There are three authors for this paper, two of whom were principal investigators and one who served on the CAB. The first author served as a co-principal investigator with sixteen years of clinical experience and eleven years of research experience in the child welfare system. The second author is also a co-principal investigator and has twenty years of experience conducting qualitative and mixed methods studies as well as twenty-five years of experience as a mental health professional and educator. Both the first and second authors volunteered with A Home Within at various points in their careers. They were able to maintain a boundary between A Home Within programming and the research by conducting the study through their affiliated educational institutions and taking the appropriate steps to mitigate potential research bias, including maintaining participant confidentiality. The third author served on the CAB and has worked as an advocate for foster youth. Specifically, she was a member of California Youth Connection, which impacted California state policy including Assembly Bill 5. She has also served as a consultant for the Breakthrough Series Collaborative on Independent living for former foster youth as well as an advocate through the Casey Family Programs’ Bay Area office.

Utilizing the CBPAR research framework, the research CAB was conceptualized and developed during phase one of the needs assessment. Specifically, we identified potential CAB members who have lived personal and/or professional experiences in foster care, informed by initial interview findings. The composition of the CAB included five former foster youths living in California, Oregon, or Texas, all of whom have both personal and professional experiences with foster youth. Three CAB members were current or previous recipients of therapy services through A Home Within. Another member was a licensed mental health professional and a volunteer clinician for A Home Within. The CAB met monthly for approximately 90 min over a period of two years with the purpose of providing feedback and helping to shape research methods, interpreting study findings, and providing recommendations for implications to both the larger field and A Home Within services more specifically.

The co-principal investigators used community mapping to identify key stakeholder groups in child welfare for participation in the study. Community mapping is an inquiry-based research method that situates learning within the context of the community in order to uncover the depth and diversity of community needs, resources, and assets (Ordoñez-Jasis & Myck-Wayne, 2012 ). We considered A Home Within staff not only as research partners but also as part of the community. We collaborated with A Home Within and CAB, gaining recruitment recommendations for both key stakeholder groups and individuals to invite for participation. All recruitment efforts occurred through email, paired with assurance that participation would be kept confidential and would not affect standing with A Home Within.

Participants

Twenty-two stakeholders in the foster care community participated in qualitative semi-structured interviews used for analysis in this study (see Table  1 for participant demographic information.) Participants lived across the United States with the majority (68%) residing on the West Coast. Over half (54.5%) of the participants identified as White; eight (36.3%) identified as persons of color, and two (9.1%) did not respond to the question about racial/ethnic identity.

Thirteen (59%) key stakeholder participants were affiliated with A Home Within. Specifically, they served either as a volunteer therapist, consultation group leader, clinical director, paid leadership role, and/or client. Of those ( n  = 9, 41%) not affiliated with A Home Within, five were mental health therapists working with current and former foster youth; four were social workers and case managers working in child welfare and/or juvenile justice. Of the eight former foster youth (36%) who participated in the study, four (18%) were current A Home Within clients. Three participants (14%) were foster parents, one (5%) was the biological parent of foster children, and two participants (9%) worked in juvenile justice.

Data Collection

The co-principal investigators conducted Zoom interviews with key stakeholders that ranged from 90 to 180 min in length. Several questions from the semi-structured interview guide focused on how to best include current and former foster youth in research. Interview questions pertaining to the current study were asked at the beginning and end of each interview to allow for an iterative process between data collection and analysis. The questions included the following:

How did you hear about this project and why did you decide to participate?

What are your perceptions of research on foster care?

Do you have any thoughts or recommendations about how to get more people to participate in research with the foster care system, particularly those that are in care to prioritize the voices of the people that were trying to serve?

Who else should we talk to in order to understand the experiences of current and former foster youth?

Do you have any recommendations related to the research?

Often, we asked follow-up questions that differed based on the participants’ context and experiences. At several points, we checked in with participants to ask about the general process of the study in progress and gained insights that are included in our findings. We did not have preconceived notions about the findings and allowed key stakeholders’ voices to characterize the data. As such, the findings and ordering of themes are presented in the way that they flowed from the conversations with interview participants.

Participants consented and provided demographic data prior to the interview via a Qualtrics survey. Throughout the interview data collection process, the co-principal investigators kept field notes and debriefed after every interview. Consistent with content analysis qualitative methods, interviews were recorded and transcribed using Otter.ai. Following, a trained research assistant reviewed each video recording and cleaned the transcripts to ensure accuracy.

Data Analysis

Our primary objective of this qualitative inquiry was to provide a participant-informed description and meaning-making of how to engage and include foster youth in research. We used thematic analysis as an independent qualitative descriptive approach, which fit with our goal of gathering descriptions from the participants themselves and not from the research team’s interpretations of them (Braun & Clarke, 2006 ; Sandelowski, 2010 ). This approach does have a guiding philosophy, even though it does not follow an explicit set of theoretical assumptions (Caelli et al., 2003 ).

For the first step of the analysis and in order to familiarize ourselves with the data, the first and second authors independently read through all the qualitative answers to get an overall sense of the data before engaging in the initial coding process. Then, during the second review of the data, the authors generated initial codes with each person writing down initial concepts, phrases, or words that were important to create the coding scheme (Clarke & Braun, 2013 ). The team then searched for themes by clustering codes into themes and subthemes (Vaismoradi et al., 2013 ). The final coding scheme consisted of four overarching themes and fourteen subthemes.

We took several steps to ensure the trustworthiness and dependability of our findings. Throughout the process, the team used peer debriefing and reaching intercoder agreement in order to bolster the credibility and trustworthiness of the findings (Curtin & Fossey, 2007 ; Saldaña, 2015 ). The authors conducted member checking, defined as taking back ideas from the research to the participants for their confirmation and clarification (Birt et al., 2016 ; Charmaz, 2006 ). Specifically, we sent findings out to nine participants, asking them to review the findings and respond to the following two questions:

Do you see your experience captured in the findings?

Is there anything you would add or change?

We gained feedback from six of the nine participants, and all members indicated that the interpretation of the data and the findings captured their perspectives. Four offered elaboration related to implications of the findings, reflected in the Discussion of this paper. Two participants shared an interest in member checking but had time conflicts and requested involvement at a different time.

Lastly, the research team elicited and received feedback from the research CAB on the study findings and its implications. A draft of the findings was sent to the CAB via email in preparation for a two-hour discussion. In the email and in the meeting, the CAB was asked:

Understanding that the findings needed to reflect the participants’ experiences, do they resonate with experiences?

Based on your professional and personal experiences, what are some possible practice, research, policy, and/or service implications of the findings?

The CAB provided us with valuable feedback in writing and orally, included in our Discussion.

Interviews with twenty-two participants supported an increased understanding of reasons stakeholders may or may not participate in research and what could possibly be done to support representative and inclusive research. Specifically, four major themes emerged across the conversations with key stakeholders including (a) general barriers to participation in research; (b) who participates in research and who does not; (c) reasons for research participation; and (d) recommendations for future research in child welfare. Within each major theme, there were two to six subthemes, described and illustrated with participant quotes below.

General Barriers Towards Participation

Not surprisingly, the key stakeholders interviewed for this study reflected on the many barriers that they believed impacted involvement in child welfare research. These barriers clustered into two larger subthemes: (a) skepticism about the motivations of research and (b) concerns about meeting the expectations of the researchers. Across this theme, participants expressed that they themselves, or other stakeholders, held a general mistrust in telling one’s story for the purpose of research. Participants frequently mentioned concerns about who research ultimately serves and how their individual stories might be used.

Skepticism About the Research Motivations

Participants held curiosity, if not skepticism, about how research questions and priorities were decided and how personal stories may then be filtered, analyzed, and shared in research. For some participants, this mistrust felt parallel, or at least related, to the experiences of not having a choice within the foster care system. One participant captured this concern, saying:

Yeah, being a kid is hard. People are telling you what to do all the time. And being a foster kid is even harder. It’s kind of amazing that people don’t stop to ask what you want. And so, research that you have to do and be told what you have to do, I don’t think it goes over very well because it’s just more of the same. (A Home Within staff)

Another former foster youth expressed exhaustion with this sameness and a desire for boundaries around who and when to share their story. They mentioned “I don’t want to always talk about me. And I think in general, foster kids don’t want that. We don’t want everything to be on us.” The last part of this quote captured a sentiment shared by others with lived experience that foster youth should not have to carry the primary responsibility for change. Specifically, participants expressed concern about the emotional and psychological work required in discussing child welfare and their conflict in wondering who this ultimately benefited—the system/s, foster youth, and/or researchers.

While discussing a mistrust of research, several participants wondered what would be done with the research findings and what lens would be used to frame their story. Some shared that these intentions are often unclear in research studies with unclear study aims. One Home Within clinical consultant highlighted their distrust, not knowing what is done with the information collected in research, sharing:

Mostly where it comes from is distrust of like, “Why? What are you going to do with this? Who are you? Why would you change anything? Like how does this change anything for me?” And it’s not as selfish as it sounds. It’s more like protecting their story because it takes a lot to walk in the world with the story they have.

Participants shared that it may be that underlying concerns and mistrust of research were, in part, related to past negative ramifications of having shared information with professionals. As one parent of a foster youth shared:

If it’s with somebody that will actually listen and not point the finger and not blame. … People want to share their stories. They’re just afraid. People get looked at as crazy. No, that didn’t happen, that doesn’t happen, there are laws to protect you. There are laws to protect your child and not everybody gets the benefits – those benefits. Some people are … some agencies are opportunists.

Simply put by one A Home Within former staff member, “you have a population of folks who tend to be exploited in various ways and are rightfully wary, and who are also probably just exhausted, right?” Across these findings, research participation was described to feel like a risk, requiring caution and some sense of safety.

Meeting Expectations of the Researchers

A second subtheme that emerged as a possible barrier to research participation was a concern about meeting the expectations of the researcher. The lack of clarity about research language, intention, and overall goals left some participants feeling a bit unclear about what researchers wanted and hesitancy about matching these expectations. One clinician reflected on the many conversations that they had with former foster youth when referring them to research, and the common question they asked was whether or not they would “do a good job.” This participant elaborated that this concern is often shared by families, case managers, and other stakeholders, who hold a sense of responsibility about how the data they provide might impact their communities.

Concern about researcher expectations was also evident during many of the interviews with participants included in this study. Throughout the interview transcripts, participants inquired if they were answering questions how we wanted them to or meeting the goals of our study. This was evident in the interview process when clients would directly ask the researchers for this study, “Did that answer your question?” As will be discussed later, this finding was also reflected as a research recommendation under the subtheme “ Offer Feedback .” There, participants recommended providing more structure and feedback to stakeholders to know whether the information they shared was in line with the research goals and, even more, was heard and held as an impactful, individual experience.

Differentiating Between Who Does and Does Not Participate

A second major theme was evident across the findings related to participants’ awareness of which key stakeholders in foster care participate and which do not. This theme was built on the previous theme that reflected stakeholders’ mistrust in research and led to conversations on how research results and findings are often not representative of the full breadth of lived and professional experiences. For example, when asked about participation in research, one clinician and A Home Within Clinical Director immediately responded, “But you won’t get the people who are the most disengaged, right? We’ll always have trouble with that.” Later, this same participant continued, reflecting on researchers’ reliance on specific individuals who may be more likely to participate that “the mentoring group that’s here, that works with young adults here, there’s a subset that really did want to be activists. And they were looking at ways to have an impact on the system, to change the system.” Another example of this theme was offered by a social worker who said, “There are pockets of folks we can always rely on, but that’s only 5–10 people, that’s not necessarily a scale of what we’re working with.”

Within this broader theme of who is included in research and who is not, two subthemes emerged: access to participants and timing of request for participation. These subthemes reflected possible assumptions about research procedures that influenced whose voices are heard and led to recommendations for increasing research access to and inclusion of a broader range of stakeholders.

Access to Participants

When asked the question whether participants had any thoughts or recommendations about how to get more people to participate in research, particularly those with lived and/or professional experience, the majority shared an immediate offer to help. Specifically, even though the intention of this question was not to ask for names for the study at hand, participants tended to brainstorm organizations and nonprofits and provide names with a willingness to help recruit. One former foster youth and social worker answered:

Well, you know [name of nonprofit] might be a nice opportunity because it’s a contained community. And I think the onsite staff could certainly facilitate that. I mean, of course you’re going to get one very particular view because with these kids, youth. … Oh, and there is a program called ___, and I think that is a national program, and those are former foster youth who become involved because they’re interested in the bigger picture.

Participants also showed an inclination to rely on nonprofits and agencies as referral sources for research and, again, to understand that such recruitment methods may only offer access to a fraction of those with lived experience in foster care. Some participants emphasized the importance of utilizing nonprofits as a preferred network for research recruitment due to their extensive knowledge and understanding of child welfare. One participant commented:

Organizations that work with young people very frequently, talking to line staff and folks that are supporting young people all the time, I think that those are people that are really important to talk to, because they just have a finger on the pulse of what’s happening in the world and how people are feeling about it. (Former Foster Youth and A Home Within client)

Several other individuals voiced concerns or challenges in the use of third parties to create connections with stakeholders. One A Home Within staff member and clinician said:

Some of these other nonprofit organizations. … Sometimes that works and sometimes it’s not so easy because people become very proprietary. Why are you taking my kids to do your research? What do I get out of it? Or, we don’t use mental health; we do something else. People get very proprietary.”

A former foster parent and foster youth advocate working at a college expanded on hesitations in connecting stakeholders and researchers, based on their first-hand experience:

I always feel a little bit protective of students when there’s an opportunity to tell your story, because I think it’s super important to get people’s voices out there. In order to raise support and funding, people want to hear people’s stories. It can also be re-traumatizing or you can feel like your traumas are being used. When we have an opportunity, where we need some student voices, I like to just put it out there widely, and with no pressure. Some students will be really, really interested in that and feel really empowered by it. But it’s definitely not all students who feel that way. I don’t want to overuse it and ask people over and over again.

Later, this same participant explained how they tend to make decisions about when to connect researchers with potential foster youth participants:

I’m always forming partnerships for people to refer to us. Right? So, for me to refer students to other programs and for them to refer to us. It’s a relationship that takes time to build. Once I see that students connect with it, and report back that it’s going well, and then another student does and another student does, and we kind of build that proof, then there’s this confidence built. If there’s someone that I don’t know, and there’s a random email, I’m not going to immediately send it on to students. Maybe we’ll meet and see where the intersections are. Then, maybe there’s a particular student who I think might be a good opportunity for them to try. They’ll try and it’ll go well, or not.

This general sense of protection of certain stakeholders was evident when talking with professionals in child welfare, yet of note, many stakeholders with lived experience shared concerns about professionals making these decisions and “gatekeeping” opportunities.

Timing of Participation

A second subtheme evident across the interviews about who participates in research and who does not, related to the timing of participation. This subtheme seemed to be particularly focused on current and former foster youth as stakeholders. Additionally, participants noted that not only does timing influence who participates, but it may also influence the data a researcher may receive from a current versus a former foster youth. To explain, one former foster youth and A Home Within client stated:

I think that’s a recommendation that I make too, folks like me, who are 34, [are] probably helpful to talk to you because I’m a bit removed from my own personal experience at this point in time [as they would be]. But I think that it’s really important to also talk to young people who are in it. In the same way, not folks who are currently being traumatized by the system … that’s not necessarily what I’m saying. But folks who are 19 or 20 [ages] struggling trying to figure it out. I think that they would have a very different perspective than somebody like me.

Other participants emphasized that a trauma lens should inform the timing of recruitment, so that researchers have an awareness that, many times, foster youth are not in a space to answer questions related to their experiences until they have a sense of safety, and that this sense of safety may come with time and distance from foster care.

Now I have my support team, I have my confidence, you know, I have my voice established. I’m able to speak up. And that’s a big part of why I’m at [child welfare nonprofit], is being able to advocate because I wasn’t able to advocate for myself at that age, sitting at a table with all these grown-ups, and they have all these degrees, and they know what they’re talking about. And sometimes it could still be intimidating. Now, because I don’t have a college degree, I go off of my personal experience. But that’s what makes me that much better. Because I do have that lived experience. And I do have that voice and I do know what works and what doesn’t. So, I think my voice is being heard now. (Former Foster Youth and A Home Within Client)

Reasons Key Stakeholders May Participate

In learning why each participant agreed to an interview for this study, discussions broadened to examine why stakeholders may participate in research on child welfare. There were two subthemes that identified contribution and sharing success as possible reasons for research participation. Of note, the data supporting this theme, and quotes offered below, were either from former foster youth about their own participation, or about foster youth, and did not necessarily extend to discussions regarding the participation of other key stakeholders.

There were a few non-foster youth interviewees that participated in this study who conveyed the general idea that foster youth stakeholders would want to participate in research, assuming that the invitation, timing and conditions were correct. One clinician shared that “my sense has been that foster kids are really happy to be asked what’s on their mind. So, I think it’s getting to them. I would see that as a roadblock rather than giving them the opportunity to talk.”

This quote highlights the importance of having access to recruitment and referral sources and builds on the previous subtheme of “ Timing of the Ask .” Specifically, the data suggested that it was more in the way one would be approached than whether they had an interest in participation.

To Contribute

When expanding on assumed interest in research, participants often reflected on the belief that many stakeholders held a strong desire to contribute. This finding was evident across stakeholder status, but particularly emphasized among current and former foster youth, and is exemplified in the quotes below.

I think that one of the reasons why I wanted to participate was just because I’m just the kind of person who has always tried to give back and use my experience to help inform better policy practice, etc. Anything that I can do to help the experiences of other people going through it, is something that I’ll always say yes to. (Former Foster Youth and A Home Within client) I’m pretty open about my experience in foster care and I know a lot of people aren’t so like, I like to step up for those who aren’t comfortable sharing things like what happened to them and stuff like that. (Former Foster Youth) I was super eager to jump in and tell you what I’m struggling with at my age now and with my mental health and the services that I’m having. So, I was really eager to be able to talk about now being a former foster youth, and how crazy it is, the way the systems are in place, and sometimes how difficult it can be not having the support and not having someone to advocate for you. I’ve really been thriving off of this, sharing my experience and talking about what I’ve gone through, how it’s helped me and what I’m still going through today… like every day is a struggle. And I call it a beautiful struggle, because we choose to make it ugly, or we choose to make it beautiful. So, I’m actually starting to involve myself in doing a lot more community speaking … doing a lot more public speaking, things like that sharing my experience, and it’s really something that since I did it with you guys [at a community event] that I’ve been wanting to do more and it’s really exciting to me to be able to talk. (Former Foster Youth)

As seen through these quotes, contribution often overlapped with advocacy and a desire to ensure others did not feel alone in their experience. One former foster youth participant shared:

It’s really important for people like you to know, so that you know that there are people out here that have gone through these things and like what things you guys can do to stop certain things from happening, maybe giving more support, you know?

To Share Success

Several participants directly discussed participation as a means to share their successes and strengths. One former foster youth and A Home Within client shared pride in telling their story:

It’s beyond being open to sharing it. I want to wear it like a crown. I’m not kidding. Yeah. I’m a badass. … Like, no matter what it’s like, I’ve survived multiple decades of domestic violence, motherfucker. What do you think, you know? You think you know, anything? Try to walk in my shoes for five minutes? You wouldn’t make it half a block?

Other participants explained that sharing their successes were not only for others’ benefit, but also for their own selves in supporting personal accountability and growth. One former foster youth explained:

I feel like I am a huge success … not only for foster care, but for family, who are really well known here and not in a positive way. I’m the only one out of 10 that graduated in high school, that’s been to college, that has my own home, that has custody of my child. Child welfare has been involved with my brothers and sisters in a really negative way. And it’s like, I know, they see my name. So, it’s really nice to be that success, and be able to empower it, you know?

This same participant then continued, “I can’t be telling people that I am this and doing this and doing that when really, I’m not a good member of the community or something like that.” This sentiment was also reflected in other interviews, suggesting that sharing success was sometimes intertwined with maintaining success. The following additional quotes further supports this point:

The reason why I wanted to do it was because I think it’s a good experience for me, and a stepping stone for me to talk about these things that affect me. It’s still a big thing for me. I still have a problem with trusting people. Just maintaining relationships are a really big thing for me. I push people away. You know I have a problem with having them there and before they can do something to me, I push them [away]. So, it’s a really big thing for me to do this and express myself and tell my story. I don’t know if that makes sense. (Former Foster Youth) I’m actually looking into doing a lot of motivational speaking. So, something that has discouraged me, is not having that degree. So, a lot of people think that because I don’t have a degree in motivational speaking or sharing my story, I didn’t know how much of an impact it would be, until I did it that first time. And I didn’t know how many people I could reach until I was pushed to do it that first time. I kind of feel like letting people know that you don’t have to have a degree to share your story. It’s more of what you went through and what you’ve experienced. ... I can tell you now when I first started here [foster youth advocacy organization], I could not do public speaking. I would shake, I’d be sick, I’d throw up…it’s bad. So practice, having a support team, having somebody that’s told me, I am the only one that knows what happened in my life. I’m the only one that knows what I’m going to say. You guys don’t even know when I mess up or when I skip a line, because I am a professional on my life and my experience and being that confident and having that support is what has caused me to thrive and be more open to doing these things. Did that answer your question? (Former Foster Youth)

These quotes build on the importance of sharing success and the potential positive influence it can have on others and self.

Recommendations for Research on Foster Care

Many of the conversations with participants led to brainstorming and sometimes direct recommendations on how to support future research on child welfare. These conversations focused on involving current and former foster youth, more so than foster parents, case managers, or others with lived and/or professional experience, in research as stakeholders. The recommendations were generously offered, organized into six subthemes.

Incentivizing Participation

Consistent across the interviews, participants suggested that the bare minimum for supporting stakeholders’ participation in research included offering compensation, not only for their time but for the psychological work required to share personal information. As one clinician and A Home Within staff member said, “I think the gift card helps.” Or, as a former foster youth highlighted, “I mean I think incentivizing it is always a good thing. … I don’t know to what scale. I always signed up for stuff where I was like, oh, I get pizza or oh you’re gonna pay me or like those kinds of things.” It was not always recommended that these incentives only be financial, but as one parent of a foster youth suggested:

Have like a pamphlet full of resources full of you know advocates, and one of the key points here is to believe what’s happening because if you dismiss what’s occurring to each family, then people just go into the little cocoon and they don’t want to talk anymore.

This participant continued to discuss how difficult it can be to tell one’s story, and how resources or referrals felt like one way, of many, for researchers to show care about what they had heard and about them. Several participants recommended that researchers offer participants a choice of incentive, including them in the decision-making process; this recommendation will be further exemplified in the recommendation subcategories “ Join as Co-Creators of Research .”

Build Relationships

The data clearly indicated that individuals were more likely to participate in research, and to have a positive experience, thereby increasing future participation, in the context of a relationship with researchers. The acknowledgement of the tendency to lack a relationship was reflected here:

Research when I have seen it done on foster youth, it is extractive, right? You know, you do not know the researchers from anybody and they show up, and they can be as nice as they damn well please, but you do not have any connection to them. (Social Worker)

Many suggested that nonprofits and agencies could help facilitate relationships, specifically with stakeholders with lived experience. Specifically, some participants emphasized the importance of building relationships with third parties to earn trust. One clinician offered the following:

I tried to in big and small ways create a sense that we’re all in this together. We’re all doing different stuff. We’re coming at it in different ways, but, you know there’s a huge need. You’re not going to meet all the needs through your organization. I’m not going to meet all of those through mine. But what can we do together? And this research will help you, will help your kids, the kids you’re serving. Maybe not help you and your organization directly, but it should help the kids you want to help.

Several participants offered direct suggestions for navigating the process, building relationships with third party recruitment sources. One clinician and A Home Within Clinical Consultant shared:

But I think the question of how you get their trust. Maybe there needs to be a pre-interview meeting, right, to get them interested and see what they think and to answer some questions. Or, I don’t know, I think it’s more about how the interviewer builds trust with people who don’t trust and who feel like they’ve been misunderstood or their words have been taken away … or worry that if they say something it’d come back to haunt them.

Offer Feedback

As noted previously, the data showed some mistrust of research, combined with wants and desires to contribute and share successes. These conversations often segued to the recommendation for researchers to offer feedback to participants in real time during the process. Participants understood the need to not bias research and still offered this recommendation as a way to convey that the researchers heard the participants’ stories and valued their willingness to share. For example, one college support professional for foster youth who also had been a foster parent offered:

I think that there’s this thing with the foster care experience, also where you have to support individuals’ caseworkers, or whatever that comes into your life, and maybe a week later, they’ve moved on, and there’s someone else. You just kind of never know. There are all these moving parts and so to share your voice, but have it just be this passing thing that you’re never connected with the results [research] ... that can be hard. I guess, from that perspective, trying to incorporate it into agencies and people who are serving foster youth who have more of a long-term relationship model, so that it can be part of that longer term relationship rather than this like random opportunity where I share all my stuff, and then, where does it go? Or, what happens to it? Does that make sense?

For other participants, the request for feedback was evident within the interview process for the current study. For example, the ending of the previous quote (“does that make sense”), as well as similar endings of previously provided quotes (e.g., “Did that answer your question?”) convey a general want for some possible confirmation. Or, when we checked in with a former foster youth participant half way through the interview and asked “how are things going,” they said:

Well, I like the way it’s gone so far because you guys throw the question and I answer it. You haven’t interrupted me. You haven’t invalidated anything I said. You haven’t given me the impression that you agree or disagree with what I’m saying, you’re just taking it all in … You’re not making me feel like I did something wrong or I’m saying something wrong. You are just taking it all in, which is kind of like … I can stay here. I can answer more questions. You’re actually interested in what I have to say whether it makes a difference or not … that’s what I’m getting.

Later, this participant concluded: “Yeah and kids want the same thing. They just want to let you know what's on their mind, even if you disagree.”

Re-engagement (or Maintained Engagement)

In addition to offering feedback in vivo during research, participants recommended circling back with participants at a later point to let them know what the researchers heard, what was shared, and what may have happened as a result of the work. It was clear that many felt that participants were asked to tell their story or parts of their story, but rarely knew what happened with their stories, how researchers interpreted them, and what the impact may have been. One participant shared:

Also making sure that you follow up with them afterwards, about anything that came up for them … if there is anything that you can do to support them, what the next steps are so they understand if they have a role moving past that interview and if you need to debrief or any of those kinds of things. (Former Foster Youth)

Participants expressed that it may increase their skepticism and reluctance to participate in research if researchers failed to maintain engagement post-data collection. Even more, some participants suggested that the experience of sharing one’s story may linger, triggering unresolved feelings, and serve as another instance wherein someone heard their story, but did not care enough to follow up. One former foster youth and foster youth advocate shared:

A lot of people would like to know what you’re using your research for. Is it to present the data to … I don’t know who funds counseling, but like whoever you’re trying to get more financial resources for counseling for foster youth who need it. If you write a grant, and even when you write a grant, as you get your money, you’re supposed to write a follow up of how you used your grant? That’s what I would think that foster youth would need as a follow-up, not just do an interview and get all this emotional labor and never follow up. I’d like to know, how has your research improved mental health services, specifically. So, I think the investment into youth is important.

Join as Co-creators of Research

Building on the importance of valued time, feedback, and relationships, the data supported a subtheme and recommendation focused on working with stakeholders as co-creators of research. This finding was evident in numerous quotes, emphasizing how critical it can be for someone who has firsthand experience to sit at the decision-making table. One A Home Within Clinical Consultant directly asked:

I mean, is there room for folks that were in the system to sit on the board in the construction of it and how are those faces present when these invitations are made? Right? Things like that I think can help. Again, it doesn’t guarantee you anything, but it just shows that this isn’t performative, you know?

Later this same participant returned to this idea, sharing:

So, when I talked earlier about how mental health is constructed, even that in of itself is like, “How do we get the folks who do the research to integrate people they’re researching” … seems very much like that gatekeeper thing, right? So, even that we struggle with that question, it puts people on the defensive, because then it’s like I have to justify why I deserve to know and be a part of people reviewing my story. That’s just a tough sell for folks. Yeah, and I don’t think anybody who has an understanding of what it’s like to not have control over their story and over their life, would willingly give up that easily? You know?

This quote captures how the question for researchers to even consider a decision of inclusion is inappropriate and off putting for many stakeholders in child welfare. Similarly, a former foster youth discussed both the importance of partnerships and the challenges they experienced with inclusion in process:

And so being a youth and being at that table, we’re kind of always told what’s best for us, what we should do, you know, what our goal should be. And it’s nice that I’m able to tell them. I’m able to share with them what works and what doesn’t work, because I’ve been there. I’ve seen what I struggled with at that age when I wasn’t able to vocalize it. (Former Foster Youth)

A former foster youth and child welfare professional shared how important the co-creation of research is at all stages of the research process, particularly when working to interpret the data and discuss the implications:

We often see policies and laws, and all of these things have been created, but it’s not like literally benefiting us, you know? They’re not looking at us as victims, who were, you know, put through a pipeline and because of the traumatic experiences that we experienced along that pipeline, how that has affected us and led us to other horrible situations … i.e, you know, prison, juvenile detention centers from school, from foster care going into placement from group homes to juvenile detention centers.

Numerous participants also underscored how critical research collaboration and co-creation is in participant recruitment. One social worker emphasized the importance of having stakeholders conduct the interviews and gather the data, to increase the likelihood of a positive experience for participants in research. Specifically, they said:

I think that is a big piece when foster parents can connect with … whoever it is trying to gather the information, for a lot of different reasons. I think it just kind of builds that rapport immediately when they know that they have some understanding of what they’re going through. I think that is a big piece to it.

Some stakeholders noted that when collaborating and co-creating research, it may be of value to offer support during the process, while individuals in this role are asked to hold both their own experiences in foster care and those of others:

It’s really hard I think to– If you haven’t dealt with your own trauma to support people in their own, and that piece I think is like one of the big lessons learned and youth engagement and youth development that I’ve kind of participated in is like you need to do a lot of pre work with people to get them up to speed on the why the how the purpose, you know, all those kinds of things that they get it and they understand the bigger picture and their role in that bigger picture. (Former Foster Youth and A Home Within Client)

Capture the Story, Not Just the Outcome

A final subtheme that emerged from the data was a request that the findings of a research study reflect a story, rather than a single outcome, or set of outcomes. The consensus was that it was important to avoid a reduction of their experience, and forgo critical context. One former foster youth emphasized the importance of not only hearing the context, but ensuring this was at the forefront of the findings:

So, the way that we share out some of this information once the research is actually completely concluded is also really important, maybe just as important as the way that the research is collected. Because if you’re not, if you’re not catering to the audience that you’re trying to reach and you’re not connecting with them, you’re not going to, you know, chances are that it’s not going to have the impact that you’re hoping to.

When considering how to capture the context in research, one participant shared:

I mean, I know that you guys know this as researchers. ... I think that a lot of times it’s very hard to use qualitative information to inform and persuade people because people love numbers and statistics and yada yada. I think that can be very hard but I think that the storytelling aspect of, of all of this, is also very important and I think that sharing perspective is also valuable … at the end of the day, to have an impact, whether that’s on policy or whatever it is. I think that there is a unique kind of value added to having young people telling the story that the research has created. I think that’s really impactful to have people who have lived experience, share that message about themselves and they can talk, instead of talking about this disassociated concept of what we took away from the survey. People can connect that back to their real life.

Later, the same participant continued:

Just be able to say that and say, you know, we learned, we evolved, we thought it through, we tested it, we looked, experimented. And here’s a better way we think of doing it. And people have to be comfortable with that kind of experimentation, rather than to assume that there’s absolute truth that will be revealed by a randomized trial. (Researcher, Social Worker)

A clinician and A Home Within Clinical Consultant offered a similar perspective, challenging the focus in research on diagnosis and pathology:

I think it starts by making it less detached. That’s the fatal flaw. And I think in research, I understand the intention behind it, but when it becomes - when you make it so clinical, when you detach it from your own experience, you can’t not look through your own experience. You can’t not see things and interpret them. But if your model comes from one that’s more normative, or at least we’ve diagnosed it as normative, how could you understand someone who does not live the same insurance, who hasn’t lived the same insurance: the habits, the ways they create security looked totally different to you. And so when you look at it, yes it looks pathological. You’re like, “Nobody I know has ever done this. And every book I’ve read is like, no this is normative.” But when you’re in a survival state, like what’s normal?

Research on foster care is complex and often requires a balance of both creativity and compromise, particularly when prioritizing the voices and perspectives of those with lived and/or professional experience (Garcia-Quiroga & Salvo Agoglia, 2020 ; Wilson & Conroy, 1999 ). Barriers exist that challenge what we know about those previously or currently in foster care, despite the consensus that this research knowledge is necessary to inform programs, policy, and practice. A clear theme across the summarized findings of this study is the need to establish relationships between stakeholders and researchers to (a) reduce barriers to participation and involvement, (b) increase inclusion and representation of stakeholder perspectives, and (c) support reciprocal learning.

Barriers to Research

A major finding of this study suggested that stakeholders may see parallels, or at least hold concerns about the similarities between their experiences in and/or with foster care and their participation in research on foster care. These concerns and hesitations among stakeholders pertained to both sharing personal content and information, as well as to the larger process of interacting with “others”—often unknown professionals—who hold perceived power in interpreting and influencing narratives about stakeholder experiences. These findings highlight the vulnerabilities that can come with disclosure, rooted in real and harmful experiences associated with foster care to both the participant and their community (Steenbakkers et al., 2016 ).

A second key finding of this study was an understanding by stakeholders that there is limited participation in research and that this impacts what is known and not known, as well as whose perspectives and experiences are represented in the literature. Participants expressed concern about an overreliance on specific partnerships in research, as well as a curiosity about whether participation may be confounded by individual affiliation or demographic identity. For example, participants noted a reliance on transition-age samples and wondered about whether retrospective experiences of past foster youth would inform current needs about experiences within a constantly changing system. Conversations centered on a critical need for increased ethnic and racial representation. Specifically, while child welfare disproportionately targets racially and ethnically minoritized individuals (Watt & Kim, 2019 ), participants consistently noted that those represented in research are more likely to identify with majority-status demographics. Across the interviews, questions arose about why this may be with conversations highlighting a compounding mistrust of research among minoritized stakeholders and the importance of anti-racist research that acknowledges and exposes the systemic and structural racism in foster care (Wilson, n.d ).

When considering participation, findings reflected an awareness that some stakeholders do not participate in research, because they may need time and space from their involvement with and/or experiences in the foster care system. One specific subtheme that emerged from the data centered on the importance of timing and when one might be asked to participate, noting that there are numerous reasons for not disclosing personal information, including not wanting to hurt those in their personal lives, self-protection from memories, and/or feeling “angry, hurt, ashamed, or otherwise uncomfortable” (Steenbakkers et al., 2016 , p. 5). Identifying and understanding these reasons require additional research to learn why these reasons can and do change over time and how they may affect research participation.

When stakeholders did share their experiences in research, findings suggested that their participation is often motivated by a desire to contribute, share success, and/or remain accountable to their success. Collectively, these motivations represent stakeholders’ interests in positive representation in the literature and the importance of balanced research that allows for opportunities to evidence both individual and collective strengths and contributions and not simply risks and challenges (Cook-Cottone & Beck, 2007 ). This mirrors other research findings (e.g., Ruff & Harrison, 2020 ; Ruff et al., 2023 ), which highlights that those with a history of foster care involvement can experience concern about the skew towards negative narratives and representations in research. Additionally, findings emphasized the importance of these strengths-based, balanced narratives coming from various stakeholder status groups and not just foster youth, to reduce the pressure youth stakeholders routinely shared of feeling isolated in holding primary advocacy roles (e.g., “We don’t want everything to be on us”).

Recommendations for Research

Participants generously shared numerous recommendations for research. Consistent across all interviews, stakeholders clearly communicated: (1) a relationship between researchers and child welfare stakeholders is imperative to reciprocal learning, and 2) building this relationship requires awareness of context and intentional inclusion. As stated by a Community Advisory Board (CAB) member, the message to prioritize across the research process is: “Let them know their voice matters.” To support relationships in research, participants highlighted the importance of inclusion at all steps of the research process and not simply data collection (Garcia-Quiroga & Salvo Agoglia, 2020 ). This perspective was clearly articulated by participants who noted that even having to justify or make a case to increase involvement was problematic, as inclusion at all stages, including the foundational moments, is the “right of rights,” recognizing stakeholders’ agency (Giorgi, 2010 ).

Recommendations for inclusion centered around the initiation and maintenance of relationships with stakeholders that support research integrity. Specifically, participants suggested inclusion in the co-creation of research questions, choice of incentive to signal respect and agency, reliance on stakeholders to collect data, co-authorships of findings, and sharing of any change or consequence to programs, policy, and practice. Participants also highlighted the value of ongoing communication during data collection that validated their experience, confirmed that their responses answered the research questions, and showed that the researchers were appropriately informed on foster care. Beyond data collection, participants noted that the experience of sharing one’s story may linger, triggering unresolved feelings for stakeholder participants, and suggested appropriate follow up. Across all interactions, participants offered recommendations that mirrored previous research (e.g., Jackson et al., 2012 ; Steenbakkers et al., 2016 ), focused on the importance of taking time to capture context and balance and resisting transactional exchanges. Participants also recommended the use of qualitative methodologies to capture necessary context and balance, and support the establishment of trust. This finding also mirrors previous research encouraging the consideration of mixed methods (e.g., Aarons et al., 2012 ) and underscores the importance of working to ensure a sense of control and agency in research participation.

When increasing inclusion in research, stakeholders recommended that researchers recognize and find ways to offer protection to participants. In addition to risks associated with sharing personal experience, participants noted ramifications associated with an over reliance on specific stakeholder groups. Specifically, our findings suggested that participants with lived experience often feel burned out by research requests and, in some situations, tokenized. Both participants and CAB members how their experiences in research changed over time and how understanding this may be of importance in both recruiting participants and in supporting participants throughout the process. Participants reflected that, at first, they tended to feel nervous and/or concerned about research involvement, wondering how others were hearing and interpreting their experiences. However, these same participants noted that eventually they developed an ability to hold psychological distance from their stories while sharing that supported regular participation. This experience inevitably changed again, when participation began to feel formulaic, detached, and as if it served an audience looking for a specific narrative. Further exploration of these experiences may serve to help understand stakeholders’ research participation as well as maybe even changes in one’s comfort sharing experiences over time. Additional investigation may also learn whether those who do not participate may (a) feel that they do not fit the standard/token criteria aligned with research interest and/or (b) be utilizing necessary boundaries around how and to whom they offer their time, resources, and stories.

Limitations and Future Directions

This study is not without limitations. As noted, participation may be confounded by participants’ affiliation with mental health treatment and/or services, by their participation in the foster care system as a whole, as well as by their age (over 18) and stakeholder status. Additional research is needed to understand unique experiences of participation in research by stakeholder status, and varying identities and affiliations. Research is also recommended that clarifies assumptions about which stakeholders should be included and whether research ought to prioritize the experience of stakeholders with personal versus professional experience.

The design of foster care increases challenges in accessing individuals in research. These cautions are well intended and prudent in protecting the privacy of children and their families, and yet they also can increase difficulties in understanding the scope, severity and experiences of professionals, families, and individuals in the system. This research explored the perceptions, experiences, and recommendations among stakeholders in child welfare. Additional studies are needed to build upon current study findings, and to further understand how to execute a primary recommendation to facilitate reciprocal, non-transactional relationships with participants.

Aarons, G. A., Fettes, D. L., Sommerfeld, D. H., & Palinkas, L. A. (2012). Mixed methods for implementation research: Application to evidence-based Practice implementation and staff turnover in community-based organizations providing child welfare services. Child Maltreatment, 17 (1), 67–79. https://doi.org/10.1177/1077559511426908

Article   Google Scholar  

Adoption and Foster Care Analysis and Reporting System (AFCARS). (2022). The Administration for Children and Families. https://www.acf.hhs.gov/cb/data-research/adoption-fostercare

Beidas, R. S., Marcus, S., Wolk, C. B., Powell, B., Aarons, G. A., Evans, A. C., Hurford, M. O., Hadley, T., Adams, D. R., Walsh, L. M., Babbar, S., Barg, F., & Mandell, D. S. (2016). A prospective examination of clinician and supervisor turnover within the context of implementation of evidence-based practices in a publicly-funded mental health system. Administration and Policy in Mental Health, 43 (5), 640–649. https://doi.org/10.1007/s10488-015-0673-6

Berrick, J. D., Frasch, K., & Fox, A. (2000). Assessing children’s experiences of out-of-home care: Methodological challenges and opportunities. Social Work Research, 24 (2), 119–127.

Birt, L., Scott, S., Cavers, D., Campbell, C., & Walter, F. (2016). Member checking: A tool to enhance trustworthiness or merely a nod to validation? Qualitative Health Research, 26 (13), 1802–1811. https://doi.org/10.1177/1049732316654870

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), 77–101.

Caelli, K., Ray, L., & Mill, J. (2003). “Clear as Mud”: Toward greater clarity in generic qualitative research. International Journal of Qualitative Methods, 2 (2), 1–13. https://doi.org/10.1177/160940690300200201

Charmaz, Kathy. (2006) Constructing grounded theory: A practical guide through qualitative analysis. Los Angeles, SAGE .

Clarke, V., & Braun, V. (2013). Successful qualitative research: A practical guide for beginners. ResearchGate , www.researchgate.net/publication/256089360_Successful_Qualitative_Research_A_Practical_Guide_for_Beginners

Cook-Cottone, C., & Beck, M. (2007). A model for life-story work: Facilitating the construction of personal narrative for foster children. Child and Adolescent Mental Health, 12 (4), 193–195. https://doi.org/10.1111/j.1475-3588.2007.00446.x

Curtin, M., & Fossey, E. (2007). Appraising the trustworthiness of qualitative studies: Guidelines for occupational therapists. Australian Occupational Therapy Journal, 54 (2), 88–94. https://doi.org/10.1111/j.1440-1630.2007.00661.x

DePanfilis, D., & Zlotnik, J. L. (2008). Retention of front-line staff in child welfare: A systematic review of research. Children and Youth Services Review, 30 (9), 995–1008.

Garcia-Quiroga, M., & Salvo Agoglia, I. (2020). Too vulnerable to participate? Challenges for meaningful participation in research with children in alternative care and adoption. International Journal of Qualitative Methods , 19 (1). https://doi.org/10.1177/1609406920958965

Gilbertson, R., & Barber, J. (2002). Obstacles to involving children and young people in foster care research. Child and Family Social Work, 7 , 253–258. https://doi.org/10.1046/j.1365-2206.2002.00251.x

Giorgi, A. (2010). Phenomenology and the practice of science. Existential Analysis, 21 (1), 3–22.

Google Scholar  

Gopalan, G., Hooley, C., Winters, A., & Stephens, T. (2019). Perceptions among child welfare staff when modifying a child mental health intervention to be implemented in child welfare services. American Journal of Community Psychology, 63 (3–4), 366–377.

Greiner, M. V., Beal, S. J., Allen, A., Patel, V., Meinzen-Derr, J., & Antommaria, A. H. M. (2018). Who speaks for me? Addressing variability in informed consent practices for minimal risk research involving foster youth. Journal of Health Disparities Research and Practice, 11 (4), 111–131.

Greiner, M. V., Beal, S. J., & Antommaria, A. H. M. (2020). Perspectives on informed consent practices for minimal-risk research involving foster youth. Pediatrics, 145 (4), e20192845. https://doi.org/10.1542/peds.2019-2845

Israel, B. A., Eng, E., Schulz, A. J., & Parker, E. A. (2005). Introduction to methods in community-based participatory research for health. Methods in Community-Based Participatory Research for Health, 3 , 26.

Jackson, Y., Gabrielli, J., Tunno, A. M., & Hambrick, E. P. (2012). Strategies for longitudinal research with youth in foster care: A demonstration of methods, barriers, and innovations. Child Youth Services Review, 34 (7), 1208–1213. https://doi.org/10.1016/j.childyouth.2012.02.007

Leake, R., Rienks, S., & Obermann, A. (2017). A deeper look at burnout in the child welfare workforce. Human Service Organizations: Management, Leadership & Governance, 41 (5), 492–502.

Ordoñez-Jasis, R., & Myck-Wayne, J. (2012). Community mapping in action: Uncovering resources and assets for young children and their families. Young Exceptional Children, 15 (3), 31–45. https://doi.org/10.1177/1096250612451756

Ruff, S. C., Linville, D., & Vasquez, N. (2023). “Resilience,” as defined by foster youth and key stakeholders. Journal of Public Child Welfare , 1–31. https://doi.org/10.1080/15548732.2023.2222665

Ruff, S. C., & Harrison, K. (2020). “Ask me what I want”: Community-based participatory research to explore transition-age foster Youth’s use of support services. Children and Youth Services Review, 108 (1), 104608.

Saldaña, J. M. (2015). The coding manual for qualitative researchers (3rd ed.). SAGE Publications .

Sandelowski, M. (2010). What’s in a name? Qualitative description revisited. Research in Nursing & Health, 33 (1), 77–84. https://doi.org/10.1002/nur.20362

Steenbakkers, A., van der Steen, S., & Grietens, H. (2016). ‘To talk or not to talk?’: Foster youth’s experiences of sharing stories about their past and being in foster care. Children and Youth Services Review, 71 , 2–9. https://doi.org/10.1016/j.childyouth.2016.10.008

Substance Abuse and Mental Health Services Administration (2014). Results from the 2013 National Survey on Drug Use and Health: Summary of National Findings, NSDUH Series H-48, HHS Publication No. (SMA) 14–4863. Rockville, MD: Substance Abuse and Mental Health Services Administration.

Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & Health Sciences, 15 (3), 398–405. https://doi.org/10.1111/nhs.12048

Watt, T., & Kim, S. (2019). Race/ethnicity and foster youth outcomes: An examination of disproportionality using the national youth in transition database. Children and Youth Services Review, 102 , 251–258. https://doi.org/10.1016/j.childyouth.2019.05.017

Wilson, L., & Conroy, J. (1999). Satisfaction of children in out-of-home care. Child Welfare, 78 (1), 53–69.

Wilson, V. (n.d.). Guiding principles for anti-racist research, the “bodycam” for racial economic injustice. Economic Policy Institute. https://www.epi.org/anti-racist-policy-research/guiding-principles-for-anti-racist-research-the-bodycam-for-racial-economic-injustice/

Download references

Acknowledgements

The authors wish to thank A Home Within for their long-term, consistent support of current and former foster youth, and the University of San Francisco for funding compensation of the student researchers of the Foster Care Research Group.

Open access funding provided by SCELC, Statewide California Electronic Library Consortium

Author information

Authors and affiliations.

Psychology Department, University of San Francisco, 2130 Fulton Street, San Francisco, CA, 94117, USA

Saralyn Ruff

Center for Transformative Healing, San Francisco, USA

Deanna Linville

University of California, Berkeley, San Francisco, USA

Quanice Hawkins

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Saralyn Ruff .

Ethics declarations

Ethical approval.

The authors declare that all procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee.

Consent to Participate

Informed consent was obtained from all individual participants involved in the study.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Ruff, S., Linville, D. & Hawkins, Q. A Qualitative Investigation of the Relationships Between Foster Care Stakeholders and Research. Glob Soc Welf (2024). https://doi.org/10.1007/s40609-024-00349-3

Download citation

Accepted : 20 August 2024

Published : 09 September 2024

DOI : https://doi.org/10.1007/s40609-024-00349-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Foster care
• Child welfare
• Foster youth
• Community-based participatory action research
• Child welfare research
• Qualitative research
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Best Practices for Reducing Bias in the Interview Process

Ilana bergelson.

Department of Urology, University of Iowa, Iowa City, USA

Elizabeth Takacs

Purpose of review.

Objective measures of residency applicants do not correlate to success within residency. While industry and business utilize standardized interviews with blinding and structured questions, residency programs have yet to uniformly incorporate these techniques. This review focuses on an in-depth evaluation of these practices and how they impact interview formatting and resident selection.

Recent Findings

Structured interviews use standardized questions that are behaviorally or situationally anchored. This requires careful creation of a scoring rubric and interviewer training, ultimately leading to improved interrater agreements and biases as compared to traditional interviews. Blinded interviews eliminate even further biases, such as halo, horn, and affinity bias. This has also been seen in using multiple interviewers, such as in the multiple mini-interview format, which also contributes to increased diversity in programs. These structured formats can be adopted to the virtual interviews as well.

There is growing literature that using structured interviews reduces bias, increases diversity, and recruits successful residents. Further research to measure the extent of incorporating this method into residency interviews will be needed in the future.

Introduction

Optimizing the criteria to rank residency applicants is a difficult task. The National Residency Matching Program (NRMP) is designed to be applicant-centric, with the overarching goal to provide favorable outcomes to the applicant while providing opportunity for programs to match high-quality candidates. From a program’s perspective, the NRMP is composed of three phases: the screening of applicants, the interview, and the creation of the rank list. While it is easy to compare candidates based on objective measures, these do not always reflect qualities required to be a successful resident or physician. Prior studies have demonstrated that objective measures such as Alpha Omega Alpha status, United States Medical Licensing Exams (USMLE), and class rank do not correlate with residency performance measures [ 1 ]. Due to the variability of these factors to predict success and recognition of the importance of the non-cognitive traits, most programs place increased emphasis on candidate interviews to assess fit [ 2 ].

Unfortunately, the interview process lacks standardization across residency programs. Industry and business have more standardized interviews and utilize best practices that include blinded interviewers, use of structured questions (situational and/or behavioral anchored questions), and skills testing. Due to residency interview heterogeneity, studies evaluating the interview as a predictor of success have failed to reliably predict who will perform well during residency. Additionally, resident success has many components, such that isolating any one factor, such as the interview, may be problematic and argues for a more holistic approach to resident selection [ 3 ]. Nevertheless, there are multiple ways the application review and interview can be standardized to promote transparency and improve resident selection.

Residency programs have begun adopting best practices from business models for interviewing, which include standardized questions, situational and/or behavioral anchored questions, blinded interviewers, and use of the multiple mini-interview (MMI) model. The focus of this review is to take a more in-depth look at practices that have become standard in business and to review the available data on the impact of these practices in resident selection.

Unstructured Versus Structured Interviews

Unstructured interviews are those in which questions are not set in advance and represent a free-flowing discussion that is conversational in nature. The course of an unstructured interview often depends on the candidate’s replies and may offer opportunities to divert away from topics that are important to applicant selection. While unstructured interviews may involve specific questions such as “tell me about a recent book you read” or “tell me about your research,” the questions do not seek to determine specific applicant attributes and may vary significantly between applicants. Due to their free-form nature, unstructured interviews may be prone to biased or illegal questions. Additionally, due to a lack of a specific scoring rubric, unstructured interviews are open to multiple biases in answer interpretation and as such generally show limited validity [ 4 ]. For the applicant, unstructured interviews allow more freedom to choose a response, with some studies reporting higher interviewee satisfaction with these questions [ 5 ].

In contrast to the unstructured interview, structured interviews use standardized questions that are written prior to an interview, are asked of every candidate, and are scored using an established rubric. Standardized questions may be behaviorally or situationally anchored [ 5 ]. Due to their uniformity, standardized interviews have higher interrater reliability and are less prone to biased or illegal questions.

Behavioral questions ask the candidate to discuss a specific response to a prior experience, which can provide insight into how an applicant may behave in the future [ 5 ]. Not only does the candidate’s response reflect a possible prediction of future behavior, it can also demonstrate the knowledge, priorities, and values of the candidate [ 5 ]. Questions are specifically targeted to reflect qualities the program is searching for (Table ​ (Table1) 1 ) [ 5 – 7 ].

Behavioral questions and character traits [ 5 – 7 ]

Situational questions require an applicant to predict how they would act in a hypothetical situation and are intended to reflect a realistic scenario the applicant may encounter during residency; this can provide insight into priorities and values [ 5 ]. For example, asking what an applicant would do when receiving sole credit for something they worked on with a colleague can provide insight into the integrity of a candidate [ 4 ]. These types of questions can be especially helpful for fellowships, as applicants would already have the clinical experience of residency to draw from [ 5 ].

Using standardized questions provides a method to recruit candidates with characteristics that ultimately correlate to resident success and good performance. Indeed, structured interview scores have demonstrated an ability to predict which students perform better with regard to communication skills, patient care, and professionalism in surgical and non-surgical specialties [ 8 •]. In fields such as radiology, non-cognitive abilities that can be evaluated in behavioral questions, such as conscientiousness or confidence, are thought to critically influence success in residency and even influence cognitive performance [ 1 ]. This has also been demonstrated in obstetrics and gynecology, where studies have shown that resident clinical performance after 1 year had a positive correlation with the rank list percentile that was generated using a structured interview process [ 9 ].

Creating Effective Structured Interviews

To be effective, standardized interview questions should be designed in a methodical manner. The first step in standardizing the interview process is determining which core values predict resident success in a particular program. To that end, educational leaders and faculty within the department should come to a consensus on the main qualities they seek in a resident. From there, questions can be formatted to elicit those traits during the interview process. Some programs have used personality assessment inventories to establish these qualities. Examples include openness to experience, humility, conscientiousness, and honesty. Further program-specific additions can be included, such as potential for success in an urban versus rural environment [ 10 ].

Once key attributes have been chosen and questions have been selected, a scoring rubric can be created. The scoring of each question is important as it helps define what makes a high-performing versus low-performing answer. Once a scoring system is determined, interviewers can be trained to review the questions, score applicant responses, and ensure they do not revise the questions during the interview [ 11 ]. Questions and the grading rubric should be further scrutinized through mock interviews with current residents, including discussing responses of the mock interviewee and modifying the questions and rubric prior to formal implementation [ 12 ]. Interviewer training itself is critical, as adequate training leads to improved interrater agreements [ 13 ]. Figure  1 demonstrates the steps to develop a behavioral interview question.

Example of standardized question to evaluate communication with scoring criteria

Rating the responses of the applicants can come with errors that ultimately reduce validity. For example, central tendency error involves interviewers not rating students at the extremes of a scale but rather placing all applicants in the middle; leniency versus severity refers to interviewers who either give all applicants high marks or give everyone low marks; contrast effects involve comparing one applicant to another rather than solely focusing on the rubric for each interviewee. These rating errors reflect the importance of training and providing feedback to interviewers [ 4 ].

Blinded Interviewers

Blinding the interviewers to the application prior to meeting with a candidate is intended to eliminate various biases within the interview process (Table ​ (Table2) 2 ) [ 14 , 15 ]. In addition to grades and test scores, aspects of the application that can either introduce or exacerbate bias include photographs, demographics, letters of recommendation, selection to medical honor societies, and even hobbies. Impressions of candidates can be formed prematurely, with the interview then serving to simply confirm (or contradict) those impressions [ 16 •]. Importantly, application blinding may also decrease implicit bias against applicants who identify as underrepresented in medicine [ 17 ].

Examples of bias [ 14 , 15 ]

Despite the proven success of these various interview tactics, their use in resident selection remains limited, with only 5% of general surgery programs using standardized interview questions and less than 20% using even a limited amount of blinding (e.g., blinding of photograph) [ 2 ]. Some programs have continued to rely on unblinded interviews and prioritize USMLE scores and course grades in ranking [ 18 ]. Due to their potential benefits and ability to standardize the interview process, it is critical that programs become familiar with the various interview practices so that they can select the best applicants while minimizing the significant bias in traditional interview formats.

Multiple Mini-interview (MMI)

The use of multiple interviews by multiple interviewers provides an opportunity to ask the applicant more varied questions and also allows for the averaging out of potential interviewer bias leading to more consistent applicant scoring and ability to predict applicant success [ 7 ]. Training of the interviewers in interviewing techniques, scoring, and avoiding bias is also likely to decrease scoring variability. Similarly, the use of the same group of interviewers for all candidates should be encouraged in order to limit variance in scoring amongst certain faculty [ 19 ].

One interview method that incorporates multiple interviewers and has had growing frequency in medical school interviews as well as residency interviews is the MMI model. This system provides multiple interviews in the form of 6–12 stations, each of which evaluates a non-medical question designed to assess specific non-academic applicant qualities [ 20 ]. While the MMI format can intimidate some candidates, others find that it provides an opportunity to demonstrate traits that would not be observed in an unstructured interview, such as multitasking, efficiency, flexibility, interpersonal skills, and ethical decision-making [ 21 ]. Furthermore, MMI has been shown to have increased reliability as shown in a study of five California medical schools that showed inter-interviewer consistency was higher for MMIs than traditional interviews which were unstructured and had a 1:1 ratio of interviewer to applicant [ 22 ].

The MMI format is also versatile enough to incorporate technical competencies even through a virtual platform. In general surgery interviews, MMI platforms have been designed to test traits such as communication and empathy but also clinical knowledge and surgical aptitude through anatomy questions and surgical skills (knot tying and suturing). Thus, MMIs are not only versatile, but also have an ability to evaluate cognitive traits and practical skills [ 23 ].

MMI also has the potential to reduce resident attrition. For example, in evaluating students applying to midwifery programs in Australia, attrition rates and grades were compared for admitted students using academic rank and MMI scores obtained before and after the incorporation of MMIs into their selection program. The authors found that when using MMIs, enrolled students had not only higher grades but significantly lower attrition rates. MMI was better suited to show applicants’ passion and commitment, which then led to similar mindsets of accepted applicants as well as a support network [ 24 ]. Furthermore, attrition rates have been found to be higher in female residents in general surgery programs [ 25 ]. Perhaps with greater diversity, which is associated with use of standardized interviews, the number of women can increase in surgical specialties and thus reduce attrition rate in this setting as well.

Impact of Interview Best Practices on Bias and Diversity

An imperative of all training programs is to produce a cohort of physicians with broad and diverse experiences representative of the patient populations they treat. To better address diversity within surgical residencies, particularly regarding women and those who are underrepresented in medicine, it is important that interviews be designed to minimize bias against any one portion of the applicant pool. Diverse backgrounds and cultures within a program enhance research, innovation, and collaboration as well as benefit patients [ 26 ]. Patients have shown greater satisfaction and reception when they share ethnicity or background with their provider, and underrepresented minorities in medicine often go on to work in underserved communities [ 27 ].

All interviewers undoubtedly have elements of implicit bias; Table ​ Table2 2 describes the common subtypes of implicit bias [ 14 ]. While it is difficult to eliminate bias in the interview process, unstructured or “traditional” interviews are more likely to risk bias toward candidates than structured interviews. Studies have demonstrated that Hispanic and Black applicants receive scores one quarter of a standard deviation lower than Caucasian applicants [ 28 ]. “Like me” bias is just one example of increased subjectivity with unstructured interviews, where interviewers prefer candidates who may look like, speak like, or share personal experiences with the interviewer [ 29 ].

Furthermore, unstructured interviews provide opportunities to ask inappropriate or illegal questions, including those that center on religion, child planning, and sexual orientation [ 30 ]. Inappropriate questions tend to be disproportionately directed toward certain groups, with women more likely to get questions regarding marital status and to be questioned and interrupted than male counterparts [ 28 , 31 ].

Structured interviews, conversely, have been shown to decrease bias in the application process. Faculty trained in behavior-based interviews for fellowship applications demonstrated that there were reduced racial biases in candidate evaluations due to scoring rubrics [ 12 ]. Furthermore, as structured questions are determined prior to the interview and involve training of interviewers, structured interviews are less prone to illegal and inappropriate questions [ 32 ]. Interviewers can ask additional questions such as “could you be more specific?” with the caveat that probing should be minimized and kept consistent between applications. This way the risk of prompting the applicant toward a response is reduced [ 4 ].

Implementing Interview Types During the Virtual Interview Process

An added complexity to creating standardized interviews is incorporating a virtual platform. Even prior to the move toward virtual interviews instituted during the COVID-19 pandemic, studies on virtual interviews showed that they provided several advantages over in-person interviews, including decreased cost, reduction in time away from commitments for applicants and staff, and ability to interview at more programs. A significant limitation, for applicants and for programs, is the inability to interact informally, which allows applicants to evaluate the environment of the hospital and the surrounding community [ 33 •]. Following their abrupt implementation in 2020 during the COVID-19 pandemic, virtual interviews have remained in place and likely will remain in place in some form into the future due to their significant benefits in reducing applicant cost and improving interview efficiency. Although these types of interviews are in their relative infancy in the resident selection process, studies have found that standardized questions and scoring rubrics that have been used in person can still be applied to a virtual interview setting without degrading interview quality [ 34 ].

The virtual format may also allow for further interview innovation in the form of standardized video interviews. For medical student applicants, the Association of American Medical Colleges (AAMC) has trialed a standardized video interview (SVI) that includes recording of applicant responses, scoring, and subsequent release to the Electronic Residency Application Service (ERAS) application. Though early data in the pilot was promising, the program was not continued after the 2020 cycle due to lack of interest [ 35 ]. There is limited evidence supporting the utility of this type of interview in residency training, and one study found that these interviews did not add significant benefit as the scores did not associate with other candidate attributes such as professionalism [ 32 ]. Similarly, a separate study found no correlation between standardized video interviews and faculty scores on traits such as communication and professionalism. Granted, there was no standardization in what the faculty asked, and they were not blinded to academic performance of the applicants [ 36 ]. While there was an evaluation of six emergency medicine programs that demonstrated a positive linear correlation between the SVI score and the traditional interview score, it was a very low r coefficient; thus the authors concluded that the SVI was not adequate to replace the interview itself [ 37 ].

Conclusions: Future Steps in Urology and Beyond

The shift to structured interviews in urology has been slow. Within the last decade, studies consistent with other specialties demonstrated that urology program directors prioritized USMLE scores, reference letters, and away rotations at the program director’s institution as the key factors in choosing applicants [ 38 ]. More recently, a survey of urology programs found < 10% blinded the recruitment team at the screening step, with < 20% blinding the recruitment team during the interview itself [ 39 ]. In 2020 our program began using structured interview questions and blinded interviewers to all but the personal statement and letters of recommendation. After querying faculty and interviewees, we have found that most interviewers do not miss the additional information, and applicants feel that they are able to have more eye contact with faculty who are not looking down at the application during the interview. Structured behavioral interview questions have allowed us to focus on the key attributes important to our program. With time we hope to see that inclusion of these metrics helps diversify our resident cohort, improve resident satisfaction with the training program, and produce successful future urologists.

Despite the slow transition in urology and other fields, there is a growing body of literature in support of standardized interviews for evaluating key candidate traits that ultimately lead to resident success and reducing bias while increasing diversity. With time, the hope is that programs will continue incorporating these types of interviews in the resident selection process.

Compliance with Ethical Standards

The authors have no financial or non-financial interests to disclose.

This article does not contain any studies with human or animal subjects performed by any of the authors.

This article is part of Topical Collection on Education

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Papers of particular interest, published recently, have been highlighted as: • Of importance