adolescent problem solving behavior

Connection denied by Geolocation Setting.

Reason: Blocked country: Russia

The connection was denied because this country is blocked in the Geolocation settings.

Please contact your administrator for assistance.

• Share full article

How to Help Teens Struggling With Mental Health

Answers to common questions about identifying and compassionately addressing issues of anxiety and depression in adolescents.

By Matt Richtel

Leer en español

Health risks in adolescence are undergoing a major shift. Three decades ago, the biggest health threats to teenagers were binge drinking, drunken driving, teenage pregnancy, cigarettes and illicit drugs. Today, they are anxiety, depression, suicide, self-harm and other serious mental health disorders.

From 2001 to 2019, the suicide rate for American youngsters from ages 10 to 19 jumped 40 percent, and emergency room visits for self-harm rose 88 percent.

Managing a mental health crisis can be challenging for teenagers and their parents. It is often unchartered territory that needs to be navigated with the utmost sensitivity. The guidance below may help.

What are the signs of an adolescent struggling with anxiety or depression?

Anxiety and depression are different but can share some indicators. First, look for some key changes in a youth’s behavior, such as disinterest in eating or participating in social activities previously enjoyed, altered sleep patterns or withdrawal from other aspects of life. It’s tricky; these behaviors can sometimes be normal teenage angst. However, a teenager in distress may express excessive worry, hopelessness or profound sadness, particularly for long periods of time.

Whether a teenager is dealing with angst or a clinical problem “is the 64 jillion dollar question,” said Stephen Hinshaw, an expert in teenage mental health issues at the University of California, Berkeley. The question is about “persistence, interference with thriving, sheer suffering (on her or his part and yours) that can help to make this difficult differentiation.”

If the lines become too blurry to tell the difference, it can help to visit a pediatrician to explore whether there is a clinical problem.

What’s the best way to start a discussion with an adolescent who may be struggling?

The counsel from experts is resounding: Be clear and direct and don’t shy from hard questions, but also approach these issues with compassion and not blame. Challenging as it may seem to talk about these issues, young people often are desperate to be heard. At the same time, talking to a parent can feel hard.

“Be gentle, be curious, and, over time, be persistent but not insistent,” Dr. Hinshaw explained. “Shame and stigma are a huge part of the equation here, and if you are outraged and judgmental, be prepared for a shutdown.”

“A good number” of teenagers “are practically begging you — without telling you so directly — to stay concerned and loving and to keep open a dialogue,” Dr. Hinshaw said.

For teenagers having trouble opening up, try working together on a shared hobby or activity without bringing up their mental health. Put them at ease, and eventually they may be more willing to share.

These issues are “typically very hard for a teen to talk about with their parent or guardian,” said Nicole Nadell, an assistant professor in pediatrics and psychiatry at Mount Sinai. “Be a patient and active listener at first, reflect back to the teen what they are saying, thinking and feeling.”

Advertisement

What can I do if I’m feeling suicidal?

If you are having thoughts of suicide or are concerned that someone you know may be having those thoughts, in the United States call or text 988 to reach the National Suicide Prevention Lifeline, or call 911, go to the emergency room, get help from an adult, or go to SpeakingOfSuicide.com/resources for a list of additional resources. Go here for resources outside the United States.

It is a sign of strength, not weakness, to seek help.

Research shows that the suicidal impulse will pass if it can be put off. Then the underlying problems can be addressed. Researchers call suicide “a permanent solution to a temporary problem.” It takes help to get through this period of excruciating pain that leads to suicidal ideation. Get that help.

I am concerned that a loved one is cutting or self-harming. What can I do?

Self-harm can include cutting, hitting oneself, burning or other forms of mutilation. These behaviors may appear to induce pain, but they are actually intended to redirect or make emotional pain go away, experts said.

Nonsuicidal self-injury is “predominately used to re-regulate,” said Emily Pluhar, a child and adolescent psychologist at Harvard Medical School. The behavior, she explained, can actually release a pain analgesic, a natural painkiller that may provide a feeling of relief. “It helps people re-regulate and feel calmer.”

The trouble is that such behavior ultimately doesn’t work to eliminate the underlying problem and then can intensify.

Cutting often happens on the wrists, ankles or thighs. “Self-harm is often hidden from parents and peers by long sleeves and secrecy. If you see evidence, try to engage in discussion — even though your teen is likely to try to minimize or hide it, out of shame,” Dr. Hinshaw said.

If you see wounds that appear to put a teenager at imminent risk, call 911 or go straight to the emergency room.

When cutting is discovered, it is vital that a parent or caregiver reacts with concern and compassionate curiosity, not alarm (unless danger is imminent). “First, be curious rather than alarmist. The best way to get your teen never to talk with you about any key problem is to be outraged or moral or judgmental,” Dr. Hinshaw said.

Some recommended tactics for starting and stoking conversation about mental health issues in general include making sure you are genuine and authentic — admit if the subject makes you nervous — and creating silence and space for a youth to express. And try a “change of settings,” some experts recommend , like a car ride or an activity, that can make conversation feel natural with less eye contact.

Then: “Validate, validate, validate,” says Dr. Pluhar, from Harvard. “You don’t have to agree with their perspective but you have to validate that their perspective matters and that you understand it.”

The ultimate goal is to help an adolescent find the root cause of the emotional pain leading to self-harm. Once your child is ready, a pediatrician or another health expert can help you find an appropriate counseling path. Evidence supports various forms of cognitive behavioral therapy, including dialectical behavioral therapy to help teach coping skills. These skills help people recognize their thinking patterns and reframe issues in healthier ways.

It is important to understand that self-harm is not the same as a suicidal ideation, which is a much broader overriding of the biological instinct to survive. That said, self-harm that goes on for a long time and that becomes more severe can be a predictor of suicidal behavior.

Are there alternatives to self-harm that can help my child manage emotions?

It may be worthwhile to suggest healthy alternatives to self-harm that your child could try. Research shows that the urge can be put off by removing from the home the object or tool used to harm, and by using simple methods like exercise. Dr. Nadell from Mount Sinai suggests a few:

Engage in intense exercise for 20 minutes

Use meditative breathing and muscle relaxation

Call a friend

Go for a walk

Keep a journal

Use ice or cold water on the body to change body temperature

How do I find the right doctor for my child? And how can I be sure my teenager has received the correct diagnosis?

Dr. Hinshaw recommends several concrete steps:

“Ask other parents, or engage in self-help/advocacy groups, to get a sense of the clinicians in your area with reputations for careful, state-of-the-art assessments/evaluations versus those who are too quick on the draw.”

“Ask the prospective assessor: How many evaluations have you performed for ADHD or anxiety or depression? How many hours does such an evaluation often take? How many, would you estimate, of those evaluations that you perform end up confirming versus disconfirming a diagnosis?”

Ask if the provider uses evidence-based rating scales filled out by parents and teachers that provide objective measures of how a young person is doing socially and academically. These measurement tools are important, experts say, because parents often can receive a skewed picture of how a young person is acting — for example, the child may appear moody and contentious at home or complain about how difficult school is, while actually performing well in classes, having friends and adapting well.

Ask: “Do you contact the school for additional information? For any condition, do you get a developmental history from the parents, from infancy onwards, about milestones, deficits, strengths and contexts that seem to accentuate versus help with the problems at hand? Do you get a family history of similar conditions?”

I’m concerned about medication for my teenager. What’s the best way to be sure that an adolescent is getting the right medication, in the right amount?

Psychiatric medications can be powerful and effective. But they can have side effects, risky interactions with drugs and withdrawal challenges. So parents should approach the issue of medication with the same clear, thoughtful inquiry that they would when seeking a therapist, pediatrician or other provider to help with mental health issues in general.

“Again,” Dr. Hinshaw said, “ask around for doctors/psychiatrists with excellent reputations along these lines. And work with a doctor/psychiatrist who strives for the lowest possible dose of the right medication for your teen’s issues.”

Ideally, said Dr. Nadell from Mount Sinai, the prescriber will be specialized as a child and adolescent psychiatrist. The challenge in many parts of the country is that specialists are unavailable or only take cash or private insurance. That means, experts said, parents should press pediatricians or primary care doctors on their experience and make sure they explain the side effects and interactions with other drugs, as well as how to tell if the medication is working and how hard it is to wean off the medication.

Remember, often the best first line of treatment for mental health issues is cognitive behavioral therapy or other nonmedical techniques. These strategies give an adolescent tools for coping with anxiety, stress and other challenges. Research shows that when medications are needed, they can be most effective when used together with such therapies.

What else can you do to help with mental health?

Experts say there are essential habits to promote mental and physical health. Sleep is huge. Young people, with developing brains, need eight to 10 hours of sleep. Lack of sleep can interfere with development, and can dramatically impact mood and the ability to learn. Physical activity is also vital to mental and physical well-being.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Problem-solving interventions and depression among adolescents and young adults: A systematic review of the effectiveness of problem-solving interventions in preventing or treating depression

Roles Conceptualization, Data curation, Formal analysis, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, United States of America

Roles Conceptualization, Writing – original draft

Affiliation Centre for Evidence and Implementation, London, United Kingdom

Roles Data curation

Roles Conceptualization, Writing – review & editing

Affiliation Department of Psychology, Virginia Commonwealth University, Richmond, VA, United States of America

Roles Conceptualization, Methodology

Roles Conceptualization, Project administration, Writing – review & editing

Affiliation Centre for Evidence and Implementation, Melbourne, Victoria, Australia

Roles Conceptualization, Formal analysis, Methodology, Writing – original draft, Writing – review & editing

Affiliation Department of Social Work, Monash University, Melbourne, Victoria, Australia

• Kristina Metz, 
• Jane Lewis, 
• Jade Mitchell, 
• Sangita Chakraborty, 
• Bryce D. McLeod, 
• Ludvig Bjørndal, 
• Robyn Mildon, 
• Aron Shlonsky

• Published: August 29, 2023
• https://doi.org/10.1371/journal.pone.0285949
• Peer Review
• Reader Comments

Problem-solving (PS) has been identified as a therapeutic technique found in multiple evidence-based treatments for depression. To further understand for whom and how this intervention works, we undertook a systematic review of the evidence for PS’s effectiveness in preventing and treating depression among adolescents and young adults. We searched electronic databases ( PsycINFO , Medline , and Cochrane Library ) for studies published between 2000 and 2022. Studies meeting the following criteria were included: (a) the intervention was described by authors as a PS intervention or including PS; (b) the intervention was used to treat or prevent depression; (c) mean or median age between 13–25 years; (d) at least one depression outcome was reported. Risk of bias of included studies was assessed using the Cochrane Risk of Bias 2.0 tool. A narrative synthesis was undertaken given the high level of heterogeneity in study variables. Twenty-five out of 874 studies met inclusion criteria. The interventions studied were heterogeneous in population, intervention, modality, comparison condition, study design, and outcome. Twelve studies focused purely on PS; 13 used PS as part of a more comprehensive intervention. Eleven studies found positive effects in reducing depressive symptoms and two in reducing suicidality. There was little evidence that the intervention impacted PS skills or that PS skills acted as a mediator or moderator of effects on depression. There is mixed evidence about the effectiveness of PS as a prevention and treatment of depression among AYA. Our findings indicate that pure PS interventions to treat clinical depression have the strongest evidence, while pure PS interventions used to prevent or treat sub-clinical depression and PS as part of a more comprehensive intervention show mixed results. Possible explanations for limited effectiveness are discussed, including missing outcome bias, variability in quality, dosage, and fidelity monitoring; small sample sizes and short follow-up periods.

Citation: Metz K, Lewis J, Mitchell J, Chakraborty S, McLeod BD, Bjørndal L, et al. (2023) Problem-solving interventions and depression among adolescents and young adults: A systematic review of the effectiveness of problem-solving interventions in preventing or treating depression. PLoS ONE 18(8): e0285949. https://doi.org/10.1371/journal.pone.0285949

Editor: Thiago P. Fernandes, Federal University of Paraiba, BRAZIL

Received: January 2, 2023; Accepted: May 4, 2023; Published: August 29, 2023

Copyright: © 2023 Metz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant methods and data are within the paper and its Supporting Information files.

Funding: This work was commissioned by Wellcome Trust and was conducted independently by the evaluators (all named authors). No grant number is available. Wellcome Trust had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. The authors declare no financial or other competing interests, including their relationship and ongoing work with Wellcome Trust. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Depression among adolescents and young adults (AYA) is a serious, widespread problem. A striking increase in depressive symptoms is seen in early adolescence [ 1 ], with rates of depression being estimated to almost double between the age of 13 (8.4%) and 18 (15.4%) [ 2 ]. Research also suggests that the mean age of onset for depressive disorders is decreasing, and the prevalence is increasing for AYA. Psychosocial interventions, such as cognitive-behavioural therapy (CBT) and interpersonal therapy (IPT), have shown small to moderate effects in preventing and treating depression [ 3 – 6 ]. However, room for improvement remains. Up to half of youth with depression do not receive treatment [ 7 ]. When youth receive treatment, studies indicate that about half of youth will not show measurable symptom reduction across 30 weeks of routine clinical care for depression [ 8 ]. One strategy to improve the accessibility and effectiveness of mental health interventions is to move away from an emphasis on Evidence- Based Treatments (EBTs; e.g., CBT) to a focus on discrete treatment techniques that demonstrate positive effects across multiple studies that meet certain methodological standards (i.e., common elements; 9). Identifying common elements allows for the removal of redundant and less effective treatment content, reducing treatment costs, expanding available service provision and enhancing scability. Furthermore, introducing the most effective elements of treatment early may improve client retention and outcomes [ 9 – 13 ].

A potential common element for depression intervention is problem-solving (PS). PS refers to how an individual identifies and applies solutions to everyday problems. D’Zurilla and colleagues [ 14 – 17 ] conceptualize effective PS skills to include a constructive attitude towards problems (i.e., a positive problem-solving orientation) and the ability to approach problems systematically and rationally (i.e., a rational PS style). Whereas maladaptive patterns, such as negative problem orientation and passively or impulsively addressing problems, are ineffective PS skills that may lead to depressive symptoms [ 14 – 17 ]. Problem Solving Therapy (PST), designed by D’Zurilla and colleagues, is a therapeutic approach developed to decrease mental health problems by improving PS skills [ 18 ]. PST focuses on four core skills to promote adaptive problem solving, including: (1) defining the problem; (2) brainstorming possible solutions; (3) appraising solutions and selecting the best one; and (4) implementing the chosen solution and assessing the outcome [ 14 – 17 ]. PS is also a component in other manualized approaches, such as CBT and Dialectical Behavioural Therapy (DBT), as well as imbedded into other wider generalized mental health programming [ 19 , 20 ]. A meta-analysis of over 30 studies found PST, or PS skills alone, to be as effective as CBT and IPT and more effective than control conditions [ 21 – 23 ]. Thus, justifying its identification as a common element in multiple prevention [ 19 , 24 ] and treatment [ 21 , 25 ] programs for adult depression [ 9 , 26 – 28 ].

PS has been applied to youth and young adults; however, no manuals specific to the AYA population are available. Empirical studies suggest maladaptive PS skills are associated with depressive symptoms in AYA [ 5 , 17 – 23 ]. Furthermore, PS intervention can be brief [ 29 ], delivered by trained or lay counsellors [ 30 , 31 ], and provided in various contexts (e.g., primary care, schools [ 23 ]). Given PS’s versatility and effectiveness, PS could be an ideal common element in treating AYA depression; however, to our knowledge, no reviews or meta-analyses on PS’s effectiveness with AYA specific populations exist. This review aimed to examine the effectiveness of PS as a common element in the prevention and treatment of depression for AYA within real-world settings, as well as to ascertain the variables that may influence and impact PS intervention effects.

Identification and selection of studies

Searches were conducted using PsycInfo , Medline , and Cochrane Library with the following search terms: "problem-solving", “adolescent”, “youth”, and” depression, ” along with filters limiting results to controlled studies looking at effectiveness or exploring mechanisms of effectiveness. Synonyms and derivatives were employed to expand the search. We searched grey literature using Greylit . org and Opengrey . eu , contacted experts in the field and authors of protocols, and searched the reference lists of all included studies. The search was undertaken on 4 th June 2020 and updated on 11 th June 2022.

Studies meeting the following criteria were included: (a) the intervention was described by authors as a PS intervention or including PS; (b) the intervention was used to treat or prevent depression; (c) mean or median age between 13–25 years; and (d) at least one depression outcome was reported. Literature in electronic format published post 2000 was deemed eligible, given the greater relevance of more recent usage of PS in real-world settings. There was no exclusion for gender, ethnicity, or country setting; only English language texts were included. Randomized controlled trials (RCTs), quasi-experimental designs (QEDs), systematic reviews/meta-analyses, pilots, or other studies with clearly defined comparison conditions (no treatment, treatment as usual (TAU), or a comparator treatment) were included. We excluded studies of CBT, IPT, Acceptance and Commitment Therapy (ACT), Dialectical Behaviour Therapy (DBT), and modified forms of these treatments. These treatments include PS and have been shown to demonstrate small to medium effects on depression [ 13 , 14 , 32 ], but the unique contribution of PS cannot be disentangled. The protocol for this review was not registered; however, all data collection forms, extraction, coding and analyses used in the review are available upon inquiry from the first author.

Study selection

All citations were entered into Endnote and uploaded to Covidence for screening and review against the inclusion/exclusion criteria. Reviewers with high inter-rater reliability (98%) independently screened the titles and abstracts. Two reviewers then independently screened full text of articles that met criteria. Duplicates, irrelevant studies, and studies that did not meet the criteria were removed, and the reason for exclusion was recorded (see S1 File for a list of excluded studies). Discrepancies were resolved by discussion with the team leads.

Data extraction

Two reviewers independently extracted data that included: (i) study characteristics (author, publication year, location, design, study aim), (ii) population (age, gender, race/ethnicity, education, family income, depression status), (iii) setting, (iv) intervention description (therapeutic or preventative, whether PS was provided alone or as part of a more comprehensive intervention, duration, delivery mode), (v) treatment outcomes (measures used and reported outcomes for depression, suicidality, and PS), and (vi) fidelity/implementation outcomes. For treatment outcomes, we included the original statistical analyses and/or values needed to calculate an effect size, as reported by the authors. If a variable was not included in the study publication, we extracted the information available and made note of missing data and subsequent limitations to the analyses.

RCTs were assessed for quality (i.e., confidence in the study’s findings) using the Cochrane Risk of Bias 2.0 tool [ 33 ] which includes assessment of the potential risk of bias relating to the process of randomisation; deviations from the intended intervention(s); missing data; outcome measurement and reported results. Risk of bias pertaining to each domain is estimated using an algorithm, grouped as: Low risk; Some concerns; or High risk. Two reviewers independently assessed the quality of included studies, and discrepancies were resolved by consensus.

We planned to conduct one or more meta-analyses if the studies were sufficiently similar. Data were entered into a summary of findings table as a first step in determining the theoretical and practical similarity of the population, intervention, comparison condition, outcome, and study design. If there were sufficiently similar studies, a meta-analysis would be conducted according to guidelines contained in the Cochrane Collaboration Handbook of Systematic Reviews, including tests of heterogeneity and use of random effects models where necessary.

The two searches yielded a total number of 874 records (after the removal of duplicates). After title and abstract screening, 184 full-text papers were considered for inclusion, of which 25 studies met the eligibility criteria and were included in the systematic review ( Fig 1 ). Unfortunately, substantial differences (both theoretical and practical) precluded any relevant meta-analyses, and we were limited to a narrative synthesis.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0285949.g001

Risk of bias assessment

Risk of bias assessments were conducted on the 23 RCTs ( Fig 2 ; assessments by study presented in S1 Table ). Risk of bias concerns were moderate, and a fair degree of confidence in the validity of study findings is warranted. Most studies (81%) were assessed as ‘some concerns’ (N = 18), four studies were ‘low risk’, and one ‘high risk’. The most frequent areas of concern were the selection of the reported result (n = 18, mostly due to inadequate reporting of a priori analytic plans); deviations from the intended intervention (N = 17, mostly related to insufficient information about intention-to-treat analyses); and randomisation process (N = 13).

https://doi.org/10.1371/journal.pone.0285949.g002

Study designs and characteristics

Study design..

Across the 25 studies, 23 were RCTs; two were QEDs. Nine had TAU or wait-list control (WLC) comparator groups, and 16 used active control groups (e.g., alternative treatment). Eleven studies described fidelity measures. The sample size ranged from 26 to 686 and was under 63 in nine studies.

Selected intervention.

Twenty interventions were described across the 25 studies ( Table 1 ). Ten interventions focused purely on PS. Of these 10 interventions: three were adaptations of models proposed by D’Zurilla and Nezu [ 20 , 34 ] and D’Zurilla and Goldfried [ 18 ], two were based on Mynors-Wallis’s [ 35 ] Problem-Solving Therapy (PST) guide, one was a problem-orientation video intervention adapted from D’Zurilla and Nezu [ 34 ], one was an online intervention adapted from Method of Levels therapy, and three did not specify a model. Ten interventions used PS as part of a larger, more comprehensive intervention (e.g., PS as a portion of cognitive therapy). The utilization and dose of PS steps included in these interventions were unclear. Ten interventions were primary prevention interventions–one of these was universal prevention, five were indicated prevention, and four were selective prevention. Ten interventions were secondary prevention interventions. Nine interventions were described as having been developed or adapted for young people.

https://doi.org/10.1371/journal.pone.0285949.t001

Intervention delivery.

Of the 20 interventions, eight were delivered individually, eight were group-based, two were family-based, one was mixed, and in one, the format of delivery was unclear. Seventeen were delivered face-to-face and three online. Dosage ranged from a single session to 21, 50-minute sessions (12 weekly sessions, then 6 biweekly sessions); the most common session formar was once weekly for six weeks (N = 5).

Intervention setting and participants.

Seventeen studies were conducted in high-income countries (UK, US, Australia, Netherlands, South Korea), four in upper-middle income (Brazil, South Africa, Turkey), and four in low- and middle-income countries (Zimbabwe, Nigeria, India). Four studies included participants younger than 13 and four older than 25. Nine studies were conducted on university or high school student populations and five on pregnant or post-partum mothers. The remaining 11 used populations from mental health clinics, the community, a diabetes clinic, juvenile detention, and a runaway shelter.

Sixteen studies included participants who met the criteria for a depressive, bipolar, or suicidal disorder (two of these excluded severe depression). Nine studies did not use depression symptoms in the inclusion criteria (one of these excluded depression). Several studies excluded other significant mental health conditions.

Outcome measures.

Eight interventions targeted depression, four post/perinatal depression, two suicidal ideation, two resilience, one ‘problem-related distress’, one ‘diabetes distress’, one common adolescent mental health problem, and one mood episode. Those targeting post/perinatal depression used the Edinburgh Postnatal Depression Scale as the outcome measure. Of the others, six used the Beck Depression Inventory (I or II), two the Children’s Depression Inventory, three the Depression Anxiety Stress Scale-21, three the Centre for Epidemiologic Studies Depression Scale, one the Short Mood and Feelings Questionnaire, one the Hamilton Depression Rating Scale, one the depression subscale on the Schedule for Affective Disorders and Schizophrenia for School-Age Children, one the Strengths and Difficulties Questionnaire, one the Youth Top Problems Score, one the Adolescent Longitudinal Interval Follow-up Evaluation and Psychiatric Status Ratings, one the Kiddie Schedule for Affective Disorders and Schizophrenia, and one the Mini International Neuropsychiatric Interview.

Only eight studies measured PS skills or orientation outcomes. Three used the Social Problem-Solving Inventory-Revised, one the Problem Solving Inventory, two measured the extent to which the nominated problem had been resolved, one observed PS in video-taped interactions, and one did not specify the measure.

The mixed findings regarding the effectiveness of PS for depression may depend on the type of intervention: primary (universal, selective, or indicated), secondary or tertiary prevention. Universal prevention interventions target the general public or a population not determined by any specific criteria [ 36 ]. Selective prevention interventions target specific populations with an increased risk of developing a disorder. Indicated prevention interventions target high-risk individuals with sub-clinical symptoms of a disorder. Secondary prevention interventions include those that target individuals diagnosed with a disorder. Finally, tertiary prevention interventions refer to follow-up interventions designed to retain treatment effects. Outcomes are therefore grouped by intervention prevention type and outcome. Within these groupings, studies with a lower risk of bias (RCTs) are presented first. According to the World Health Organisation guidelines, interventions were defined as primary, secondary or tertiary prevention [ 36 ].

Universal prevention interventions

One study reported on a universal prevention intervention targeting resilience and coping strategies in US university students. The Resilience and Coping Intervention, which includes PS as a primary component of the intervention, found a significant reduction in depression compared to TAU (RCT, N = 129, moderate risk of bias) [ 37 ].

Selective prevention interventions

Six studies, including five RCTs and one QED, tested PS as a selective prevention intervention. Two studies investigated the impact of the Manage Your Life Online program, which includes PS as a primary component of the intervention, compared with an online programme emulating Rogerian psychotherapy for UK university students (RCT, N = 213, moderate risk of bias [ 38 ]; RCT, N = 48, moderate risk of bias [ 39 ]). Both studies found no differences in depression or problem-related distress between groups.

Similarly, two studies explored the effect of adapting the Penn Resilience Program, which includes PS as a component of a more comprehensive intervention for young people with diabetes in the US (RCT, N = 264, moderate risk of bias) [ 40 , 41 ]. The initial study showed a moderate reduction in diabetes distress but not depression at 4-, 8-, 12- and 16-months follow-up compared to a diabetes education intervention [ 40 ]. The follow-up study found a significant reduction in depressive symptoms compared to the active control from 16- to 40-months; however, this did not reach significance at 40-months [ 41 ].

Another study that was part of wider PS and social skills intervention among juveniles in state-run detention centres in the US found no impacts (RCT, N = 296, high risk of bias) [ 42 ]. A QED ( N = 32) was used to test the effectiveness of a resilience enhancement and prevention intervention for runaway youth in South Korea [ 43 ]. There was a significant decrease in depression for the intervention group compared with the control group at post-test, but the difference was not sustained at one-month follow-up.

Indicated prevention interventions

Six studies, including five RCTs and one QED, tested PS as an indicated prevention intervention. Four of the five RCTs tested PS as a primary component of the intervention. A PS intervention for common adolescent mental health problems in Indian high school students (RCT, N = 251, low risk of bias) led to a significant reduction in psychosocial problems at 6- and 12 weeks; however, it did not have a significant impact on mental health symptoms or internalising symptoms compared to PS booklets without counsellor treatment at 6- and 12-weeks [ 31 ]. A follow-up study showed a significant reduction in overall psychosocial problems and mental health symptoms, including internalizing symptoms, over 12 months [ 44 ]. Still, these effects no longer reached significance in sensitivity analysis adjusting for missing data (RCT, N = 251, low risk of bias). Furthermore, a 2x2 factorial RCT ( N = 176, moderate risk of bias) testing PST among youth mental health service users with a mild mental disorder in Australia found that the intervention was not superior to supportive counselling at 2-weeks post-treatment [ 30 ]. Similarly, an online PS intervention delivered to young people in the Netherlands to prevent depression (RCT, N = 45, moderate risk of bias) found no significant difference between the intervention and WLC in depression level 4-months post-treatment [ 45 ].

One RCT tested PS approaches in a more comprehensive manualized programme for postnatal depression in the UK and found no significant differences in depression scores between intervention and TAU at 3-months post-partum (RCT, N = 292, moderate risk of bias) [ 46 ].

A study in Turkey used a non-equivalent control group design (QED, N = 62) to test a nursing intervention against a PS control intervention [ 47 ]. Both groups showed a reduction in depression, but the nursing care intervention demonstrated a larger decrease post-intervention than the PS control intervention.

Secondary prevention interventions

Twelve studies, all RCTs, tested PS as a secondary prevention intervention. Four of the 12 RCTs tested PS as a primary component of the intervention. An intervention among women in Zimbabwe (RCT, N = 58, moderate risk of bias) found a larger decrease in the Edinburgh Postnatal Depression Scale score for the intervention group compared to control (who received the antidepressant amitriptyline and peer education) at 6-weeks post-treatment [ 48 ]. A problem-orientation intervention covering four PST steps and involving a single session video for US university students (RCT, N = 110, moderate risk of bias), compared with a video covering other health issues, resulted in a moderate reduction in depression post-treatment; however, results were no longer significant at 2-weeks, and 1-month follow up [ 49 ].

Compared to WLC, a study of an intervention for depression and suicidal proneness among high school and university students in Turkey (RCT, N = 46, moderate risk of bias) found large effect sizes on post-treatment depression scores for intervention participants post-treatment compared with WLC. At 12-month follow-up, these improvements were maintained compared to pre-test but not compared to post-treatment scores. Significant post-treatment depression recovery was also found in the PST group [ 12 ]. Compared to TAU, a small but high-quality (low-risk of bias) study focused on preventing suicidal risk among school students in Brazil (RCT, N = 100, low risk of bias) found a significant, moderate reduction in depression symptoms for the treatment group post-intervention that was maintained at 1-, 3- and 6-month follow-up [ 50 ].

Seven of the 12 RCTs tested PS as a part of a more comprehensive intervention. Two interventions targeted mood episodes and were compared to active control. These US studies focused on Family-Focused Therapy as an intervention for mood episodes, which included sessions on PS [ 51 , 52 ]. One of these found that Family-Focused Therapy for AYA with Bipolar Disorder (RCT, N = 145, moderate risk of bias) had no significant impact on mood or depressive symptoms compared to pharmacotherapy. However, Family-Focused Therapy had a greater impact on the proportion of weeks without mania/hypomania and mania/hypomania symptoms than enhanced care [ 53 ]. Alternatively, while the other study (RCT, N = 127, low risk of bias) found no significant impact on time to recovery, Family-Focused Therapy led to significantly longer intervals of wellness before new mood episodes, longer intervals between recovery and the next mood episode, and longer intervals of randomisation to the next mood episode in AYA with either Bipolar Disorder (BD) or Major Depressive Disorder (MDD), compared to family and individual psychoeducation [ 52 ].

Two US studies used a three-arm trial to compare Systemic-behavioural Family Therapy (SBFT) with elements of PS, to CBT and individual Non-directive Supportive therapy (NST) (RCT, N = 107, moderate risk of bias) [ 53 , 54 ]. One study looked at whether the PS elements of CBT and SFBT mediated the effectiveness of these interventions for the remission of MDD. It found that PS mediated the association between CBT, but not SFBT, and remission from depression. There was no significant association between SBFT and remission status, though there was a significant association between CBT and remission status [ 53 ]. The other study found no significant reduction in depression post-treatment or at 24-month follow-up for SBFT [ 54 ].

A PS intervention tested in maternal and child clinics in Nigeria RCT ( N = 686, moderate risk of bias) compared with enhanced TAU involving psychosocial and social support found no significant difference in the proportion of women who recovered from depression at 6-months post-partum [ 55 ]. However, there was a small difference in depression scores in favour of PS averaged across the 3-, 6-, 9-, and 12-month follow-up points. Cognitive Reminiscence Therapy, which involved recollection of past PS experiences and drew on PS techniques used for 12-25-year-olds in community mental health services in Australia (RCT, N = 26, moderate risk of bias), did not reduce depression symptoms compared with a brief evidence-based treatment at 1- or 2-month follow-up [ 56 ]. Additionally, the High School Transition Program in the US (RCT, N = 497, moderate risk of bias) aimed to prevent depression, anxiety, and school problems in youth transitioning to high school [ 57 ]. There was no reduction in the percentage of intervention students with clinical depression compared to the control group. Similarly, a small study focused on reducing depression symptoms, and nonadherence to antiretroviral therapy in pregnant women with HIV in South Africa (RCT, N = 23, some concern) found a significant reduction in depression symptoms compared to TAU, with the results being maintained at the 3-month follow-up [ 58 ].

Reduction in suicidality

Three studies measured a reduction in suicidality. A preventive treatment found a large reduction in suicidal orientation in the PS group compared to control post-treatment. In contrast, suicidal ideation scores were inconsistent at 1-,3- and 6- month follow-up, they maintained an overall lower score [ 50 ]. Furthermore, at post-test, significantly more participants in the PS group were no longer at risk of suicide. No significant differences were found in suicide plans or attempts. In a PST intervention, post-treatment suicide risk scores were lower than pre-treatment for the PST group but unchanged for the control group [ 12 ]. An online treatment found a moderate decline in ideation for the intervention group post-treatment compared to the control but was not sustained at a one-month follow-up [ 49 ].

Mediators and moderators

Eight studies measured PS skills or effectiveness. In two studies, despite the interventions reducing depression, there was no improvement in PS abilities [ 12 , 52 ]. One found that change in global and functional PS skills mediated the relationship between the intervention group and change in suicidal orientation, but this was not assessed for depression [ 50 ]. Three other studies found no change in depression symptoms, PS skills, or problem resolution [ 38 – 40 ]. Finally, CBT and SBFT led to significant increases in PS behaviour, and PS was associated with higher rates of remission across treatments but did not moderate the relationship between SBFT and remission status [ 53 ]. Another study found no changes in confidence in the ability to solve problems or belief in personal control when solving problems. Furthermore, the intervention group was more likely to adopt an avoidant PS style [ 46 ].

A high-intensity intervention for perinatal depression in Nigeria had no treatment effect on depression remission rates for the whole sample. Still, it was significantly effective for participants with more severe depression at baseline [ 55 ]. A PS intervention among juvenile detainees in the US effectively reduced depression for participants with higher levels of fluid intelligence, but symptoms increased for those with lower levels [ 42 ].The authors suggest that individuals with lower levels of fluid intelligence may have been less able to cope with exploring negative emotions and apply the skills learned.

This review has examined the evidence on the effectiveness of PS in the prevention or treatment of depression among 13–25-year-olds. We sought to determine in what way, in which contexts, and for whom PS appears to work in addressing depression. We found 25 studies involving 20 interventions. Results are promising for secondary prevention interventions, or interventions targeting clinical level populations, that utilize PS as the primary intervention [ 12 , 47 – 49 ]. These studies not only found a significant reduction in depression symptoms compared to active [ 48 , 49 ] and non-active [ 12 , 47 ] controls but also found a significant reduction in suicidal orientation and ideation [ 12 , 47 , 49 ]. These findings are consistent with meta-analyses of adult PS interventions [ 21 , 22 , 23 ], highlighting that PS interventions for AYA can be effective in real-world settings.

For other types of interventions (i.e., universal, selective prevention, indicated prevention), results were mixed in reducing depression. The one universal program was found to have a small, significant effect in reducing depression symptoms compared to a non-active control [ 37 ]. Most selective prevention programs were not effective [ 39 , 40 , 56 ], and those that did show small, significant effects had mixed outcomes for follow-up maintenance [ 41 , 42 ]. Most indicated prevention programs were not effective [ 30 , 31 , 45 – 47 ], yet a follow-up study showed a significant reduction in internalizing symptoms at 12-month post-treatment compared to an active control [ 44 ]. Given that these studies targeted sub-clinical populations and many of them had small sample sizes, these mixed findings may be a result of not having sufficient power to detect a meaningful difference.

Our review found limited evidence about PS skills as mediator or moderator of depression. Few studies measured improvements in PS skills; fewer still found interventions to be effective. The absence of evidence for PS abilities as a pathway is puzzling. It may be that specific aspects of PS behaviours and processes, such as problem orientation [ 59 ], are relevant. Alternatively, there may be a mechanism other than PS skills through which PS interventions influence depression.

Studies with PS as part of a wider intervention also showed mixed results, even amongst clinical populations. Although there was no clear rationale for the discrepancies in effectiveness between the studies, it is possible that the wider program dilutes the focus and impact of efficacious therapeutic elements. However, this is difficult to discern given the heterogeneity in the studies and limited information on study treatments and implementation factors. A broad conclusion might be that PS can be delivered most effectively with clinical populations in its purest PS form and may be tailored to a range of different contexts and forms, a range of populations, and to address different types of problems; however, this tailoring may reduce effectiveness.

Although the scale of impact is broadly in line with the small to moderate effectiveness of other treatments for youth depression [ 6 ], our review highlights shortcomings in study design, methods, and reporting that would allow for a better understanding of PS effectiveness and pathways. Studies varied in how well PS was operationalised. Low dosage is consistent with usage described in informal conversations with practitioners but may be insufficient for effectiveness. Fidelity was monitored in only half the studies despite evidence that monitoring implementation improves effectiveness [ 60 ]. There were references to implementation difficulties, including attrition, challenges in operationalizing online interventions, and skills of those delivering. Furthermore, most of the studies had little information about comorbidity and no analysis of whether it influenced outcomes. Therefore, we were unable to fully examine and conceptualize the ways, how and for whom PS works. More information about study populations and intervention implementation is essential to understand the potential of PS for broader dissemination.

Our review had several limitations. We excluded studies that included four treatments known to be effective in treating depression among AYA (e.g., CBT) but where the unique contribution of PS to clinical outcome could not be disentangled. Furthermore, we relied on authors’ reporting to determine if PS was included: details about operationalization of PS were often scant. Little evidence addressing the fit, feasibility, or acceptability of PS interventions was found, reflecting a limited focus on implementation. We included only English-language texts: relevant studies in other languages may exist, though our post-2000 inclusion criteria may limit this potential bias due to improved translation of studies to English over the years. Finally, the heterogeneity of study populations, problem severity, comparison conditions, outcome measures, and study designs, along with a relatively small number of included studies, limits confidence in what we can say about implementation and treatment outcomes.

Overall, our review indicates that PS may have the best results when implemented its purest form as a stand-alone treatment with clinical level AYA populations; tailoring or imbedding PS into wider programming may dilute its effectiveness. Our review also points to a need for continued innovation in treatment to improve the operationalizing and testing of PS, especially when included as a part of a more comprehensive intervention. It also highlights the need for study methods that allow us to understand the specific effects of PS, and that measure the frequency, dosage, and timing of PS to understand what is effective for whom and in what contexts.

Supporting information

S1 file. list of excluded studies..

https://doi.org/10.1371/journal.pone.0285949.s001

S2 File. PRISMA checklist.

https://doi.org/10.1371/journal.pone.0285949.s002

S1 Table. Individual risk of bias assessments using cochrane RoB2 tool by domain (1–5) and overall (6).

https://doi.org/10.1371/journal.pone.0285949.s003

Acknowledgments

All individuals that contributed to this paper are included as authors.

• 1. Dietz LJ, Silk J, Amole M. Depressive disorders. In: Ollendick TH, White SW, White BA, editors. The Oxford Handbook of Clinical Child and Adolescent Psychology. Oxford University Press, New York, NY; 2019. p. 280–297.
• View Article
• PubMed/NCBI
• Google Scholar
• 14. Nezu AM, Nezu CM, D’Zurilla T. Problem-solving therapy: A treatment manual. Springer Publishing Company. 2012.
• 15. D’Zurilla TJ, Nezu AM. Problem-solving therapy. In: Handbook of Cognitive-Behavioral Therapies. Third. New York, NY: Guilford Press; 2010. p. 197–225.
• 17. D’Zurilla TJ, Nezu AM, Maydeu-Olivares A. Social Problem Solving: Theory and Assessment. In: Chang EC D’Zurilla TJ, Sanna LJ, editors. Social problem solving: Theory, research, and training: American Psychological Association; 2004. p. 11–27.
• 20. D’Zurilla TJ, Nezu AM. (2007). Problem-solving therapy: A positive approach to clinical intervention (3rd edition) New York: Spring Publishing Company.
• 34. D’Zurilla TJ, Nezu AM. Problem-solving therapy: A social competence approach to clinical intervention. 2nd ed. New York: Springer. 1999.
• 35. Mynors-Wallis LM. Problem-solving treatment for anxiety and depression: A practical guide. Oxford University Press: Oxford. 2005.
• 62. Brugha T. (1995). Social support and psychiatric disorder: Overview of evidence. In Brugha T. (Ed.), Social Support and Psychiatric Disorder : Research Findings and Guidelines for Clinical Practice (Studies in Social and Community Psychiatry, pp. 1–38). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511526749.002
• 66. Guerra N. G. (1995). Viewpoints : A Guide to Conflict Resolution and Decision Making for Adolescents . Research Press, 2612 North Mattis Avenue, Champaign, IL 61821.
• 67. Guerra N. G., & Williams K. R. (2012). Implementing evidence-based practices for juvenile justice prevention and treatment in communities. In Grigorenko E. L. (Ed.), Handbook of juvenile forensic psychology and psychiatry (pp. 297–308). New York: Springer
• 68. Nezu A. M., Maguth Nezu C., & D’Zurilla T. J. (2013). Problem-solving therapy : A treatment manual . Springer Publishing Co.
• 69. Robin A. L., & Foster S. L. (1989). Negotiating parent-adolescent conflict: A behavioral-family systems approach. New York, NY: Guilford Press.
• 70. VAZQUEZ F. L., OTERO P., BLANCO V., & TORRES A. (2015). Terapia de solucion de problemas para la depresion. Una breve guıa practica en grupo. [Problem-solving therapy for depression: A brife guide for group practice]. Madrid: Alianza Editorial.

Behavioral Problems in Adolescents

• Specific Behavioral Disorders |
• Violence and Gang Membership |

Adolescence is a time for developing independence. Typically, adolescents exercise their independence by questioning or challenging, and sometimes breaking, rules. Parents and doctors must distinguish occasional errors of judgment, which are typical and expected of this age group, from a pattern of misbehavior that requires professional intervention. The severity and frequency of infractions are guides. For example, regular drinking, frequent episodes of fighting, absenteeism from school without permission (truancy), and theft are much more significant than isolated episodes of the same activities. Other warning signs of a possible behavioral disorder include deterioration of performance at school and running away from home. Of particular concern are adolescents who cause serious injury to themselves or others or who use a weapon in a fight.

Because adolescents are much more independent and mobile than they were as children, they are often out of the direct physical control of adults. In these circumstances, adolescents' behavior is determined by their own decision-making, which is not yet mature. Parents guide rather than directly control their adolescents' actions. Adolescents who feel warmth and support from their parents and whose parents convey clear expectations regarding their children’s behavior and show consistent limit setting and monitoring are less likely to develop serious problems.

Authoritative parenting is a parenting style in which children participate in establishing family expectations and rules. This style of parenting involves limit setting, which is important for healthy adolescent development. Authoritative parenting, as opposed to authoritarian-style parenting (in which parents make decisions with minimal input from their children) or permissive parenting (in which parents set few limits) is most likely to promote mature behaviors.

Authoritative parenting uses a system of graduated privileges, in which adolescents initially are given small bits of responsibility, such as caring for a pet, doing household chores, purchasing clothing, decorating their room, or managing an allowance. If adolescents handle a responsibility or privilege well over a period of time, more responsibilities and more privileges, such as going out with friends without parents and driving, are granted. By contrast, poor judgment or lack of responsibility leads to loss of privileges. Each new privilege requires close monitoring by parents to make sure adolescents comply with the agreed-upon rules.

Some parents and their adolescents clash over almost everything. In these situations, the core issue is really control. Adolescents want to feel they can make or contribute to decisions about their lives, and parents are afraid to allow their children to make bad decisions. In these situations, everyone may benefit from the parents picking their battles and focusing their efforts on the adolescent's actions (such as attending school and complying with household responsibilities) rather than on expressions (such as dress, hairstyle, and preferred entertainment).

Adolescents whose behavior is dangerous or otherwise unacceptable despite their parents' best efforts may need professional intervention. Substance use is a common trigger of behavioral problems, and substance use disorders require specific treatment. Behavioral problems also may be symptoms of learning disabilities , depression , or other mental health disorders . Such disorders typically require counseling, and adolescents who have mental health disorders may benefit from treatment with drugs. If parents are not able to limit an adolescent’s dangerous behavior, they may request help from the court system and be assigned to a probation officer who can help enforce reasonable household rules.

(See also Introduction to Problems in Adolescents .)

Specific Behavioral Disorders

Disruptive behavioral disorders are common during adolescence.

Attention-deficit/hyperactivity disorder (ADHD) is the most common mental health disorder of childhood and often persists into adolescence and adulthood. However, adolescents who have difficulty paying attention may instead have another disorder, such as depression or a learning disability

Other common disruptive behaviors of childhood include oppositional defiant disorder and conduct disorder . These disorders are typically treated with psychotherapy for the child and advice and support for parents.

Violence and Gang Membership

Children occasionally engage in physical confrontation and bullying , including cyberbullying. During adolescence, the frequency and severity of violent interactions may increase. Although episodes of violence at school are highly publicized, adolescents are much more likely to be involved in violent episodes (or more often the threat of violence) at home and outside of school. Many factors contribute to an increased risk of violence for adolescents, including

Developmental problems

Intense corporal punishment (such as punching or beating) inflicted on the child

Caregivers with substance use disorders

Gang membership

Access to firearms

Substance use

There is little evidence to suggest a relationship between violence and genetic defects or chromosomal abnormalities.

Gang membership has been linked with violent behavior. Youth gangs are self-formed associations made up of 3 or more members, typically ranging in age from 13 to 24. Gangs usually adopt a name and identifying symbols, such as a particular style of clothing, the use of certain hand signs, certain tattoos, or graffiti. Some gangs require prospective members to perform random acts of violence before membership is granted.

Increasing youth gang violence has been blamed at least in part on gang involvement in drug distribution and drug use. Firearms and other weapons are frequent features of gang violence.

Violence prevention begins in early childhood with violence-free discipline. Limiting exposure to violence through media and video games may also help because exposure to these violent images has been shown to desensitize children to violence and cause children to accept violence as part of their life. School-age children should have access to a safe school environment. Older children and adolescents should not have access to weapons and should be taught to avoid high-risk situations (such as places or settings where others have weapons or are using alcohol or drugs) and to use strategies to defuse tense situations.

All victims of violence should be encouraged to talk to parents, teachers, and even their doctor about problems they are having.

• Cookie Preferences

Copyright © 2024 Merck & Co., Inc., Rahway, NJ, USA and its affiliates. All rights reserved.

Process Evaluation of a Problem-Solving Approach for Analyzing Literacy Practices within a Multi-Tiered System of Supports Framework

• Original Paper
• Open access
• Published: 06 May 2024

Cite this article

You have full access to this open access article

• Amy Murdoch   ORCID: orcid.org/0000-0003-4138-9166 1 ,
• Julie Q. Morrison 2 &
• Wendy Strickler 1  

46 Accesses

Explore all metrics

A Multi-Tiered System of Supports (MTSS) framework features a structured problem-solving process and the use of assessment data to develop, identify, and evaluate the impact of instruction and intervention to meet the needs of all students proactively. The purpose of this process evaluation was to examine the implementation of a novel problem-solving approach for analyzing literacy practices across the tiers of an MTSS framework (i.e., core instruction, strategic intervention). The aim of the initiative was to build the capacity of teachers to provide effective instruction based on the science of reading in two elementary schools. The findings from this process evaluation study provide evidence that a problem-solving approach for analyzing literacy practices resulted in improvements in the core curriculum, instruction, and intervention supports. Implications for improvement efforts at the school district and state department of education levels are discussed.

Similar content being viewed by others

Constructing Successful School Models for Inclusive Literacy: Supporting Student Success Through Strategic Instruction in Diverse Settings

Problem Analysis at Tier 2: Using Data to Find the Category of the Problem

Data-based decision-making, the problem-solving model, and response to intervention in the minneapolis public schools.

Avoid common mistakes on your manuscript.

Research on how children learn to read and why some young learners have difficulty developing as proficient readers has informed the development of evidence-based instructional approaches and interventions to promote reading proficiency. This vast, interdisciplinary body of knowledge is known as the science of reading (The Reading League, 2022 ). The science of reading emphasizes the importance of explicit, systematic, and sequential instruction targeting the five essential components of reading: phonemic awareness, phonics, fluency, vocabulary, and comprehension (National Reading Panel, 2000 ; Spear-Swerling, 2018 ).

Effective instruction based on the science of reading is beneficial for the approximately 40% of students for whom learning to read with general instruction develops fairly easily, but essential for approximately 60% of students for whom learning to read is more difficult (Foorman et al., 2016 ; National Reading Panel, 2000 ). The majority of these emergent readers do not have a learning disability, but simply require explicit, systematic, and sequential instruction to become proficient readers. The term “instructional casualties” has been used to refer to struggling readers who did not receive adequate, scientifically based reading instruction and consequently risked being misidentified as a student with a learning disability (Lyon et al., 2001 ). The impact on the lives of these young students is devastating. Students who do not develop reading proficiency are more likely to be retained a grade in school, drop out of high school, and enter the juvenile justice system (Fien et al., 2021 ; Reynolds et al., 2002 ).

Decades of dismal reading outcomes indicate that the national educational landscape is strewn with instructional casualties. Only 33% of fourth-grade students performed at or above the proficient level in reading in 2022 (National Assessment of Educational Progress [NAEP], 2022 ). This reading crisis is particularly dire among students who are educationally marginalized - in particular, students from diverse racial and ethnic groups, economically disadvantaged students, English Language Learners, and students with disabilities. Only 19% of fourth-grade students eligible for the National School Lunch Program scored at or above the proficient level in reading in 2022 (NAEP, 2022 ). Likewise, only 17% of Black fourth-grade students and 21% of Hispanic students performed at or above the proficient level in reading (NAEP, 2022 ).

Evidence-Based Interventions within a Multi-Tiered System of Supports Framework

An MTSS framework is a systems-level approach to ensuring supports for all students proactively based on academic or behavioral skill need. The core features of an MTSS framework are: (1) universal screening; (2) data-based decision making and problem solving; (3) continuous progress monitoring; (4) a continuum of evidence-based practices across tiers (core instruction with universal support, targeted intervention, and intensive intervention); and (5) a focus on fidelity of implementation (McIntosh & Goodman, 2016 ).

An MTSS framework aims to meet the needs of all students by considering the influence of ecological variables such as instruction, curriculum, and classroom environment on student outcomes. Through the prevention-focused early intervention approach, with an emphasis on effective core instruction and instructional supports through data-based decision making, MTSS facilitates meeting the needs of marginalized students (Albritton et al., 2016 ). Research indicates that an MTSS framework, when implemented with fidelity, holds promise for addressing several key outcomes including (1) increasing student achievement (VanDerHeyden et al., 2007 ); (2) detecting and intervening early with students at risk for academic problems (Al Otaiba & Torgeson, 2007 ); (3) improving the means by which students are determined to be eligible for special education services (Barrett, 2023 ); (4) reducing disproportional representation of minoritized students receiving special education (VanDerHeyden & Witt, 2005 ); and (5) meeting the needs of struggling readers cost-effectively (Morrison et al., 2020 ).

MTSS Reading Implementation Fidelity Challenges

The core components of MTSS need to be implemented fully in an integrated and consistent manner for the positive outcomes to be attained (Keller-Margulis, 2012 ; Noell & Gansle, 2006 , Sanetti & Luh, 2019 ). Yet implementation fidelity can be difficult to achieve because it requires educators to fundamentally change how they support learners not only by advancing their knowledge and skills in scientifically-based reading approaches (Kilpatrick, 2015 ), but also developing their competencies in the timely use of data to inform instructional decision making (Daly et al., 2007 ; Kratochwill et al., 2007 ), and adopting an ecological view of learning by seeking to, “eliminate contextual variables as a viable explanation for academic failure” (Vaughn & Fuchs, 2003 , p. 142).

Many challenges to MTSS implementation have been identified in the research literature (see Table 1 ). The failure to implement evidence-based reading interventions with fidelity is the most frequently recognized limitation (Sanetti & Luh, 2019 ). Reading intervention researchers contend that the evidence-based interventions that produce significant word-reading outcomes (Torgesen et al., 2001 ; Vellutino et al., 1996 ) risk being diluted or dropped within an MTSS framework (Vellutino et al., 2008 ). Furthermore, when MTSS implementation efforts fail to address weaknesses in the core instruction, a high proportion of students may meet the criteria for more intensive intervention resulting in more school resources dedicated to intensive intervention, when the resources may have been more effectively used to improve core instruction (Fuchs & Fuchs, 2017 ).

Increasing MTSS Implementation Fidelity with Training and Coaching

Building teachers’ capacity to provide effective instruction and intervention based on the science of reading within an MTSS framework involves a fundamental shift in how teachers provide reading instruction (Spear-Swerling, 2018 ) and gather, monitor, and respond to reading assessment data (Kratochwill et al., 2007 ). Training alone is insufficient. Research on teacher professional learning has shown that newly learned practices are crude compared to the performance by a master practitioner, fragile in the face of reactions from others, and incomplete when translated to the school setting (Joyce & Showers, 2002 ). As a result, training coupled with ongoing and embedded coaching is essential for promoting competent usage of evidence-based instructional practices (Joyce & Showers, 2002 ). Training and coaching function as critical drivers of implementation (Blase et al., 2013 ; Fixsen et al., 2013 ), but more research is needed on models for training and coaching to promote sustained use of evidence-based reading instruction and intervention within an MTSS framework.

Purpose of the Study

Evidence-based instruction and interventions emerging from the science of reading are key to addressing the needs of struggling readers and closing the opportunity gap for educationally marginalized learners (Fletcher et al., 2004 ). Yet, weak reading intervention fidelity and inconsistent fidelity of implementation of the core components of MTSS have resulted in the proliferation of initiatives that feature only the surface manifestations of an MTSS framework (e.g., sorting students into tiers based on universal screening data) grafted on top of traditional practices and routines not aligned with the science of reading (Hall, 2018 ; Kilpatrick, 2015 ; Sabnis et al., 2020 ). The purpose of this study was to evaluate the effectiveness of an MTSS initiative designed to build the capacity of teachers to provide effective instruction and intervention based on the science of reading. In particular, process evaluation examined the implementation of a novel problem-solving approach for analyzing literacy practices across an MTSS framework (i.e., core instruction and strategic intervention). The Promoting Achievement in Reading Through Needs-driven Evidence-based Reading Structures (PARTNERS) Project aimed to provide comprehensive professional learning (i.e., training and coaching) in a problem-solving approach to examining the tiers of literacy instruction to improve early literacy outcomes for students in kindergarten through second grade. This process evaluation examined teacher team outcomes during the project’s third year of implementation, which focused on strengthening the core reading curriculum and instruction (Tier 1) and strategic intervention (Tier 2). The study addressed the following process evaluation question: To what extent did the PARTNERS Project increase teacher teams’ capacity to analyze and improve the core curriculum and instruction and intervention supports?

The methods used to evaluate the effectiveness of the PARTNERS Project were reviewed by the Institutional Review Board. Data from the LAP-G presented in Figures 1 , 2 and 3 are not publicly available in order to protect teacher participants’ privacy. The data are available from the corresponding author upon reasonable request. The PARTNERS Project and its evaluation were funded as a Model Demonstration Project by the U.S. Department of Education, Office of Special Education Programs.

Results of the Annual Administration of the LAP-G Tier 1 and Tier 2 at Loweland School: K – 2 Teacher Team

Results of the Annual Administration of the LAP-G Tier 1 and Tier 2 at St. Mark School: K – 1 Teacher Team

Results of the Annual Administration of the LAP-G Tier 1 and Tier 2 at St. Mark School: Grade 2 – 3 Teacher Team

Participants and Setting

Loweland School (a pseudonym) is an elementary school in a public school district classified as Urban-High Student Poverty and Average Student Population according to the state department of education. Loweland School had an enrollment of 278 students in kindergarten through Grade 6, of which 100% were classified economically disadvantaged. In the year prior to the PARTNERS Project, only 21.2% of the third-grade students scored at or above the proficient level on the state-mandated achievement test.

St. Mark School (a pseudonym) is a non-public, Catholic School serving an urban, predominately Hispanic community. St. Mark School had an enrollment of 215 students in preschool through Grade 8, of which 94% were economically disadvantaged. State-mandated achievement test data were not available for non-public schools. However, Acadience screening data collected at the onset of the study indicated that only 25% of the third-grade students were at benchmark for reading at the middle of the year checkpoint. Demographic information for each school’s student population is provided in Table 2 .

Loweland School formed a K – 2 teacher team that consisted of four teachers, an intervention teacher, and the principal. St. Mark School formed two teams, a K – 1 teacher team with eight members, and a grades 2 – 3 teacher team with six members. At St. Mark School, the principal, English Language Learner (ELL) teacher, reading intervention teacher, and the Title I teacher all served on both the K – 1 team and the grades 2 – 3 team. Table 3 provides information regarding the gender, race/ethnicity, years of teaching experience at the start of the PARTNERS Project, and any relevant training each participant had received prior to the start of the PARTNERS Project. Given the opportunity to participate in the PARTNERS Project, all of the participants expressed a willingness to be involved. The teachers were encouraged, but not required, to participate in the project Table 3 .

The PARTNERS Project

The PARTNERS Project provided comprehensive professional learning (i.e., training and coaching) to teams of teachers in a problem-solving, data-driven process whereby they evaluated their own instructional program and identified areas of needed improvement to align with the science of reading. The problem-solving approach was operationalized in the Literacy Analysis and Planning Guide (LAP-G). A description of the LAP-G and the professional learning supports are provided in this section.

Literacy Analysis and Planning Guide Tool

The Literacy Analysis and Planning Guide (LAP-G) provided for an analysis of evidence-based literacy practices across each tier of an MTSS framework (i.e., core instruction, strategic intervention, and intensive intervention). The core components of evidence-based literacy instruction analyzed by the LAP-G process at Tier 1 were (1) Screening; (2) Instructional Materials by Essential Component—Core and Supplemental; (3) Implementation of Tier 1 Instruction; and (4) Differentiated Instruction. The reliability, validity, and comprehensiveness of the school’s screening system was operationalized by 11 items on the LAP-G (see Table 4 ). The second core component pertained to the instructional materials (the core curriculum and supplemental materials) used to address each of the essential components of reading: Phonological awareness, phonics, reading fluency, vocabulary, comprehension, and writing (see Table 5 ). Implementation of Tier 1 instruction was the third core component. Six items on the LAP-G operationalize how implementation is assessed through multiple measures (e.g., permanent product review, direct observation) and multiple data sources (see Table 6 ). The fourth core component was differentiated instruction. Seven items on the LAP-G operationalize differentiated instruction (see Table 7 ).

The core components of evidence-based literacy intervention analyzed by the LAP-G process at Tier 2 were (1) Assessments—Intervention-based Diagnostics and Progress monitoring; (2) Instructional Materials for Each Intervention Program; and (3) General Considerations: Effective Intervention Design, Professional Development, Implementation Checks. The core components of evidence-based literacy intervention at Tier 3 were (1) Assessment; (2) Designing Tier 3 Supports—Collaborative Problem Solving, Intervention Components; (3) Effective Implementation of Tier 3 Interventions—Effective Implementation, Appropriate Placement in Tier 3, Professional Learning for Tier 3. A copy of the LAP-G with the core components of evidence-based literacy interventions at Tier 2 and 3 is available upon request to the corresponding author.

The LAP-G engaged a team of educators in a problem-solving process whereby the effectiveness of each tier of instruction/intervention was evaluated. The tool utilizes a five-step collaborative problem-solving process: (1) Define and Analyze Needs: Collect Initial Information; (2) Define and Analyze Needs: Summarize, Analyze, and Prioritize; (3) Plan Support; (4) Implement Plan; and (5) Evaluate. Teams began by documenting and reviewing student data to identify areas of concern with the support of their PARTNERS Project consultant who served as a facilitator. Priority areas of opportunity were then identified and action plans were developed to improve upon current practices (plan development and implementation). The LAP-G process was completed annually in April/May to evaluate progress and determine next steps.

Professional Learning: Training and Coaching

An overview of the scope of the professional learning provided through the PARTNERS Project is presented in Table 8 . At Loweland Elementary, the need for changes in the core curriculum and instructional practices was identified in the first year of PARTNERS Project implementation with its focus on Tier 1 instruction. Teachers participated in training in Language Essentials for Teachers of Reading and Spelling (LETRS) to build foundational understanding of the science of reading. LETRS has been shown by research to be an effective professional development program for reading teachers (Garet et al., 2008 ). Kindergarten and first-grade teachers at Loweland identified a need to focus on supplemental phonics instruction in Tier 1. A review of curriculum aligned with the science of reading and matched to the instructional need resulted in the selection of Superkids. The Superkids Reading Program meets the criteria for which programs can be called evidence-based established by the federal Every Student Succeeds Act ( https://media.zaner-bloser.com/reading/superkids-reading-program/pdfs/R1727_SK_EvidenceforESSA.pdf ). Teachers at Loweland were trained in effective literacy instruction and high-fidelity curriculum implementation in the use of Superkids at the kindergarten, first and second grade levels. Training also included a two-day Summer Institute focused on Superkids and data-based decision making prior to implementation. Due to teacher requests during a monthly teacher team PARTNERS meeting, the Superkids consultant was brought back for a day of modeling lessons and program-specific coaching the following spring.

At St. Mark School, the need to strengthen the core instructional program and use of differentiation in instruction were identified as priorities at the kindergarten and first-grade levels. Teachers participated in training in Language Essentials for Teaching Reading and Spelling (LETRS) to build foundational understanding of the science of reading. Teachers at St. Mark School identified a need to focus on supplemental phonics instruction in Tier 1. Superkids was adopted as the curriculum prior to the start of the PARTNERS Project, but there was a recognized need for additional professional learning. St. Mark School teachers participated alongside Lowland Elementary teachers for the 2-day Summer Institute focused on Superkids and data-based decision making. Kindergarten and first-grade teachers at St. Mark School also participated in ongoing training in the valid and reliable use of Acadience measures beginning in their first year of engagement with the PARTNERS Project. In Year 2, the LAP-G process showed gaps in the phonics instruction with Superkids. The teacher teams reviewed alternative curriculums and selected the 95% Group Phonics Lesson Library. In addition to training in this new curriculum, the teams at St. Mark School focused on writing skills in Years 2-3 and reading comprehension in Year 3. Given that the student population was predominately Hispanic, the PARTNERS Project professional learning included a year-long book study on supporting the early literacy skills of English Language Learners.

At both schools, PARTNERS Project consultants observed teacher instruction and assessed implementation fidelity using a fidelity checklist co-designed with the teachers. Observations were conducted weekly in Years 2 and 3, such that each teacher was observed at least once a month. The collaborative coaching model included individualized performance feedback shared in person or via email following each observation.

Measure and Analysis

The LAP-G tool was developed for use in the PARTNERS Project based on a prototype developed by the first author and colleagues. The development of the LAP-G was informed by an extensive review of the science of reading research and the literature on MTSS. Preliminary content validation was established through expert review. A panel of nine individuals with expertise in the science of reading were asked to review the LAP-G overall and by tier on three dimensions: quality, relevance, and usefulness (QRU). Definitions for QRU were based on the U.S. Department of Education, Office for Special Education Program’s Government Performance and Results Act (GPRA) measures (Moore & Lammert, 2019 ). Quality was defined as the degree to which the tool is grounded with current research or policy. On a 10-point scale where 10 represented the highest level of quality, the mean ratings were high for the LAP-G overall ( M = 8.8, SD = 1.28), Tier 1 ( M = 9.0, SD = 1.41), Tier 2 ( M = 9.2, SD = 1.30), and Tier 3 ( M = 9.1, SD = 1.1). Relevance was defined as the degree to which the tool addresses current educational problems or issues. On a 10-point scale where 10 represented the highest level of relevance, the mean ratings were high for the LAP-G overall ( M = 9.1, SD = 1.36), Tier 1 ( M = 9.3, SD = 1.66), Tier 2 ( M = 9.2, SD = 1.64), and Tier 3 ( M = 9.7, SD = 0.7). Usefulness was defined as the degree to which the tool could be readily and successfully used by practitioners (i.e., ease of use, suitability). On a 10-point scale where 10 represented the highest level of usefulness, the mean ratings were high for the LAP-G overall ( M = 9.1, SD = 1.36), Tier 1 ( M = 8.9, SD = 1.62), Tier 2 ( M = 8.9, SD = 1.62), and Tier 3 ( M = 9.1, SD = 1.1). Teacher team members were also asked to complete a survey to assess their perceptions of the Quality, Relevance, and Usefulness of the LAP-G and solicit qualitative data regarding their acceptability of the tool.

In using the LAP-G, teacher teams complete a problem-solving process led by their PARTNERS Project consultant who had a primary role in the development of the tool and served as a coach in this project. The problem-solving process involved an examination of many sources of information including Acadience screening data, assessment schedules, and decision rules for the screening section; sample lesson plans for the Tier 1 section; weekly schedules to examine time allotted for instruction and classroom observation data for the classroom environment section; and sample intervention programs for the Tier 2 section. After reviewing these student screening data and permanent products, each of the core components was scored on a 3-point scale, where 3 represented “Strong evidence/No need to problem solve,” 2 “Mixed or inconsistent evidence/Possible area for problem solving,” and 1 represented “No evidence/An area in need of problem solving.” Many of the LAP-G items included a checklist of essential elements that were required to be evident for a score of a 3 for that core component. The number of possible points for each component was based on the number of items and not the number of essential elements. In the Tier 1 section, the number of possible points for the core components was: Screening (33 points), Instructional Materials by Essential Component – Core and Supplemental (18 points), Implementation of Tier 1 Instruction (18 points), and Differentiated Instruction (21 points). The number of possible points for the Tier 2 section by core component was as follows: Assessments—Intervention-based Diagnostics and Progress Monitoring (30 points), Instructional Materials for Each Intervention Program (12 points), and General Considerations: Effective Intervention Design, Professional Development, Implementation Checks (64 points). Only the Tier 1 and Tier 2 sections had been completed by the teacher teams during the period of time reported on in this study.

Design and Procedures

The evaluation of the PARTNERS Project used descriptive research methods. The LAP-G was completed annually in April/May during the first three years of the project. The results were calculated as the percentage of possible points earned for each of the core components in the Tier 1 section and the Tier 2 sections.

The results of this process evaluation indicate that the PARTNERS Project increased teachers’ capacity to implement key MTSS practices pertaining to reading. Over the course of the first three years of the PARTNERS Project, implementation gains were made at Tier 1 and Tier 2, as measured by the LAP-G.

At Loweland School, Tier 1: Screening remained stable from Year 1 to Year 2 at 81.8% and increased to 87.9% in Year 3. Instructional Materials by Essential Component increased from 63.9% in Year 1 to 83.3% in Years 2 and 3. Implementation of Tier 1 Instruction was more variable with a decrease from 94.4% in Year 1 to 77.8% in Year 2 before a slight increase to 88.9% in Year 3. This dip is attributed to greater teacher understanding of the components of the science of reading leading to more accurate rating of practices in Year 2 relative to Year 1, as well as a possible implementation dip while learning the newly adopted phonics program. Differentiated Instruction saw a steady increase from 42.9% in Year 1 to 61.9% in Year 2 and 69.0% in Year 3.

The Tier 2 section was completed in Years 2 and 3 of the PARTNERS Project at Loweland. Tier 2 Assessments increased from 70.4% in Year 2 to 92.6% in Year 3. Instructional Materials for Each Intervention Program remained stable at 83.3% both years. General Considerations/Implementation increased from 59.5% to 72.6%. Loweland School’s results on the annual administration of the LAP-G are presented graphically in Figure 1 .

The K-1 teacher team at St. Mark School demonstrated marked gains in Tier 1. Tier 1 Screening increased sharply from 39.4% in Year 1 to 100% in Years 2 and 3. Instructional Materials by Essential Component increased from 33.3% in Year 1 to 86.1% in Year 2 to 100% in Year 3. Implementation of Tier 1 Instruction increased sharply from 44.4% in Year 1 to 100% in Years 2 and 3. Differentiated Instruction increased from 61.9% in Year 1 to 100% in Years 2 and 3.

The K – 1 teacher team at St. Mark School also demonstrated marked gains in Tier 2 evidence-based literacy practices. Tier 2 Assessments increased sharply from 33.3% in Year 1 to 100% in Years 2 and 3. Instructional Materials for Each Intervention Program increased from 33.3% in Year 1 to 91.7% in Year 2 to 100% in Year 3. General Considerations/Implementation increased sharply from 21.9% in Year 1 to 96.9% in Year 2 to 100% in Year 3. Through this process, the K – 1 team focused on increasing the coordination of the interventionists delivering a seamless continuum of supports. St. Mark School’s K – 1 teacher team results are presented graphically in Figure 2 .

The Grade 2 – 3 teacher team at St. Mark School demonstrated similarly positive gains on the annual administration of the LAP-G at Tier 1. Tier 1 Screening increased sharply from 39.4% in Year 1 to 100% in Years 2 and 3. Instructional Materials by Essential Component increased from 33.3% in Year 1 to 72.2% in Years 2 and 3. Implementation of Tier 1 Instruction increased from 50.0% in Year 1 to 72.2% in Year 2 to 83.3% in Year 3. Differentiated Instruction increased sharply from 52.4% in Year 1 to 100% in Years 2 and 3.

The Grade 2 – 3 teacher team at St. Mark School also demonstrated marked gains in Tier 2 evidence-based literacy practices. Tier 2 Assessments increased sharply from 33.3% in Year 1 to 100% in Years 2 and 3. Instructional Materials for Each Intervention Program increased from 33.3% in Year 1 to 91.7% in Years 2 to 100% in Year 3. General Considerations/Implementation increased sharply from 21.9% in Year 1 to 96.9% in Year 2 to 100% in Year 3. St. Mark School’s Grade 2 – 3 teacher team results are presented graphically in Figure 3 .

Through this process at St. Mark School, a screening system was installed, the core instruction program at K – 1 was determined to be focused on skills that were too advanced given the students’ actual instructional levels, LETRS training was initiated to increase their knowledge and skills in the science of reading, and both teams focused on small-group differentiated instruction. Mid-year changes in Year 1 were made to the supplemental instructional program at K – 1 when the Acadience benchmark data indicated a stronger focus on phonics was needed.

Qualitative data regarding teachers’ perceptions of the acceptability of the PARTNERS Project and the LAP-G process were gathered from 100% of the teachers on St. Mark’s K – 1 and Grades 2 – 3 teams at the end of their first year of implementation. Teachers’ comments provide additional evidence to support the contribution of the PARTNERS Project on increasing teachers’ capacity to implement key MTSS practices pertaining to reading:

I have high hopes that the LAP-G will do everything that is listed above. I am excited about the progress that has been made this year with the Superkids curriculum and am looking forward to more progress next year!

It provided a systematic way to look at strengths and needs.

Still learning about the LAP-G, but I think it's a wonderful system and learning tool to help promote early literacy inside the classroom and provide the teachers with extra support to see areas of needs.

The teachers’ comments provided at the end of the first year of implementation also highlight the teachers’ recognition that they are still developing their skill fluency and require the scaffolded support of their PARTNERS Project consultant and the sustained investment of their school’s administration.

I believe the LAP-G does a great job at pin-pointing areas of need, which makes it easier to plan ahead and supplement those areas for improvement. However, without the guidance of [PARTNERS Project consultant], I'm not sure how successful I'd be filling it out on my own.

I hope the administration will be able to work with the PARTNERS work to provide instructional materials & training needed to continue the positive direction PARTNERS & our teachers have taken this year!

One only needs to consider the decades of substandard reading outcomes to recognize that we have a national reading crisis. Given that research shows upwards of 95% of all students are capable of becoming proficient readers (Foorman et al., 2003 ; Simos et al., 2002 ; Torgesen, 2007 ), low rates of reading proficiency are indicative of an inadequate core curriculum, instruction, and tiered supports to target student needs. The travesty of instructional casualties is particularly devastating among educationally marginalized students. Whereas children from affluent families can pursue private tutoring to compensate for weaknesses in early literacy instruction, families with fewer financial resources often find their children falling farther behind due to unmet instructional needs.

Advances in the science of reading have produced a robust evidence base for effective curriculum and instruction. MTSS provides a framework within which literacy instruction aligned with the science of reading can be provided to address the needs of students proactively. Fidelity of implementation is essential to realizing the potential of evidence-based instruction delivered within an MTSS framework. However, schools often lack the capacity to implement research-based practices with fidelity. The PARTNERS Project was designed to build the capacity of teachers to strengthen the core reading curriculum, instruction, and tiered intervention supports for students in kindergarten through second grade.

As presented in Table 8 , the PARTNERS Project’s training and coaching supports varied from school to school to meet the unique needs, opportunities, and challenges at each school. A highly engaged and effective principal at St. Mark School committed time and energy to drive PARTNERS Project implementation. As a result, the K – 1 teacher team was able to strengthen phonics instruction at Tier 1 and Tier 2 in the first year of implementation, which enabled them to focus on writing instruction, comprehension, and Tier 3 intervention in the second year of implementation. The lack of continuity of effective, invested leadership at Loweland Elementary created challenges in fostering engagement among the teachers in the PARTNERS Project. These varied experiences of the PARTNERS Project highlight the importance of leadership as a driver of meaningful systems change.

The results of this study provide evidence that a problem-solving process focused on evaluating the core curriculum and instruction and intervention supports based on the science of reading and delivered within an MTSS framework can increase the capacity of teacher teams to implement evidence-based literacy practices. Evidence of the effectiveness of the LAP-G problem-solving process was demonstrated at two elementary schools serving educationally marginalized students. Anecdotal evidence based on the observations of the PARTNERS consultants indicates that teachers gained the knowledge and skills to analyze and improve their reading instruction. Classroom observations showed teachers using the targeted instructional practices, for example replacing word walls with sound walls, directing students to sound out known words instead of guessing, prompting students to retrieve previously learned skills and knowledge such as why a given word (e.g., white) has a long i sound (recalling the “silent e” or “magic e” rule—e makes the vowel say its name). Teachers demonstrated increased knowledge and intention in the questions they would ask, such as why a certain word appeared in the curriculum before the sounds had been taught. Finally, the principal of St. Mark School commented that the teachers are now quick to provide the needed instructional support to help students when the learner first shows signs of struggling, whereas before the PARTNERS Project, special education referral and retention seemed like the only ways of dealing with students who were falling behind.

The findings from this process evaluation are consistent with advances in implementation science, which emphasizes training and coaching as crucial for supporting high-fidelity practices, in sharp contrast to the flawed “train and hope” (Stokes & Baer, 1977 ) approach to professional learning. This study extends the research literature by describing the problem-solving process (focused on evaluating weaknesses in the core curriculum and instruction) and professional learning supports needed to build the capacity for teachers to improve Tier 1 instruction to improve outcomes and prevent instructional casualties.

Limitations of the Study

Several limitations need to be considered when interpreting the results of this study. First, this evaluation employed descriptive research methods to show changes in teachers’ capacity to provide scientifically based reading instruction and intervention based on only one measure, the LAP-G, used with teachers at two schools. Additional research is needed to triangulate these findings and validate the impact of the PARTNERS Project on teachers’ literacy instructional practices and the impact on student reading outcomes across a larger sample of schools.

A second limitation of the evaluative study was the lack of a research-validated instrument for conducting classroom observations of teachers’ instructional practices. As part of the PARTNERS Project, PARTNERS Project consultants observed teacher instruction and assessed implementation fidelity using a fidelity checklist that was developed collaboratively with the teachers based on the targeted instructional practices identified through the LAP-G problem-solving process and focused on in the coaching cycle. In addition, interobserver agreement should have been measured to determine the reliability of the observation data collection. The lack of evidence regarding the reliability and validity of the classroom observation is a limitation of this study.

As a third limitation, in the 3rd year of implementation the PARTNERS Project focused only on strengthening the core reading curriculum and instruction (Tier 1) and targeted intervention (Tier 2). In the 4th year of the project, the focus will extend to intensive intervention (Tier 3). Thus, the outcomes reported represent changes that focused on Tiers 1 and 2 only, an incomplete application of the PARTNERS Project.

A final limitation of the study was the historical threat to validity of the COVID-19 pandemic. The first 2 full years of PARTNERS Project implementation (2020 – 2021 and 2021 – 2022) coincided with significant disruptions to teaching and learning, created by the public health crisis. The demands of physical distancing, health insecurity, financial hardship, and dramatically reduced access for families to school-based instruction, specialized instruction and behavioral health services, and social supports (i.e., school lunch, after-school care) created unprecedented challenges for school communities (Schaffer et al., 2021 ). The positive outcomes attained in 2021 – 2022 through the implementation of the PARTNERS Project focused on Tier 1 instruction are all the more noteworthy for having been achieved during the latter part of the pandemic. Future research should examine the effectiveness of the PARTNERS Project, or a similar initiative, to improve reading instruction in postpandemic conditions.

Implications of the Study

The PARTNERS Project does not assert a single curriculum or instructional program, if selected, will meet the needs of all learners. Just as a functional assessment is needed to develop a hypothesis regarding how and why an individual student is struggling to read (Daly et al., 2005 , 2006 ), an analysis of the instructional environment and resulting learning outcomes is needed to determine specific evidence-based literacy practices to be targeted for improvement. In essence, the LAP-G problem-solving process serves as an autopsy examining the fatal flaws in the core curriculum and instruction and tiered system of interventions resulting in mass instructional casualties.

The PARTNERS Project engages teams of teachers in a problem-solving, data-driven process whereby they evaluate their own instructional program and identify areas of needed improvement to align with the science of reading. With considerable training and coaching, teacher teams can be equipped to shift their efforts to correcting deficits in instruction (rather than exclusively focusing on remediating academic skill deficits in students). The results of this study have significant implications for school districts and state departments of education urgently seeking to align the core curriculum and instruction with the science of reading and prevent instructional casualties among our most educationally marginalized students.

Al Otaiba, S., & Torgeson, J. (2007). Effects from intensive standardized kindergarten and first-grade interventions for the prevention of reading difficulties. In S. E. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Handbook of response to intervention (pp. 212–222). Springer.

Chapter   Google Scholar  

Albritton, K., Anhalt, K., & Terry, N. P. (2016). Promoting equity for our nation’s youngest students: School psychologists as agents of social justice in early childhood settings. School Psychology Forum, 10 (3), 237–250.

Google Scholar  

Barrett, C. A. (2023). Best practices in applying multiple models to assessment and identification of learning disabilities. In P. L. Harrison, S. L. Proctor, & A. Thomas (Eds.), Best practices in school psychology: Data-based and collaborative decision making (7 th ed., pp. 227–239). National Association of School Psychologists.

Blase, K. A., Fixsen, D. L., Van Dyke, M., & Duda, M. A. (2013). Implementation drivers: Best practices for coaching . National Implementation Research Network. http://nirn.fpg.unc.edu

Codding, R. S., & Lane, K. L. (2015). A spotlight on treatment intensity: An important and often overlooked component of intervention inquiry. Journal of Behavioral Education, 24 , 1–10.

Article   Google Scholar  

Daly, E. J., III, Bonfiglio, C. M., Mattson, T., Persampieri, M., & Foreman-Yates, K. (2005). Refining the experimental analysis of academic skills deficits: Part I. An investigation of variables that affect generalized oral reading performance. Journal of Applied Behavior Analysis , 38 (4), 485–497.

Daly, E. J., III, Bonfiglio, C. M., Mattson, T., Persampieri, M., & Foreman-Yates, K. (2006). Refining the experimental analysis of academic skills deficits: Part II. Use of brief experimental analysis to evaluate reading fluency treatments. Journal of Applied Behavior Analysis , 39 (3), 323–331.

Daly, E. J., III., Martens, B. K., Barnett, D., Witt, J. C., & Olson, S. C. (2007). Varying intervention delivery in response-to-intervention: Confronting and resolving challenges with measurement, instruction, and intensity. School Psychology Review, 36 , 562–581.

Fien, H., Chard, D. J., & Baker, S. K. (2021). Can the evidence revolution and multi-tiered systems of support improve education equity and reading achievement? Reading Research Quarterly, 56 (1), S105–S118. https://doi.org/10.1002/rrq.391

Fixsen, D., Blase, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79 , 213–230. https://doi.org/10.1177/001440291307900206

Fletcher, J. M., Coulter, W. A., Reschly, D. J., & Vaugh, S. (2004). Alternative approaches to the definition and identification of learning disabilities: Some questions and answers. Annals of Dyslexia, 54 , 304–331.

Article   PubMed   Google Scholar  

Foorman, B., Beyler, N., Borradaile, K., Coyne, M., Denton, C. A., Dimino, J., Furgeson, J., Hayes, L., Henke, J., Justice, L., Keating, B., Lewis, W., Sattar, S., Streke, A., Wagner, R., & Wissel, S. (2016). Foundational skills to support reading for understanding in kindergarten through 3rd grade (NCEE 2016-4008). National Center for Education Evaluation and Regional Assistance (NCEE), Institute of Education Sciences, U.S. Department of Education. http://whatworks.ed.gov .

Foorman, B. R., Breier, J. I., & Fletcher, J. M. (2003). Interventions aimed at improving reading success: An evidence-based approach. Developmental Neuropsychology, 24 (3), 613–639. https://doi.org/10.1080/87565641.2003.9651913

Fuchs, D., & Fuchs, L. S. (2017). Critique of the national evaluation of response to intervention: A case for simpler frameworks. Exceptional Children, 83 , 255–268. https://doi.org/10.1177/0014402917693580

Garet, M. S., Cronen, S., Eaton, M., Kurki, A., Ludwig, M., Jones, W., Uekawa, K., Falk, A., Bloom, H., Doolittle, F., Zhu, P., & Sztenjnberg, L. (2008). The impact of two professional development interventions on early reading instruction and achievement (NCEE 2008-4030). National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Hall, S. (2018). 10 success factors for literacy intervention: Getting results with MTSS in elementary school . Association for Supervision & Curriculum Development.

Joyce, B. R., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Association for Supervision & Curriculum Development.

Keller-Margulis, M. A. (2012). Fidelity of implementation framework: A critical need for response to intervention models. Psychology in the Schools, 49 , 342–352. https://doi.org/10.1002/pits.21602

Kilpatrick, D. A. (2015). Essentials of assessing, preventing, and overcoming reading difficulties . John Wiley & Sons.

Kratochwill, T. R., Volpiansky, P., Clements, M., & Ball, C. (2007). Professional development in implementing and sustaining multitier prevention models: Implications for response to intervention. School Psychology Review, 36 , 618–631.

Long, A. C. J., Hagermoser, Sanetti, & L. M., Collier-Meek, M. A., Gallucci, J, Altschaefl, M., & Kratochwill, T. R. (2016). An exploratory investigation of teachers’ intervention planning and perceived implementation barriers. Journal of School Psychology, 55 , 1–26.

Lyon, G. R., Fletcher, J. M., Shaywitz, S. E., Shaywitz, B. A., Torgesen, J. K., Wood, F. B., Schulte, A., & Olson, R. (2001). Rethinking learning disabilities. In C. E. Finn, R. A. J. Rotherham, & C. R. Hokanson (Eds.), Rethinking special education for a new century (pp. 259–287). Thomas B. Fordham Foundation & Progressive Policy Institute.

Maras, M. A., Splett, J. W., Reinke, W. M., Stormont, M., & Herman, K. C. (2014). School practitioners’ perspectives on planning, implementing, and evaluating evidence-based practices. Child & Youth Services Review, 47 , 314–322.

McIntosh, K., & Goodman, S. (2016). Integrated multi-tiered systems of support: Blending RTI and PBIS. Guilford Press.

Moore, H., & Lammert, J. (2019). Making project measures meaningful: Quality, relevance, usefulness, and beyond! Westat Center to Improve Program & Project Performance.

Morrison, J. Q., Hawkins, R. O., & Collins, T. A. (2020). Evaluating the cost-effectiveness of the Dyslexia Pilot Project: A multi-tiered system of support for early literacy. Psychology in the Schools, 57 (4), 522–539. https://doi.org/10.1002/pits.22343

National Assessment of Educational Progress. (2022). https://www.nationsreportcard.gov/

National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction . National Institutes of Health.

Noell, G. H., & Gansle, K. A. (2006). Assuring the form has substance: Treatment plan implementation as the foundation of assessing response to intervention. Assessment for Effective Intervention, 32 , 32–39.

The Reading League. (2022). The science of reading: A defining guide .

Reynolds, A. J., Temple, J. A., Robertson, D. L., & Mann, E. A. (2002). Age 21 cost-benefit analysis of the Title I Chicago Child-Parent Centers. Educational Evaluation & Policy Analysis , 24 (4), 267–303. https://www.jstor.org/stable/3594119

Sabnis, S., Castillo, J. M., & Wolgemuth, J. R. (2020). RTI, equity, and the return to the status quo: Implications for Consultants. Journal of Educational & Psychological Consultation, 30 (3), 285–313. https://doi.org/10.1080/10474412.2019.1674152

Sanetti, L. M. H., & Luh, H.-J. (2019). Fidelity of implementation in the field of learning disabilities. Learning Disability Quarterly, 42 (4), 204–216.

Schaffer, G. E., Power, E. M., Fisk, A. K., & Trolian, T. L. (2021). Beyond the four walls: The evolution of school psychological services during the COVID-19 outbreak. Psychology in the Schools, 58 (7), 1246–1265. https://doi.org/10.1002/pits.22543

Article   PubMed   PubMed Central   Google Scholar  

Simos, P. G., Fletcher, J. M., Bergman, E., Breier, J. I., Foorman, B. R., Castillo, E. M., Davis, R. N., Fitzgerald, M., & Papanicolaou, A. C. (2002). Dyslexia-specific brain activation profile becomes normal following successful remedial training. Neurology, 58 , 1203–1213.

Spear-Swerling, L. (2018). Structured literacy and typical literacy practices: Understanding differences to create instructional opportunities. Teaching Exceptional Children, 51 (3), 201–211. https://doi.org/10.1177/0040059917750160

Stokes, T. F., & Baer, D. M. (1977). An implicit technology of generalization. Journal of Applied Behavior Analysis, 10 (2), 349–367. https://doi.org/10.1901/jaba.1977.10-349

Torgesen, J. K. (2007). Recent discoveries on remedial interventions for children with dyslexia. In M. J. Snowling & C. Hume (Eds.), The science of reading: A handbook (pp. 521–537). Blackwell.

Torgesen, J. K., Alexander, A. W., Wagner, R. K., Rashotte, C. A., Voeller, K. K. S. & Conway, T. (2001). Intensive remedial instruction for children with severe reading disabilities: Immediate and long-term outcomes from two instructional approaches. Journal of Learning Disabilities, 34 (1), 33–58, 78.

VanDerHeyden, A. M. (2013). Universal screening may not be for everyone: Using a threshold model as a smarter way to determine risk. School Psychology Review, 42 , 402–414.

VanDerHeyden, A. M., & Witt, J. C. (2005). Quantifying context in assessment: Capturing the effect of base rates on teacher referral and a problem-solving model of identification. School Psychology Review, 34 , 161–183.

VanDerHeyden, A. M., Witt, J. C., & Gilbertson, D. (2007). A multi-year evaluation of the effects of a response to intervention (RTI) model on identification of children for special education. Journal of School Psychology, 45 (2), 225–256. https://doi.org/10.1016/j.jsp.2006.11.004

Vaughn, S., & Fuchs, L. S. (2003). Redefining learning disabilities as inadequate response to instruction: The promise and potential problems. Learning Disabilities Research & Practice, 18 , 137–146. https://doi.org/10.1111/1540-5826.00070

Vellutino, F. R., Scanlon, D. M., Sipay, E. R., Small, S. G., Pratt, A., Chen, R., & Denkla, M. B. (1996). Cognitive profiles of difficult-to-remediate and readily remediated poor readers: Early intervention as a vehicle for distinguishing between cognitive and experiential deficits as basic causes of specific reading disability. Journal of Educational Psychology, 88 , 601–638.

Vellutino, F. R., Scanlon, D. M., Zhang, H., & Schatschneider, C. (2008). Using response to kindergarten and first grade intervention to identify children at-risk for long-term reading difficulties. Reading & Writing: An Interdisciplinary Journal, 21 , 437–480.

Download references

This research project was funded by a U.S. Department of Education, Office for Special Education Program (OSEP) Model Demonstration Grant for the Early Identification of Students with Dyslexia in Elementary Schools.

Author information

Authors and affiliations.

Reading Science Program, Mount Saint Joseph University, Cincinnati, OH, USA

Amy Murdoch & Wendy Strickler

School Psychology Program, University of Cincinnati, CECH-School of Human Services, P.O. Box 210068, Cincinnati, OH, 45221-0068, USA

Julie Q. Morrison

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Julie Q. Morrison .

Ethics declarations

Conflicts of interest.

The authors declare that there is no conflict of interest to disclose.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Murdoch, A., Morrison, J.Q. & Strickler, W. Process Evaluation of a Problem-Solving Approach for Analyzing Literacy Practices within a Multi-Tiered System of Supports Framework. Behav. Soc. Iss. (2024). https://doi.org/10.1007/s42822-024-00166-5

Download citation

Accepted : 12 March 2024

Published : 06 May 2024

DOI : https://doi.org/10.1007/s42822-024-00166-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Science of reading
• Program evaluation
• Find a journal
• Publish with us
• Track your research

Adolescence

Effortful helping in teenagers at risk for psychopathy, new research shows adolescents with conduct problems are less willing to help..

Updated May 10, 2024 | Reviewed by Davia Sills

• What Changes During Adolescence?
• Find counselling to support kids and teens
• Adolescents with conduct problems often display antisocial behavior.
• People with conduct problems and high callous-unemotional traits are at risk of developing psychopathy.
• Research finds those at risk of developing psychopathy are especially unwilling to make an effort to help.
• The data highlight differences between individuals and new targets for research on behavioral interventions.

This post was written by Anne Gaule, Ph.D., and Essi Viding, Ph.D., with edits from Patricia Lockwood, Ph.D., and Jo Cutler, Ph.D.

Helping other people—be it your friend moving house or a colleague with their work—often requires effort. However, research has shown that helping others or engaging in what psychologists call "prosocial" behaviors has a range of positive impacts on our social relationships, our physical and mental well-being, and even our longevity.

This is as true when we are young as it is in adulthood. There is good evidence that when young people engage in prosocial behaviors, this helps them to form the building blocks they need to establish good relationships with others, which, in turn, can be protective against common mental health and behavior problems .

However, there is a group of young people that engage in antisocial behavior, have difficulty in their social relationships, and also appear to show worryingly low levels of prosocial behaviors. This pattern of behaviors in adolescence is termed "conduct problems." A new study explores how teenagers with conduct problems engage in effortful prosocial behaviors and highlights important factors we may need to consider when designing interventions to support this vulnerable group.

Individual differences matter

As outlined above, research indicates that young people with conduct problems show low levels of prosocial behaviors. However, just because someone behaves antisocially does not mean that they will never behave prosocially. Individual differences among adolescents with conduct problems (or psychological characteristics that define who we are and how we process information about the world) have not received a lot of attention when it comes to prosocial behavior. These differences matter—especially when we are trying to design effective interventions.

One source of individual differences that is thought to be very important when considering how conduct problems develop and are expressed is a person’s level of what researchers call callous-unemotional traits. These traits include having difficulty empathizing with other people, lacking remorse for poor behavior, and not placing importance on social relationships. Research has shown that adolescents with conduct problems who also show high levels of these traits have poorer treatment outcomes and are at a greater risk for continued antisocial behavior and psychopathy in adulthood.

Given the social and psychological benefits that prosocial behaviors can have during development, understanding how these traits influence prosocial behavior in young people with conduct problems could be hugely important.

The importance of effort

When we consider prosocial behaviors in young people with conduct problems and how these impact their social relationships, it is important to think about what people generally value when it comes to prosocial behavior. Of course, this probably varies from person to person, but as a general rule, it seems as though people care a lot about effort. For example, if a friend is moving house and you wish to support them, simply saying that you will help probably is not enough to contribute to the good relationship you have with that person—you have to actually show up and help with the packing!

Supporting this, a research study carried out in 2018 found that people often care more about the personal sacrifice involved in a prosocial action than about the social benefit produced by the action itself, as they take this sacrifice to be an indication of a person’s moral character.

Therefore, if we want to get a full picture of prosocial behaviors, we need to understand how willing people are to put in effort to help others. This is especially important in the context of social relationships and understanding how willingness to help others may vary between people, such as those with conduct problems. Surprisingly, however, this is rarely looked at in research studies—which often look at whether people donate money to help others.

Building our understanding of prosocial behavior in adolescents with conduct problems

In our recent study, published in the Journal of Child Psychology and Psychiatry , we tested a sample of 94 adolescent boys between the ages of 11 and 16 from mainstream and specialist provision schools in the UK. The teenagers played a game where they could squeeze a hand-held gripper to earn points towards gift vouchers—with rounds where they played for themselves and prosocial rounds where they played to earn vouchers for another boy at a different school.

On each round of the game, the boys had to first choose if they wanted to have a go at squeezing the gripper to get more points towards vouchers, or if they would rather rest. If they chose to exert effort for a higher number of points, they were then asked to squeeze the gripper with the required force in order to earn their points.

We found that boys who met our criteria for having conduct problems chose less prosocially than the boys without conduct problems. However, once choices had been made, there was a distinction within the group of boys with conduct problems related to levels of callous-unemotional traits. Boys with conduct problems who were also rated by their teachers as being high in callous-unemotional traits put in considerably less effort to help others (relative to for themselves) compared to both boys without conduct problems and boys with conduct problems with lower levels of these traits.

So, while boys with conduct problems overall were less likely to choose to help someone else, once the choice to be helpful was made, only those boys who had both conduct problems and callous unemotional traits made less effort to follow through.

Implications

Our findings underscore the importance of nuance when trying to understand social behavior. By looking at prosocial effort as well as prosocial choices, and by also accounting for differences among adolescents with conduct problems, we have demonstrated a number of things. First, it is not enough to just study prosocial choices; knowing about people’s willingness to make effort for others is also important if we want to fully understand adolescents’ social lives.

Furthermore, even among adolescents with conduct problems, there is substantial variation in prosociality. If we want to promote more prosocial behavior in adolescents who have behavioral problems, we need to understand the precise difficulties that they have, which may need to be targeted specifically in interventions. Our study clearly shows that we should not treat all the young people in this vulnerable group the same way and that some of them may need more help than others.

Gaule, A., Martin, P., Lockwood, P. L., Cutler, J., Apps, M., Roberts, R., Phillips, H., Brown, K., McCrory, E. J., & Viding, E. (2024). Reduced prosocial motivation and effort in adolescents with conduct problems and callous-unemotional traits. Journal of Child Psychology and Psychiatry . https://doi.org/10.1111/jcpp.13945

Johnson, S. G. B. (2020). Dimensions of Altruism: Do Evaluations of Prosocial Behavior Track Social Good or Personal Sacrifice? OSF. https://doi.org/10.31234/osf.io/r85jv

Memmott-Elison, M. K., & Toseeb, U. (2023). Prosocial behavior and psychopathology: An 11-year longitudinal study of inter- and intraindividual reciprocal relations across childhood and adolescence. Development and Psychopathology , 35 (4), 1982–1996. https://doi.org/10.1017/S0954579422000657

Post, S. G. (2005). Altruism, Happiness, and Health: It’s Good to Be Good. International Journal of Behavioral Medicine , 12 (2), 66–77.

Patricia Lockwood, Ph.D., is a Wellcome Trust/Royal Society Sir Henry Dale Fellow (Associate Professor) and Jacobs Foundation Research Fellow at the University of Birmingham, UK, where she leads the Social Decision Neuroscience Lab. Jo Cutler, Ph.D., is a Postdoctoral Research Fellow in the Social Decision Neuroscience Lab at the University of Birmingham, UK.

• Find a Therapist
• Find a Treatment Center
• Find a Psychiatrist
• Find a Support Group
• Find Online Therapy
• International
• New Zealand
• South Africa
• Switzerland
• Asperger's
• Bipolar Disorder
• Chronic Pain
• Eating Disorders
• Passive Aggression
• Personality
• Goal Setting
• Positive Psychology
• Stopping Smoking
• Low Sexual Desire
• Relationships
• Child Development
• Therapy Center NEW
• Diagnosis Dictionary
• Types of Therapy

At any moment, someone’s aggravating behavior or our own bad luck can set us off on an emotional spiral that threatens to derail our entire day. Here’s how we can face our triggers with less reactivity so that we can get on with our lives.

• Emotional Intelligence
• Gaslighting
• Affective Forecasting
• Neuroscience

Contamination bias in the estimation of child maltreatment causal effects on adolescent internalizing and externalizing behavior problems

Background: When unaddressed, contamination in child maltreatment research, in which some proportion of children recruited for a nonmaltreated comparison group are exposed to maltreatment, downwardly biases the significance and magnitude of effect size estimates. This study extends previous contamination research by investigating how a dual-measurement strategy of detecting and controlling contamination impacts causal effect size estimates of child behavior problems. Methods: This study included 634 children from the LONGSCAN study with 63 cases of confirmed child maltreatment after age 8 and 571 cases without confirmed child maltreatment. Confirmed child maltreatment and internalizing and externalizing behaviors were recorded every 2 years between ages 4 and 16. Contamination in the nonmaltreated comparison group was identified and controlled by either a prospective self-report assessment at ages 12, 14, and 16 or by a one-time retrospective self-report assessment at age 18. Synthetic control methods were used to establish causal effects and quantify the impact of contamination when it was not controlled, when it was controlled for by prospective self-reports, and when it was controlled for by retrospective self-reports. Results: Rates of contamination ranged from 62% to 67%. Without controlling for contamination, causal effect size estimates for internalizing behaviors were not statistically significant. Causal effects only became statistically significant after controlling contamination identified from either prospective or retrospective reports and effect sizes increased by between 17% and 54%. Controlling contamination had a smaller impact on effect size increases for externalizing behaviors but did produce a statistically significant overall effect, relative to the model ignoring contamination, when prospective methods were used. Conclusions: The presence of contamination in a nonmaltreated comparison group can underestimate the magnitude and statistical significance of causal effect size estimates, especially when investigating internalizing behavior problems. Addressing contamination can facilitate the replication of results across studies.