research & reviews a journal of health professions

Change Password

Heading 1 Example Heading 1 Example Login Create Account Reset Password

Go to Dashboard

Welcome to the Manuscript Submission Site

To begin, log in with your user ID and password.

If you are unsure about whether or not you have an account, or have forgotten your password, go to the Reset Password screen.

• Journals List
• Author Guidelines
• Manuscript Submission Guideline
• Site Support

Edit Profile

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Can Med Educ J
• v.12(3); 2021 Jun

Writing, reading, and critiquing reviews

Écrire, lire et revue critique, douglas archibald.

1 University of Ottawa, Ontario, Canada;

Maria Athina Martimianakis

2 University of Toronto, Ontario, Canada

Why reviews matter

What do all authors of the CMEJ have in common? For that matter what do all health professions education scholars have in common? We all engage with literature. When you have an idea or question the first thing you do is find out what has been published on the topic of interest. Literature reviews are foundational to any study. They describe what is known about given topic and lead us to identify a knowledge gap to study. All reviews require authors to be able accurately summarize, synthesize, interpret and even critique the research literature. 1 , 2 In fact, for this editorial we have had to review the literature on reviews . Knowledge and evidence are expanding in our field of health professions education at an ever increasing rate and so to help keep pace, well written reviews are essential. Though reviews may be difficult to write, they will always be read. In this editorial we survey the various forms review articles can take. As well we want to provide authors and reviewers at CMEJ with some guidance and resources to be able write and/or review a review article.

What are the types of reviews conducted in Health Professions Education?

Health professions education attracts scholars from across disciplines and professions. For this reason, there are numerous ways to conduct reviews and it is important to familiarize oneself with these different forms to be able to effectively situate your work and write a compelling rationale for choosing your review methodology. 1 , 2 To do this, authors must contend with an ever-increasing lexicon of review type articles. In 2009 Grant and colleagues conducted a typology of reviews to aid readers makes sense of the different review types, listing fourteen different ways of conducting reviews, not all of which are mutually exclusive. 3 Interestingly, in their typology they did not include narrative reviews which are often used by authors in health professions education. In Table 1 , we offer a short description of three common types of review articles submitted to CMEJ.

Three common types of review articles submitted to CMEJ

Type of Review	Description	Examples of published HPE articles using review methodology
Systematic Review	Often associated with Cochrane Reviews, this type of review aims to answer a narrowly focused question and uses a predetermined structured method to search, screen, select, appraise and summarize findings.	Tang KS, Cheng DL, Mi E, Greenberg PB. Augmented reality in medical education: a systematic review. Can Med Ed J. 2020;11(1):e81. And in this issue: of the CMEJ: Bahji A, Smith J, Danilewitz M, Crockford D, el-Guebaly N, Stuart H. Towards competency-based medical education in addictions psychiatry: a systematic review. . 2021; 12(3) 10.36834/cmej.69739
Scoping Review	Aims to quickly map a research area, documenting key concepts, sources of evidence, methodologies used. Typically, scoping reviews do not judge the quality of the papers included in the review. They tend to produce descriptive accounts of a topic area.	Kalun P, Dunn K, Wagner N, Pulakunta T, Sonnadara R. Recent evidence on visual-spatial ability in surgical education: A scoping review. . 2020 Dec;11(6):e111. Refer to Cacchione and Arksey and O’Malley and for more details.
(Critical) Narrative Review	Narrative reviews are expert interpretations and critiques of previously published studies. They are not intended to be exhaustive in their review of evidence, but rather synthetic and generative. Research questions can be narrow or broad and are often theoretically derived. They may constitute a synthesis of existing models or schools of thoughts or generate a new interpretation or way of thinking.	Examples of authors applying (Critical) Narrative reviews: Ng, S. L., Kinsella, E. A., Friesen, F., & Hodges, B. (2015). Reclaiming a theoretical orientation to reflection in medical education research: a critical narrative review. (5), 461–475. 10.1111/medu.12680 For more information: Greenhalgh, T., Thorne, S., & Malterud, K. (2018). Time to challenge the spurious hierarchy of systematic over narrative reviews? (6), e12931–n/a. 10.1111/eci.12931 Ferrari, R. Writing narrative style literature reviews. . 2015;24(4):230-235. doi:10.1179/2047480615Z.000000000329

More recently, authors such as Greenhalgh 4 have drawn attention to the perceived hierarchy of systematic reviews over scoping and narrative reviews. Like Greenhalgh, 4 we argue that systematic reviews are not to be seen as the gold standard of all reviews. Instead, it is important to align the method of review to what the authors hope to achieve, and pursue the review rigorously, according to the tenets of the chosen review type. Sometimes it is helpful to read part of the literature on your topic before deciding on a methodology for organizing and assessing its usefulness. Importantly, whether you are conducting a review or reading reviews, appreciating the differences between different types of reviews can also help you weigh the author’s interpretation of their findings.

In the next section we summarize some general tips for conducting successful reviews.

How to write and review a review article

In 2016 David Cook wrote an editorial for Medical Education on tips for a great review article. 13 These tips are excellent suggestions for all types of articles you are considering to submit to the CMEJ. First, start with a clear question: focused or more general depending on the type of review you are conducting. Systematic reviews tend to address very focused questions often summarizing the evidence of your topic. Other types of reviews tend to have broader questions and are more exploratory in nature.

Following your question, choose an approach and plan your methods to match your question…just like you would for a research study. Fortunately, there are guidelines for many types of reviews. As Cook points out the most important consideration is to be sure that the methods you follow lead to a defensible answer to your review question. To help you prepare for a defensible answer there are many guides available. For systematic reviews consult PRISMA guidelines ; 13 for scoping reviews PRISMA-ScR ; 14 and SANRA 15 for narrative reviews. It is also important to explain to readers why you have chosen to conduct a review. You may be introducing a new way for addressing an old problem, drawing links across literatures, filling in gaps in our knowledge about a phenomenon or educational practice. Cook refers to this as setting the stage. Linking back to the literature is important. In systematic reviews for example, you must be clear in explaining how your review builds on existing literature and previous reviews. This is your opportunity to be critical. What are the gaps and limitations of previous reviews? So, how will your systematic review resolve the shortcomings of previous work? In other types of reviews, such as narrative reviews, its less about filling a specific knowledge gap, and more about generating new research topic areas, exposing blind spots in our thinking, or making creative new links across issues. Whatever, type of review paper you are working on, the next steps are ones that can be applied to any scholarly writing. Be clear and offer insight. What is your main message? A review is more than just listing studies or referencing literature on your topic. Lead your readers to a convincing message. Provide commentary and interpretation for the studies in your review that will help you to inform your conclusions. For systematic reviews, Cook’s final tip is most likely the most important– report completely. You need to explain all your methods and report enough detail that readers can verify the main findings of each study you review. The most common reasons CMEJ reviewers recommend to decline a review article is because authors do not follow these last tips. In these instances authors do not provide the readers with enough detail to substantiate their interpretations or the message is not clear. Our recommendation for writing a great review is to ensure you have followed the previous tips and to have colleagues read over your paper to ensure you have provided a clear, detailed description and interpretation.

Finally, we leave you with some resources to guide your review writing. 3 , 7 , 8 , 10 , 11 , 16 , 17 We look forward to seeing your future work. One thing is certain, a better appreciation of what different reviews provide to the field will contribute to more purposeful exploration of the literature and better manuscript writing in general.

In this issue we present many interesting and worthwhile papers, two of which are, in fact, reviews.

Major Contributions

A chance for reform: the environmental impact of travel for general surgery residency interviews by Fung et al. 18 estimated the CO 2 emissions associated with traveling for residency position interviews. Due to the high emissions levels (mean 1.82 tonnes per applicant), they called for the consideration of alternative options such as videoconference interviews.

Understanding community family medicine preceptors’ involvement in educational scholarship: perceptions, influencing factors and promising areas for action by Ward and team 19 identified barriers, enablers, and opportunities to grow educational scholarship at community-based teaching sites. They discovered a growing interest in educational scholarship among community-based family medicine preceptors and hope the identification of successful processes will be beneficial for other community-based Family Medicine preceptors.

Exploring the global impact of the COVID-19 pandemic on medical education: an international cross-sectional study of medical learners by Allison Brown and team 20 studied the impact of COVID-19 on medical learners around the world. There were different concerns depending on the levels of training, such as residents’ concerns with career timeline compared to trainees’ concerns with the quality of learning. Overall, the learners negatively perceived the disruption at all levels and geographic regions.

The impact of local health professions education grants: is it worth the investment? by Susan Humphrey-Murto and co-authors 21 considered factors that lead to the publication of studies supported by local medical education grants. They identified several factors associated with publication success, including previous oral or poster presentations. They hope their results will be valuable for Canadian centres with local grant programs.

Exploring the impact of the COVID-19 pandemic on medical learner wellness: a needs assessment for the development of learner wellness interventions by Stephana Cherak and team 22 studied learner-wellness in various training environments disrupted by the pandemic. They reported a negative impact on learner wellness at all stages of training. Their results can benefit the development of future wellness interventions.

Program directors’ reflections on national policy change in medical education: insights on decision-making, accreditation, and the CanMEDS framework by Dore, Bogie, et al. 23 invited program directors to reflect on the introduction of the CanMEDS framework into Canadian postgraduate medical education programs. Their survey revealed that while program directors (PDs) recognized the necessity of the accreditation process, they did not feel they had a voice when the change occurred. The authors concluded that collaborations with PDs would lead to more successful outcomes.

Experiential learning, collaboration and reflection: key ingredients in longitudinal faculty development by Laura Farrell and team 24 stressed several elements for effective longitudinal faculty development (LFD) initiatives. They found that participants benefited from a supportive and collaborative environment while trying to learn a new skill or concept.

Brief Reports

The effect of COVID-19 on medical students’ education and wellbeing: a cross-sectional survey by Stephanie Thibaudeau and team 25 assessed the impact of COVID-19 on medical students. They reported an overall perceived negative impact, including increased depressive symptoms, increased anxiety, and reduced quality of education.

In Do PGY-1 residents in Emergency Medicine have enough experiences in resuscitations and other clinical procedures to meet the requirements of a Competence by Design curriculum? Meshkat and co-authors 26 recorded the number of adult medical resuscitations and clinical procedures completed by PGY1 Fellow of the Royal College of Physicians in Emergency Medicine residents to compare them to the Competence by Design requirements. Their study underscored the importance of monitoring collection against pre-set targets. They concluded that residency program curricula should be regularly reviewed to allow for adequate clinical experiences.

Rehearsal simulation for antenatal consults by Anita Cheng and team 27 studied whether rehearsal simulation for antenatal consults helped residents prepare for difficult conversations with parents expecting complications with their baby before birth. They found that while rehearsal simulation improved residents’ confidence and communication techniques, it did not prepare them for unexpected parent responses.

Review Papers and Meta-Analyses

Peer support programs in the fields of medicine and nursing: a systematic search and narrative review by Haykal and co-authors 28 described and evaluated peer support programs in the medical field published in the literature. They found numerous diverse programs and concluded that including a variety of delivery methods to meet the needs of all participants is a key aspect for future peer-support initiatives.

Towards competency-based medical education in addictions psychiatry: a systematic review by Bahji et al. 6 identified addiction interventions to build competency for psychiatry residents and fellows. They found that current psychiatry entrustable professional activities need to be better identified and evaluated to ensure sustained competence in addictions.

Six ways to get a grip on leveraging the expertise of Instructional Design and Technology professionals by Chen and Kleinheksel 29 provided ways to improve technology implementation by clarifying the role that Instructional Design and Technology professionals can play in technology initiatives and technology-enhanced learning. They concluded that a strong collaboration is to the benefit of both the learners and their future patients.

In his article, Seven ways to get a grip on running a successful promotions process, 30 Simon Field provided guidelines for maximizing opportunities for successful promotion experiences. His seven tips included creating a rubric for both self-assessment of likeliness of success and adjudication by the committee.

Six ways to get a grip on your first health education leadership role by Stasiuk and Scott 31 provided tips for considering a health education leadership position. They advised readers to be intentional and methodical in accepting or rejecting positions.

Re-examining the value proposition for Competency-Based Medical Education by Dagnone and team 32 described the excitement and controversy surrounding the implementation of competency-based medical education (CBME) by Canadian postgraduate training programs. They proposed observing which elements of CBME had a positive impact on various outcomes.

You Should Try This

In their work, Interprofessional culinary education workshops at the University of Saskatchewan, Lieffers et al. 33 described the implementation of interprofessional culinary education workshops that were designed to provide health professions students with an experiential and cooperative learning experience while learning about important topics in nutrition. They reported an enthusiastic response and cooperation among students from different health professional programs.

In their article, Physiotherapist-led musculoskeletal education: an innovative approach to teach medical students musculoskeletal assessment techniques, Boulila and team 34 described the implementation of physiotherapist-led workshops, whether the workshops increased medical students’ musculoskeletal knowledge, and if they increased confidence in assessment techniques.

Instagram as a virtual art display for medical students by Karly Pippitt and team 35 used social media as a platform for showcasing artwork done by first-year medical students. They described this shift to online learning due to COVID-19. Using Instagram was cost-saving and widely accessible. They intend to continue with both online and in-person displays in the future.

Adapting clinical skills volunteer patient recruitment and retention during COVID-19 by Nazerali-Maitland et al. 36 proposed a SLIM-COVID framework as a solution to the problem of dwindling volunteer patients due to COVID-19. Their framework is intended to provide actionable solutions to recruit and engage volunteers in a challenging environment.

In Quick Response codes for virtual learner evaluation of teaching and attendance monitoring, Roxana Mo and co-authors 37 used Quick Response (QR) codes to monitor attendance and obtain evaluations for virtual teaching sessions. They found QR codes valuable for quick and simple feedback that could be used for many educational applications.

In Creation and implementation of the Ottawa Handbook of Emergency Medicine Kaitlin Endres and team 38 described the creation of a handbook they made as an academic resource for medical students as they shift to clerkship. It includes relevant content encountered in Emergency Medicine. While they intended it for medical students, they also see its value for nurses, paramedics, and other medical professionals.

Commentary and Opinions

The alarming situation of medical student mental health by D’Eon and team 39 appealed to medical education leaders to respond to the high numbers of mental health concerns among medical students. They urged leaders to address the underlying problems, such as the excessive demands of the curriculum.

In the shadows: medical student clinical observerships and career exploration in the face of COVID-19 by Law and co-authors 40 offered potential solutions to replace in-person shadowing that has been disrupted due to the COVID-19 pandemic. They hope the alternatives such as virtual shadowing will close the gap in learning caused by the pandemic.

Letters to the Editor

Canadian Federation of Medical Students' response to “ The alarming situation of medical student mental health” King et al. 41 on behalf of the Canadian Federation of Medical Students (CFMS) responded to the commentary by D’Eon and team 39 on medical students' mental health. King called upon the medical education community to join the CFMS in its commitment to improving medical student wellbeing.

Re: “Development of a medical education podcast in obstetrics and gynecology” 42 was written by Kirubarajan in response to the article by Development of a medical education podcast in obstetrics and gynecology by Black and team. 43 Kirubarajan applauded the development of the podcast to meet a need in medical education, and suggested potential future topics such as interventions to prevent learner burnout.

Response to “First year medical student experiences with a clinical skills seminar emphasizing sexual and gender minority population complexity” by Kumar and Hassan 44 acknowledged the previously published article by Biro et al. 45 that explored limitations in medical training for the LGBTQ2S community. However, Kumar and Hassen advocated for further progress and reform for medical training to address the health requirements for sexual and gender minorities.

In her letter, Journey to the unknown: road closed!, 46 Rosemary Pawliuk responded to the article, Journey into the unknown: considering the international medical graduate perspective on the road to Canadian residency during the COVID-19 pandemic, by Gutman et al. 47 Pawliuk agreed that international medical students (IMGs) do not have adequate formal representation when it comes to residency training decisions. Therefore, Pawliuk challenged health organizations to make changes to give a voice in decision-making to the organizations representing IMGs.

In Connections, 48 Sara Guzman created a digital painting to portray her approach to learning. Her image of a hand touching a neuron showed her desire to physically see and touch an active neuron in order to further understand the brain and its connections.

--!>

	in a scored and prestigious database of scientific journals. This section allows you to place information about the journal, editors, and publisher, etc. from all over the world.		Area: Evaluated version: online
The journal is indexed in:
SJIFactor.com
Basic information
Main title
ISSN	2277-6192 (E) / 2348-9537 (P)
URL
Country
Scientific
Frequency	3 times a year
License	All rights reserved - no automatic licence
Texts availability	Paid
Contact Details
Prof. Nirmal Kumar Ganguly
Padma Bhushan’, Honorary Senior Research Professor (Clinical Research), Institute of Liver & Biliary Sciences, New Delhi-110070, India. Professor (GRIPMER), The Ganga Ram Institute for Postgraduate Medical Education & Research, Sir Ganga Ram Hospital, New Delhi-110060, India. Visiting Professor of Eminence, Translational Health Science & Technology Institute, Faridabad-121001
STM Journals, An Imprint Of Consortium E-Learning Network Pvt. Ltd.
A-118, 1st Floor, Sector-63, Noida, U.P. India, Pin – 201301
Journal's description

• Open access
• Published: 14 September 2024

Role of sex and training characteristics on exercise effects on cardiovascular aging: protocol for a systematic review with meta-analysis of randomized trials

• Emmanuel Gomes Ciolac   ORCID: orcid.org/0000-0002-2397-8162 1 ,
• Jana Babjakova 2 ,
• Raphael Martins de Abreu 3 , 4 ,
• Su-Jie Mao 5 ,
• Guoping Qian 5 ,
• Vanessa Teixeira do Amaral 1 ,
• Bartlomiej Wrzesinski 5 ,
• Artur Junio Togneri Ferron 1 ,
• Zbigniew Ossowski 5 ,
• Fabiane Valentini Francisqueti-Ferron 1   nAff6 ,
• Seda Cansu Yeniğün 7 ,
• Bianca Fernandes 1 ,
• Luis Monteiro Rodrigues 8 ,
• Rahima Gabulova 9 on behalf of

the PhysAgeNet (Network on Evidence-Based Physical Activity in Old Age)

Systematic Reviews volume  13 , Article number:  234 ( 2024 ) Cite this article

Metrics details

Cardiovascular diseases remain a leading global cause of mortality worldwide especially in older adults. Although it is known that regular exercise reduces cardiovascular diseases incidence, its effects on specific cardiovascular aging parameters considering the influence of sex and different exercise designs are still not fully understood. Therefore, this systematic review and meta-analysis aims to evaluate the effects of different physical exercise protocols on age-related cardiovascular outcomes in older adults.

This systematic review and meta-analysis will be reported in agreement with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. Articles will be eligible if they are randomized controlled trials with a primary objective of evaluating the chronic effects of exercise interventions on cardiovascular aging parameters. Search strategy will be performed from the inception to September 30th, 2023, in the following electronic databases: MEDLINE (Ovid), SCOPUS (Elsevier), Embase, Sport Discus (EBSCO), Cochrane Central Register of Controlled Trials (CENTRAL), and Web of Science Core Collection (Clarivate Analytics). Data will be extracted and managed through Research Electronic Data Capture (REDCap) software. The Tool for the assEssment of Study qualiTy and reporting in EXercise (TESTEX) will be used to assess the methodological quality of included studies. Additionally, the quality of the findings will be evaluated using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) profiler. Meta-analysis based on the random-effects model will be performed (if deemed suitable, considering the methodological and clinical heterogeneity of the studies) to estimate the effects of exercise training on cardiovascular aging variables (i.e., cardiac output; arterial stiffness; stroke volume; endothelial function; and carotid intima-media thickness). Heterogeneity will be assessed with the I 2 statistics, while the publication bias will be assessed based on Egger’s test.

To the best of our knowledge, this will be the first systematic review and meta-analysis to investigate the impact of sex and training protocols on the cardiovascular aging parameters. Moreover, the findings of this systematic review and meta-analysis will provide evidence for health professionals in the management of elderly patients in order to optimize the exercise prescription to face the cardiovascular alterations related to the aging process, considering the effects of different protocols according to sex.

Systematic review registration

PROSPERO CRD42023441015 .

Peer Review reports

Cardiovascular disease (CVD) is the leading cause of death worldwide. In 2020, nearly 20 million deaths worldwide (95% uncertainty interval (UI), 17.53–20.24 million) were due to CVD, accounting for 37% of deaths due to non-communicable diseases (NCDs) in people under 70 years of age [ 1 ]. Although the age-standardized prevalence of CVD has almost stabilized over the past decade, the crude prevalence has increased by 29.01% (95% UI, 27.73–30.38%), totaling 607.64 million (95% UI, 568.07–644.85 million) cases in 2020 [ 1 ]. This increase in crude prevalence can be explained by the increase in life expectancy, as age-related changes in the cardiovascular system are involved in the pathophysiology of all CVD [ 2 , 3 ].

Cardiac aging is characterized by a decrease in systolic/diastolic ventricular function, increased ventricular stiffness, cardiomyocyte hypertrophy, increased fibrosis, and other alterations [ 4 , 5 ]. These age-related cardiac changes ultimately lead to a decrease in stroke volume and cardiac output and can lead to coronary artery disease and other CVDs [ 4 , 5 ]. At the same time, vascular aging is characterized by changes in the functional and structural components of the arterial wall, manifested primarily in arterial stiffness, endothelial dysfunction, and intima thickness [ 6 , 7 ]. These age-related vascular changes are associated with the development of hypertension, atherosclerosis, stroke, heart failure, and other diseases [ 6 , 7 , 8 , 9 ]. With the older population expected to more than double by 2050 [ 10 ], global burden of CVD will also increase in the following years [ 11 ]. Therefore, strategies to reduce the impact of aging on the cardiovascular system are of paramount importance for the prevention and treatment of CVD as well as for reducing its health and economic consequences.

It is important to note that several environmental and/or lifestyle factors may affect cardiovascular aging [ 2 , 3 ]. Among lifestyle factors, regular participation in physical activity or exercise programs is well established and is inversely related to the incidence of CVD disease and mortality [ 1 ], including in older adults [ 12 ]. In this context, regular physical activity through participation in structured exercise programs (mainly aerobic and resistance training programs) is a key recommendation in several guidelines for the prevention and treatment of CVD [ 13 , 14 , 15 ] and for the health promotion and prevention/treatment of disease in older adults [ 16 ]. In relation to cardiovascular aging, there are several systematic reviews showing the effectiveness of exercise interventions on improving cardiac output [ 17 , 18 ], stroke volume [ 18 ], arterial stiffness [ 17 , 19 , 20 , 21 , 22 , 23 , 24 ], endothelial function [ 25 ], and carotid intima-media thickness [ 22 ]. However, only a few assessed the effects of exercise intervention in older individuals, and only on arterial stiffness [ 19 , 20 , 21 , 23 ], which is an important bias because aging process may affect cardiovascular adaptation to exercise [ 26 ]. In addition, these systematic reviews have included only a specific population (i.e., obese) [ 23 ] and/or have not assessed the effects of important mediating and moderating variables that may influence the effect of exercise training on cardiovascular aging parameters [ 19 , 20 , 21 , 23 ].

Among the mediating and moderating variables, the adequate representation of women, and the comparison between the sexes should be considered to minimize the under representation of women in cardiovascular disease randomized clinical trials (RCTs) [ 27 ], mainly in older age [ 28 ]. CVD is highly prevalent in women, and the burden of CVD in women has stagnated during the past decade [ 27 ]. In addition, the sex-specific mechanisms of cardiovascular aging and CVD remain poorly understood [ 27 ]. It appears that sex chromosomes and hormones influence cardiovascular function and cardiovascular risk factors throughout life [ 29 , 30 ], leading to poorer outcomes in women, particularly at older ages [ 31 ]. In this context, differences between women and men in the effects of physical exercise on cardiovascular outcomes have been reported [ 30 , 31 ], and sex/gender-specific studies are gaining increasing attention and highlighting crucial implications, including cardiac adaptation to exercise and CVD prevention/rehabilitation [ 31 , 32 ].

The characteristics of the exercise intervention, defined by the frequency, intensity, type, and time (FITT) principle, are another factor that should be considered when systematically reviewing the effects of exercise interventions on cardiovascular aging. For example, there is limited evidence from systematic reviews with meta-analyzes suggesting that exercise type (aerobic, resistance or combined) may influence the effects of exercise on cardiac output [ 17 ] and arterial stiffness [ 17 , 19 , 24 , 33 , 34 ], while exercise intensity (low-, moderate-, or high-intensity) may influence the effects on arterial stiffness [ 33 ] and endothelial function [ 25 ]. However, only one study has systematically reviewed the effect of exercise in older adults, and arterial stiffness was the only arterial aging outcome [ 19 ]. Furthermore, exercise type cannot be reduced to aerobic, resistance, or combined interventions. There is some evidence that other types of exercise such as stretching, yoga, and dance [ 21 , 35 , 36 ] improve several cardiovascular risk factors in older adults, but their effects on cardiovascular aging, as well as compared with other exercise modalities, remain to be determined. Indeed, the effects of all components of the FITT principle on the effect of exercise on cardiovascular aging parameters in older women and men also remain to be determined.

Our purpose is therefore to perform a systematic review with meta-analysis to assess the effect of exercise training compared with a control intervention on cardiovascular aging parameters (i.e., cardiac output, stroke volume, arterial stiffness, endothelial function, and carotid intima-media thickness) in older individuals. Our specific aim is to quantify the moderating effects of sex (women vs. men) and exercise training characteristics (i.e., FITT principle variables) on cardiovascular aging parameters, to help identify the optimal exercise dose (frequency, intensity, type and time) required to improve cardiovascular aging in older women and men.

This systematic review will be carried out and reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [ 37 ]. The present protocol is registered in the PROSPERO repository (CRD42023441015) and follows the PRISMA Protocols statement (Supplement 1) [ 38 ].

Eligibility criteria

The present review will include only full-text scientific documents of RCTs (i.e., peer-reviewed journal publications, non-peer-reviewed preprints, or PhD/MSc dissertations) investigating the effects of exercise intervention(s) on cardiovascular aging parameters in humans that were written in English, Portuguese, Spanish, Chinese, Polish, Slovak, Azerbaijani, or Turkish. The Population-Intervention-Comparator-Outcomes-Study design (PICOS) criteria will be used to select the studies eligible for inclusion in the review (Table  1 ).

We will include studies that assessed women and/or men (description of results separated by sex), healthy or with noncommunicable chronic diseases, with a mean age of ≥ 60 years (minimum age ≥ 50 years), that provided a detailed description of the participants (i.e., age, number or percentage of women/men, BMI, healthy status and comorbidities, and medication). The corresponding authors of studies including both men and women, in which participants’ characteristics and results were not presented separately by sex, will be asked to provide the data separately by sex.

Intervention

Studies should have any type of chronic exercise intervention (i.e., repeated exercise sessions during a short or long-term follow-up) to be included in the review. As defined by the American College of Sports Medicine [ 39 ], exercise comprises a planned, structured, and repetitive bodily movement done to improve or maintain one or more components of physical fitness and may include aerobic, resistance, and alternative exercise programs, such as Tai Chi, Qigong, dance, exergaming, or balance. Any organ-specific exercise intervention such as respiratory or pelvic muscle training as well as relaxation, meditation, or stretching/flexibility interventions will not be included. Studies assessing only the acute response to exercise or not describing the type, duration and frequency of exercise intervention will not be included.

To be included in the review, studies should have a control group with no treatment/intervention, usual care with no exercise interventions (e.g., only medical treatment or healthy habits counseling), sham intervention (e.g., stretching, etc.), or non-active interventions (e.g., health education, recreational activities).

The primary outcomes will be changes between pre- and post-intervention in cardiac output and arterial stiffness. The secondary outcomes will be changes between pre- and post-intervention in stroke volume, endothelial function, and carotid intima-media thickness. To be eligible for this review, studies should have measured, at baseline and at the end of intervention/control follow-up, any of the primary or secondary outcomes, by using a validated method.

Study design

We will include only RCTs.

Search strategy and study selection

Search strategy will be performed from the inception to August 20th, 2024, in the following electronic databases: MEDLINE (Ovid), SCOPUS (Elsevier), Embase, Sport Discus (EBSCO), Cochrane Central Register of Controlled Trials (CENTRAL), and Web of Science Core Collection (Clarivate Analytics). The reference list of the major textbooks, review articles, and all included studies will be hand searched in order to identify other potentially eligible studies. Finally, we will also undertake searches on online gray literature to minimize publication bias (i.e., Google Scholar). The search strategy will include a combination of terms for the key concepts “older adults,” “exercise,” “cardiovascular system,” and “clinical trial.” The full search strategy for MEDLINE and other databases are described in Table  2 and Supplement 2, respectively.

All manuscripts retrieved from each search database will be exported as “.ris” or “.nbiib” files. The files will be imported into Rayyan, a web-based application for systematic review screening ( https://rayyan.qcri.org/welcome ), and a single researcher will remove the duplicates and duplicates by using the application’s duplicate function. Then, at least two independent reviewers will screen the titles and abstracts of all studies for eligibility. All studies that meet the inclusion criteria, and those with insufficient information in the abstract, will be retrieved for full text screening by two independent reviewers. Disagreements between reviewers, both at abstract or full-text screening, will be discussed with another independent reviewer (E.G.C) to obtain consensus.

Data extraction and management

After study selection, all data will be extracted and managed by using REDCap. The data extraction form will be designed by the review authors, and data extraction will be piloted with a small number of studies (at least 10). If necessary, it will be modified prior to the data extraction of all studies. Data extraction for each individual study will be performed independently by two reviewers. Any disagreement in data extraction will be solved by consensus. If consensus is not achieved, a third reviewer will independently extract the data. The data extraction will include report characteristics (title, first author, study location, year of publication/availability), characteristics of the RCT (study design [parallel group, crossover, number of arms, etc.], total study duration, sequence generation, allocation sequence concealment, blinding, intention to treat, primary and secondary outcomes), participants’ characteristics (health status, sample size at beginning and end of the study, mean age, sex, and physical activity level), intervention(s) characteristics (all FITT variables), control intervention characteristics (type, duration, orientations), outcomes (primary and secondary), and moderators. Outcomes and moderators are detailed below.

In case of missing or unclear data, the authors of the specific study will be contacted by email, up to two times, to provide the required information. If there is no response within 4 weeks, the data we will be considered unobtainable. For data presented in graphs, we will contact the authors to obtain the data or use Web Plot Digitizer (if there is no response within 4 weeks).

The primary outcomes will be cardiac output (both at rest and/or during exercise) and arterial stiffness. The secondary outcomes will be stroke volume (both at rest and/or during exercise), endothelial function, and carotid intima-media thickness. The primary and secondary outcomes should be measured by a validated method, which include:

Cardiac output: acetylene rebreathing; bioreactance; cardiac MRI; CO 2 rebreathing; echocardiography; finger photoplethysmography; impedance cardiography; inert gas rebreathing; radionuclide ventriculography; or thoracic impedance

Arterial stiffness: aortic pulse wave velocity (PWV); carotid-femoral PWV; brachial-ankle PWV; cardio-ankle vascular index (CAVI); or waveform reflection (i.e., augmentation index, and/or augmentation index adjusted to 75 bpm)

Stroke volume: acetylene rebreathing; bioreactance; cardiac MRI; CO 2 rebreathing; echocardiography; finger photoplethysmography; impedance cardiography; inert gas rebreathing; radionuclide ventriculography; or thoracic impedance

Endothelial function: brachial artery flow-mediated dilation (FMD); radial artery FMD; or peripheral arterial tonometry by the reactive hyperemia index (RHI)

Carotid intima-media thickness: ultrasound data processing; or echo-tracking

Mean and standard deviation (SDs) at baseline and follow-up will be extracted from intervention(s) and control groups for each primary or secondary outcome. The effect of exercise on primary and secondary outcomes will be first examined without sex separation. Subgroup analyses by sex (women vs . men) will be then conducted to investigate the sex differences in outcomes. Any significant differences in the effects of exercise and physical activity on cardiovascular aging between males and females will be reported, and potential explanations for these differences will be explored.

The primary moderators of the present review will be the exercise intervention characteristics (i.e., FITT principle variables). Thus, we will extract information regarding exercise interventions’ frequency (i.e., number of sessions per week), intensity (i.e., absolute and/or relative objective and/or subjective parameters of exercise intensity), type (i.e., exercise modality(ies) performed), and duration (i.e., duration of the exercise sessions, duration of each exercise modality, and duration of the exercise program) as well as about the progression of these variables throughout the follow-up. To try to identify the optimal exercise dose (frequency, intensity, type and time) required to improve cardiovascular aging, we will conduct analyses for the following potential moderators: (1) frequency of exercise sessions, (2) exercise intensity, (3) type(s) of exercise(s), (4) exercise session(s) duration, (5) workload of exercise sessions (e.g., intensity X duration, metabolic costs…), (6) program duration, (7) density of exercise sessions per week (frequency X intensity X volume), (8) progression of exercise frequency/intensity/volume, (9) type of control group, and (10) use of behavioral interventions.

We will also extract information about individual characteristics that are used in previously published systematic reviews and meta-analyses and/or are hypothesized to modulate the effect size of exercise on cardiovascular aging. Possible moderator variables related to this will include the following: (1) population type (e.g., ethnicity, health status, type of comorbidity…), (2) mean age, (3) cardiovascular aging parameters at baseline, (4) fitness and/or physical activity level at baseline, (5) prevention level (e.g., primary, secondary, tertiary…), (6) education, and (7) socioeconomic status. Other possible moderators that will be extracted from the studies will be the type of publication (e.g., peer-reviews journal article, preprint, thesis, etc.) and sample size of intervention(s) and control group.

The above-mentioned moderator variables will only be included in the meta-analysis if sufficient studies are available (at least three studies for of each level of a categorical moderator description/analysis and at least eight studies for a meta-regression).

Assessment of study quality, risk of bias , and overall quality of evidence

The methodological quality of the included studies will be assessed by the Tool for the assEssment of Study qualiTy and reporting in EXercise (TESTEX) [ 40 ], which has 12 criteria (maximum score of 15 points) pertaining to study quality (i.e., eligibility criteria specified, randomization specified, allocation concealment, groups similar at baseline, and blinding of assessor for the primary outcome) and study reporting (i.e., outcome measures assessed in 85% of patients, intention-to-treat analysis, between-group statistical comparisons reported, point measures and measures of variability for all reported outcome measures, activity monitoring in control groups, relative exercise intensity remained constant, and exercise volume and energy expenditure). A higher score reflects better quality. Two reviewers will assess independently to rate each criterion, and inter-observer agreement will be determined using Kappa statistics [ 41 ]. In case of disagreement of rating, the agreement will be solved by two different reviewers. Studies will not be excluded based on their quality.

The overall quality of evidence will be reported using Grading of Recommendations Assessment, Development and Evaluation (GRADE) [ 42 ], which is structured in five domains: risk of bias, imprecision, inconsistency, indirectness, and publication bias. Each outcome will be assessed for the overall certainty in evidence and classified into four levels: high, moderate, low, or very low.

Data analysis

Statistical analyses will be performed using Comprehensive Meta-Analysis software (CMA, version 2.2.064, Biostat, NJ, USA). Descriptive data for each of the individual studies will be reported as means ± standard deviations (SD). Continuous outcomes will be used to analyze as post-intervention changes from baseline using mean difference (MD). Standardized mean differences (SMD, i.e., mean weighted difference and its 95% confidence intervals [CI]) will be used if the outcomes were collected using different measurements across studies. Data expressed using the standard error of the mean (SEM) will be first converted to SD. For binary outcomes, the odds ratio (OR) and its corresponding 95% CI will be calculated. The Mantel–Haenszel method will be employed for pooled estimates in binary outcomes, with the choice between a fixed-effect or random-effects model based on the level of heterogeneity. Pooled estimates of the effect of exercise training on cardiac aging outcomes (i.e., cardiac output, arterial stiffness, stroke volume, endothelial function, and carotid intima-media thickness) will be obtained using a random-effects model with significance set at P  < 0.05 (two-tailed), as we expected heterogeneity in the methodology of the studies. The random-effects model will use the DerSimonian and Laird method to estimate the heterogeneity variance ( τ 2 ), which allows for the assumption that the true effects might vary across studies.

Sub-group analyses, including sensitivity and meta-regression approaches, will be performed to explore potential moderators, such as sex differences, baseline fitness levels, and study design factors, on cardiovascular aging outcomes. Meta-regression will be conducted using a mixed-effects model, where the heterogeneity variance will be estimated using restricted maximum likelihood (REML). Covariates, including age, gender, duration of intervention, and baseline health status, will be examined as potential moderator variables Heterogeneity will be assessed using the I 2 statistics where I 2 values between 25 and 50% represents small amounts of inconsistency, 50 and 75% represents medium inconsistency, and above 75% represents large amounts of inconsistency [ 43 ]. Cochran’s Q test will also be performed to test for heterogeneity. If substantial heterogeneity is detected, sources of heterogeneity will be explored through subgroup analyses and meta-regression.

Publication bias will be examined by visual inspection of the different funnel plots’ asymmetry. The Egger’s test [ 44 ] will indicate the presence of a significant publication bias. Duval and Tweedie’s trim and fill procedure will be applied to estimate the effect of publication bias on the results [ 43 ]. Sensitivity analyses will be conducted to assess the robustness of the results by excluding studies with high risk of bias, studies with extreme effect sizes, or studies contributing disproportionately to the heterogeneity.

Previous systematic reviews have investigated the effects of exercise interventions on several cardiovascular aging parameters, including cardiac output [ 17 , 18 ], stroke volume [ 18 ], arterial stiffness [ 17 , 19 , 20 , 21 , 22 , 23 , 24 ], endothelial function [ 25 ], and carotid intima-media thickness [ 22 ]. However, in older individuals, there are systematic effects assessing the effect of exercise interventions only on arterial stiffness [ 19 , 20 , 21 , 23 ]. Although these systematic reviews found positive effects of exercise interventions on arterial stiffness of older individuals, they did not investigate if the benefits are different between sexes nor investigated the impact of most moderators that will be investigated in the present review.

According to our knowledge, this is the first systematic review with meta-analysis aiming to assess the effects of chronic exercise on different cardiovascular aging parameters in older individuals. In addition, it will also be the first to investigate the role of sex (men vs. women) and training characteristics (i.e., frequency, intensity, type and time) on the effects of chronic exercise effects on cardiovascular aging. Thus, this review will contribute to the evidence base of exercise on cardiovascular aging, which may help the development of specific exercise recommendations (i.e., frequency, intensity, type and time) to improve cardiovascular aging according to sex.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Tsao CW, Aday AW, Almarzooq ZI, et al. Heart Disease and Stroke Statistics—2023 update: a report from the American Heart Association. Circulation. 2023;147(8):e93–621. https://doi.org/10.1161/CIR.0000000000001123 .

Article   PubMed   Google Scholar  

Fontana L. Interventions to promote cardiometabolic health and slow cardiovascular ageing. Nat Rev Cardiol. 2018;15(9):566–77. https://doi.org/10.1038/s41569-018-0026-8 .

Gude NA, Broughton KM, Firouzi F, Sussman MA. Cardiac ageing: extrinsic and intrinsic factors in cellular renewal and senescence. Nat Rev Cardiol. 2018;15(9):523–42. https://doi.org/10.1038/s41569-018-0061-5 .

Nakayama H, Nishida K, Otsu K. Macromolecular degradation systems and cardiovascular aging. Circ Res. 2016;118(10):1577–92. https://doi.org/10.1161/CIRCRESAHA.115.307495 .

Pandey A, Kraus WE, Brubaker PH, Kitzman DW. Healthy aging and cardiovascular function: invasive hemodynamics during rest and exercise in 104 healthy volunteers. JACC Heart Fail. 2020;8(2):111–21. https://doi.org/10.1016/j.jchf.2019.08.020 .

Laurent S. Defining vascular aging and cardiovascular risk. J Hypertens. 2012;30. https://journals.lww.com/jhypertension/Fulltext/2012/06001/Defining_vascular_aging_and_cardiovascular_risk.2.aspx .

Climie RE, Alastruey J, Mayer CC, et al. Vascular ageing - moving from bench towards bedside. Eur J Prev Cardiol Published online. February 2023. https://doi.org/10.1093/eurjpc/zwad028 .

Article   Google Scholar  

Alexander Y, Osto E, Schmidt-Trucksäss A, et al. Endothelial function in cardiovascular medicine: a consensus paper of the European Society of Cardiology Working Groups on Atherosclerosis and Vascular Biology, Aorta and Peripheral Vascular Diseases, Coronary Pathophysiology and Microcirculation, and Thr. Cardiovasc Res. 2021;117(1):29–42. https://doi.org/10.1093/cvr/cvaa085 .

Chirinos JA, Segers P, Hughes T, Townsend R. Large-artery stiffness in health and disease: JACC State-of-the-Art Review. J Am Coll Cardiol. 2019;74(9):1237–63. https://doi.org/10.1016/j.jacc.2019.07.012 .

Article   PubMed   PubMed Central   Google Scholar  

United Nations. World population ageing 2019 (st/esa/ser. a/444). Department of Economic and Social Affairs PD, editor New York, USA 2020. Published online 2020. Accessed April 17, 2023. https://www.un.org/development/desa/pd/news/world-population-ageing-2019-0 .

Global Burden of Disease Collaborative Network. Global Burden of Disease Study 2019 (GBD 2019) Results.; 2020. Accessed April 17, 2023. http://ghdx.healthdata.org/gbd-results-tool .

Cunningham C, O’ Sullivan R, Caserotti P, Tully MA. Consequences of physical inactivity in older adults: a systematic review of reviews and meta-analyses. Scand J Med Sci Sports. 2020;30(5):816–27. https://doi.org/10.1111/sms.13616 .

Visseren FLJ, MacH F, Smulders YM, et al. 2021 ESC guidelines on cardiovascular disease prevention in clinical practice. Eur Heart J. 2021;42(34):3227–337. https://doi.org/10.1093/eurheartj/ehab484 .

Williams B, Mancia G, Spiering W, et al. 2018 ESC/ESH guidelines for the management of arterial hypertension. Eur Heart J. 2018;39(33):3021–104. https://doi.org/10.1093/eurheartj/ehy339 .

Ambrosetti M, Abreu A, Corrà U, et al. Secondary prevention through comprehensive cardiovascular rehabilitation: From knowledge to implementation. 2020 update. A position paper from the Secondary Prevention and Rehabilitation Section of the European Association of Preventive Cardiology. Eur J Prev Cardiol. 2021;28(5):460–95. https://doi.org/10.1177/2047487320913379 .

Izquierdo M, Merchant RA, Morley JE, et al. International Exercise Recommendations in Older Adults (ICFSR): Expert Consensus Guidelines. Journal of Nutrition, Health and Aging. 2021;25(7):824–53. https://doi.org/10.1007/s12603-021-1665-8 .

Zhang Y, Qi L, Xu L, et al. Effects of exercise modalities on central hemodynamics, arterial stiffness and cardiac function in cardiovascular disease: systematic review and meta-analysis of randomized controlled trials. PLoS One. 2018;13(7). https://doi.org/10.1371/journal.pone.0200829 .

Astorino TA, Causer E, Hazell TOMJ, Arhen BB, Gurd BJ. Change in central cardiovascular function in response to intense interval training: a systematic review and meta-analysis. Med Sci Sports Exerc. 2022;54(12):1991–2004. https://doi.org/10.1249/MSS.0000000000002993 .

Lan YS, Khong TK, Yusof A. Effect of exercise on arterial stiffness in healthy young, middle-aged and older women: a systematic review. Nutrients. 2023;15(2). https://doi.org/10.3390/nu15020308 .

Manojlović M, Protić-Gava B, Maksimović N, et al. Effects of combined resistance and aerobic training on arterial stiffness in postmenopausal women: a systematic review. Int J Environ Res Public Health. 2021;18(18). https://doi.org/10.3390/ijerph18189450 .

Kato M, Green FN, Hotta K, et al. The efficacy of stretching exercises on arterial stiffness in middle-aged and older adults: a meta-analysis of randomized and non-randomized controlled trials. Int J Environ Res Public Health. 2020;17(16):1–15. https://doi.org/10.3390/ijerph17165643 .

Huang C, Wang J, Deng S, She Q, Wu L. The effects of aerobic endurance exercise on pulse wave velocity and intima media thickness in adults: a systematic review and meta-analysis. Scand J Med Sci Sports. 2016;26(5):478–87. https://doi.org/10.1111/sms.12495 .

Montero D, Roberts CK, Vinet A. Effect of aerobic exercise training on arterial stiffness in obese populations: a systematic review and meta-analysis. Sports Med. 2014;44(6):833–43. https://doi.org/10.1007/s40279-014-0165-y .

Sardeli AV, GáSpari AF, Chacon-Mikahil MP. Acute, short-, and long-term effects of different types of exercise in central arterial stiffness: a systematic review and meta-analysis. J Sports Med Phys Fitness. 2018;58(6):923–32. https://doi.org/10.23736/S0022-4707.17.07486-2 .

Ashor AW, Lara J, Siervo M, et al. Exercise modalities and endothelial function: a systematic review and dose–response meta-analysis of randomized controlled trials. Sports Med. 2015;45(2):279–96. https://doi.org/10.1007/s40279-014-0272-9 .

Ciolac EG, Roberts CK, Rodrigues Da Silva JM, Guimarães GV. Age affects exercise-induced improvements in heart rate response to exercise. Int J Sports Med. 2014;35(5). https://doi.org/10.1055/s-0033-1351332 .

Vogel B, Acevedo M, Appelman Y, et al. The Lancet women and cardiovascular disease Commission: reducing the global burden by 2030. Lancet. 2021;397(10292):2385–438. https://doi.org/10.1016/S0140-6736(21)00684-X .

Daitch V, Turjeman A, Poran I, et al. Underrepresentation of women in randomized controlled trials: a systematic review and meta-analysis. Trials. 2022;23(1). https://doi.org/10.1186/s13063-022-07004-2 .

Gerdts E, Sudano I, Brouwers S, et al. Sex differences in arterial hypertension: a scientific statement from the ESC Council on Hypertension, the European Association of Preventive Cardiology, Association of Cardiovascular Nursing and Allied Professions, the ESC Council for Cardiology Practice, and the ESC Working Group on Cardiovascular Pharmacotherapy. Eur Heart J. 2022;43(46):4777–88. https://doi.org/10.1093/eurheartj/ehac470 .

Seals DR, Nagy EE, Moreau KL. Aerobic exercise training and vascular function with ageing in healthy men and women. J Physiol. 2019;597(19):4901–14. https://doi.org/10.1113/JP277764 .

Vaccarezza M, Papa V, Milani D, et al. Sex/gender-specific imbalance in CVD: could physical activity help to improve clinical outcome targeting CVD molecular mechanisms in women? Int J Mol Sci. 2020;21(4). https://doi.org/10.3390/ijms21041477 .

Ghisi GL de M, Kin R. SM, Price J, et al. Women-focused cardiovascular rehabilitation: an International Council of Cardiovascular Prevention and Rehabilitation clinical practice guideline ∗ . Canadian Journal of Cardiology. 2022;38(12):1786–1798. https://doi.org/10.1016/j.cjca.2022.06.021 .

Ashor AW, Lara J, Siervo M, Celis-Morales C, Mathers JC. Effects of exercise modalities on arterial stiffness and wave reflection: a systematic review and meta-analysis of randomized controlled trials. PLoS One. 2014;9(10). https://doi.org/10.1371/journal.pone.0110034 .

Lopes S, Afreixo V, Teixeira M, et al. Exercise training reduces arterial stiffness in adults with hypertension: a systematic reviewand meta-analysis. J Hypertens. 2021;39(2):214–22. https://doi.org/10.1097/HJH.0000000000002619 .

Brandani JZ, Mizuno J, Ciolac EG, Monteiro HL. The hypotensive effect of Yoga’s breathing exercises: a systematic review. Complement Ther Clin Pract. 2017;28. https://doi.org/10.1016/j.ctcp.2017.05.002 .

Rodrigues-Krause J, Farinha JB, Krause M, Reischak-Oliveira Á. Effects of dance interventions on cardiovascular risk with ageing: Systematic review and meta-analysis. Complement Ther Med. 2016;29:16–28. https://doi.org/10.1016/j.ctim.2016.09.004 .

Page MJ, McKenzie JE, Bossuyt PM, The PRISMA, et al. statement: an updated guideline for reporting systematic reviews. The BMJ. 2020;2021:372. https://doi.org/10.1136/bmj.n71 .

Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Revista Espanola de Nutricion Humana y Dietetica. 2016;20(2):148–60. https://doi.org/10.1186/2046-4053-4-1 .

ACSM’s guidelines for exercise testing and prescription, 11th Edition. ProtoView. 2021;2021(8).

Smart NA, Waldron M, Ismail H, et al. Validation of a new tool for the assessment of study quality and reporting in exercise training studies: TESTEX. Int J Evid Based Healthc. 2015;13(1):9–18. https://doi.org/10.1097/XEB.0000000000000020 .

Viera A, Garrett J. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37:360–3.

PubMed   Google Scholar  

Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94. https://doi.org/10.1016/j.jclinepi.2010.04.026 .

Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–58. https://doi.org/10.1002/sim.1186 .

Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315(7109):629. https://doi.org/10.1136/bmj.315.7109.629 .

Download references

Acknowledgements

This publication is based upon work from EU COST Action CA20104 – Network on evidence-based physical activity in old age (PhysAgeNet), supported by COST (European Cooperation in Science and Technology).

The fees for the publication of the protocol are supported by COST Action CA20104. EGC was supported by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq #310572/2021–5).

Author information

Fabiane Valentini Francisqueti-Ferron

Present address: Medical School, São Paulo State University (Unesp), Botucatu, Brazil

Authors and Affiliations

Exercise and Chronic Disease Research Laboratory, Department of Physical Education, School of Sciences, São Paulo State University (Unesp), Av. Eng. Luiz Edmundo Carrijo Coube 14-01, Bauru, Sao Paulo, 17033-360, Brazil

Emmanuel Gomes Ciolac, Vanessa Teixeira do Amaral, Artur Junio Togneri Ferron, Fabiane Valentini Francisqueti-Ferron & Bianca Fernandes

Faculty of Medicine, Comenius University in Bratislava, Institute of Hygiene, Bratislava, Slovakia

Jana Babjakova

Department of Physiotherapy, LUNEX University, International University of Health, Exercise & Sports S.A, Differdange, Luxembourg

Raphael Martins de Abreu

LUNEX ASBL Luxembourg Health & Sport Sciences Research Institute, Differdange, Luxembourg

Faculty of Physical Culture, Gdansk University of Physical Education and Sport, Gdansk, Poland

Su-Jie Mao, Guoping Qian, Bartlomiej Wrzesinski & Zbigniew Ossowski

Faculty of Health Sciences, Akdeniz University, Kumluca , Antalya, Turkey

Seda Cansu Yeniğün

Research Center for Biosciences & Health Technologies (CBIOS), Universidade Lusófona, Lisbon, Portugal

Luis Monteiro Rodrigues

Department of Family Medicine, Azerbaijan Medical University, Baku, Azerbaijan

Rahima Gabulova

You can also search for this author in PubMed   Google Scholar

Contributions

EGC coordinated and supervised all activities and participated in the study conception and design, manuscript drafting, and reviewing. JB participated in the study conception and design, manuscript drafting, and reviewing. RMA, SJM, GQ, and VTA participated in the study conception and design and manuscript drafting. BW, AJTF, ZO, FVFF, SCY, and BF participated in the study conception and manuscript drafting. LMR and RG participated in the study conception and manuscript reviewing. All authors read and approved the final version of the manuscript.

Corresponding author

Correspondence to Emmanuel Gomes Ciolac .

Ethics declarations

Ethics approval and consent to participate.

This review does not require approval from an Ethics Committee and any consent to participate.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2. , rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Ciolac, E.G., Babjakova, J., de Abreu, R.M. et al. Role of sex and training characteristics on exercise effects on cardiovascular aging: protocol for a systematic review with meta-analysis of randomized trials. Syst Rev 13 , 234 (2024). https://doi.org/10.1186/s13643-024-02644-8

Download citation

Received : 24 August 2023

Accepted : 22 August 2024

Published : 14 September 2024

DOI : https://doi.org/10.1186/s13643-024-02644-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Cardiovascular system
• Health promotion
• Physical activity

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Methodology
• Open access
• Published: 10 September 2024

Developing a process for assessing the safety of a digital mental health intervention and gaining regulatory approval: a case study and academic’s guide

• Rayan Taher 1 ,
• Charlotte L. Hall 2 ,
• Aislinn D Gomez Bergin 2 , 3 ,
• Neha Gupta 4 ,
• Clare Heaysman 5 ,
• Pamela Jacobsen 6 ,
• Thomas Kabir 7 ,
• Nayan Kalnad 4 ,
• Jeroen Keppens 8 ,
• Che-Wei Hsu 9 ,
• Philip McGuire 10 ,
• Emmanuelle Peters 11 ,
• Sukhi Shergill 12 ,
• Daniel Stahl 13 ,
• Ben Wensley Stock 14 &
• Jenny Yiend   ORCID: orcid.org/0000-0002-1967-6292 1  

Trials volume  25 , Article number:  604 ( 2024 ) Cite this article

185 Accesses

Metrics details

The field of digital mental health has followed an exponential growth trajectory in recent years. While the evidence base has increased significantly, its adoption within health and care services has been slowed by several challenges, including a lack of knowledge from researchers regarding how to navigate the pathway for mandatory regulatory approval. This paper details the steps that a team must take to achieve the required approvals to carry out a research study using a novel digital mental health intervention. We used a randomised controlled trial of a digital mental health intervention called STOP (Successful Treatment of Paranoia) as a worked example.

The methods section explains the two main objectives that are required to achieve regulatory approval (MHRA Notification of No Objection) and the detailed steps involved within each, as carried out for the STOP trial. First, the existing safety of digital mental health interventions must be demonstrated. This can refer to literature reviews, any feasibility/pilot safety data, and requires a risk management plan. Second, a detailed plan to further evaluate the safety of the digital mental health intervention is needed. As part of this we describe the STOP study’s development of a framework for categorising adverse events and based on this framework, a tool to collect adverse event data.

We present literature review results, safety-related feasibility study findings and the full risk management plan for STOP, which addressed 26 possible hazards, and included the 6-point scales developed to quantify the probability and severity of typical risks involved when a psychiatric population receives a digital intervention without the direct support of a therapist. We also present an Adverse Event Category Framework for Digital Therapeutic Devices and the Adverse Events Checklist—which assesses 15 different categories of adverse events—that was constructed from this and used in the STOP trial.

Conclusions

The example shared in this paper serves as a guide for academics and professionals working in the field of digital mental health. It provides insights into the safety assessment requirements of regulatory bodies when a clinical investigation of a digital mental health intervention is proposed. Methods, scales and tools that could easily be adapted for use in other similar research are presented, with the expectation that these will assist other researchers in the field seeking regulatory approval for digital mental health products.

Peer Review reports

The field of digital mental health interventions (DMHIs) has followed an exponential growth trajectory in recent years [ 1 ]. DMHIs typically involve mental health interventions, such as cognitive behavioural therapy, delivered via digital technologies, such as smartphones, and can either be completed as self-directed interventions or blended alongside synchronous (e.g., face-to-face or videoconference) or asynchronous (e.g., email or text message) clinical support [ 2 ]. The main benefit of these interventions is delivering evidence-based care to a large number of people with limited clinical resources [ 3 ]. While the evidence base has increased significantly, the adoption of these interventions within health and care services has been slowed by several challenges, including a lack of knowledge from researchers regarding how to navigate the pathway for mandatory regulatory approval. In the UK, DMHIs must meet the standard of evidence set by the National Institute of Health and Care Excellence (NICE) for adoption within the National Health Service (NHS) [ 4 ]. For DMHIs that are developed to diagnose, prevent, monitor, treat, or alleviate a mental health condition, this may include regulation as a “Software as a Medical Device” (SaMD) by the Medicines and Healthcare products Regulatory Agency (MHRA) [ 5 ]. The regulatory process ensures that devices used within the health and social care context are safe and effective.

In some cases, research will involve digital therapeutics that are already in use and carry a CE or UKCA mark. In this case, the therapeutic’s safety and effectiveness has already been established (and is maintained either through self-certification by the manufacturer or, for higher risk devices, through the use of a “Notified Body”: a government-approved organisation that ensures the device continues to conform to the required standards). However, early-stage digital therapeutics will not yet bear a CE/UKCA mark and are therefore required to obtain a specific form of regulatory approval from the MHRA (called “Notification of No Objection”; NoNO) before being used in research, in addition to the usual ethical approvals [ 6 ]. The NoNO regulatory process requires that safety and effectiveness data collection are the primary purpose of a clinical investigation, with the overall aim being to establish whether the benefits of the device outweigh its risks. This places a number of constraints and requirements upon how researchers design their investigations and write their protocols, the most obvious being that rigorous safety assessment is paramount. The present paper is intended to help academics who are interested in digital therapeutics, but unfamiliar with medical device safety assessment, to navigate a course through this complex regulatory field.

Although the research proposal for which NoNO is sought will, as already explained, need to have safety as a primary outcome, obtaining NoNO also requires the research team to demonstrate the safety of their device before the proposed investigation can be approved [ 6 ]. To understand this apparent contradiction, it is crucial to appreciate that safety assessment is considered an inherently iterative process: preliminary safety data must be presented in order to justify collecting more detailed safety data. This can be done by providing a summary of the existing device safety information using all possible sources (e.g. prototypes, user testing, pilot or feasibility data, qualitative information); a risk management plan (identifying all possible hazards, their potential impact and mitigations) and a detailed plan for safety assessment in the proposed clinical investigation (such as collecting and assessing any untoward medical occurrences [ 7 ], usually called adverse events (AEs)) [ 6 ].

However, researchers investigating DMHIs face specific challenges when proposing a safety assessment plan. Notably, MHRA guidance was developed in consideration of medical devices used in clinical contexts such as surgery and pharmacological interventions and was not designed to accommodate the unique safety considerations relevant to DMHIs. Additionally, the guidelines used in research for assessing the safety of DMHIs are borrowed from the medical and pharmaceutical fields, such as the “International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use—Good Clinical Practice” (ICH-GCP) guidelines [ 8 ]. These medical guidelines do not transfer well to assessing the safety of both digital and non-digital psychological interventions because of the fundamental differences between pharmacological and psychological approaches to treatment [ 8 ]. For example, biological responses to medicines usually occur rapidly and can be objectively measured, whereas psychological responses to therapy rely heavily on patients’ self-reported symptoms, can be hard to disentangle from other contextual factors, and intervention effects can take days, weeks or even months to emerge. As others have also identified, using medical definitions and processes to assess the safety of non-medical interventions such as DMHIs and behaviour change interventions can be unhelpful. It can overcomplicate the process of safety assessment, and lead to missing important harms [ 8 , 9 , 10 ].

These concerns have already been raised in trials assessing the safety of behaviour change interventions [ 8 ]. For example, in a qualitative study on recording harms in RCTs for behaviour change interventions, experts emphasised the need for harm recording to be proportionate and focused on harms that are plausibly linked (i.e. related) to the intervention under study [ 34 ]. It is likely that medical processes are being used to assess the safety of DMHIs, because there are no regulatory or standard safety assessment processes in place for face-to-face mental health interventions [ 9 ]. This is surprising given that most adverse events/side effects are common to both face-to-face mental health interventions and DMHIs (e.g. short-term deterioration, novel symptoms, and non-response) [ 10 , 11 , 12 ]. The one area that differs is, of course, technical and device-related harms.

Two recent reviews found that the identification and categorisation of AEs in DMHI trials was inconsistent and often inadequate [ 3 , 11 ]. This was similar to findings of a review on safety assessment in non-pharmacological psychological, behavioural and lifestyle interventions [ 8 ]. It is essential that harmonised standards tailored specifically to the needs of DMHIs are developed. Support mechanisms can then be implemented to assist manufacturers and researchers to understand and adhere to these guidelines . In the absence of these, the purpose of this paper is to share a worked example of how our clinical trial team successfully applied and received the MHRA’s NoNO for STOP (Successful Treatment for Paranoia).

STOP is a mobile app DMHI that uses Cognitive Bias Modification for paranoia (CBM-pa) to reduce symptoms of paranoia [ 12 , 13 ]. STOP consists of 12 weekly sessions of about 40 min each. In each session, the user is presented with 40 ambiguous scenarios that could be interpreted in a paranoid manner. Users are then guided to reevaluate each scenario in a non-paranoid way by completing words and answering questions designed to suggest alternative meanings. The goal is to gradually retrain the brain to assume non-paranoid meanings of ambiguous situations that occur in daily life, which has been shown to reduce paranoid symptoms. More information about STOP and its development is provided elsewhere [ 13 ]. Using STOP as our example, we aimed to provide valuable insights to other research teams undertaking clinical investigations of DMHIs, particularly those requiring regulatory (e.g., MHRA) approval and guidance in the assessment of safety in DMHI research.

Participants

The work presented in this paper was collaboratively completed by academics and clinicians in the field of digital mental health, an expert regulatory consultant, device manufacturers (Avegen), a clinical trials unit at a university, and representatives from an organisation working with experts by experience (the McPin Foundation). Participants varied at each stage; more detail is provided below per task. See Appendix A for the full list of participants.

To assess the safety of STOP (ISRCTN17754650) and obtain the MHRA’s NoNO the STOP team needed to achieve two main objectives:

Demonstrate existing safety.

Evaluate safety within the proposed research for which approval was being sought.

Demonstrating existing safety

This objective was achieved by completing three separate tasks; an empirical feasibility study; relevant literature searches and the creation of a comprehensive Risk Management Plan.

Feasibility study

The research team needed to present current safety data relevant to STOP such as previous publications, feasibility or pilot studies from the same or similar devices/ interventions. To achieve this, the team referred to a previously conducted study that had assessed the intervention’s feasibility and safety as a desktop intervention [ 12 ]. The feasibility study included two arms: treatment (CBM-pa which is a 6-session version of the therapeutic intervention used in STOP but delivered using a desktop computer) and an active control (a version of the same 6 session desktop programme with the same design and format as CBM-pa, except the content was neutral and should not trigger paranoid thoughts) [ 12 ]. CBM-pa works in the same way as STOP (see “ Background ”) by presenting users with a scenario that could be interpreted in a paranoid way and then, using word tasks and questions, helping participants to interpret the scenario in a nonparanoid, benign way [ 12 ]. Sixty-three outpatients with clinically significant paranoia participated in the feasibility study and were randomised to either the treatment or control group [ 12 ].

The feasibility study assessed safety by measuring whether presenting participants with these potentially paranoia-inducing scenarios was distressing or provoking for them, using visual analogue scales (VAS) completed before and after every session and measuring state anxiety, sadness, paranoia and friendliness. These data were used as proxy safety data relevant to STOP because STOP uses the same content and procedures as CBM-pa but delivered in a different format (mobile app vs. desktop) and the sample size of the feasibility studies are usually small [ 12 , 13 ].

Literature searches and regulatory databases

Second, three members of the research team (RT, CH, JY) conducted two literature searches to identity any published safety data that might be relevant to STOP or any equivalent intervention. These literature reviews are different to those used in academia and thus follow a different structure [ 14 ]. The team worked with an independent regulatory consultant on these to make sure we followed industry and regulatory standards. See Appendix B for more details around the methodology used in these two separate literature reviews. As part of this review, FDA databases were also searched for similar devices and any reported adverse events.

Risk management plan

The risk management plan was created collaboratively by key members of the STOP trial academic team and other project stakeholders, including members from a Lived Experience Advisory Panel, members of the software manufacturer and a regulatory consultant (see Appendix A, column 1, for full details).

Under this step, and by using the knowledge that arose from the feasibility study and the review, we developed a risk management plan. This step is required to demonstrate to regulators that the research team has listed all possible hazards, documented what harms might result from each hazard and identified actions or changes that will mitigate every risk entry as far as possible. Regulators require a comprehensive risk analysis specific to the product, showing a clear understanding of the following stepwise process: hazard, harm, initial risk rating, risk controls/ mitigations, revised risk ratings, identification of residual risks and final demonstration that the expected product benefits outweigh the identified residual risks, which should have been reduced as far as possible. In STOP’s case, a residual risk matrix (likelihood × severity) demonstrated that there were no residual medium or high risks.

The risk management plan described above was implemented by carrying out the following activities:

Hazard identification

To develop a risk management plan, the team first needed to develop a list of all potential hazards that participants taking part in the trial could be exposed to and articulate the harms that could result. Based on ISO 14971, which is a standard for risk management for medical devices, hazards are defined as “a potential source of harm” and harms are defined as “injury or damage to the health of people, or damage to property or the environment” [ 15 ]. This was done during a 1-h consensus meeting with an expert regulatory consultant, two representatives from the manufacturer, two clinicians, two academics, and one representative of a user organisation. During the consensus meeting all participants brainstormed possible hazards and articulated, through discussion, the harm it could lead to. The meeting resulted in a comprehensive spreadsheet of hazards and corresponding harms. The spreadsheet was compiled and circulated for members to review, revise and populate with any further suggestions.

Hazard analysis

After identifying hazards, a hazard analysis needs to be performed. While manufacturers may be experienced in providing this for the technical side of their products, the majority of hazards for DMHIs will be related to clinical risks. Most manufacturers will be unable to assess these and will require the clinical and academic team to become conversant with applying and interpreting risk assessment procedures. To this end, the STOP clinical and academic team received training in risk assessment from a regulatory consultant who worked with them to implement the process outlined below.

The first step in a hazard analysis is to quantify the probability (likelihood) and severity (impact) of each identified hazard. First, one must assess the probability that each hazard will lead to the specified harm. One must assume that the hazard has occurred and then ask oneself “how likely is the harm to now happen?”. Some of these may be fairly standard assessments or known within the digital industry, for example, should a participant stare at the screen for longer than advised, how likely is it that they will experience physical side effects such as eyestrain, fatigue or headache? However, in many cases nuanced clinical judgements are required to make this assessment. For example, how likely is it that the participant’s condition will worsen in the short term as a result of engaging with the content of the therapy? Second, one must assess the severity and impact, should that harm occur. For example, were eye strain, fatigue or headache to occur as a result of using the device how severe could those effects be at their worst? Severity in risk assessment can be operationalised in these terms:

The duration of harmful effects.

The level of intervention or support needed in response to the effects.

The possibility and extent of any long-lasting or permanent impact.

Central to the STOP hazard analysis was the need to create bespoke probability and severity scales relevant to the clinical therapeutic context. These were carefully devised by consensus discussion between the regulatory expert consultant and members of the clinical academic team to agree the most appropriate exact thresholds and wording for both dimensions. The application of the preliminary hazard analysis for STOP was a quantitative assessment of each individual risk entry (i.e., hazard and corresponding harm) against the criteria for probability and severity outlined above. The product of these two scores yields a “risk score”. It is important to note that these risk scores were based on expert consensus estimations derived from their knowledge of the literature and field experience. In line with standard practice in the field of risk analysis, no formal validation was conducted.

Risk control and re-evaluation

In common with all risk assessments, the next stage was to work through each of the identified risks outlining all the “risk control” actions (i.e., mitigations) that could be taken to reduce the identified risk to participants as far as possible. Each risk is then re-evaluated in terms of its probability and severity yielding a revised (“post risk control”) risk score. Finally, a risk acceptability management plan is implemented where various actions are specified for any residual risks that cannot be reduced any further, for example adding “warnings” and “cautions” to device details and labelling. These serve to alert the user to important residual risks that cannot be addressed in any other way. One residual risk that is common to mental health interventions is the possibility of users being distressed when presented with information that relates to the mental health condition they are living with.

In the case of STOP, residual risks were managed using a variety of processes, depending on the nature of the risk. This included, for example, warnings (e.g. “If negative feelings or symptoms worsen as result of using this app for more than a day, please contact your support team and cease use of the STOP app until advised further”), fortnightly check-in phone calls with researchers; a dedicated, in-app 24-h study helpline number and use of an inbuilt mood-tracking algorithm to trigger researcher alerts. Further details of these are provided below.

Evaluating the safety of a DMHI within the proposed research study

After demonstrating the current safety of STOP as seen in section A of this paper, the team needed to demonstrate how the safety of STOP would be assessed in the proposed clinical trial. Any assessment of safety will involve collecting data about the occurrence of adverse events, both related and unrelated, to the trial. For STOP, we planned to do this proactively and regularly in line with recent recommendations [ 11 ]. We therefore needed an overarching framework to organise and classify the large quantity of adverse event information that was likely given the larger sample size (273) and length of time each participant would spend in the trial (6 months). We therefore devised an adverse event classification framework as follows.

AE classification framework development

Literature review

First a brief narrative literature review (conducted within the limited, 60-day time window of the regulatory approval pathway) was carried out to identify any publications in the last 10 years [03/23/2012–03/23/2022] that discussed how adverse events were assessed, coded or categorised in psychiatric populations receiving psychological interventions (digital or non-digital). We combined the categories and definitions identified in the outputs of the literature review to create a first working draft of a classification framework.

Expert consultation

We then carried out an expert consultation involving key members of the STOP trial academic team and other project stakeholders. This included the McPin Foundation, key members of the software manufacturer, a regulatory consultant and key external members of the trial committees. Full details are given in Appendix A. The classification framework working draft was shared with this group to review and comment upon. The group was invited to edit, remove or add categories or examples. Where any conflicts or differences of opinion emerged, these were resolved by group discussion and consensus using virtual meeting and/or email communications. This resulted in a finalised ‘Adverse Event Category Framework for Digital Therapeutic Devices’ which is provided in the “ Results ” section.

Proposed safety plan for the trial

To appropriately and sufficiently assess the safety of a DMHI, regulators expect to see safety positioned as the primary outcome in the proposed study, alongside efficacy. In the STOP trial, this was done by adjustment of the protocol in three ways.

First, we built-in proactive, fortnightly collection of AE data for each participant throughout the entire trial (including throughout the follow-up period) in both arms, using a custom designed checklist based upon the Adverse Events Category Framework for digital devices described above. Even though collecting AE data in both arms is resource-intensive, it is important, as shown in a recent systematic review [ 11 ]. These data enable researchers to statistically compare the prevalence of AEs in the treatment and control arms, allowing for conclusions about the safety of the DMHI. The checklist was developed from the framework, customised to the STOP trial and designed to be administered by researchers during a 10-min phone or video interview with participants. Customisation included adding introductory scripting, one or more prompt questions under each adverse event category, examples of typical events for researchers’ reference and reordering/ grouping categories and questions to optimise efficiency and acceptability of the delivery. According to the ICH-GCP guidelines, all AE data need to be categorised based on seriousness, severity, relatedness and expectedness [ 16 ]. This was done following standard guidance widely available across clinical trials units (see Appendix C for further details). The checklist was devised to incorporate the first three of these evaluations (seriousness, severity, relatedness). By definition, any event that fell within one of the listed Adverse Event Categories was considered “expected” (i.e. anticipated). Items that had not been foreseen and were therefore classed as “unexpected” were listed under the “Other” category heading. The resulting Adverse Events Checklist for the STOP trial is presented in the “ Results ” section.

In response to regulatory safety concerns, the frequency of AE data collection calls was increased to once a week for any participants identified as high risk. High-risk participants were identified at baseline using a cut-off score on a Persuadability/Suggestibility scale [ 17 ] (higher suggestibility can lead to higher risk, as the intervention aims to foster nonparanoid and trusting thoughts) and a suicide risk assessment, and throughout the trial using a suicide assessment that was administered on a weekly/biweekly basis. In addition, researchers recorded any AE that was spontaneously reported by the participant at any other contact. Note that it is crucial to collect AE data using identical methods for both the intervention and control groups even if the trial is unblinded as the control group serves as an important baseline for adverse event occurrences.

Second, we built in safety monitoring within the device. An algorithm was used to trigger an alert to researchers whenever a participant had a worsening of state mood on self-reported levels of paranoia, anxiety or sadness across a weekly treatment session (using visual analogue scale in-app pre/post session assessments; see Supplementary File 1) on 3 consecutive occasions. Researchers would then make a follow-up call to check in on the participant, collect further information and safety data and decide if follow-up action was needed (for example alerting a GP or clinical care team).

Third, we added a specific outcome measure relevant to safety, namely the Negative Effects Questionnaire (NEQ) administered once at the end of the intervention (end of treatment). The NEQ is a 20-item self-report measure [ 18 ]. It was developed using the results from Rozental et al., (2014)’s consensus statement on the negative effects of internet interventions [ 18 ], and studies aimed at investigating the negative effects of psychotherapy [ 18 , 19 ]. It is used to collect data on the negative effects experienced by patients/users during treatment, their severity and whether they were related to the intervention or other circumstances [ 19 ]. The NEQ is a reliable and valid measure with an internal consistency of α  = 0.95 [ 19 ].

Demonstrating the existing safety of the DMHI

The feasibility study main outcome paper reported no adverse events or serious adverse events and an “absence of evidence of any harmful effects on state mood and the practicality of the protocol as delivered” [ 12 ] The results from the VAS showed that there was no evidence of significant short-term detrimental effects on anxiety, sadness, paranoia or friendliness in the intervention group compared to the control group, suggesting that the intervention did not exacerbate negative mood, or pose any risk of harm to patients with distressing paranoia [ 12 ]. These data are provided in Supplementary File 1. The STOP study team used these combined findings to argue in support of the safety of STOP based on its similarity in therapeutic content to CBM-pa.

Literature search and regulatory databases

Results of literature review 1 (Device use or experience):

The search for the first literature review resulted in 14 included studies. See Appendix D for the respective PRISMA flowchart. Results showed evidence that cognitive impairment in this population does not affect engagement with digital interventions [ 20 ]. There was evidence suggesting that digital interventions are effective at improving social functioning [ 21 ], memory [ 22 ], educational and vocational attainment [ 23 ], personal recovery [ 24 ], and alleviating loneliness [ 25 ] in psychotic disorders. Some digital interventions used in this population aimed to monitor symptoms such as sleep [ 26 ], and psychotic symptoms [ 24 ]. The results of a previous literature search [ 27 ] showed that there were three digital mental health interventions that have been developed to improve symptoms in individuals struggling with psychosis [ 21 , 23 , 25 ]. The review included eight papers on smartphone-based interventions for psychosis, of which three were protocols, two were feasibility studies, two were pilot RCTs and only one was an RCT with a sample of 36 participants. This RCT found that participants who used Actissit (a Cognitive Behavioural Therapy based app for psychosis) plus treatment as usual experienced better improvements psychotic symptoms compared to those who used a symptom monitoring app plus treatment as usual [ 28 ].

The data on the use or experience of digital therapies to monitor, reduce symptoms or improve recovery in this population were promising but still limited. Larger randomised controlled trials are needed. There was no study on the use or experience of digital mental health interventions in a sample specifically defined by paranoid symptomatology except for the feasibility study precursor to the STOP [ 11 ]. For that, the literature search criteria was expanded to include devices that address psychosis in general to find comparable studies.

Results of literature review 2 (Device safety):

The search for the second literature review resulted in five included studies. See Appendix D for the respective PRISMA flowchart. Although the literature on the safety of digital mental health interventions targeting paranoia/psychosis is limited, all the current studies demonstrated positive safety outcomes [ 21 , 23 ]. A number of studies assessed the safety of the Horyzons—an online social media-based intervention that was designed to enhance social functioning in individuals with a first episode of psychosis; the studies found Horyzons safe to use (no incidents) and Horyzons users reported feeling safe and empowered [ 23 , 29 , 30 ]. A social media-based intervention called (MOMENTUM), which aims to improve social functioning in “at high-risk mental state” young individuals, was found to be safe to use [ 29 , 30 ]. A randomised controlled trial ( N  = 36) of Actissit—a CBT-informed mobile phone app for people who have experienced psychosis—found it safe to use (no serious adverse events) [ 28 ]. Finally, a randomised clinical trial ( N  = 41) assessing the EMPOWER app (Early signs Monitoring to Prevent relapse in psychosis and prOmote Wellbeing, Engagement and Recovery) reported 9 adverse events that were related to the app such as increased feelings of paranoia, increased fear of relapse and technical issues [ 31 ]. Findings were in line with those of a systematic review on the digital interventions for early psychosis where all eight smartphone-based interventions under study were found to be safe [ 27 ].

The clinical data appraisal tools for both literature reviews are provided in Appendix C.

Regulatory databases

There were no safety concerns raised from the review of the regulatory databases.

In total 26 unique hazards and their corresponding harms were identified, which are listed in full in Appendix E.

The team defined likelihood/probability and severity for the proposed study as explained in the “ Methods ” section. Table 1 shows the final operationalised definition of probability and Table  2 shows the equivalent for severity.

Afterwards, the team used these definitions to rate the probability and severity of every identified hazard. Probability refers to the likelihood that the identified hazardous situation will lead to the specified harm . A risk score was calculated for each hazard entry by multiplying the probability and severity scores.

Under this step, the study team identified all measures they could take to reduce each risk entry as much as possible, listing these as “risk controls”. They then recalculated new probability, severity and risk scores under the assumption that the stated risk controls were effective. Before implementing risk-control strategies, the highest risk score was 20 out of 36. After applying these strategies, the highest risk score was reduced to 9 out of 36. These quantitative ratings and products are shown in Appendix F, which constitutes the final STOP Hazard Analysis, and was a key requirement of the submission for regulatory approval. One new insight that emerged from the consultation was the need for a product recall feature that could operate at either an individual or entire cohort level. This requirement was a crucial safety attribute, for use in the unlikely event that access to the app had to be immediately terminated. At the individual level access could be revoked by account deactivation. At the cohort level, the technical team implemented a “recall switch” feature in the app to enable the recall, and a corresponding participant facing message. Footnote 1

The literature review and the expert consultation that we conducted as described in the “ Methods ” section resulted in a finalised “Adverse Event Category Framework for Digital Therapeutic Devices” (See Table  3 ) . This framework was informed by three key publications arising from the literature review that discussed a range of classes of negative effects in psychotherapy [ 18 , 32 , 33 ]. Our framework was then used to identify the “anticipated” AEs for the STOP trial. As such, it might be applicable to all DMHIs that are in the form of a mobile app. The AEs collected are not exclusive to STOP but might not all be relevant or be comprehensive of all possible AEs for another DMHI. Professionals testing a DMHI delivered in a different format (virtual reality for example) and/or targeting a different population would need to make suitable adjustments and might even wish to incorporate additional AE categories specific to their device’s safety profile. However, this framework is recommended as a potentially useful starting point for any DMHI.

Some of the categories such as technical malfunction might be less clear to clinicians, as they do not directly relate to the therapeutic component of the device. When assessing adverse events (AEs), it is essential to evaluate the entire device, not just the treatment component. This includes potential risks from using any mobile app. Furthermore, the MHRA approval mandates monitoring all aspects of the approved research for safety, covering study procedures, intervention and the device. One learning that came out of discussions during the development of the AE framework with other professionals was the need to have a separate AE category for “device deficiency” that is distinct from “technical malfunction”. Device deficiency is defined as “an inadequacy of a medical device related to its identity, quality, durability, reliability, safety or performance, such as malfunction, misuse or use error and inadequate labelling” [ 34 ]. This differentiation was highlighted by some of the academics on the team with experience in other DMHIs, to align with the requirements and terminology used by the regulatory framework.

The trial is still ongoing at the time of writing and a full report of the STOP safety evaluation will be published as part of the trial outcomes. Here we present the tools we developed to aid STOP safety data collection, as described in the “ Methods ” sections of this paper. We also outline the final STOP safety analysis plan, which was subject to rigorous review and revision as part of the regulatory approval process.

Safety data collection

The Adverse Events Checklist used by researchers to proactively collect fortnightly (or weekly for more vulnerable participants) AE data is presented in Appendix G. Participants were asked each prompt question in turn to identify and record details of any adverse that had happened since the last researcher contact. For every event recorded researchers completed the remaining columns of the checklist to record a free text event description and to determine its seriousness, relatedness, expectedness and severity. The checklist will be administered every week rather than fortnightly with high-risk patients to mitigate any risks. It is likely that administering the checklist in these patients more than the rest might lead to a higher number of reported AEs. This will be taken into account in the analyses using sensitivity analyses.

Safety data analysis

The complete statistical analysis plan (SAP) for the STOP trial underwent a number of iterations in review with the regulators before approval was achieved. In terms of safety specifically, the approved plan included the following. Any AE/SAE involving the target clinical symptoms (paranoia) will be analysed separately from other AE/SAEs, due to the assessed (small) likelihood that the device could trigger these symptoms. This risk was singled out for separate analysis because it was the one of most concern to clinicians and regulators. Formal statistical analyses are unlikely due to small numbers of observations but the incidence rate of AEs (total number of those having the event divided by the person-months at risk) and the ratio of incidence rates of AEs between the two treatment arms per time period will be reported to allow detection of any safety concerns within the treatment arm.

Analysis of the checklist data will produce a list of adverse events along with frequencies, seriousness, relatedness and possible methods of prevention/mitigation. Additionally, demographic and clinical characteristics of those who experienced adverse events will identify patients who might be at a higher risk. Comparative statistical tests will be used to analyse the NEQ data between the treatment and control arms using a linear regression approach.

An overview of the pathway followed by the STOP team from start to finish is provided in Table  4 . This shows the purpose of each step, some brief details on what it included and pointers allowing the reader to more easily navigate to relevant sections of the present paper and associated resources.

This paper details the steps that the STOP study team took to thoroughly assess the safety of a DMHI and achieve regulatory approval to conduct an RCT (MHRA’s NoNO). The example shared in this paper serves as a guide for academics and other professionals in the field. It provides a roadmap for the essential prerequisites, requirements and expectations regarding safety when seeking regulatory approval to conduct research with DMHIs. A fuller understanding of this pathway will significantly benefit research teams, clinicians and developers involved in the process of developing and delivering novel DMHIs.

There are various key concepts and practical takeaways outlined in this paper. The overarching requirement is to compile an evidence-based argument that the benefit of the proposed device outweighs its risk to users, and this can only be done convincingly by the fullest consideration and quantification of that risk. The process by which one might do this can be broken down into various discrete steps. Figure  1 demonstrates the process model presented in this paper.

The process model of “How to demonstrate the safety of as-yet untested DMHI?

First, it is important to establish the safety of the DMHI even before testing its efficacy. This could be done by looking at the safety data of “equivalent” interventions that have been used in a similar population, studying the literature and/or conducting a feasibility/pilot study to assess the preliminary safety of the intervention. It is noteworthy that in the UK devices exclusively developed and used (either clinically or for research) within a single institution are exempt from formal regulatory approval requirements [ 35 ] which can provide an appropriate setting for gathering early-stage safety data. Second, conducting a comprehensive risk analysis specific to each DMHI is crucial [ 36 ]. This involves identifying all the potential hazards that are relevant to that DMHI, assessing any potential harm (likelihood and severity), calculating a risk rating per identified hazard, implementing risk control measures, reassessing risk, calculating a final post-risk score, denoting and reporting any residual risk and finally demonstrating that the expected benefits outweigh the identified risks in a quantifiable manner. Third, the safety of a new and untested DMHI needs to be evaluated as a primary outcome within the proposed research. It needs to hold the same importance as efficacy/effectiveness, irrespective of the academic research agenda. A safety evaluation plan needs to be integrated within the study protocol or presented separately as a standalone study.

Fourth, a helpful component of any safety evaluation is the use of a framework for organising the data to be collected, given the likely breadth of possible adverse events. The Adverse Event Category Framework for Digital Therapeutic Devices provides one such possibility. At a more practical level, this must be supplemented by a structured approach to collecting and evaluating individual adverse events. The Adverse Events Checklist (provided in appendices) received regulatory approval for use in the STOP trial and could usefully serve as a guide for others. By incorporating categorisation of each entry on the key dimensions of seriousness, severity, relatedness and expectedness, it allows a research team to more easily demonstrate their intended compliance with reporting requirements. It also facilitates gathering a richer dataset around negative effects that will go on to permit a more comprehensive analysis than previous traditional practices [ 11 ]. A scoping review on the recording of harms in RCTs of behaviour change interventions has mapped out the categories of harms found in that literature [ 35 ]. As might be expected, there is some overlap with our AE category framework, such as physical and psychological harms, which is reassuring and validatory. In contrast, group-level harms (such as the impact of a behaviour change intervention on culture, environment or health equity) feature strongly in the scoping review but are absent from our framework, which focused exclusively on individual participant-level harms data. It will be important for future studies to consider whether macro-level harms relevant to behaviour change interventions might also be relevant to DMHI interventions.

It is important to highlight the time involved in the processes summarised in this paper. In the present worked example acquiring regulatory approval (MHRA NoNO) took approximately 6 months and the authors’ recommendation is to allow a timeframe of up to 9 months, if working from a position of relatively little prior knowledge and experience. This timeline is necessary to allow for the involvement of clinical and technical experts, patient groups and regulatory consultants. In the present worked example employing a regulatory consultant played a vital role in ensuring compliance with all regulatory requirements and smooth passage through regulatory review. Their knowledge of the complex regulatory landscape provided a key interface between software developers and the academic team to ensure that the requisite information was compiled and presented in a manner compliant with the appropriate national and international standards [ 37 ]. Academic teams are advised to routinely cost such expertise into research projects involving medical devices, unless equivalent institutional support is already available.

Limitations

It is important to be aware that this paper provides an example of how one DMHI assessed safety and achieved regulatory approval. The experiences of other DMHIs will most likely differ. Thus, it is important to view this process flexibly and adapt it to each DMHI. Furthermore, this example is UK-centric. Even though the process described might be helpful for DMHIs applying for regulatory approval outside the UK, professionals need to be aware of the needs of their specific regulatory environment.

The example provided in this paper can be adapted by other professionals in the digital mental health field to help them navigate complex regulatory processes. Prioritising and emphasising safety and regulatory compliance allows researchers to contribute to the responsible development of DMHIs. Ensuring that the benefit of these interventions outweighs any risks that they carry is important for building confidence and trust among clinicians, patients and academics. The systematic approach to safety evaluation outlined here sets a valuable precedent for assessing the safety of DMHIs.

Availability of data and materials

All data generated or analysed during this study are included in this published article in the form of tables and appendices.

“ You can no longer use STOP as the product has been withdrawn. You will shortly be contacted by a member of the research team who will explain and offer further support if required. In the meantime, if you require more urgent assistance, please contact the study helpline at 020 784 80,425 or clinical support email at [email protected]”.

Abbreviations

Adverse event

Digital mental health intervention

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use—Good Clinical Practice

Medicines and Healthcare products Regulatory Agency

Negative Effects Questionnaire

National Health Service

National Institute of Health and Care Excellence

• Notification of No Objection

Preferred Reporting Items for Systematic Reviews and Meta-Analyses RCT: randomised controlled trial

Randomised controlled trial

Serious adverse event

Software as a Medical Device

Philippe TJ, Sikder N, Jackson A, et al. Digital health interventions for delivery of mental health care: systematic and comprehensive meta-review. JMIR Ment Health. 2022;9(5):e35159. https://doi.org/10.2196/35159 .

Article   PubMed   PubMed Central   Google Scholar  

Sanderson C, Kouzoupi N, Hall CL. Technology matters: the human touch in a digital age – a blended approach in mental healthcare delivery with children and young people. Child Adolesc Ment Health. 2020;25(2):120–2. https://doi.org/10.1111/camh.12385 .

Article   PubMed   Google Scholar  

Gómez Bergin AD, Valentine AZ, Rennick Egglestone S, Slade M, Hollis C, Hall CL. How are adverse events identified and categorised in trials of digital mental health interventions? A narrative scoping review of trials in the ISRCTN registry (Preprint). JMIR Ment Health. https://doi.org/10.2196/42501 . Published online February 22, 2022.

National Institute for Health and Care Excellence (NICE). Evidence standards framework for digital health technologies. https://www.nice.org.uk/corporate/ecd7/resources/evidence-standards-framework-for-digital-health-technologies-pdf-1124017457605 .

MHRA. Notify the MHRA about a clinical investigation for a medical device. https://www.gov.uk/guidance/notify-mhra-about-a-clinical-investigation-for-a-medical-device .

MHRA. Clinical investigations of medical devices – compiling a submission to MHRA. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1097797/Guidance_for_mfrs_-_compiling_a_submission_to_MHRA_-_May_2021.pdf .

Guideline IH. Clinical safety data management: definitions and standards for expedited reporting E2A. In: International conference on harmonisation of technical requirements for registration of pharmaceuticals for human use 1994 Oct. 1994.

Google Scholar  

Papaioannou D, Cooper C, Mooney C, Glover R, Coates E. Adverse event recording failed to reflect potential harms: a review of trial protocols of behavioral, lifestyle and psychological therapy interventions. J Clin Epidemiol. 2021;136:64–76. https://doi.org/10.1016/j.jclinepi.2021.03.002 .

Papaioannou D, Sprange K, Hamer-Kiwacz S, Mooney C, Moody G, Cooper C. Recording harms in randomised controlled trials of behaviour change interventions: a qualitative study of UK clinical trials units and NIHR trial investigators. Trials. 2024;25(1):163. https://doi.org/10.1186/s13063-024-07978-1 .

Papaioannou D, Hamer-Kiwacz S, Mooney C, et al. Recording harms in randomized controlled trials of behavior change interventions: a scoping review and map of the evidence. J Clin Epidemiol. 2024;169:111275. https://doi.org/10.1016/j.jclinepi.2024.111275 .

Taher R, Hsu CW, Hampshire C, et al. The safety of digital mental health interventions: systematic review and recommendations. JMIR Ment Health. 2023;10:e47433. https://doi.org/10.2196/47433 .

Yiend J, Lam CLM, Schmidt N, et al. Cognitive bias modification for paranoia (CBM-pa): a randomised controlled feasibility study in patients with distressing paranoid beliefs. Psychol Med. 2023;53(10):4614–26. https://doi.org/10.1017/S0033291722001520 .

Hsu C, Stahl D, Mouchlianitis E, Peters E, Vamvakas G, Keppens J, Watson M, Schmidt N, Jacobsen P, McGuire P, Shergill S, Kabir T, Hirani T, Yang Z, Yiend J. User-Centered Development of STOP (Successful Treatment for Paranoia): Material Development and Usability Testing for a Digital Therapeutic for Paranoia. JMIR Hum Factors. 2023;10:e45453. https://humanfactors.jmir.org/2023/1/e45453 . https://doi.org/10.2196/45453 .

European Commission. Clinical evaluation: a guide for manufacturers and notified bodies under directives 93/42/EEC and 90/385/EEC. 2016.

ISO 14971: Medical devices Application of risk management to medical devices. https://www.iso.org/standard/72704.html .

International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH). Integrated addendum to ICH E6(R1): guideline for good clinical practice E6(R2). https://database.ich.org/sites/default/files/E6_R2_Addendum.pdf .

Kotov RI, Bellman SB, Watson DB. Multidimensional Iowa Suggestibility Scale (MISS) Brief Manual. 2004.

Rozental A, Andersson G, Boettcher J, et al. Consensus statement on defining and measuring negative effects of internet interventions. Internet Interv. 2014;1(1):12–9. https://doi.org/10.1016/j.invent.2014.02.001 .

Article   Google Scholar  

Rozental A, Kottorp A, Forsström D, et al. The negative effects questionnaire: psychometric properties of an instrument for assessing negative effects in psychological treatments. Behav Cogn Psychother. 2019;47(5):559–72. https://doi.org/10.1017/S1352465819000018 .

Bell IH, Lim MH, Rossell SL, Thomas N. Ecological momentary assessment and intervention in the treatment of psychotic disorders: a systematic review. Psychiatr Serv. 2017;68(11):1172–81. https://doi.org/10.1176/appi.ps.201600523 .

Biagianti B, Schlosser D, Nahum M, Woolley J, Vinogradov S. Creating Live Interactions to Mitigate Barriers (CLIMB): a mobile intervention to improve social functioning in people with chronic psychotic disorders. JMIR Ment Health. 2016;3(4):e52. https://doi.org/10.2196/mental.6671 .

Reeder C, Huddy V, Cella M, et al. A new generation computerised metacognitive cognitive remediation programme for schizophrenia (CIRCuiTS): a randomised controlled trial. Psychol Med. 2017;47(15):2720–30. https://doi.org/10.1017/S0033291717001234 .

Article   PubMed   PubMed Central   CAS   Google Scholar  

Alvarez-Jimenez M, Koval P, Schmaal L, et al. The Horyzons project: a randomized controlled trial of a novel online social therapy to maintain treatment effects from specialist first-episode psychosis services. World Psychiatry. 2021;20(2):233–43. https://doi.org/10.1002/wps.20858 .

Lewis S, Ainsworth J, Sanders C, et al. Smartphone-enhanced symptom management in psychosis: open, randomized controlled trial. J Med Internet Res. 2020;22(8):e17019. https://doi.org/10.2196/17019 .

Lim MH, Gleeson JFM, Rodebaugh TL, et al. A pilot digital intervention targeting loneliness in young people with psychosis. Soc Psychiatry Psychiatr Epidemiol. 2020;55(7):877–89. https://doi.org/10.1007/s00127-019-01681-2 .

Meyer N, Joyce DW, Karr C, et al. The temporal dynamics of sleep disturbance and psychopathology in psychosis: a digital sampling study. Psychol Med. 2022;52(13):2741–50. https://doi.org/10.1017/S0033291720004857 .

Rus-Calafell M, Schneider S. Are we there yet?!—a literature review of recent digital technology advances for the treatment of early psychosis. Mhealth. 2020;6:3–3. https://doi.org/10.21037/mhealth.2019.09.14 .

Bucci S, Barrowclough C, Ainsworth J, et al. Actissist: proof-of-concept trial of a theory-driven digital intervention for psychosis. Schizophr Bull. 2018;44(5):1070–80. https://doi.org/10.1093/schbul/sby032 .

Alvarez-Jimenez M, Gleeson J, Bendall S, et al. SU125. Momentum: a novel online social media, mindfulness, and strengths-based intervention to promote functional recovery in ultra high risk (UHR) patients. Schizophr Bull. 2017;43(suppl_1):S206–7. https://doi.org/10.1093/schbul/sbx024.121 .

Article   PubMed Central   Google Scholar  

Alvarez-Jimenez M, Gleeson JF, Bendall S, et al. Enhancing social functioning in young people at Ultra High Risk (UHR) for psychosis: a pilot study of a novel strengths and mindfulness-based online social therapy. Schizophr Res. 2018;202:369–77. https://doi.org/10.1016/j.schres.2018.07.022 .

Article   PubMed   CAS   Google Scholar  

Bradstreet S, Allan S, Gumley A. Adverse event monitoring in mHealth for psychosis interventions provides an important opportunity for learning. J Ment Health. 2019;28(5):461–6. https://doi.org/10.1080/09638237.2019.1630727 .

Linden M. How to define, find and classify side effects in psychotherapy: from unwanted events to adverse treatment reactions. Clin Psychol Psychother. 2013;20(4):286–96. https://doi.org/10.1002/cpp.1765 .

Gullickson KM, Hadjistavropoulos HD, Dear BF, Titov N. Negative effects associated with internet-delivered cognitive behaviour therapy: an analysis of client emails. Internet Interv. 2019;18. https://doi.org/10.1016/j.invent.2019.100278 .

UK Goverment. Glossary. https://assets.publishing.service.gov.uk/media/6143184cd3bf7f05b694d6cc/Glossary_PDF.pdf .

MHRA. In-house manufacture of medical devices in Great Britain. https://www.gov.uk/government/publications/in-house-manufacture-of-medical-devices/in-house-manufacture-of-medical-devices .

Khinvasara T, Ness S, Tzenios N. Risk management in medical device industry. J Eng Res Rep. 2023;25(8):130–40. https://doi.org/10.9734/jerr/2023/v25i8965 .

Medical Device Regulation. MEDDEV Guidance List. https://www.medical-device-regulation.eu/meddev-guidance-list-download/ .

Download references

Acknowledgements

We acknowledge the contributions made to this work by the McPin Foundation. We thank Andrew Gumley, Alex Kenny, Caroline Murphy, Sumiti Saharan, Carolina Sportelli and Chris Taylor for their input to consultations carried out as part of the work reported here.

This work was supported by the Medical Research Council Biomedical Catalyst: Developmental Pathway Funding Scheme (DPFS), MRC Reference: MR/V027484/1. We would also like to express our gratitude to the National Institute for Health and Care Research (NIHR) Biomedical Research Centre hosted at South London and Maudsley NHS Foundation Trust in partnership with King’s College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health and Social Care, the ESRC or King’s College London. For the purposes of open access, the author has applied a Creative Commons Attribution (CC, BY) licence to any Accepted Author Manuscript version arising from this submission.

Author information

Authors and affiliations.

Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK

Rayan Taher & Jenny Yiend

Institute of Mental Health, School of Medicine, National Institute for Health and Care Research MindTech MedTech Co-Operative, University of Nottingham, Nottingham, UK

Charlotte L. Hall & Aislinn D Gomez Bergin

School of Computer Science, University of Nottingham, Nottingham, UK

Aislinn D Gomez Bergin

Avegen Health, London, UK

Neha Gupta & Nayan Kalnad

London Institute for Healthcare Engineering, King’s College London, London, UK

Clare Heaysman

Department of Psychology, Addiction and Mental Health Group, Bath Centre for Mindfulness and Compassion, University of Bath, Bath, UK

Pamela Jacobsen

McPin Foundation, London, UK

Thomas Kabir

Department of Informatics, King’s College London, London, UK

Jeroen Keppens

Department of Psychological Medicine, Dunedin School of Medicine, University of Otago, Otago, New Zealand

Che-Wei Hsu

Department of Psychiatry, University of Oxford, Oxford, UK

Philip McGuire

Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK

Emmanuelle Peters

Kent and Medway Medical School, Canterbury, Kent, UK

Sukhi Shergill

Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK

Daniel Stahl

Wensley Stock Ltd, Wiltshire, UK

Ben Wensley Stock

You can also search for this author in PubMed   Google Scholar

Contributions

RT, CH and JY conducted the literature reviews. CWH, TK, CH, PM, EP, SS, BW, DS and JY were involved in the development of the risk management plan. RT, CWH, TK, CH, PM, EP, SS, BW, DS and JY were involved in the development of the Adverse Events Checklist. RT, AB, CH and JY wrote up this paper. All the authors read and approved the final manuscript.

Corresponding author

Correspondence to Jenny Yiend .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Taher, R., Hall, C.L., Bergin, A.D.G. et al. Developing a process for assessing the safety of a digital mental health intervention and gaining regulatory approval: a case study and academic’s guide. Trials 25 , 604 (2024). https://doi.org/10.1186/s13063-024-08421-1

Download citation

Received : 30 November 2023

Accepted : 21 August 2024

Published : 10 September 2024

DOI : https://doi.org/10.1186/s13063-024-08421-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Regulatory approval
• Digital mental health
• Digital mental health interventions
• Adverse events
• Medical device
• Software as a medical device

ISSN: 1745-6215

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Open access
• Published: 14 September 2024

"No Papers, No Treatment": a scoping review of challenges faced by undocumented immigrants in accessing emergency healthcare

• Sezer Kisa 1   na1 &
• Adnan Kisa 2 , 3   na1  

International Journal for Equity in Health volume  23 , Article number:  184 ( 2024 ) Cite this article

1 Altmetric

Metrics details

Undocumented immigrants face many obstacles in accessing emergency healthcare. Legal uncertainties, economic constraints, language differences, and cultural disparities lead to delayed medical care and thereby exacerbate health inequities. Addressing the healthcare needs of this vulnerable group is crucial for both humanitarian and public health reasons. Comprehensive strategies are needed to ensure equitable health outcomes.

This study aimed to identify and analyze the barriers undocumented immigrants face in accessing emergency healthcare services and the consequences on health outcomes.

We used a scoping review methodology that adhered to established frameworks. Utilizing MEDLINE/PubMed, Embase, Web of Science, PsychoInfo, and the Cumulative Index to Nursing and Allied Health Literature (CINAHL), we identified 153 studies of which 12 focused on the specific challenges that undocumented immigrants encounter when accessing emergency healthcare services based on the inclusion and exclusion criteria.

The results show that undocumented immigrants encounter significant barriers to emergency healthcare, including legal, financial, linguistic, and cultural challenges. Key findings were the extensive use of emergency departments as primary care due to lack of insurance and knowledge of alternatives, challenges faced by health professionals in providing care to undocumented migrants, increased hospitalizations due to severe symptoms and lack of healthcare access among undocumented patients, and differences in emergency department utilization between irregular migrants and citizens. The findings also serve as a call for enhanced healthcare accessibility and the dismantling of existing barriers to mitigate the adverse effects on undocumented immigrants' health outcomes.

Conclusions

Undocumented immigrants' barriers to emergency healthcare services are complex and multifaceted and therefore require multifaceted solutions. Policy reforms, increased healthcare provider awareness, and community-based interventions are crucial for improving access and outcomes for this vulnerable population. Further research should focus on evaluating the effectiveness of these interventions and exploring the broader implications of healthcare access disparities.

Introduction

People who live without legal authorization in a foreign country form a significant global demographic [ 1 ]. The terms "immigrant" and "migrant" are often used interchangeably in this context; however, "immigrant" typically refers to individuals who move to another country with the intention of permanent settlement, whereas "migrant" can refer to those who move temporarily, often for work, and may not intend to stay permanently [ 2 ]. Estimates suggest there are approximately 281 million international migrants worldwide, a substantial portion of whom lack legal status in their host countries [ 3 ]. For instance, in the United States alone, it is estimated that there are around 10.5 million undocumented immigrants, representing about 3.2% of the total U.S. population [ 4 ]. Similarly, in the European Union, there are an estimated 3.9 to 4.8 million undocumented migrants [ 5 ].These individuals face many obstacles in accessing healthcare. Such obstacles include lack of health insurance, fear of deportation, ineligibility for government programs, and language and cultural differences [ 1 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 ]. Addressing their healthcare needs is crucial not only from a humanitarian perspective but also for public health, as their exclusion from healthcare systems has serious consequences [ 15 , 16 ].

Studies found that financial barriers to healthcare included high out-of-pocket payments, high service prices, fragmented financial support, limited funding capacity, fear of deportation, and delayed referral [ 12 , 17 ]. Geographic challenges also play a role, with many migrants living in areas where healthcare facilities are either overwhelmed or scarce. These barriers hinder not only access to routine care but also emergency services, contributing to wider public health concerns [ 7 , 12 , 17 , 18 , 19 ].

In emergency care situations, undocumented immigrants face even greater challenges. They often avoid essential treatment due to financial problems and fear of legal actions [ 1 , 6 , 10 , 12 , 17 , 18 ]. Even when they do seek emergency care, they often encounter language and cultural differences that can lead to misunderstandings and inappropriate treatment [ 7 , 12 ]. This avoidance of essential care not only endangers their health but also affects the health of the community [ 10 , 11 , 13 ].

Although extensive searches were conducted, no systematic reviews were found that specifically addressed the difficulties undocumented immigrants have in accessing emergency care. The phrase "No Papers, No Treatment," used in the title of this study, reflects the harsh reality that undocumented immigrants often face when seeking healthcare. This phrase, which has been echoed in various advocacy platforms and public discussions, encapsulates the severe barriers to care that this population experiences. This scoping review aims to bridge this gap by examining those very challenges. The objectives of this review are threefold: 1) to identify the specific barriers encountered; 2) to understand the reported consequences of these barriers on undocumented immigrants; and 3) to examine the solutions that have been proposed to improve their access to emergency care. By undertaking this study, we aim to provide a foundational understanding of the complexities involved in access to emergency healthcare for undocumented immigrants, thereby contributing to the body of knowledge and suggesting pathways for future research and policy development. This is the first study to address this neglected issue in healthcare research and policy.

Methodology

This scoping review was designed by integrating the methodologies described by Arksey and O'Malley (2005) [ 20 ] and further refined by Levac et al. (2010) [ 21 ]. The research team consisted of two reviewers, who are also the authors of this work. These reviewers formulated the main research objectives and outlined the review by defining the search terms, identifying the databases for the literature search, and establishing the inclusion and exclusion criteria. We selected the MEDLINE/PubMed, Embase, Web of Science, PsychoInfo, and Cumulative Index to Nursing and Allied Health Literature (CINAHL) databases due to their extensive coverage of medical, psychological, and health literature. The search terms were chosen to cover a wide array of relevant components ("emergency" OR "emergency care") AND ("undocumented immigrants" OR "illegal immigrants" OR "unauthorized immigrants" OR "undocumented migrants" OR "irregular migrants"). This ensured the inclusion of literature that specifically addressed barriers faced by undocumented immigrants in accessing emergency care.

The search and selection processes were conducted by both reviewers. Duplicates were removed, followed by two parallel and separate screenings of titles and abstracts by each reviewer. The full-text review and data extraction were also performed independently by each reviewer, with any disagreements resolved through discussion. Our scoping review did not include a formal quality assessment of the included studies, in line with Arksey & O'Malley's (2005) [ 20 ] recommendations for scoping reviews. We limited our review to peer-reviewed research articles that examined undocumented immigrants' barriers to emergency care and were published in English up to February 29, 2024. Studies were excluded if they did not focus on undocumented immigrants in accessing emergency care, were not related to undocumented immigrants, were not based on empirical research, or were published in languages other than English. This extensive selection process resulted in a total of 12 studies for the final review (Fig.  1 ).

All findings were entered in EndNote (version 21). The data from the included studies, which related to characteristics such as author, publication year, study design and participants, sample size, study purpose, and key findings were extracted and charted by the first author in Excel to address the research objectives.

This review uncovered 12 studies on emergency care use by undocumented individuals in the United States [ 13 , 18 , 22 , 23 , 24 ], Switzerland [ 25 ], Denmark [ 9 ], French Guiana [ 10 ], Israel [ 19 ], Norway [ 15 , 26 ], and Spain [ 16 ]. The methodologies of the studies varied. They encompassed six cross-sectional surveys [ 10 , 13 , 18 , 19 , 22 , 24 ], one prospective cohort design [ 25 ], one historical cohort study [ 15 ], one case-control study [ 23 ], one observational cross-sectional study [ 26 ], and two qualitative studies [ 9 , 16 ]. Notably, the study by Jiménez-Lasserrotte et al. (2023) included valuable insights from nurses who were directly involved in the care of child migrants, highlighting their critical role in health and social triage, as well as in addressing the immediate health needs of this vulnerable population. Sample sizes varied significantly across these studies, ranging from small-scale qualitative interviews with 12 participants [ 9 ] to large-scale analyses involving over half a million individuals [ 19 ]. The studies were published between 1996 and 2023.

Key findings were the excessive use of emergency departments for primary care due to lack of insurance and knowledge of alternatives [ 22 ], challenges faced by health professionals in providing care to undocumented migrants [ 9 ], increased hospitalizations due to severe symptoms, and lack of healthcare access [ 10 , 23 ], and differences in emergency department utilization between irregular migrants and citizens [ 19 ] (Table  1 ).

Barriers to accessing emergency healthcare

Barriers to accessing emergency care were broadly categorized under six themes: linguistic, financial, legal, cultural, health literacy, and other (Table  2 ).

Lack of health insurance [ 9 , 10 , 13 , 19 , 22 , 23 , 24 , 25 ], restricted medical benefits [ 22 ], high costs associated with healthcare [ 10 , 25 ], financial constraints due to unemployment or underemployment [ 19 ]; and exclusion from general practitioner and reimbursement schemes [ 15 ] were reported as the financial barriers to emergency care.

Most of the legal barriers were related to one's undocumented status and lack of entitlements, such as a health insurance card or identity number [ 9 , 10 , 15 , 16 , 19 , 22 , 23 , 25 , 26 ]. Fear of being reported to authorities [ 13 , 22 , 24 ] was mentioned in three studies. Administrative hurdles and systemic healthcare challenges, which include complications due to lack of proper documentation or previous medical records and the inefficiencies within the healthcare system itself, were also reported [ 9 , 15 , 26 ].

Transportation issues and lack of childcare were among the other barriers that prevented timely access to emergency healthcare [ 18 ]. Geographical remoteness and the complexity of health insurance systems [ 10 ], the patchwork system of safety net care (which is especially relevant to emergency renal disease care and the inconsistency in healthcare policies) [ 23 ], and structural vulnerabilities such as poor working and living conditions [ 15 , 26 ], were other assorted factors affecting the migrants’ accessibility and utilization of healthcare services.

Consequences of barriers

The costs of these identified barriers were increased reliance on emergency departments as primary care sources, higher rates of unfunded visits, and delays in treatment [ 22 ]; unintended pregnancies, delayed prenatal care, increased exposure to violence during pregnancy [ 25 ]; and limited access resulting in neglect of preventive care and excessive emergency service use [ 13 , 18 ]. The researchers also identified disparities such as: unequal access to primary care, delayed treatment, and administrative burdens [ 9 ]; fears leading to delayed healthcare access and higher emergency severity [ 24 ]; extended emergency department stays and lower hospitalization rates for non-severe conditions [ 19 ]; substandard antenatal care and related risks [ 15 , 26 ]; more severe conditions upon hospital arrival and higher hospitalization rates [ 10 ]; and specific issues such as increased emergent dialysis usage and associated costs [ 23 ] (Table  3 ).

Suggested solutions

The studies advocate for systemic changes to improve healthcare accessibility and quality for undocumented immigrants. Free or low-cost services and culturally appropriate education [ 25 ], increased social and economic resources [ 13 ], information dissemination through trusted sources [ 18 ], legal clarification and language support [ 9 ], patient education about confidentiality and health rights [ 24 ], initiatives to better healthcare access for undocumented migrants and affordable insurance options [ 10 ], and inclusive Medicaid policies [ 23 ] were all recommended. Furthermore, comprehensive care that addresses health, social, and emotional aspects, with culturally adapted and coordinated approaches, were also suggested [ 16 , 19 ] (Table  3 ).

Research gaps and future directions

The studies identified several significant gaps and future research needs in healthcare access for undocumented immigrants. These include understanding the impacts of legislative measures [ 22 ], access to care without documentation [ 13 , 25 ], improving prenatal care, variations in emergency room use, effects of information sources, and structural impacts on healthcare-seeking behaviors [ 18 ]. Other urgent areas for research are the impact of fear on healthcare access, ensuring understanding of a patient's rights and confidentiality, exploring health needs in regions with significant migrant populations, understanding intersections of immigration status with ethnicity in care disparities, and focusing on healthcare access and community care strategies for migrants [ 9 , 19 , 23 ]. Finally, investigating comprehensive care pathways, uncovering structural vulnerabilities that affect health coverage, and developing enhanced protocols for vulnerable migrant populations are imperative for future healthcare improvement and policy development [ 10 , 24 ] (Table  3 ). Additionally, there is a notable lack of qualitative insight from undocumented immigrants/migrants themselves regarding their experiences and perspectives on accessing emergency healthcare. Future research should prioritize capturing these first-hand accounts to better understand the nuanced challenges faced by this population and to inform more effective and empathetic policy interventions.

This scoping review aimed to identify and synthesize research on the challenges faced by undocumented immigrants in accessing emergency healthcare. The objectives were to identify specific barriers to care, understand the consequences of those barriers, and explore proposed solutions to improve access. Despite differences in methodologies, participants, and regional focus, the studies highlighted the urgent need for systemic reform to improve healthcare accessibility for undocumented populations.

Barriers to accessing emergency care

Ensuring equitable access to safe, well-organized, and high-quality emergency care services for all individuals in need can help mitigate health disparities [ 27 ]. However, several barriers were found that prevent undocumented immigrants from accessing emergency care. Most significantly, the fear of deportation led immigrants to avoid healthcare facilities [ 23 , 24 ]. Asch et al. found that individuals who feared seeing a doctor lest they get reported to the immigration authorities were nearly four times more prone to delaying care for over two months, increasing the risk of disease transmission [ 28 ]. Brenner et al. noted that deportation fears forced undocumented immigrants with end-stage renal disease (ESRD) to seek emergency care only when their condition became life-threatening [ 29 ].

Cultural and linguistic barriers further complicate these challenges. Many immigrants rely on social media or friends for health information due to a lack of trust in healthcare systems [ 24 ]. Granero-Molina et al. [ 30 ] note that health providers struggle to provide care due to language barriers and cultural misunderstandings [ 30 ]. Additionally, transportation issues, childcare responsibilities, and systemic inefficiencies hinder timely access to care, particularly in emergencies [ 15 , 18 , 26 ].

Structural vulnerabilities also play a role, as immigrants often live and work in environments that limit their access to healthcare [ 15 , 26 ]. DuBard and Massing emphasize that healthcare access for undocumented immigrants is further impeded by the complexity of health insurance systems [ 31 ]. These systemic barriers result in a system where undocumented immigrants rely on emergency departments, leading to overcrowding and increased costs [ 22 , 23 ]. Hsia and Gil-González note that legal ambiguities and administrative barriers exacerbate challenges in providing consistent healthcare access to undocumented immigrants [ 32 ].

Barriers to emergency care have many consequences for undocumented immigrants. Relying on emergency departments for primary care leads to delays in treatment, worsening conditions, and higher hospitalization rates [ 10 , 22 ]. Pregnant and undocumented women risk delayed prenatal care and exposure to violence [ 15 , 25 , 26 ]. Limited access to primary care results in untreated conditions becoming acute emergencies [ 19 ]. For patients with chronic conditions such as ESRD, limited access to regular hemodialysis forces them to rely on emergency departments for emergency-only hemodialysis EOHD, resulting in higher morbidity, mortality, and costs [ 23 , 33 ]. Patients receiving EOHD often experience severe symptoms such as hyperkalemia and uremia before seeking emergency care [ 34 ]. Clinicians providing EOHD also report significant morale distress due to the substandard care they have to provide [ 33 , 35 ]. In addition, cultural barriers during emergency triage contribute to inadequate care for undocumented immigrants, particularly those arriving by small boats in Europe [ 30 ]. Although our study did not specifically examine mental health conditions, it is well-documented that undocumented immigrants frequently experience significant mental health challenges due to the stress of living in uncertain conditions. This is particularly concerning in emergency department settings, where overcrowding and limited resources often result in inadequate mental health care for this vulnerable population.

Proposed solutions

Addressing these challenges requires systemic improvements to healthcare access and quality for undocumented immigrants. Cervantes et al. [ 34 ] argue that enhancing access to primary and preventive care through free or low-cost services and culturally appropriate education can help reduce the reliance on emergency departments for non-emergency conditions [ 34 ]. Nandi et al. (2008) [ 13 ]emphasized the need for increased social and economic resources.

Legal clarification and policy changes that explicitly include undocumented immigrants in healthcare systems are essential. Improved access to primary care, coupled with patient education about their rights and the confidentiality of healthcare services, can alleviate fears related to immigration status [ 9 , 24 ]. Affordable health insurance options and inclusive Medicaid (a joint federal and state program in the United States that provides health coverage to eligible low-income individuals and families) policies would significantly improve access to care and reduce the financial burden on safety-net programs [ 10 , 23 ]. Brenner et al. (2021) [ 29 ] argue that systemic efforts to improve public health, reduce the effects of injury and illness, and secure access to emergency and basic health care for all must involve policies that prioritize care over immigration enforcement.

Programs that enhance access to primary care and consider broader inclusion policies can improve outcomes for undocumented immigrants [ 19 ]. The inclusion of diverse healthcare provider perspectives, such as those of nurses, as seen in Jiménez-Lasserrotte et al. (2023), is crucial for developing comprehensive care strategies that address the unique needs of undocumented populations. Addressing structural vulnerabilities, including working and living conditions, is essential for improving healthcare access and quality. Accessible antenatal care and comprehensive healthcare that addresses physical, social, and emotional needs are crucial for vulnerable populations [ 16 ]. Addressing legislative barriers and reducing administrative burdens, as highlighted by the challenges faced in Spain, is also essential for ensuring equitable healthcare access [ 32 ]. By focusing on these systemic changes, healthcare systems can better accommodate the needs of undocumented immigrants, ensuring they receive the necessary care without unnecessary legal and administrative obstacles. Cultural mediation can help to bridge gaps in understanding between healthcare providers and undocumented immigrants [ 30 ].

Significant research gaps remain in understanding the full extent of healthcare challenges faced by undocumented immigrants. Further research is needed to understand the impact of legislative measures on healthcare access [ 22 ]. Additionally, studies should explore the influence of one's undocumented status on healthcare access and outcomes, especially in prenatal care [ 13 , 25 ]. Comprehensive studies on emergency room use, information sources, and structural barriers to healthcare are needed [ 18 ].

More comprehensive studies on healthcare access and quality for undocumented immigrants are required to inform effective policies [ 9 ]. Addressing the impact of fear on healthcare access, along with strategies to ensure that immigrants understand their rights, is critical [ 24 ]. Research should focus on developing effective community care strategies to overcome healthcare barriers for migrant populations [ 19 ]. Understanding the structural vulnerabilities affecting health coverage is imperative for future care improvement and policy development [ 15 , 26 ]. Further research should also explore the impact of administrative barriers and the challenges of policy implementation, as seen in Spain, to develop more effective solutions [ 32 ]. Additionally, research should prioritize examining the mental health challenges faced by undocumented immigrants, particularly in emergency settings. Given the limited resources in emergency departments, there is a critical need for targeted interventions that address these mental health needs to improve care and outcomes for this population.

Limitations

This review has several limitations. First, a restriction to English-language publications may have excluded important studies published in other languages and limited the global representativeness of our findings. Second, the exclusion of gray literature sources, such as reports and conference abstracts, may have overlooked valuable insights, restricting the breadth and depth of our review. Third, the heterogeneous methodologies employed across included studies introduced variability and could have complicated direct comparison and synthesis of findings. These limitations emphasize the need for careful interpretation and draw attention to areas where methodological improvements are needed in future research.

In conclusion, this comprehensive review found a diverse range of barriers faced by undocumented immigrants in accessing emergency healthcare services. Legal, financial, linguistic, cultural, and systemic factors collectively contribute to adverse health outcomes and strain emergency healthcare systems. Proposed solutions encompass policy initiatives such as enacting inclusive healthcare policies, together with community-based interventions like culturally tailored education and improved information dissemination. Further research is needed to understand the intersectionality of barriers, evaluate the effectiveness of proposed interventions, and assess the impact of legislative measures on healthcare access. By dismantling structural barriers, fostering cultural competency, and prioritizing the healthcare needs of undocumented immigrants, policymakers and practitioners can advance health equity agendas and foster a more inclusive healthcare landscape. Overall, addressing the diverse barriers to emergency healthcare access for undocumented immigrants is crucial for promoting health equity and improving public health outcomes. We will only achieve a truly healthy society when all its members, documented and otherwise, receive the care they need and deserve.

Availability of data and materials

All data generated or analyzed during this study are included in this published article.

Data availability

No datasets were generated or analysed during the current study.

Martinez O, Wu E, Sandfort T, Dodge B, Carballo-Dieguez A, Pinto R, et al. Evaluating the impact of immigration policies on health status among undocumented immigrants: a systematic review. J Immigr Minor Health. 2015;17(3):947–70.

Article   PubMed   PubMed Central   Google Scholar  

(IOM). IOfM. Glossary on Migration. 2019. Available from: https://publications.iom.int/books/international-migration-law-ndeg34-glossary-migration .

Affairs. UNDoEaS. International migration 2020 highlights. United Nations. 2020. Available from: https://www.un.org/development/desa/pd/news/international-migration-2020 .

Center PR. Facts on U.S. immigrants. 2020. Available from: https://www.pewresearch.org/race-and-ethnicity/2020/08/20/facts-on-u-s-immigrants/ .

Migration. IOf. World migration report 2020; 2020.

Clifford N, Blanco N, Bang SH, Heitkemper E, Garcia AA. Barriers and facilitators to healthcare for people without documentation status: a systematic integrative literature review. J Adv Nurs. 2023;79(11):4164–95.

Article   PubMed   Google Scholar  

Hacker K, Anies M, Folb BL, Zallman L. Barriers to health care for undocumented immigrants: a literature review. Risk Manag Healthc Policy. 2015;8:175–83.

Jacquez F, Vaughn L, Zhen-Duan J, Graham C. Health care use and barriers to care among latino immigrants in a new migration area. J Health Care Poor Underserved. 2016;27(4):1761–78.

Jensen NK, Norredam M, Draebel T, Bogic M, Priebe S, Krasnik A. Providing medical care for undocumented migrants in Denmark: what are the challenges for health professionals? BMC Health Serv Res. 2011;11:154.

Jolivet A, Cadot E, Angénieux O, Florence S, Lesieur S, Lebas J, Chauvin P. Use of an emergency department in Saint-Laurent du Maroni, French guiana: does being undocumented make a difference? J Immigr Minor Health. 2014;16(4):586–94.

Metcalf M, Comey D, Hines D, Chavez-Reyes G, Moyce S. "Because We Are Afraid": voices of the undocumented in a new immigrant destination in the United States. J Public Health Policy. 2024.;45(2):367–77.

Molina RL, Beecroft A, Pazos Herencia Y, Bazan M, Wade C, DiMeo A, et al. Pregnancy care utilization, experiences, and outcomes among undocumented immigrants in the united states: a scoping review. Womens Health Issues. 2024:34(4):370–80.

Nandi A, Galea S, Lopez G, Nandi V, Strongarone S, Ompad DC. Access to and use of health services among undocumented Mexican immigrants in a US urban area. Am J Public Health. 2008;98(11):2011–20.

Vargas Bustamante A, Fang H, Garza J, Carter-Pokras O, Wallace SP, Rizzo JA, Ortega AN. Variations in healthcare access and utilization among Mexican immigrants: the role of documentation status. J Immigr Minor Health. 2012;14(1):146–55.

Eick F, Vallersnes OM, Fjeld HE, Sørbye IK, Storkås G, Ekrem M, et al. Use of non-governmental maternity services and pregnancy outcomes among undocumented women: a cohort study from Norway. BMC Pregnancy Childbirth. 2022;22(1):789.

Jiménez-Lasserrotte MDM, Artés-Navarro R, Granero-Molina J, Fernández-Medina IM, Ruiz-Fernández MD, Ventura-Miranda MI. Experiences of healthcare providers who provide emergency care to migrant children who arriving in Spain by small boats (Patera): a qualitative study. Children (Basel). 2023;10(6):1079.

PubMed   PubMed Central   Google Scholar  

Cabral J, Cuevas AG. Health inequities among latinos/hispanics: documentation status as a determinant of health. J Racial Ethn Health Disparities. 2020;7(5):874–9.

Akincigil A, Mayers RS, Fulghum FH. Emergency room use by undocumented Mexican immigrants. J Sociol Soc Welf. 2011;38(4):33–50.

Google Scholar  

Shachaf S, Davidovitch N, Halpern P, Mor Z. Utilization profile of emergency department by irregular migrants and hospitalization rates: lessons from a large urban medical center in Tel Aviv, Israel. Int J Equity Health. 2020;19(1):56.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69.

Chan TC, Krishel SJ, Bramwell KJ, Clark RF. Survey of illegal immigrants seen in an emergency department. West J Med. 1996;164(3):212–6.

CAS   PubMed   PubMed Central   Google Scholar  

Madden EF, Qeadan F. Dialysis hospitalization inequities by hispanic ethnicity and immigration status. J Health Care Poor Underserved. 2017;28(4):1509–21.

Maldonado CZ, Rodriguez RM, Torres JR, Flores YS, Lovato LM. Fear of discovery among Latino immigrants presenting to the emergency department. Acad Emerg Med. 2013;20(2):155–61.

Wolff H, Epiney M, Lourenco AP, Costanza MC, Delieutraz-Marchand J, Andreoli N, et al. Undocumented migrants lack access to pregnancy care and prevention. BMC Public Health. 2008;8:93.

Eick F, Vallersnes OM, Fjeld HE, Sørbye IK, Ruud SE, Dahl C. Use of emergency primary care among pregnant undocumented migrants over ten years: an observational study from Oslo, Norway. Scand J Prim Health Care. 2023;41(3):317–25.

Organization WH. Emergency care systems for universal health coverage: ensuring timely care for the acutely ill and injured. Draft resolution proposed by Argentina, Ecuador, Eswatini, Ethiopia, Israel, the European Union and its Member States and the United States of America. 2019.

Asch S, Frayne S, Waitzkin H. To discharge or not to discharge: ethics of care for an undocumented immigrant. J Health Care Poor Underserved. 1995;6(1):3–9.

Article   CAS   PubMed   Google Scholar  

Brenner JM, Blutinger E, Ricke B, Vearrier L, Kluesner NH, Moskop JC. Ethical issues in the access to emergency care for undocumented immigrants. J Am Coll Emerg Phys Open. 2021;2(3):e12461.

Granero-Molina J, Jiménez-Lasserrrotte MDM, Fernández-Sola C, Hernández-Padilla JM, Sánchez Hernández F, López DE. Cultural issues in the provision of emergency care to irregular migrants who arrive in spain by small boats. J Transcult Nurs. 2019;30(4):371–9.

DuBard CA, Massing MW. Trends in emergency Medicaid expenditures for recent and undocumented immigrants. JAMA. 2007;297(10):1085–92.

Hsia RY, Gil-González D. Perspectives on Spain’s legislative experience providing access to healthcare to irregular migrants: a qualitative interview study. BMJ Open. 2021;11(8):e050204.

Cervantes L, Richardson S, Raghavan R, Hou N, Hasnain-Wynia R, Wynia MK, et al. Clinicians’ perspectives on providing emergency-only hemodialysis to undocumented immigrants: a qualitative study. Ann Intern Med. 2018;169(2):78–86.

Cervantes L, Tuot D, Raghavan R, Linas S, Zoucha J, Sweeney L, et al. Association of emergency-only vs standard hemodialysis with mortality and health care use among undocumented immigrants with end-stage renal disease. JAMA Intern Med. 2018;178(2):188–95.

Welles CC, Cervantes L. Barriers to providing optimal dialysis care for undocumented immigrants: policy challenges and solutions. Semin Dial. 2020;33(1):52–7.

Download references

Open access funding provided by OsloMet - Oslo Metropolitan University

Author information

Sezer Kisa and Adnan Kisa contributed equally to this work.

Authors and Affiliations

Department of Nursing and Health Promotion, Faculty of Health Sciences, Oslo Metropolitan University, Oslo, Norway

School of Health Sciences, Kristiania University College, Oslo, Norway

Department of International Health and Sustainable Development, Tulane University, New Orleans, USA

You can also search for this author in PubMed   Google Scholar

Contributions

AK, SK: Conceptualization, Methodology, acquisition of data, interpretation of data;  AK: Writing- Original draft preparation, SK: reviewing and editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sezer Kisa .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Kisa, S., Kisa, A. "No Papers, No Treatment": a scoping review of challenges faced by undocumented immigrants in accessing emergency healthcare. Int J Equity Health 23 , 184 (2024). https://doi.org/10.1186/s12939-024-02270-9

Download citation

Received : 12 June 2024

Accepted : 06 September 2024

Published : 14 September 2024

DOI : https://doi.org/10.1186/s12939-024-02270-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Emergency healthcare
• Health equity
• Public health
• Undocumented immigrants

International Journal for Equity in Health

ISSN: 1475-9276

• General enquiries: [email protected]

Research & Reviews: A Journal of Health Professions

• Other Journals
• For Readers
• For Authors
• For Librarians
• Announcements
• Author Guideline
• Referencing Pattern
• Sample Research Paper
• Editorial Board
• Manuscript Withdrawal Policy
• STM Copyright Licensing Form
• Publication Ethics & Malpractice Statement

Socio-Economic Status and Patient Rehabilitation in India: A Community Physiotherapist’s Odyssey

Rehabilitation of patients belonging to lower socio-economic status in India can be challenging. This is due to many reasons. Some of the commonly found causes are lack of adequate health care facilities, patient overload and lack of skilled physiotherapists and lack of awareness among the patients. Given the time and resource constraints in a country like India, it is important to look for cost-effective solutions that could help in alleviation of the symptoms of these patients. One of the cost-effective ways to tackle these issues could be greater emphasis on community-based rehabilitation as opposed to institution-based rehabilitation. There are various advantages of community-based rehabilitation, taking into consideration its cost-effectiveness and time-saving strategies. It can thus be concluded that community-based rehabilitation could be viewed as an effective solution to tackle health issues affecting larger populations, especially those that lack access to institution-based health care facilities.

Keywords:  Rehabilitation, health care, physiotherapist, community-based rehabilitation, institution-based rehabilitation

• There are currently no refbacks.

      

• STM Journals
• Special Issues
• Conferences
• Editorial Board Members
• Reviewers Board Members
• Advisory Panel
• Indexing Bodies
• For Authors
• For Reviewers
• For Editors
• For Advisory Board
• Special Issue Guidelines
• Peer-Review Policy
• Manuscript Submission and Processing
• Publication Ethics and Virtue
• Article Processing Charge
• Editorial Policy
• Advertising Policy
• STM Website and Link Policy
• Distribution and dessemination of Research
• Informed consent Policy
• DOI Payment

"Connect with colleagues and showcase your academic achievements."

"Unleashing the potential of your words"

 "Explore a vast collection of books and broaden your horizons."

 "Empower yourself with the knowledge and skills needed to succeed."

"Collaborate with like-minded professionals and share your knowledge."

"Learn from experts and engage with a community of learners."

• ICDR Group of Companies

• Training Programs

Research & Reviews: A Journal of Health Professions

ISSN: 2277-6192

Journal Menu

Editors overview.

WEBSITE DISCLAIMER

Last updated: 2022-06-15

The information provided by STM Journals (“Company”, “we”, “our”, “us”) on https://journals.stmjournals.com / (the “Site”) is for general informational purposes only. All information on the Site is provided in good faith, however, we make no representation or warranty of any kind, express or implied, regarding the accuracy, adequacy, validity, reliability, availability, or completeness of any information on the Site.

UNDER NO CIRCUMSTANCE SHALL WE HAVE ANY LIABILITY TO YOU FOR ANY LOSS OR DAMAGE OF ANY KIND INCURRED AS A RESULT OF THE USE OF THE SITE OR RELIANCE ON ANY INFORMATION PROVIDED ON THE SITE. YOUR USE OF THE SITE AND YOUR RELIANCE ON ANY INFORMATION ON THE SITE IS SOLELY AT YOUR OWN RISK.

EXTERNAL LINKS DISCLAIMER

The Site may contain (or you may be sent through the Site) links to other websites or content belonging to or originating from third parties or links to websites and features. Such external links are not investigated, monitored, or checked for accuracy, adequacy, validity, reliability, availability, or completeness by us.

WE DO NOT WARRANT, ENDORSE, GUARANTEE, OR ASSUME RESPONSIBILITY FOR THE ACCURACY OR RELIABILITY OF ANY INFORMATION OFFERED BY THIRD-PARTY WEBSITES LINKED THROUGH THE SITE OR ANY WEBSITE OR FEATURE LINKED IN ANY BANNER OR OTHER ADVERTISING. WE WILL NOT BE A PARTY TO OR IN ANY WAY BE RESPONSIBLE FOR MONITORING ANY TRANSACTION BETWEEN YOU AND THIRD-PARTY PROVIDERS OF PRODUCTS OR SERVICES.

PROFESSIONAL DISCLAIMER

The Site can not and does not contain medical advice. The information is provided for general informational and educational purposes only and is not a substitute for professional medical advice. Accordingly, before taking any actions based on such information, we encourage you to consult with the appropriate professionals. We do not provide any kind of medical advice.

Content published on https://journals.stmjournals.com / is intended to be used and must be used for informational purposes only. It is very important to do your analysis before making any decision based on your circumstances. You should take independent medical advice from a professional or independently research and verify any information that you find on our Website and wish to rely upon.

THE USE OR RELIANCE OF ANY INFORMATION CONTAINED ON THIS SITE IS SOLELY AT YOUR OWN RISK.

AFFILIATES DISCLAIMER

The Site may contain links to affiliate websites, and we may receive an affiliate commission for any purchases or actions made by you on the affiliate websites using such links.

TESTIMONIALS DISCLAIMER

The Site may contain testimonials by users of our products and/or services. These testimonials reflect the real-life experiences and opinions of such users. However, the experiences are personal to those particular users, and may not necessarily be representative of all users of our products and/or services. We do not claim, and you should not assume that all users will have the same experiences.

YOUR RESULTS MAY VARY.

The testimonials on the Site are submitted in various forms such as text, audio, and/or video, and are reviewed by us before being posted. They appear on the Site verbatim as given by the users, except for the correction of grammar or typing errors. Some testimonials may have been shortened for the sake of brevity, where the full testimonial contained extraneous information not relevant to the general public.

The views and opinions contained in the testimonials belong solely to the individual user and do not reflect our views and opinions.

ERRORS AND OMISSIONS DISCLAIMER

While we have made every attempt to ensure that the information contained in this site has been obtained from reliable sources, STM Journals is not responsible for any errors or omissions or the results obtained from the use of this information. All information on this site is provided “as is”, with no guarantee of completeness, accuracy, timeliness, or of the results obtained from the use of this information, and without warranty of any kind, express or implied, including, but not limited to warranties of performance, merchantability, and fitness for a particular purpose.

In no event will STM Journals, its related partnerships or corporations, or the partners, agents, or employees thereof be liable to you or anyone else for any decision made or action taken in reliance on the information in this Site or for any consequential, special or similar damages, even if advised of the possibility of such damages.

GUEST CONTRIBUTORS DISCLAIMER

This Site may include content from guest contributors and any views or opinions expressed in such posts are personal and do not represent those of STM Journals or any of its staff or affiliates unless explicitly stated.

LOGOS AND TRADEMARKS DISCLAIMER

All logos and trademarks of third parties referenced on https://journals.stmjournals.com / are the trademarks and logos of their respective owners. Any inclusion of such trademarks or logos does not imply or constitute any approval, endorsement, or sponsorship of STM Journals by such owners.

Should you have any feedback, comments, requests for technical support, or other inquiries, please contact us by email: [email protected] .