rubrics for case study report

Rubric Best Practices, Examples, and Templates

A rubric is a scoring tool that identifies the different criteria relevant to an assignment, assessment, or learning outcome and states the possible levels of achievement in a specific, clear, and objective way. Use rubrics to assess project-based student work including essays, group projects, creative endeavors, and oral presentations.

Rubrics can help instructors communicate expectations to students and assess student work fairly, consistently and efficiently. Rubrics can provide students with informative feedback on their strengths and weaknesses so that they can reflect on their performance and work on areas that need improvement.

How to Get Started

Best practices, moodle how-to guides.

• Workshop Recording (Fall 2022)
• Workshop Registration

Step 1: Analyze the assignment

The first step in the rubric creation process is to analyze the assignment or assessment for which you are creating a rubric. To do this, consider the following questions:

• What is the purpose of the assignment and your feedback? What do you want students to demonstrate through the completion of this assignment (i.e. what are the learning objectives measured by it)? Is it a summative assessment, or will students use the feedback to create an improved product?
• Does the assignment break down into different or smaller tasks? Are these tasks equally important as the main assignment?
• What would an “excellent” assignment look like? An “acceptable” assignment? One that still needs major work?
• How detailed do you want the feedback you give students to be? Do you want/need to give them a grade?

Step 2: Decide what kind of rubric you will use

Types of rubrics: holistic, analytic/descriptive, single-point

Holistic Rubric. A holistic rubric includes all the criteria (such as clarity, organization, mechanics, etc.) to be considered together and included in a single evaluation. With a holistic rubric, the rater or grader assigns a single score based on an overall judgment of the student’s work, using descriptions of each performance level to assign the score.

Advantages of holistic rubrics:

• Can p lace an emphasis on what learners can demonstrate rather than what they cannot
• Save grader time by minimizing the number of evaluations to be made for each student
• Can be used consistently across raters, provided they have all been trained

Disadvantages of holistic rubrics:

• Provide less specific feedback than analytic/descriptive rubrics
• Can be difficult to choose a score when a student’s work is at varying levels across the criteria
• Any weighting of c riteria cannot be indicated in the rubric

Analytic/Descriptive Rubric . An analytic or descriptive rubric often takes the form of a table with the criteria listed in the left column and with levels of performance listed across the top row. Each cell contains a description of what the specified criterion looks like at a given level of performance. Each of the criteria is scored individually.

Advantages of analytic rubrics:

• Provide detailed feedback on areas of strength or weakness
• Each criterion can be weighted to reflect its relative importance

Disadvantages of analytic rubrics:

• More time-consuming to create and use than a holistic rubric
• May not be used consistently across raters unless the cells are well defined
• May result in giving less personalized feedback

Single-Point Rubric . A single-point rubric is breaks down the components of an assignment into different criteria, but instead of describing different levels of performance, only the “proficient” level is described. Feedback space is provided for instructors to give individualized comments to help students improve and/or show where they excelled beyond the proficiency descriptors.

Advantages of single-point rubrics:

• Easier to create than an analytic/descriptive rubric
• Perhaps more likely that students will read the descriptors
• Areas of concern and excellence are open-ended
• May removes a focus on the grade/points
• May increase student creativity in project-based assignments

Disadvantage of analytic rubrics: Requires more work for instructors writing feedback

Step 3 (Optional): Look for templates and examples.

You might Google, “Rubric for persuasive essay at the college level” and see if there are any publicly available examples to start from. Ask your colleagues if they have used a rubric for a similar assignment. Some examples are also available at the end of this article. These rubrics can be a great starting point for you, but consider steps 3, 4, and 5 below to ensure that the rubric matches your assignment description, learning objectives and expectations.

Step 4: Define the assignment criteria

Make a list of the knowledge and skills are you measuring with the assignment/assessment Refer to your stated learning objectives, the assignment instructions, past examples of student work, etc. for help.

  Helpful strategies for defining grading criteria:

• Collaborate with co-instructors, teaching assistants, and other colleagues
• Brainstorm and discuss with students
• Can they be observed and measured?
• Are they important and essential?
• Are they distinct from other criteria?
• Are they phrased in precise, unambiguous language?
• Revise the criteria as needed
• Consider whether some are more important than others, and how you will weight them.

Step 5: Design the rating scale

Most ratings scales include between 3 and 5 levels. Consider the following questions when designing your rating scale:

• Given what students are able to demonstrate in this assignment/assessment, what are the possible levels of achievement?
• How many levels would you like to include (more levels means more detailed descriptions)
• Will you use numbers and/or descriptive labels for each level of performance? (for example 5, 4, 3, 2, 1 and/or Exceeds expectations, Accomplished, Proficient, Developing, Beginning, etc.)
• Don’t use too many columns, and recognize that some criteria can have more columns that others . The rubric needs to be comprehensible and organized. Pick the right amount of columns so that the criteria flow logically and naturally across levels.

Step 6: Write descriptions for each level of the rating scale

Artificial Intelligence tools like Chat GPT have proven to be useful tools for creating a rubric. You will want to engineer your prompt that you provide the AI assistant to ensure you get what you want. For example, you might provide the assignment description, the criteria you feel are important, and the number of levels of performance you want in your prompt. Use the results as a starting point, and adjust the descriptions as needed.

Building a rubric from scratch

For a single-point rubric , describe what would be considered “proficient,” i.e. B-level work, and provide that description. You might also include suggestions for students outside of the actual rubric about how they might surpass proficient-level work.

For analytic and holistic rubrics , c reate statements of expected performance at each level of the rubric.

• Consider what descriptor is appropriate for each criteria, e.g., presence vs absence, complete vs incomplete, many vs none, major vs minor, consistent vs inconsistent, always vs never. If you have an indicator described in one level, it will need to be described in each level.
• You might start with the top/exemplary level. What does it look like when a student has achieved excellence for each/every criterion? Then, look at the “bottom” level. What does it look like when a student has not achieved the learning goals in any way? Then, complete the in-between levels.
• For an analytic rubric , do this for each particular criterion of the rubric so that every cell in the table is filled. These descriptions help students understand your expectations and their performance in regard to those expectations.

Well-written descriptions:

• Describe observable and measurable behavior
• Use parallel language across the scale
• Indicate the degree to which the standards are met

Step 7: Create your rubric

Create your rubric in a table or spreadsheet in Word, Google Docs, Sheets, etc., and then transfer it by typing it into Moodle. You can also use online tools to create the rubric, but you will still have to type the criteria, indicators, levels, etc., into Moodle. Rubric creators: Rubistar , iRubric

Step 8: Pilot-test your rubric

Prior to implementing your rubric on a live course, obtain feedback from:

• Teacher assistants

Try out your new rubric on a sample of student work. After you pilot-test your rubric, analyze the results to consider its effectiveness and revise accordingly.

• Limit the rubric to a single page for reading and grading ease
• Use parallel language . Use similar language and syntax/wording from column to column. Make sure that the rubric can be easily read from left to right or vice versa.
• Use student-friendly language . Make sure the language is learning-level appropriate. If you use academic language or concepts, you will need to teach those concepts.
• Share and discuss the rubric with your students . Students should understand that the rubric is there to help them learn, reflect, and self-assess. If students use a rubric, they will understand the expectations and their relevance to learning.
• Consider scalability and reusability of rubrics. Create rubric templates that you can alter as needed for multiple assignments.
• Maximize the descriptiveness of your language. Avoid words like “good” and “excellent.” For example, instead of saying, “uses excellent sources,” you might describe what makes a resource excellent so that students will know. You might also consider reducing the reliance on quantity, such as a number of allowable misspelled words. Focus instead, for example, on how distracting any spelling errors are.

Example of an analytic rubric for a final paper

Example of a holistic rubric for a final paper, single-point rubric, more examples:.

• Single Point Rubric Template ( variation )
• Analytic Rubric Template make a copy to edit
• A Rubric for Rubrics
• Bank of Online Discussion Rubrics in different formats
• Mathematical Presentations Descriptive Rubric
• Math Proof Assessment Rubric
• Kansas State Sample Rubrics
• Design Single Point Rubric

Technology Tools: Rubrics in Moodle

• Moodle Docs: Rubrics
• Moodle Docs: Grading Guide (use for single-point rubrics)

Tools with rubrics (other than Moodle)

• Google Assignments
• Turnitin Assignments: Rubric or Grading Form

Other resources

• DePaul University (n.d.). Rubrics .
• Gonzalez, J. (2014). Know your terms: Holistic, Analytic, and Single-Point Rubrics . Cult of Pedagogy.
• Goodrich, H. (1996). Understanding rubrics . Teaching for Authentic Student Performance, 54 (4), 14-17. Retrieved from   
• Miller, A. (2012). Tame the beast: tips for designing and using rubrics.
• Ragupathi, K., Lee, A. (2020). Beyond Fairness and Consistency in Grading: The Role of Rubrics in Higher Education. In: Sanger, C., Gleason, N. (eds) Diversity and Inclusion in Global Higher Education. Palgrave Macmillan, Singapore.

Site Search

• How to Search
• Advisory Group
• Editorial Board
• OEC Fellows
• History and Funding
• Using OEC Materials
• Collections
• Research Ethics Resources
• Ethics Projects
• Communities of Practice
• Get Involved
• Submit Content
• Open Access Membership
• Become a Partner

Undergraduate Case Analysis Rubric

This rubric can be used for guiding undergraduate case analysis for the course " Genomics, Ethics, and Society ."

Related Resources

Submit Content to the OEC   Donate

This material is based upon work supported by the National Science Foundation under Award No. 2055332. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Case Study - Rubric

• Available Topics
• Top Documents
• Recently Updated
• Internal KB

Rubric example: a case study

• Investigation and Research Discussions
• Case Study - Description
• Case Study - Example
• Affordances of Online Discussions
• Steps for Building an Online Asynchronous Discussion
• Using Online Discussions to Increase Student Engagement

Case Studies

Case studies (also called "case histories") are descriptions of real situations that provide a context for engineers and others to explore decision-making in the face of socio-technical issues, such as environmental, political, and ethical issues. case studies typically involve complex issues where there is no single correct answer--a student analyzing a case study may be asked to select the "best" answer given the situation 1 . a case study is not a demonstration of a valid or "best" decision or solution. on the contrary, unsuccessful or incomplete attempts at a solution are often included in the written account. 2.

The process of analyzing a case study encourages several learning tasks:

Exploring the nature of a problem and circumstances that affect a decision or solution

Learning about others' viewpoints and how they may be taken into account

Learning about one's own viewpoint

Defining one's own priorities

Making one's own decisions to solve a problem

Predicting outcomes and consequences 1

Student Learning Outcomes in Ethics

Most engineering case studies available pertain to engineering ethics. After a two year study of education in ethics sponsored by the Hastings Center, an interdisciplinary group agreed on five main outcomes for student learning in ethics:

Sensitivity to ethical issues, sometimes called "developing a moral imagination," or the awareness of the needs of others and that there is an ethical point of view;

Recognition of ethical issues or the ability to see the ethical implications of specific situations and choices;

Ability to analyze and critically evaluate ethical dilemmas, including an understanding of competing values, and the ability to scrutinize options for resolution;

Ethical responsibility, or the ability to make a decision and take action;

Tolerance for ambiguity, or the recognition that there may be no single ideal solution to ethically problematic situations 2 .

These outcomes would make an excellent list of attributes for designing a rubric for a case analysis.

Ideas for Case Study Assignments

To assign a case analysis, an instructor needs

skill in analyzing a case (and the ability to model that process for students)

skill in managing classroom discussion of a case

a case study

a specific assignment that will guide students' case analyses, and

a rubric for scoring students' case analyses.

Below are ideas for each of these five aspects of teaching with case studies. Another viewpoint is to consider how not to teach a case study .

1. Skill in analyzing a case

For many engineering instructors, analyzing cases is unfamiliar. Examining completed case analyses could help develop case analysis skills. As an exercise for building skill in analyzing cases, use the generic guidelines for case analysis assignments (#4 below) to carefully review some completed case analyses. A few completed case analyses are available:

Five example analyses of an engineering case study

Case study part 1 [Unger, S. The BART case: ethics and the employed engineer. IEEE CSIT Newsletter. September 1973 Issue 4, p 6.]

Case study part 2 [Friedlander, G. The case of the three engineers vs. BART. IEEE Spectrum. October 1974, p. 69-76.]

Case study part 3 [Friedlander, G. Bigger Bugs in BART? IEEE Spectrum . March 1973. p32,35,37.]

Case study with an example analysis

2. Skill in managing classroom discussion of a case

Managing classroom discussion of a case study requires planning.

Suggestions for using engineering cases in the classroom

Guidelines for leading classroom discussion of case studies

3. Case studies

Case studies should be complex enough and realistic enough to be challenging, yet be manageable within the time frame. It is time-consuming to create case studies, but there are a large number of engineering case studies online.

Online Case Libraries

Case Studies in Technology, Ethics, Environment, and Public Policy

Teaching Engineering Ethics: A Case Study Approach

The Online Ethics Center for Engineering and Science

Ethics Cases

The Engineering Case Library

Cases and Teaching Tips

4. A specific assignment that will guide students' case analyses

There are several types of case study assignment:

Nine approaches to using case studies for teaching

Written Case Analysis

Case Discussions

Case analyses typically include answering questions such as:

What kinds of problems are inherent in the situation?

Describe the socio-technical situation sufficiently to enable listeners (or readers) to understand the situation faced by the central character in the case.

Identify and characterize the issue or conflict central to the situation. Identify the parties involved in the situation. Describe the origins, structure, and trajectory of the conflict.

Evaluate the strengths and weaknesses of the arguments made by each party.

How would these problems affect the outcomes of the situation?

Describe the possible actions that could have been taken by the central character in the case.

Describe, for each possible action, what the potential outcomes might be for each party involved.

Describe what action was actually taken and the outcomes for each party involved.

How would you solve these problems? Why?

Describe the action you would take if you were the central character in the case. Explain why.

What should the central character in the situation do? Why?

Describe the action you think that the central character in the case should take. Explain why.

What can be learned from this case?

Delineate the lessons about ethical (or other) issues in engineering that are illuminated by this case.

This list is adapted from two online case analysis assignments by McGinn from 3 & 4 ):

5. A rubric for scoring students' case analyses

Case studies help students explore decision-making in the face of issues. Thus, for an engineering ethics case study, the outcomes that can be assessed by scoring case analyses are a) sensitivity to ethical issues, b) recognition of ethical issues, c) the ability to analyze and critically evaluate ethical dilemmas, d) the ability to make an ethical decision and take action, and e) tolerance for ambiguity. Scoring rubrics for ethics case analyses should address these outcomes, not basic knowledge of the ethical standards of the profession. Professional standards can best be assessed by a traditional graded exam in which students must demonstrate, for example, which practices are ethically acceptable versus which are in violation of ethical standards given a hypothetical scenario 5 .

Making Scoring/Grading Useful for Assessment

General principles for making scoring/grading useful for assessment ( rubrics )

Example rubrics

For a written analysis of a case study in engineering

For a written analysis of a case study in general #1

For a written analysis of a case study in general #2

For a written and oral analysis of a case study by a group

For an oral analysis of a case study by a group

For a written analysis of a case study on ethics

For a self-assessment of learning from a case study

• school Campus Bookshelves
• menu_book Bookshelves
• perm_media Learning Objects
• login Login
• how_to_reg Request Instructor Account
• hub Instructor Commons
• Download Page (PDF)
• Download Full Book (PDF)
• Periodic Table
• Physics Constants
• Scientific Calculator
• Reference & Cite
• Tools expand_more
• Readability

selected template will load here

This action is not available.

6.1: Rubrics for Exams and Group Projects in Ethics

• Last updated
• Save as PDF
• Page ID 20524

• William Frey and Jose a Cruz-Cruz
• University of Puerto Rico - Mayaguez

Introduction

This module provides a range of assessment rubrics used in classes on engineering and computer ethics. Rubrics will help you understand the standards that will be used to assess your writing in essay exams and group projects. They also help your instructor stay focused on the same set of standards when assessing the work of the class. Each rubric describes what counts as exceptional writing, writing that meets expectations, and writing that falls short of expectations in a series of explicit ways. The midterm rubrics break this down for each question. The final project rubrics describe the major parts of the assignment and then break down each part according to exceptional, adequate, and less than adequate. These rubrics will help you to understand what is expected of you as you carry out the assignment, provide a useful study guide for the activity, and familiarize you with how your instructor has assessed your work.

Course Syllabi

Syllabus for Environments of the Organization: ADMI4016_F10.docx

Syllabus for Business, Society, and Government: ADMI6055_F10.docx

Syllabus for Business Ethics: Business Ethics Spring 2007.doc

Syllabus for Business Ethics, Spring 2008: Syllabus_S08_W97.doc

Business Ethics Syllabus Presentation - This presentation was given on the first day of class in Business Ethics, Fall 2007. It summarizes the course objectives, grading events, and also provides a PowerPoint slide of the College of Business Administration's Statement of Values: BE_Intro_F07.ppt

Rubrics Used in Connexions Modules Published by Author

Ethical theory rubric.

This first rubric assesses essays that seek to integrate ethical theory into problem-solving. It looks at a rights-based approach consistent with deontology, a consequentialist approach consistent with utilitarianism, and virtue ethics. The overall context is a question presenting a decision scenario followed by possible solutions. The point of the essay is to evaluate a solution in terms of a given ethical theory.

Ethical Theory Integration Rubric - This rubric breaks down the assessment of an essay designed to integrate the ethical theories of deontology, utilitarianism, and virtue into a decision-making scenario: EE_Midterm_S05_Rubric.doc

Decision-Making / Problem-Solving Rubric

This next rubric assesses essays that integrate ethical considerations into decision making by means of three tests, reversibility, harm/beneficence, and public identification. The tests can be used as guides in designing ethical solutions or they can be used to evaluate decision alternatives to the problem raised in an ethics case or scenario. Each theory partially encapsulates an ethical approach: reversibility encapsulates deontology, harm/beneficence utilitarianism, and public identification virtue ethics. The rubric provides students with pitfalls associated with using each test and also assesses their set up of the test, i.e., how well they build a context for analysis.

Integrating Ethics into Decision-Making through Ethics Tests - Attached is a rubric in MSWord that assesses essays that seek to integrate ethical considerations into decision-making by means of the ethics tests of reversibility, harm/beneficence, and public identification: CE_Rubric_S06.doc

Ethics Bowl Follow-Up Exercise Rubric

Student teams in Engineering Ethics at UPRM compete in two Ethics Bowls where they are required to make a decision or defend an ethical stance evoked by a case study. Following the Ethics Bowl, each group is responsible for preparing an in-depth case analysis on one of the two cases they debated in the competition. The following rubric identifies ten components of this assignment, assigns points to each, and provides feedback on what is less than adequate, adequate, and exceptional. This rubric has been used for several years to evaluate these group projects.

In-Depth Case Analysis Rubric - This rubric will be used to assess a final, group written, in-depth case analysis. It includes the three frameworks referenced in the supplemental link provided above: EE_FinalRubric_S06.doc

Rubric for Good Computing / Social Impact Statements Reports

This rubric provides assessment criteria for the Good Computing Report activity that is based on the Social Impact Statement Analysis described by Chuck Huff at www.computingcases.org. (See link) Students take a major computing system, construct the socio-technical system which forms its context, and look for potential problems that stem from value mismatches between the computing system and its surrounding socio-technical context. The rubric characterizes less than adequate, adequate, and exceptional student Good Computing Reports.

Good Computing Report Rubric - This figure provides the rubric used to assess Good Computing Reports in Computer Ethics classes: CE_FinalRubric_S06.doc

Computing Cases provides a description of a Social Impact Statement report that is closely related to the Good Computing Report. Value material can be accessed by looking at the components of a Socio-Technical System and how to construct a Socio-Technical System Analysis.

Business Ethics Midterm Rubric Spring 2008 - Clicking on this link will open the rubric for the business ethics midterm exam for spring 2008: Midterm Rubric Spring 2008.doc

Study Materials for Business Ethics

This section provides models for those who would find the Jeopardy game format useful for helping students learn concepts in business ethics and the environments of the organization. It incorporates material from modules in the Business Course and from Business Ethics and Society, a textbook written by Anne Lawrence and James Weber and published by McGraw-Hill. Thanks to elainefitzgerald.com for the Jeopardy template.

Jeopardy: Business Concepts and Frameworks: Jeopardy1Template.pptx Jeopardy2.pptx

Privacy, Property, Free Speech, Responsibility: Jeopardy_3.pptx

Jeopardy for EO Second Exam: Jeopardy4a.pptx

Jeopardy 5: Jeopardy5.pptx

Jeopardy 6: Jeopardy6.pptx

Jeopardy 7: Jeopardy7.pptx

• help_outline help

iRubric: Case Study rubric

• Case study rubric

• Open access
• Published: 11 May 2024

The reliability of the College of Intensive Care Medicine of Australia and New Zealand “Hot Case” examination

• Kenneth R. Hoffman 1 , 2 ,
• David Swanson 3 ,
• Stuart Lane 4 ,
• Chris Nickson 1 , 2 ,
• Paul Brand 5 &
• Anna T. Ryan 3  

BMC Medical Education volume  24 , Article number:  527 ( 2024 ) Cite this article

Metrics details

High stakes examinations used to credential trainees for independent specialist practice should be evaluated periodically to ensure defensible decisions are made. This study aims to quantify the College of Intensive Care Medicine of Australia and New Zealand (CICM) Hot Case reliability coefficient and evaluate contributions to variance from candidates, cases and examiners.

This retrospective, de-identified analysis of CICM examination data used descriptive statistics and generalisability theory to evaluate the reliability of the Hot Case examination component. Decision studies were used to project generalisability coefficients for alternate examination designs.

Examination results from 2019 to 2022 included 592 Hot Cases, totalling 1184 individual examiner scores. The mean examiner Hot Case score was 5.17 (standard deviation 1.65). The correlation between candidates’ two Hot Case scores was low (0.30). The overall reliability coefficient for the Hot Case component consisting of two cases observed by two separate pairs of examiners was 0.42. Sources of variance included candidate proficiency (25%), case difficulty and case specificity (63.4%), examiner stringency (3.5%) and other error (8.2%). To achieve a reliability coefficient of > 0.8 a candidate would need to perform 11 Hot Cases observed by two examiners.

The reliability coefficient for the Hot Case component of the CICM second part examination is below the generally accepted value for a high stakes examination. Modifications to case selection and introduction of a clear scoring rubric to mitigate the effects of variation in case difficulty may be helpful. Increasing the number of cases and overall assessment time appears to be the best way to increase the overall reliability. Further research is required to assess the combined reliability of the Hot Case and viva components.

Peer Review reports

Credentialling medical specialists requires defined performance standards [ 1 , 2 ] and traditionally relies upon high stakes examinations to assess trainees against those standards [ 3 , 4 , 5 ]. These examinations substitute for controlling quality of care by attempting to control progression through training programs for the safety of both patients and society. Specialist colleges are also expected to provide transparent and fair assessment processes, to ensure defensible decisions are made regarding trainee progression and specialist credentialling [ 6 ].

The College of Intensive Care Medicine of Australia and New Zealand (CICM) second part examination was introduced in 1979 and has undergone many revisions [ 3 ]. It has two components: a written examination and, if completed successfully, an oral examination. The oral examination includes an eight-station viva assessment and two clinical “Hot Case” assessments. This Hot Case component targets the highest level of assessment on Miller’s Pyramid [ 7 ], ‘Does’, requiring candidates to be assessed in the workplace performing real-world tasks. Of the candidates who have passed the written examination successfully, only 35% pass both Hot Cases [ 8 ]. It is therefore important to evaluate both the validity of inferences from this examination component and the reliability or reproducibility of the results [ 9 ].

Reliability describes the degree to which variation in scores reflects true variability in candidates’ proficiency, rather than measurement error. This is dependent on the task, examiner stringency and assessment context [ 10 ]. Reliability can be quantified using the reliability coefficient, with 0 representing a completely unreliable assessment and 1 representing a completely reliable assessment. The minimum standard generally considered acceptable for high stakes medical examinations is a reliability coefficient greater than 0.8 [ 11 , 12 , 13 , 14 ].

Generalisability theory (G-theory) provides the statistical basis for combining multiple sources of variance into a single analysis [ 15 ]. This enables the calculation of an overall reliability coefficient and calculation of the contribution from candidates, cases and examiners to examination reliability. G-theory also provides the basis for conducting decision studies (D-studies) that statistically project reliability based on alternate assessment designs.

To date, no information on the reliability of the CICM second part examination has been published. Given the implications of incorrect credentialling decisions for trainees, patients and society, the Hot Case reliability coefficient should be quantified.

Examination format

The second part examination prior to COVID-19 was held twice yearly with candidates invited to the oral component in a single Australian city. Trainees complete two Hot Cases within metropolitan intensive care units (ICU) with 20 min allocated for each: 10 min to examine an ICU patient, followed by 10 min with paired examiners to present their findings and answer questions regarding investigations and clinical management.

Format changes occurred during the COVID-19 pandemic. The first oral examination was cancelled in 2020, with trainees deferring to the second sitting. Additionally, travel restrictions meant candidates sat the Hot Case component in their home city with local examiners from the second sitting in 2020 to the second sitting in 2021. From 2022 onwards, the oral examination has been held in Sydney, Melbourne, or both.

Hot Cases are marked out of 10 by two CICM examiners using a rating scale that scores candidates based on how comfortable examiners would be supervising them. An acceptable pass standard (5/10) indicates an examiner is comfortable to leave the candidate in charge of the ICU with minimal supervision. There is no specific scoring rubric, although examiner pairs cooperatively determine clinical signs that should be identified, nominate investigations and imaging to show a candidate, and specify discussion questions. Expected levels of knowledge, interpretation and clinical management are defined prospectively. An automatic fail for the entire oral examination is triggered if candidates fail both Hot Cases and obtain a Hot Case component mark < 40% of the possible marks.

Examiner calibration

Examiners undergo calibration training prior to the examination. They independently score the candidate, then discuss their individual scores and rationale. Examiners can then amend their score before recording final scores in the examination database. Each Hot Case is marked by separate pairs of examiners, to prevent bias from a candidates first case performance influencing their second case score. Following the examination, results are presented to the whole examiner cohort for further discussion and explanation.

Data collection

The CICM provided access to their examination database from the second sitting of 2012 (2012-2) through to the first sitting of 2022 (2022-1). For each de-identified candidate, the written mark, overall Hot Case mark, viva mark, and overall examination mark were obtained. The Hot Case specific data included the cases used, examiners present and individual examiner marks, with a total of four scores per candidate (two examiner scores for each Hot Case).

Analysis was restricted to 2019-1 to 2022-1 due to data recording inconsistency providing insufficient data for G-theory analysis. Additionally, changes occurred from 2019-1 with the introduction of the Angoff standard setting method [ 16 , 17 ] for the written examination. This altered final score calculation with the written examination functioning as a barrier examination, although the written score no longer contributes to the final examination score. Candidates were included if they sat the oral examination for the first time in 2019 or later and, if they failed, subsequent attempts were recorded.

Statistical analysis

Statistical analysis used Microsoft Excel and SPSS. Continuous examination scores were summarised using mean and standard deviation. Categorical variables were reported as counts and percentages. Frequency distributions (histograms) were used to graph overall examination component results. A p-value of < 0.05 indicated statistical significance. Comparisons of examiner marks and relationships between examination components were analysed with Pearson’s correlation coefficient and visually represented with scatterplots.

G-theory analysis was used to calculate an overall reliability coefficient for the Hot Case examination, and the factors contributing to variance. As examiners observed multiple candidates and candidates performed multiple Hot Cases, the design was partially crossed. However, as the case identification numbers used in the examination were recorded variably, the initial design was modified to treat cases as nested within candidates for the analysis. The variance factors being analysed included candidate proficiency, examiner stringency, case to case performance variability (case specificity) and other unspecified measurement error. These were reported with variance components, square roots of variance components and percentage of total variance. G-theory was used to conduct D-studies exploring the impact of alternate assessment designs on overall generalisability coefficients and associated standard errors of measurement. The D-study calculated the generalisability coefficient based on the equation listed in Fig.  1 .

Generalisability coefficient equation

Overall, there were 889 candidate oral examination attempts from 2012-2 to 2022-1. After exclusion of candidate oral examination attempts prior to the 2019-1 sitting, exclusion of candidates with first attempts prior to 2019-1 and exclusion of one candidate with missing Hot Case scores, there were 296 candidate oral examination attempts analysed. This included 166 first attempts, 100 s and 30 third attempts. This resulted in 592 Hot Case results and 1184 individual examiner Hot Case scores. The recruitment, exclusion and analysis of the sample are presented in Fig.  2 .

CONSORT style diagram demonstrating the sample size from data request through to the sample available for analysis

The mean and standard deviation of individual examiner Hot Case scores from all examiners was 5.17 and 1.65 respectively. Of the 1184 Hot Case individual examiner scores, 645 (54.5%) achieved a score 5 or greater, and 539 (45.5%) scored less than 5. The distribution of individual examiner Hot Case scores is presented in Fig.  3 . First attempt candidates scored higher than those repeating (5.25 (SD1.63) vs. 4.89 (SD1.66) p  = < 0.01).

Histogram showing individual examiner Hot Case scores for all attempts

Scores on each Hot Case are calculated as the mean of the two individual examiner Hot Case scores. Overall, 312 of 592 Hot Cases were passed (52.7%). The correlation coefficient between candidates first and second Hot Cases was low at 0.30 (Fig.  4 ).

The correlation between each candidate’s first and second Hot Case scores. A jitter function was applied to spread overlying points

The correlation coefficient between examiners observing the same case (inter-rater agreement) was high at 0.91 (Fig.  5 ).

The summary of sources of variance for individual examiner Hot Case scores is presented in Table  1 .

Comparison between Hot Case scores from the first and second examiners. A jitter function was applied to spread overlying points

The overall generalisability coefficient of the Hot Case component including two separate cases observed by two examiners each was 0.42.

The results for the D-studies are presented in Table  2 . To achieve a generalisability coefficient of 0.8 or greater, 11 Hot Cases with two examiners would be needed. A graph comparing the generalisability coefficients for one and two examiners is presented in Fig.  6 .

Generalisability coefficients with a variable number of cases comparing examination designs with one and two examiners observing each case

The current examination format with two Hot Cases observed by two examiners has a reliability coefficient of 0.42. To achieve the widely accepted standard for high stakes examinations of a reliability coefficient of > 0.8 requires each candidate to sit 11 Hot Cases with two examiners.

These results are similar to The Royal Australasian College of Physicians (RACP) 60-minute long case examination observed by two examiners which has a reliability coefficient of 0.38 [ 18 ]. When the assessment time is lengthened with two long cases and four short cases, the RACP achieved a reliability coefficient of 0.71. The RACP continues to use long case examinations, as they are valued by examiners and trainees as an authentic measure of competence with an educational impact from examination preparation [ 18 ]. Educational impact is commonly cited as a reason to retain clinical examinations [ 4 , 19 , 20 ].

G-theory analysis demonstrates that examiners appear well calibrated, as examiner variance was responsible for only 3.5% of overall variance in Hot Case scores. Therefore, adding additional examiners would not substantially improve reliability. However, this conclusion may be affected by the extent of discussion between the examiners prior to recording their amended final scores. If discussion influences the opinions of an examiner strongly, it is likely there will be higher correlation between examiner scores. To evaluate this effect, independent examiner scores would need to be recorded prior to discussion, with clear guidelines around acceptable amendments to scores.

The finding that the majority of Hot Case variance (63.4%) arises from case variation is consistent with anecdotal reports from examination candidates who describe case difficulty as a “lucky dip”. This finding is consistent with the poor correlation (0.30) between candidates’ first and second Hot Cases. Whilst examiners preview the Hot Case patient, there is no formal method of quantifying and adjusting for the difficulty of each case. According to Kane’s Validity Framework [ 21 ], it is difficult to argue that the assessment is valid if the initial scoring and subsequent generalisation of those scores are based more on case specificity than candidate proficiency, particularly when the implications of the results are significant for candidates and patient safety. The CICM has introduced the Angoff method [ 16 ] for the written examination to account for variation in question difficulty and an appropriate standard setting method for the Hot Case component may mitigate this degree of case variability to some extent. The CICM has avoided the use of norm referenced assessments where candidates are compared with their peers so that all candidates deemed competent are eligible to pass. This is appropriate given the low number of candidates in each sitting, the low number of candidates taken to each case and high variability in case difficulty.

Case specificity is the concept that candidate performance is dependent on the case used and is a major issue in specialist credentialling examinations [ 4 ]. Problem solving ability and clinical reasoning are based on prior clinical experience, areas of particular interest and background knowledge. Candidate performance may be highly case specific, meaning limited numbers of examination cases have detrimental effects on reliability [ 4 , 5 , 22 ]. In the literature, increasing case numbers or overall assessment time is commonly proposed as a method of obtaining more generalisable results [ 6 , 18 , 23 , 24 ]. However, having a candidate pass overall, but clearly fail a component of a credentialling examination may be difficult to justify as defensible from the perspective of patient safety and societal obligations.

The individual examiner Hot Case scores (5.17, SD 1.65) are close to the 50% pass fail boundary. This makes examiners’ decision making difficult, with potentially small differences in performance determining a pass or fail. This is demonstrated in the histogram in Fig.  3 , with a large proportion of trainees scoring a 4.5 or 5, the junction between a pass and a fail. This dichotomisation should be supported by a clear rubric defining what constitutes a minimally competent performance. This will also give candidates clearer performance expectations and may mitigate variability due to case difficulty and specificity by defining expected competencies which are independent of the case difficulty.

Assessing the quality of future care using examination performance as a substitute marker of competence has limitations [ 11 ]. There are concerns from a validity point of view regarding decision making based on short periods of assessment [ 6 , 9 , 10 , 18 , 25 ]. As such, credentialling examinations should focus on identifying low end outliers, a possible true risk to patients and society without further training. Rather than failing candidates with a borderline performance, the focus should be on increasing the sample size to guide decision making. Additional Hot Cases for those with a borderline performance on the oral examination is a possible solution, to increase the reliability for defensible decision making. Summative Hot Cases performed during the training program, but not at the time of the final examination, is another option to increase available data through a transition to a programmatic style of longitudinal assessment.

Restricting the analysis for candidates who sat the written from the 2019-1 sitting onwards was necessitated by the quality of the available dataset. This aided analysis as the Angoff method was introduced for the written paper in 2019 [ 17 ] with the written score no longer counting toward the overall examination score. Candidates are now considered to have passed or failed the written, and then to pass the oral examination they require > 50% from the Hot Case component (worth 30 marks) and viva component (worth 40 marks) combined. This results in a higher benchmark to pass the examination overall, as previously a strong written mark could contribute to an overall pass despite a weaker oral performance.

This research fills a gap in the current understanding of credentialling intensive care physicians. However, it should be taken in context of the overall assessment process. If high stakes assessment requires a reliability coefficient of > 0.8, this value should be the benchmark for the combined oral examination including the Hot Cases and viva component. Further research is required to assess how the Hot Case component and the viva component interact to form the overall reliability of the oral examination.

The strengths of this study include the originality, the predefined statistical plan, the large cohort and the collaboration with the CICM to provide previously unexamined data for an independent analysis. Additionally, the use of descriptive statistics, G-theory analysis and D-studies provides a comprehensive picture of the Hot Case examination reliability in its current format.

Study limitations include dataset consistency issues that restricted the study period, the focus specifically on the Hot Case component without an in-depth analysis of the other components of the examination, the focus on traditional psychometric evaluation and the potential overestimation of examiner calibration due to revision of examiner scores after discussion. Evaluating examination performance without external measures of candidate ability is a research design that focuses on the examination itself. Assessment research is often not truly focussed on candidate competence as this is very difficult to study, so it inevitably evaluates the process rather than the product. As such, identifying poor reliability as a weakness of the Hot Case examination does not detract from potential validity in the overall examination process.

Several implications and unanswered questions remain. Firstly, examiners appear well calibrated, but discussion and score amendment may be significant. Secondly, with additional examiner time, reliability could be increased by challenging candidates with borderline results with additional cases upon which decisions are made. Thirdly, this research highlights the importance of a scoring rubric and robust processes for data capture. Finally, further research is required to assess how the Hot Case and viva examination interact to test the overall reliability of the oral examination. This should be supported by research aiming to assess the validity of the Hot Case as a method of evaluating clinical competence by comparing it with other forms of assessment and workplace competency.

Hot Cases have long been a method of assessment in ICU training in Australia and New Zealand, with perceived benefits from the perspective of stakeholder acceptance and educational impact. Changes to the current examination format to increase reliability would solidify its role in the credentialling process by addressing concerns within the ICU community.

The reliability of the CICM Hot Case examination is less than the generally accepted standard for a high stakes credentialling examination. Further examiner training is unlikely to improve the reliability as the examiners appear to be well calibrated. Modifications to case selection and the introduction of a clear scoring rubric to mitigate the effects of variation in case difficulty may be helpful, but are unlikely to improve reliability substantially due to case specificity. Increasing the number of cases and overall assessment time appears to be the best way to increase the overall reliability. Further research is required to assess how the Hot Case and viva results interact to quantify the reliability of the oral examination in its entirety, and to evaluate the validity of the examination format in making credentialling decisions.

Data availability

The datasets analysed during the current study are not publicly available, but are available from the corresponding author on reasonable request.

Burch VC, Norman GR, Schmidt HG, van der Vleuten CP. Are specialist certification examinations a reliable measure of physician competence? Adv Health Sci Educ Theory Pract. 2008;13(4):521–33. https://doi.org/10.1007/s10459-007-9063-5 .

Article   Google Scholar  

Norcini J, Anderson MB, Bollela V, Burch V, Costa MJ, Duvivier R, Hays R, Palacios Mackay MF, Roberts T, Swanson D. Consensus framework for good assessment. Med Teach. 2018;40(11):1102–9. https://doi.org/10.1080/0142159X.2018.1500016 .

Lee RP, Venkatesh B, Morley P. Evidence-based evolution of the high stakes postgraduate intensive care examination in Australia and New Zealand. Anaesth Intensive Care. 2009;37(4):525–31. https://doi.org/10.1177/0310057X0903700422 .

Turnbull J, Turnbull J, Jacob P, Brown J, Duplessis M, Rivest J. Contextual considerations in summative competency examinations: relevance to the long case. Acad Med. 2005;80(12):1133–7. https://doi.org/10.1097/00001888-200512000-00014 .

Memon MA, Joughin GR, Memon B. Oral assessment and postgraduate medical examinations: establishing conditions for validity, reliability and fairness. Adv Health Sci Educ Theory Pract. 2010;15(2):277–89. https://doi.org/10.1007/s10459-008-9111-9 .

Lane AS, Roberts C, Khanna P. Do we know who the Person with the Borderline score is, in standard-setting and decision-making. Health Prof Edu. 2020;6(4):617–25. https://doi.org/10.1016/j.hpe.2020.07.001 .

Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 Suppl):S63–7. https://doi.org/10.1097/00001888-199009000-00045 . PMID: 2400509.

College of Intensive Care Medicine of Australia and New Zealand. Second part examination: previous exam reports [Internet]. CICM. 2022 [updated 2023; cited 2023 Oct 30]. https://www.cicm.org.au/CICM_Media/CICMSite/Files/Exams/2022-1-SP-Exam-Report.pdf .

Hoffman K, Nickson CP, Ryan AT, Lane S. Too hot to handle? The validity and reliability of the college of intensive Care Medicine Hot Case examination. Crit Care Resusc. 2022;24(1):87–92. https://doi.org/10.51893/2022.1.L .

Van Der Vleuten C. The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ. 1996;1:41–67. https://doi.org/10.1007/BF00596229 .

Bloch R, Norman G. Generalizability theory for the perplexed: a practical introduction and guide: AMEE Guide 68. Med Teach. 2012;34(11):960–92. https://doi.org/10.3109/0142159X.2012.703791 .

Moonen-van Loon JM, Overeem K, Donkers HH, van der Vleuten CP, Driessen EW. Composite reliability of a workplace-based assessment toolbox for postgraduate medical education. Adv Health Sci Educ Theory Pract. 2013;18(5):1087–102. https://doi.org/10.1007/s10459-013-9450-z .

Crossley J, Davies H, Humphris G, Jolly B. Generalisability: a key to unlock professional assessment. Med Educ. 2002;36(10):972–8. https://doi.org/10.1046/j.1365-2923.2002.01320.x .

Weller JM, Castanelli DJ, Chen Y, Jolly B. Making robust assessments of specialist trainees’ workplace performance. Br J Anaesth. 2017;118(2):207–14. https://doi.org/10.1093/bja/aew412 .

Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N. The dependability of behavioral measurements: theory of generalizability for scores and profiles. New York: Wiley; 1972.

Google Scholar  

Angoff WH. Scales, norms, and equivalent scores. Educational Measurement. Second Edition. Washington: American Coucil on Education; 1971.

Karcher C. The Angoff method in the written exam of the College of Intensive Care Medicine of Australia and New Zealand: setting a new standard. Crit Care Resusc. 2019;21(1):6–8.

Wilkinson TJ, Campbell PJ, Judd SJ. Reliability of the long case. Med Educ. 2008;42(9):887–93. https://doi.org/10.1111/j.1365-2923.2008.03129.x .

Tey C, Chiavaroli N, Ryan A. Perceived educational impact of the medical student long case: a qualitative study. BMC Med Educ. 2020;20:257. https://doi.org/10.1186/s12909-020-02182-6 .

Sim J, Daniel E. The long case as an assessment tool of clinical skills in summative assessment: a necessary evil. Int Med J. 2015;22:537–40.

Kane MT. Validating the interpretations and uses of test scores. J Educ Meas. 2013;50:1–73.

Swanson DB, Norman GR, Linn RL. Performance-based assessment: lessons from the health professions. Educ Res. 1995;24:5e11.

Wass V, Jolly B. Does observation add to the validity of the long case? Med Educ. 2001;35(8):729–34. https://doi.org/10.1046/j.1365-2923.2001.01012.x .

Van Der Vleuten C, Schuwirth L. Assessing professional competence: from methods to programmes. Med Educ. 2005;39(3):309–17. https://doi.org/10.1111/j.1365-2929.2005.02094.x .

Dijkstra J, Galbraith R, Hodges B, et al. Expert validation of fit-for-purpose guidelines for designing programmes of assessment. BMC Med Educ. 2012;1712:20. https://doi.org/10.1186/1472-6920-12-20 .

Download references

The CICM provided staff time for data extraction and de-identification prior to release. There was no additional funding provided for the research.

Author information

Authors and affiliations.

Intensive Care Unit, The Alfred Hospital, Melbourne, Australia

Kenneth R. Hoffman & Chris Nickson

Department of Epidemiology and Preventative Medicine, School of Public Health, Monash University, Melbourne, Australia

Department of Medical Education, Melbourne Medical School, University of Melbourne, Melbourne, Australia

David Swanson & Anna T. Ryan

Sydney Medical School, The University of Sydney, Sydney, Australia

Stuart Lane

College of Intensive Care Medicine of Australia and New Zealand, Melbourne, Australia

You can also search for this author in PubMed   Google Scholar

Contributions

KH conceived and designed the study, analysed and interpreted the data and drafted and revised the manuscript. DS designed the study, performed the analysis, interpreted the data and revised the manuscript. SL contributed to the conception, design and interpretation of the study and revised the manuscript. CN contributed to the conception, design and interpretation of the study and revised the manuscript. PB contributed to data acquisition and analysis and revised the manuscript. AR contributed to the conception, design and interpretation of the study and revised the manuscript.

Corresponding author

Correspondence to Kenneth R. Hoffman .

Ethics declarations

Ethical approval and consent to participate.

Ethics approval was provided by the University of Melbourne Low Risk Human Ethics Committee (Project number 2022-23964-28268-3). The consent requirement was waived by the University of Melbourne Low Risk Human Ethics Committee as analysis was retrospective using de-identified data with no foreseeable risk to participants in accordance with the Australian National Statement on Ethical Conduct in Human Research 2023.

Consent for publication

Not applicable.

Competing interests

KH (Fellow of the CICM), AR (No conflicts of interest), SL (Fellow of the CICM, CICM Second Part examiner 2011-2023, CICM Second Part examination committee 2019-2023, Chair of the CICM Second Part examination panel 2020-2023, CICM First Part examiner 2012-2019), CN (Fellow of the CICM, CICM First Part examiner 2017-2023, CICM First Part examination committee 2019-2023, CICM Supervisor of Training 2018-2023), DS (No conflicts of interest), PB (Employed by the CICM in the position of Information, Communication and Technology Manager).

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hoffman, K.R., Swanson, D., Lane, S. et al. The reliability of the College of Intensive Care Medicine of Australia and New Zealand “Hot Case” examination. BMC Med Educ 24 , 527 (2024). https://doi.org/10.1186/s12909-024-05516-w

Download citation

Received : 21 February 2024

Accepted : 03 May 2024

Published : 11 May 2024

DOI : https://doi.org/10.1186/s12909-024-05516-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Intensive care
• Examination
• Credentialling
• Reliability
• Generalisability theory

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

The fishy death of Red Lobster

Endless Shrimp didn't sink the seafood chain. Wall Street did.

With the chain on the verge of bankruptcy, it has become abundantly clear that Red Lobster letting customers eat all the shrimp their hearts desire was not a great business idea . It's also not the reason the restaurant is in a deep financial mess .

In mid-April, Bloomberg reported the debt-laden seafood chain and home of beloved cheddar biscuits was considering filing for Chapter 11 bankruptcy protection. Red Lobster is being bogged down by increased labor costs and expensive leases on its restaurants. Some observers were quick to blame the financial woes on its decision last year to make its "Endless Shrimp" promotion, which used to be an occasional, limited-time offering, permanent. The move was not a smart one. While Red Lobster increased traffic somewhat, people coming in to chow down on all-you-can-eat shrimp was a money bleeder. The company blamed Endless Shrimp for its $11 million losses in the third quarter of 2023, and in the fourth quarter, the picture got even worse, with the restaurant chain seeing $12.5 million in operating losses.

But the story about what's gone wrong with Red Lobster is much more complicated than a bunch of stoners pigging out on shrimp (and, later, lobster ) en masse. The brand has been plagued by various problems — waning customer interest, constant leadership turnover, and, as has become a common tale, private equity's meddling in the business.

"If anything, the Endless Shrimp deals are probably as much a symbol of just either desperation or poor management or both," Jonathan Maze, the editor in chief of Restaurant Business Magazine, said.

Red Lobster first opened in Lakeland, Florida, in 1968 and was acquired by the food conglomerate General Mills in 1970. General Mills then spun the chain off in 1995 along with the rest of its restaurant division, which also included Olive Garden, as Darden Restaurants. In 2014, amid flagging sales and pressure from investors, Darden sold Red Lobster for $2.1 billion to Golden Gate Capital, a San Francisco private-equity firm.

If anything, the Endless Shrimp deals are probably as much a symbol of just either desperation or poor management or both.

To raise enough cash to make the deal happen, Golden Gate sold off Red Lobster's real estate to another entity — in this case, a company called American Realty Capital Properties — and then immediately leased the restaurants back. The next year, Red Lobster bought back some sites, but many of its restaurants were suddenly strapped with added rent expenses. Even if Darden had kept Red Lobster, it's not clear it would have taken a different route: A press release from the time says it had contacted buyers to explore such a transaction. But in Maze's view, the sale of the real estate was sort of an original sin for Red Lobster's current troubles. He compared it to throwing out a spare parachute — chances are, you'll be OK, but if the first parachute fails, you're in deep trouble.

"The thing that private equity does is just unload assets and monetize assets. And so they effectively paid for the purchase of Red Lobster by selling the real estate," he said. "It'll probably be fine, generally, but there's going to come a time in which your sales fall, your profitability is challenged, and your debt looks too bad, and then suddenly those leases are going to look awfully ugly."

That time, according to recent reporting, is now. With struggling sales and operational losses, the leases are an added headache that is helping push the company to the brink, though bankruptcy may help Red Lobster get some wiggle room on them.

Eileen Appelbaum, a codirector of the Center for Economic and Policy Research, a progressive think tank, and a longtime private-equity critic, said in 2014 that private equity wouldn't be the solution to Red Lobster's ills. She isn't surprised about how this is all turning out.

"Once they sell the real estate, then the private-equity company is golden, and they've made their money back and probably more than what they paid," she said, noting that this was a common theme in other restaurants and retailers and adding: "The retail apocalypse is all about having your real estate sold out from under you so that you have to pay the rent in good times and in bad."

After the real estate move, Golden Gate sold 25% of the company in 2016 to Thai Union, a Thailand seafood company, for $575 million and unloaded the rest of the company to an investor group called the Seafood Alliance, of which Thai Union was a part, in 2020. Golden Gate likely came out ahead, but the same can't be said for Thai Union, which also controls the Chicken of the Sea brand. It is now looking to get out of its stake in Red Lobster and took a one-time charge of $530 million on its investment in the fourth quarter of last year. In 2021, Red Lobster refinanced its debt, with one of its new lenders being Fortress Investment Group, an investment-management group and private-equity firm. According to Bloomberg, it's one of the "key lenders" involved in debt negotiations now.

Beyond the pandemic-related troubles that hit restaurants across the country , analysts and experts say that Red Lobster's particular problems are attributable to a mix of poor brand positioning and unstable leadership. The seafood-restaurant business is a tough one in the US, and people who are hankering for lobster or fish are increasingly going to steak houses that offer those options, said Darren Tristano, the CEO and founder of Foodservice Results, a food-industry consultancy.

"What's truly happened with Red Lobster is that the consumer base has changed and Red Lobster hasn't," he said. "Red Lobster isn't losing to a competitor in their space — they're losing to competitors outside their space."

John Gordon, a restaurant analyst in San Diego, said Red Lobster had been on the decline for 20 years but that it didn't "fall on the knife" until Thai Union got it. "They were totally unprepared to hold a casual-dining restaurant," he said. Kim Lopdrup, Red Lobster's longtime CEO, retired in 2021, and since then, the restaurant hasn't had much in the way of stable leadership. His successor resigned after only a matter of months, and the role remained vacant for more than a year before someone else was appointed. He's left, too, and now Jonathan Tibus, an expert in restructuring, is at the helm.

"One of the problems is that Thai Union just had no credibility in terms of recruiting a new CEO," Gordon said.

Essentially, Red Lobster finds itself in a landscape where there just aren't a lot of bright spots. Add on the weight of the debt and lease obligations the company's private-equity owners saddled the brand with, and a turnaround becomes a gargantuan task.

"It's hard to blame leadership when you have a problem that is unsolvable — I mean, getting the consumer back in the door, increasing traffic. All-you-can-eat shrimp can only do so much," Tristano said.

Red Lobster did not respond to a request for comment for this story. Golden Gate declined to comment. Thai Union pointed to a press release about its intention to exit its investment and said it didn't wish to comment further.

One bad promotion should not doom a restaurant chain like that.

As to what drove Red Lobster to the edge, it's clear that despite not being a very good idea, the blame doesn't fall on Endless Shrimp. Years of changing tastes, tough industry conditions, and poor brand management all contributed to the chain's difficult position. But plenty of other restaurants have faced similar issues and aren't on the verge of bankruptcy. What separates Red Lobster is a decade of private-equity and investor tampering. Pinging from owner to owner makes it hard to settle on a turnaround vision. The company faces challenges that necessitate a long-term view that requires patience — the kind that the short-term-focused Wall Street often struggles to tackle. Whether Red Lobster can turn it around from here remains to be seen: Even if it files for bankruptcy protection, the chain may not disappear. Plenty of companies go bankrupt and keep on keeping on.

"You've got to at least be able to pay your bills, and what's happened over the last five years is the cost of operating a restaurant has taken off," Maze said. "One bad promotion should not doom a restaurant chain like that."

Emily Stewart is a senior correspondent at Business Insider, writing about business and the economy.

About Discourse Stories

Through our Discourse journalism, Business Insider seeks to explore and illuminate the day’s most fascinating issues and ideas. Our writers provide thought-provoking perspectives, informed by analysis, reporting, and expertise. Read more Discourse stories here .

Related stories

More from Retail

Most popular

• Main content

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

Research: Negotiating Is Unlikely to Jeopardize Your Job Offer

• Einav Hart,
• Julia Bear,
• Zhiying (Bella) Ren

A series of seven studies found that candidates have more power than they assume.

Job seekers worry about negotiating an offer for many reasons, including the worst-case scenario that the offer will be rescinded. Across a series of seven studies, researchers found that these fears are consistently exaggerated: Candidates think they are much more likely to jeopardize a deal than managers report they are. This fear can lead candidates to avoid negotiating altogether. The authors explore two reasons driving this fear and offer research-backed advice on how anxious candidates can approach job negotiations.

Imagine that you just received a job offer for a position you are excited about. Now what? You might consider negotiating for a higher salary, job flexibility, or other benefits , but you’re apprehensive. You can’t help thinking: What if I don’t get what I ask for? Or, in the worst-case scenario, what if the hiring manager decides to withdraw the offer?

• Einav Hart is an assistant professor of management at George Mason University’s Costello College of Business, and a visiting scholar at the Wharton School. Her research interests include conflict management, negotiations, and organizational behavior.
• Julia Bear is a professor of organizational behavior at the College of Business at Stony Brook University (SUNY). Her research interests include the influence of gender on negotiation, as well as understanding gender gaps in organizations more broadly.
• Zhiying (Bella) Ren is a doctoral student at the Wharton School of the University of Pennsylvania. Her research focuses on conversational dynamics in organizations and negotiations.

Partner Center

• Share full article

Advertisement

Supported by

Study Suggests Genetics as a Cause, Not Just a Risk, for Some Alzheimer’s

People with two copies of the gene variant APOE4 are almost certain to get Alzheimer’s, say researchers, who proposed a framework under which such patients could be diagnosed years before symptoms.

By Pam Belluck

Scientists are proposing a new way of understanding the genetics of Alzheimer’s that would mean that up to a fifth of patients would be considered to have a genetically caused form of the disease.

Currently, the vast majority of Alzheimer’s cases do not have a clearly identified cause. The new designation, proposed in a study published Monday, could broaden the scope of efforts to develop treatments, including gene therapy, and affect the design of clinical trials.

It could also mean that hundreds of thousands of people in the United States alone could, if they chose, receive a diagnosis of Alzheimer’s before developing any symptoms of cognitive decline, although there currently are no treatments for people at that stage.

The new classification would make this type of Alzheimer’s one of the most common genetic disorders in the world, medical experts said.

“This reconceptualization that we’re proposing affects not a small minority of people,” said Dr. Juan Fortea, an author of the study and the director of the Sant Pau Memory Unit in Barcelona, Spain. “Sometimes we say that we don’t know the cause of Alzheimer’s disease,” but, he said, this would mean that about 15 to 20 percent of cases “can be tracked back to a cause, and the cause is in the genes.”

The idea involves a gene variant called APOE4. Scientists have long known that inheriting one copy of the variant increases the risk of developing Alzheimer’s, and that people with two copies, inherited from each parent, have vastly increased risk.

The new study , published in the journal Nature Medicine, analyzed data from over 500 people with two copies of APOE4, a significantly larger pool than in previous studies. The researchers found that almost all of those patients developed the biological pathology of Alzheimer’s, and the authors say that two copies of APOE4 should now be considered a cause of Alzheimer’s — not simply a risk factor.

The patients also developed Alzheimer’s pathology relatively young, the study found. By age 55, over 95 percent had biological markers associated with the disease. By 65, almost all had abnormal levels of a protein called amyloid that forms plaques in the brain, a hallmark of Alzheimer’s. And many started developing symptoms of cognitive decline at age 65, younger than most people without the APOE4 variant.

“The critical thing is that these individuals are often symptomatic 10 years earlier than other forms of Alzheimer’s disease,” said Dr. Reisa Sperling, a neurologist at Mass General Brigham in Boston and an author of the study.

She added, “By the time they are picked up and clinically diagnosed, because they’re often younger, they have more pathology.”

People with two copies, known as APOE4 homozygotes, make up 2 to 3 percent of the general population, but are an estimated 15 to 20 percent of people with Alzheimer’s dementia, experts said. People with one copy make up about 15 to 25 percent of the general population, and about 50 percent of Alzheimer’s dementia patients.

The most common variant is called APOE3, which seems to have a neutral effect on Alzheimer’s risk. About 75 percent of the general population has one copy of APOE3, and more than half of the general population has two copies.

Alzheimer’s experts not involved in the study said classifying the two-copy condition as genetically determined Alzheimer’s could have significant implications, including encouraging drug development beyond the field’s recent major focus on treatments that target and reduce amyloid.

Dr. Samuel Gandy, an Alzheimer’s researcher at Mount Sinai in New York, who was not involved in the study, said that patients with two copies of APOE4 faced much higher safety risks from anti-amyloid drugs.

When the Food and Drug Administration approved the anti-amyloid drug Leqembi last year, it required a black-box warning on the label saying that the medication can cause “serious and life-threatening events” such as swelling and bleeding in the brain, especially for people with two copies of APOE4. Some treatment centers decided not to offer Leqembi, an intravenous infusion, to such patients.

Dr. Gandy and other experts said that classifying these patients as having a distinct genetic form of Alzheimer’s would galvanize interest in developing drugs that are safe and effective for them and add urgency to current efforts to prevent cognitive decline in people who do not yet have symptoms.

“Rather than say we have nothing for you, let’s look for a trial,” Dr. Gandy said, adding that such patients should be included in trials at younger ages, given how early their pathology starts.

Besides trying to develop drugs, some researchers are exploring gene editing to transform APOE4 into a variant called APOE2, which appears to protect against Alzheimer’s. Another gene-therapy approach being studied involves injecting APOE2 into patients’ brains.

The new study had some limitations, including a lack of diversity that might make the findings less generalizable. Most patients in the study had European ancestry. While two copies of APOE4 also greatly increase Alzheimer’s risk in other ethnicities, the risk levels differ, said Dr. Michael Greicius, a neurologist at Stanford University School of Medicine who was not involved in the research.

“One important argument against their interpretation is that the risk of Alzheimer’s disease in APOE4 homozygotes varies substantially across different genetic ancestries,” said Dr. Greicius, who cowrote a study that found that white people with two copies of APOE4 had 13 times the risk of white people with two copies of APOE3, while Black people with two copies of APOE4 had 6.5 times the risk of Black people with two copies of APOE3.

“This has critical implications when counseling patients about their ancestry-informed genetic risk for Alzheimer’s disease,” he said, “and it also speaks to some yet-to-be-discovered genetics and biology that presumably drive this massive difference in risk.”

Under the current genetic understanding of Alzheimer’s, less than 2 percent of cases are considered genetically caused. Some of those patients inherited a mutation in one of three genes and can develop symptoms as early as their 30s or 40s. Others are people with Down syndrome, who have three copies of a chromosome containing a protein that often leads to what is called Down syndrome-associated Alzheimer’s disease .

Dr. Sperling said the genetic alterations in those cases are believed to fuel buildup of amyloid, while APOE4 is believed to interfere with clearing amyloid buildup.

Under the researchers’ proposal, having one copy of APOE4 would continue to be considered a risk factor, not enough to cause Alzheimer’s, Dr. Fortea said. It is unusual for diseases to follow that genetic pattern, called “semidominance,” with two copies of a variant causing the disease, but one copy only increasing risk, experts said.

The new recommendation will prompt questions about whether people should get tested to determine if they have the APOE4 variant.

Dr. Greicius said that until there were treatments for people with two copies of APOE4 or trials of therapies to prevent them from developing dementia, “My recommendation is if you don’t have symptoms, you should definitely not figure out your APOE status.”

He added, “It will only cause grief at this point.”

Finding ways to help these patients cannot come soon enough, Dr. Sperling said, adding, “These individuals are desperate, they’ve seen it in both of their parents often and really need therapies.”

Pam Belluck is a health and science reporter, covering a range of subjects, including reproductive health, long Covid, brain science, neurological disorders, mental health and genetics. More about Pam Belluck

The Fight Against Alzheimer’s Disease

Alzheimer’s is the most common form of dementia, but much remains unknown about this daunting disease..

How is Alzheimer’s diagnosed? What causes Alzheimer’s? We answered some common questions .

A study suggests that genetics can be a cause of Alzheimer’s , not just a risk, raising the prospect of diagnosis years before symptoms appear.

Determining whether someone has Alzheimer’s usually requires an extended diagnostic process . But new criteria could lead to a diagnosis on the basis of a simple blood test .

The F.D.A. has given full approval to the Alzheimer’s drug Leqembi. Here is what to know about i t.

Alzheimer’s can make communicating difficult. We asked experts for tips on how to talk to someone with the disease .

Pfizer reports patient death in Duchenne gene therapy study

• Medium Text

• Company Pfizer Inc Follow

Sign up here.

Reporting by Pratik Jain in Bengaluru; Editing by Shailesh Kuber

Our Standards: The Thomson Reuters Trust Principles. New Tab , opens new tab

Business Chevron

In the Market: Financiers fret over 'leverage on leverage' in private credit

A report from the halls of the Beverly Hilton at the Milken Institute conference in Los Angeles.