case study for objective

How to Write a Case Study: Bookmarkable Guide & Template

7 Pieces of Content Your Audience Really Wants to See [New Data]

How to Market an Ebook: 21 Ways to Promote Your Content Offers

How to Write a Listicle [+ Examples and Ideas]

What Is a White Paper? [FAQs]

What is an Advertorial? 8 Examples to Help You Write One

How to Create Marketing Offers That Don't Fall Flat

20 Creative Ways To Repurpose Content

16 Important Ways to Use Case Studies in Your Marketing

11 Ways to Make Your Blog Post Interactive

Showcase your company's success using these free case study templates.

Marketing software that helps you drive revenue, save time and resources, and measure and optimize your investments — all on one easy-to-use platform

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 21, Issue 1
• What is a case study?
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Roberta Heale 1 ,
• Alison Twycross 2
• 1 School of Nursing , Laurentian University , Sudbury , Ontario , Canada
• 2 School of Health and Social Care , London South Bank University , London , UK
• Correspondence to Dr Roberta Heale, School of Nursing, Laurentian University, Sudbury, ON P3E2C6, Canada; rheale{at}laurentian.ca

https://doi.org/10.1136/eb-2017-102845

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

What is it?

Case study is a research methodology, typically seen in social and life sciences. There is no one definition of case study research. 1 However, very simply… ‘a case study can be defined as an intensive study about a person, a group of people or a unit, which is aimed to generalize over several units’. 1 A case study has also been described as an intensive, systematic investigation of a single individual, group, community or some other unit in which the researcher examines in-depth data relating to several variables. 2

Often there are several similar cases to consider such as educational or social service programmes that are delivered from a number of locations. Although similar, they are complex and have unique features. In these circumstances, the evaluation of several, similar cases will provide a better answer to a research question than if only one case is examined, hence the multiple-case study. Stake asserts that the cases are grouped and viewed as one entity, called the quintain . 6  ‘We study what is similar and different about the cases to understand the quintain better’. 6

The steps when using case study methodology are the same as for other types of research. 6 The first step is defining the single case or identifying a group of similar cases that can then be incorporated into a multiple-case study. A search to determine what is known about the case(s) is typically conducted. This may include a review of the literature, grey literature, media, reports and more, which serves to establish a basic understanding of the cases and informs the development of research questions. Data in case studies are often, but not exclusively, qualitative in nature. In multiple-case studies, analysis within cases and across cases is conducted. Themes arise from the analyses and assertions about the cases as a whole, or the quintain, emerge. 6

Benefits and limitations of case studies

If a researcher wants to study a specific phenomenon arising from a particular entity, then a single-case study is warranted and will allow for a in-depth understanding of the single phenomenon and, as discussed above, would involve collecting several different types of data. This is illustrated in example 1 below.

Using a multiple-case research study allows for a more in-depth understanding of the cases as a unit, through comparison of similarities and differences of the individual cases embedded within the quintain. Evidence arising from multiple-case studies is often stronger and more reliable than from single-case research. Multiple-case studies allow for more comprehensive exploration of research questions and theory development. 6

Despite the advantages of case studies, there are limitations. The sheer volume of data is difficult to organise and data analysis and integration strategies need to be carefully thought through. There is also sometimes a temptation to veer away from the research focus. 2 Reporting of findings from multiple-case research studies is also challenging at times, 1 particularly in relation to the word limits for some journal papers.

Examples of case studies

Example 1: nurses’ paediatric pain management practices.

One of the authors of this paper (AT) has used a case study approach to explore nurses’ paediatric pain management practices. This involved collecting several datasets:

Observational data to gain a picture about actual pain management practices.

Questionnaire data about nurses’ knowledge about paediatric pain management practices and how well they felt they managed pain in children.

Questionnaire data about how critical nurses perceived pain management tasks to be.

These datasets were analysed separately and then compared 7–9 and demonstrated that nurses’ level of theoretical did not impact on the quality of their pain management practices. 7 Nor did individual nurse’s perceptions of how critical a task was effect the likelihood of them carrying out this task in practice. 8 There was also a difference in self-reported and observed practices 9 ; actual (observed) practices did not confirm to best practice guidelines, whereas self-reported practices tended to.

Example 2: quality of care for complex patients at Nurse Practitioner-Led Clinics (NPLCs)

The other author of this paper (RH) has conducted a multiple-case study to determine the quality of care for patients with complex clinical presentations in NPLCs in Ontario, Canada. 10 Five NPLCs served as individual cases that, together, represented the quatrain. Three types of data were collected including:

Review of documentation related to the NPLC model (media, annual reports, research articles, grey literature and regulatory legislation).

Interviews with nurse practitioners (NPs) practising at the five NPLCs to determine their perceptions of the impact of the NPLC model on the quality of care provided to patients with multimorbidity.

Chart audits conducted at the five NPLCs to determine the extent to which evidence-based guidelines were followed for patients with diabetes and at least one other chronic condition.

The three sources of data collected from the five NPLCs were analysed and themes arose related to the quality of care for complex patients at NPLCs. The multiple-case study confirmed that nurse practitioners are the primary care providers at the NPLCs, and this positively impacts the quality of care for patients with multimorbidity. Healthcare policy, such as lack of an increase in salary for NPs for 10 years, has resulted in issues in recruitment and retention of NPs at NPLCs. This, along with insufficient resources in the communities where NPLCs are located and high patient vulnerability at NPLCs, have a negative impact on the quality of care. 10

These examples illustrate how collecting data about a single case or multiple cases helps us to better understand the phenomenon in question. Case study methodology serves to provide a framework for evaluation and analysis of complex issues. It shines a light on the holistic nature of nursing practice and offers a perspective that informs improved patient care.

• Gustafsson J
• Calanzaro M
• Sandelowski M

Competing interests None declared.

Provenance and peer review Commissioned; internally peer reviewed.

Read the full text or download the PDF:

• Design for Business
• Most Recent
• Presentations
• Infographics
• Data Visualizations
• Forms and Surveys
• Video & Animation
• Case Studies
• Digital Marketing
• Design Inspiration
• Visual Thinking
• Product Updates
• Visme Webinars
• Artificial Intelligence

How to Create a Case Study + 14 Case Study Templates

Written by: Brian Nuckols

When it comes to high impact marketing content, case studies are at the top of the list for helping show off your brand’s stuff. 

In this post, I’ve put together a few high-level case study design tips as well as 14 professionally designed case study templates that you can use to start designing beautiful case studies today. 

Let’s begin! 

Here’s a short selection of 12 easy-to-edit case study templates you can edit, share and download with Visme. View more templates below:

What is a Case Study?

A case study is a way for you to demonstrate the success you’ve already had with existing clients. When you create a case study, you explore how previous clients have used your product or service to reach their goals. 

In particular, a case study highlights a specific challenge or goal one of your clients was struggling with before they discovered your product. 

It then demonstrates how your work has assisted them on the journey towards overcoming the challenge or accomplishing the goal. 

A case study’s outcome is typically to share the story of a company’s growth or highlight the increase of metrics the company tracks to understand success. 

The case study includes an analysis of a campaign or project that goes through a few steps from identifying the problem to how you implemented the solution. 

How to Write a Case Study

When it comes to adding irresistible design to your content from the start, using a helpful tool is a great start. Sign up for a free Visme account and start highlighting your own client success stories using one of our case study templates today. 

Also, while you’re beginning to transition your case study workflow to include a professional design tool, it’s helpful to review some high level principles you can incorporate into your case study. 

We’ll start by reviewing some of the critical style tips and structural elements to include in your case study before progressing to a more detailed design section. 

Pinpoint Your Main Message

When designing an impactful case study, it’s essential to stay clear on the metrics that you’re highlighting. The process of overcoming business challenges is a dynamic process with many moving parts. 

If you do not stay focused on what matters in your case study, you risk obscuring the big win your client experienced by using your product or service. 

This is why you need to focus on a single message or metric. This is often called the north star metric . 

The north star metric is the single most crucial rate, count or ratio that helped your client move closer towards their goals or overcame an obstacle. 

While north star metrics are context dependent, a useful heuristic you can utilize is to figure out the most predictive metric of your client’s long term success. 

In the template I’ll highlight below, cost per lead was the north star metric that The College for Adult Learning needed to optimize. 

Build relationships with customers and drive sales growth

• Reach out to prospects with impressive pitch decks and proposals that convert
• Monitor clients' level of engagement to see what they are most interested in
• Build a winning sales playbook to maximize your sales team's efficiency

Sign up. It’s free.

Use Emotionally Rich Language 

Recently researchers at Presado did an interesting study to understand the types of language that help readers take action. They broke the content included in marketing assets into several categories, including functional, emotional and descriptive. 

In the most successful pieces of content, the researchers found that emotionally coded words were present in 61% of the content’s total volume.

This research shows the benefit of using emotionally engaging content in your case study. While it’s essential to focus on the concrete evidence of how you helped your client get from where they started to a successful outcome, do not forget to highlight the emotional journey. 

Use Data For Concrete Evidence 

Once you’ve decided on the north star metric to highlight and you choose the emotional response you want to reinforce in your case study, it’s important to use actual data from the project to share the concrete results your product helped to achieve. 

To make sure your audience can follow your line of thinking, make sure the data in your case study is precise. If you track data across time, your readers must know whether you chose to track by month or years. 

If there are any apparent trends, you can use color to highlight specific areas in a chart. 

If you want to dig deeper into using data to tell compelling stories, check out our video data storytelling tips to improve your charts and graphs. 

In the template below, The College for Adult Learning case study is an excellent example of how these elements can work together. 

Cost per lead was a critical north star metric, so we chose to emphasize the increase in revenue and a decrease in cost per lead. 

Additionally, the background section uses emotionally rich language by highlighting how the school helps students get ahead with their career goals. Also, the factual data is the centerpiece of this page in the case study.

If you’re ready to share how you impacted a client, use the College for Adult Learning case study template right now! 

Include All Necessary Parts of a Case Study

After you’ve interviewed your client and you’re getting ready to start writing, it’s important to remember each piece you need to cover.

All good case studies consist of five parts: Introduction, Challenge, Solution, Benefit and Result.

While you don’t necessarily need to label each section like that, be sure that the flow makes sense and covers each section fully to give your audience the full scope of your case study.

RELATED: 15 Real-Life Case Study Examples & Best Practices

14 Case Study Templates

Now that we have explored some of the high level strategies you can use to create a business case study, we will transition to 14 case study design templates you can use with Visme. 

1. Fuji Xerox Australia Case Study Template

Use the Fuji Xerox case study template to showcase the concrete results you achieved for your clients. It has sections where you can explain the goals you started with and the results you achieved. 

2. College for Adult Learning Case Study Template

As we’ve explored already, the College for Adult Learning template has sections where you can embrace a data driven storytelling approach while also connecting with your audience using emotionally rich language. 

Utilize the professionally designed business case study to connect with your audience. 

3. Intel Case Study Template

The Intel case study has beautiful visual elements and gives you space to share the project’s context and the goals you set out to achieve. It also allows you to get concrete with the results you achieved. 

You can always use the Visme Brand Kit to incorporate your unique brand colors into this stunning design. 

4. Bit.ly Case Study Template

Bit.ly is a marketing product that helps brands track how they are doing with campaign results. The bit.ly business case study template showcases how they drove impressive results for an eCommerce business. 

You can modify the professionally designed case study template to illustrate the key results you drive for your clients. 

5. NVISIONCenters Case Study Template

The NVISIONCenters case study template is an excellent example of how powerful it is to pair beautiful designs with the results you generate for your clients. In this case study, we see how you can transform your past accomplishments into a powerful marketing asset. 

6. Adobe Case Study Template

The Adobe case study is an exciting example of a business case study because it does a great job illustrating how you can use a specific result to create a powerful marketing asset. 

Adobe had a particular goal of branding to position itself as a leader for the future of digital marketing. LinkedIn sponsored messages was an effective tactic to drive the outcome Adobe needed. 

You can use the Adobe case study template to demonstrate the success of your most effective tactics. 

7. Inkjet Wholesale Case Study Template

The Inkjet wholesale case study template is an excellent choice if you want to experiment with your case study’s visual element. The roadmap to objectives diagram is a powerful graphic that illustrates the journey of a successful campaign. 

8. Neutrogena Case Study Template

If you have a strong visual brand to tell your case study’s story with visuals, the Neutrogena template is a great choice. It is already designed with plenty of space to highlight your visuals. 

When it is all said and done, you have the results section to complete a successful client partnership story.

9. Weebly Case Study Template

The Weebly case study template is your choice if you want to add visual flair to your case study. The beautiful layout is a testament to the power of pairing minimal design with an exciting statistic. 

10. Patagonia Case Study Template

The Patagonia case study is a perfect example of how crucial it is to make design choices based on your brand’s unique personality. 

It is a fantastic choice if you have a project to showcase featuring a brand with a distinct brand aesthetic.  

11. Think With Google Case Study Template

The Think With Google case study template tells the story of a mobile game that needed to create more engagement on their app. 

It is a visually impactful case study design template that you can use to tell a compelling story about your results. 

12. Kleenex Case Study Template

This case study template is the perfect way to show off search marketing results for a client or other highly specific KPIs that you managed to accomplish.

Insert the initial challenge followed by your company’s solution and adjust the included data visualization tools to showcase your specific results.

13. Customer Experience Presentation Case Study Template

The presentation case study template is an excellent choice for blending beautiful visual elements with the ability to give detailed information about the results you generated, as well as showcasing that data in a unique format. 

If you are ready to show how the unique features of your product or service drove real world business results then it is a good choice for your case study. 

14. Webinar Presentation Case Study Template

One small business saw incredible results when using Visme to optimize their webinar workflow. They saved 100 hours of their precious time by incorporating our collaborative design tools. 

We designed the small business template using those results as an example. When you have an eye catching effect to showcase to your audience, you can use this template as a starting point. 

Case Study Design Tips

Now that we’ve explored the 14 templates you can use with Visme to create your case study, let’s take a look at some practical design tips that will take your content to the next level. 

Be Brief In Your Case Study 

In discussions about writing with style, brevity is a common topic. However, it’s also an important design principle. 

Brevity in design is when you find the best way to perform your intended objective in as few steps as possible. 

When designing your case study, make sure you do not add extraneous visual elements where they are not needed. Instead, think of the effect you want to have on your reader and try to do it simply. 

Describe Your Vision Clearly

Earlier in this article, I wrote about the north star metric, your case study’s emotional effect and using data to make the case study concrete. Your design choices should serve to reinforce these primary goals. 

Clarity in design is when all of the visual elements add up to a whole. 

A great example of this is in the small business case study template where the shapes, typography and color scheme all emphasize the main idea that Visme helps the reader save time. 

Create A Consistent Style 

Visual consistency is a fundamental design principle that you can not afford to ignore in your case study. It will help you increase readability and make sure your audience does not get frustrated with jarring visual elements. 

In short, a consistent style is when you use a uniform color scheme, typography and the same kinds of visual elements throughout the case study. 

Use A Case Study Template For Readability

Readability is a crucial element of design, especially for case studies that are experienced on mobile devices. Contrast is an impactful readability principle. 

Make sure any contrasting colors you chose are easy on the eye and your reader does not have to strain to read your case study. 

Use Proper Alignment In Your Case Study 

Alignment is one of the principles of design that sets professionally designed business case study templates apart. Great designers have an intuitive eye for the mathematically based ratios of proximity invisible in sound design and an eyesore in lousy design. 

The good news is that you do not have to be a mathematician nor a professional designer to have a perfect alignment for your case study. Visme utilizes an easy to use drag and drop design tool that helps you achieve proper alignment in your case study. 

Let Your Brand Personality Speak

When we make intentional design decisions, we want to create a positive emotional experience for our audience. One of the best ways to do that is to make decisions that showcase your brand’s unique personality .

Is the case study you are creating like a well dressed business person who is serious, trustworthy and capable of doing a great job? Is it more like an extravert at a party bouncing from person to person lighting up the room? 

There is no right answer, but you need to infuse your viewpoint into the case study you create if you want to create a unique design. 

Start Designing Your Case Study Today 

A professionally designed case study template will help you create a stunning case study. While reviewing some high level design strategies is an important step, a tool like Visme will help you make a real impact on your audience.

If you’re ready to create your next case study, get started with Visme today .

Design beautiful visual content you can be proud of.

Trusted by leading brands

Recommended content for you:

Create Stunning Content!

Design visual brand experiences for your business whether you are a seasoned designer or a total novice.

About the Author

Brian Nuckols is a writer working in Pittsburgh, Pennsylvania. He enjoys communicating visionary ideas in clear, action oriented language. When he’s not working on content for a transformative company you can find him analyzing dreams, creating music, and writing poetry.

What is the Case Study Method?

Simply put, the case method is a discussion of real-life situations that business executives have faced.

On average, you'll attend three to four different classes a day, for a total of about six hours of class time (schedules vary). To prepare, you'll work through problems with your peers.

How the Case Method Creates Value

Often, executives are surprised to discover that the objective of the case study is not to reach consensus, but to understand how different people use the same information to arrive at diverse conclusions. When you begin to understand the context, you can appreciate the reasons why those decisions were made. You can prepare for case discussions in several ways.

Case Discussion Preparation Details

In self-reflection.

The time you spend here is deeply introspective. You're not only working with case materials and assignments, but also taking on the role of the case protagonist—the person who's supposed to make those tough decisions. How would you react in those situations? We put people in a variety of contexts, and they start by addressing that specific problem.

In a small group setting

The discussion group is a critical component of the HBS experience. You're working in close quarters with a group of seven or eight very accomplished peers in diverse functions, industries, and geographies. Because they bring unique experience to play you begin to see that there are many different ways to wrestle with a problem—and that’s very enriching.

In the classroom

The faculty guides you in examining and resolving the issues—but the beauty here is that they don't provide you with the answers. You're interacting in the classroom with other executives—debating the issue, presenting new viewpoints, countering positions, and building on one another's ideas. And that leads to the next stage of learning.

Beyond the classroom

Once you leave the classroom, the learning continues and amplifies as you get to know people in different settings—over meals, at social gatherings, in the fitness center, or as you are walking to class. You begin to distill the takeaways that you want to bring back and apply in your organization to ensure that the decisions you make will create more value for your firm.

How Cases Unfold In the Classroom

Pioneered by HBS faculty, the case method puts you in the role of the chief decision maker as you explore the challenges facing leading companies across the globe. Learning to think fast on your feet with limited information sharpens your analytical skills and empowers you to make critical decisions in real time.

To get the most out of each case, it's important to read and reflect, and then meet with your discussion group to share your insights. You and your peers will explore the underlying issues, compare alternatives, and suggest various ways of resolving the problem.

How to Prepare for Case Discussions

There's more than one way to prepare for a case discussion, but these general guidelines can help you develop a method that works for you.

Preparation Guidelines

Read the professor's assignment or discussion questions.

The assignment and discussion questions help you focus on the key aspects of the case. Ask yourself: What are the most important issues being raised?

Read the first few paragraphs and then skim the case

Each case begins with a text description followed by exhibits. Ask yourself: What is the case generally about, and what information do I need to analyze?

Reread the case, underline text, and make margin notes

Put yourself in the shoes of the case protagonist, and own that person's problems. Ask yourself: What basic problem is this executive trying to resolve?

Note the key problems on a pad of paper and go through the case again

Sort out relevant considerations and do the quantitative or qualitative analysis. Ask yourself: What recommendations should I make based on my case data analysis?

Case Study Best Practices

The key to being an active listener and participant in case discussions—and to getting the most out of the learning experience—is thorough individual preparation.

We've set aside formal time for you to discuss the case with your group. These sessions will help you to become more confident about sharing your views in the classroom discussion.

Participate

Actively express your views and challenge others. Don't be afraid to share related "war stories" that will heighten the relevance and enrich the discussion.

If the content doesn't seem to relate to your business, don't tune out. You can learn a lot about marketing insurance from a case on marketing razor blades!

Actively apply what you're learning to your own specific management situations, both past and future. This will magnify the relevance to your business.

People with diverse backgrounds, experiences, skills, and styles will take away different things. Be sure to note what resonates with you, not your peers.

Being exposed to so many different approaches to a given situation will put you in a better position to enhance your management style.

Frequently Asked Questions

What can i expect on the first day, what happens in class if nobody talks, does everyone take part in "role-playing".

How to Make Case Study Videos in 10 Steps [Examples Included]

Confidence in your brand is important, but it’s only the beginning. To make a real impact, you need to back up your claims with solid proof. That’s where case study videos come in. Let your satisfied customers do the talking giving new leads an authentic view into your products and services. Let’s look at how to create case study videos easily.

Article Last Updated: August 23, 2024

What is a Case Study Video?

Types of case study videos, why are case study videos important, how and where to use your case study video, how to create an impactful case study video.

Who doesn’t enjoy a captivating story? That’s likely why case study videos have become so popular. They’re more than just stories as they offer a deep dive into real-world scenarios, featuring genuine people and authentic businesses. Through these videos, companies showcase the real impact of their products and services, whether it’s through documenting product development , cultural shifts, or community impact.

To bring these stories to life, tools like Zight can be incredibly useful. With Zight’s features like screen recording , GIF creation , and easy file sharing , you can capture every moment and detail seamlessly. Imagine using Zight to record a customer’s success journey or create engaging visuals to complement your narrative. It’s all about making your case study videos as compelling and impactful as possible.

The question is, what does a good case study video look like? Our guide below will cover every aspect of case study videos, from their purpose, creating compelling videos, exploring what makes them successful, and sharing practical case study video examples and tips to help you craft impactful case study videos that resonate and drive results. Let’s get into it.

A case study video is a type of video content that demonstrates how other people are successfully using and benefiting from a product. It focuses on real customer success stories to show the value of a company’s products or services. In a crowded market of claims and promises, these videos serve as credible proof that your business delivers on its promises.

The strength of a case study video comes from its relatability. When potential customers see themselves in your stories, it fosters a true connection. Seeing real people handle challenges makes your business appear more trustworthy and your solutions more appealing. After all, who could offer a more credible opinion to potential customers than someone who’s experienced your services firsthand?

How is a Video Case Study Different From a Written Case Study?

Both written and video case studies aim to convert customers, but video case studies have several specific advantages:

• More Persuasive : While written case studies require readers to interpret the message, video case studies present it directly from the customer, making the impact more immediate and convincing.
• More Engaging : Videos captivate with dynamic visuals, vibrant colors, and sound, making them far more engaging than dense blocks of text. Who wouldn’t prefer a lively video over a long read?
• Higher Conversion Rates : Video marketing campaigns are very effective. It’s not surprising, given how seamlessly videos can be optimized for mobile devices—something text-heavy content struggles with.

Like in any film or video genre, you’ll notice certain styles and tones that recur frequently. This is also true for case study videos, where you’ll come across several common types as you explore case study videos.

That said, there are different types of case study videos that your business can produce, with different levels of complexity. Each type of case study video has a specific customer problem and appeals to different aspects of your audience’s decision-making process. Depending on your objectives and the topic, choosing the right style of case study video can effectively communicate the message you want to share.

1. Customer Testimonials

This type of video is quite simple to make and is one of the easiest case study videos to make. In customer testimonial videos, you interview your happy customers about their experience with your business and its impact on their lives. Since it involves just a straightforward interview with the customer, you only need one filming location and minimal editing to create the video.

Product or service review case study videos provide a thorough look at your offering’s features, functionality, and benefits. They offer an objective assessment and serve as valuable resources for new customers.

Target Audience : These videos target potential customers who are researching your product or service and need detailed information to make an informed decision.

Testimonial Video Example:

This video is a standout example of customer testimonials. Instead of simply listing features, the interviews highlight the challenges the company faced and how Zoom provided effective solutions . The video’s concise length keeps viewers engaged while still delivering a complete and compelling story in one location.

2. Customer Reviews

Customer reviews are authentic insights that highlight a product’s real-world performance. Much like a customer testimonial video, a customer review video features a happy customer discussing your product or service. However, there’s a key difference: in a customer review video, the customer focuses more on the specific features of the product or service, rather than just the value it provided them.

Depending on your approach, such videos may include footage of the customer using your product on camera. Generally, most case study video testimonials follow a Q&A style of storytelling .

Creating a customer review video is straightforward. The interview portion requires just one shoot location and minimal editing. If you decide to add footage of the product in action, the shoot and editing process will be more complex.

Target Audience : These videos are aimed at potential customers who are actively researching your product or service. They provide detailed information to help them make an informed purchase decision.

Customer Reviews Case Study Video Example:

This customer review case study video features Lana Blakely who explains how Notion has transformed her personal and professional life. She breaks down specific features like databases, templates, and task management tools, showing real-life examples of how she uses the app to stay organized. The video includes screen recordings of how she navigates the Notion workspace, providing viewers with a visual understanding of how the platform functions. Any potential customer actively looking for Notion will find information about the tool and can be able to make an informed decision.

3. Case Study Narrative

This is the most complex type of case study video. A case study narrative video involves on-camera interviews with customers and B-roll visuals, such as footage of the customer using your product or your team engaging with the customer. Additionally, these videos often incorporate graphics and text overlays. Due to its complexity, creating this type of video content demands more shoot time, strategic planning, and extensive editing .

Narrative case study videos focus on storytelling , aiming to engage viewers emotionally by presenting a compelling narrative highlighting a customer’s journey from problem to solution, often emphasizing the transformative aspects.

Target Audience : Narrative case study videos are particularly effective for creating an emotional connection with viewers, engaging a wide range of audiences, including those in the awareness and consideration stages.

Case Study Narrative Video Example:

This video by LLLLITL is a case study of Dove’s “Turn Your Back” campaign, which was designed to raise awareness about the issue of body image. The video uses powerful storytelling to connect with viewers on an emotional level.

Case study videos can significantly enhance your video marketing strategy , particularly for B2B companies . They provide a rich, multi-faceted way to showcase a product or service and offer benefits beyond financial gains. Here is why they are important:

• Credibility and Trust : Case study videos provide authentic success stories that show how your products or services have made a real difference for customers. This helps build trust and credibility with potential clients.
• Engagement : Videos naturally draw people in more effectively than text or static images. With a case study video, you can tell a compelling story that keeps your audience engaged and interested.
• Demonstration of Expertise : These videos allow you to highlight your industry knowledge and position yourself as an expert. By showcasing real-world results, you establish your business as a reliable solution provider.
• Problem-Solution Narrative : Case study videos often follow a clear problem-solving structure, helping potential customers relate to the challenges and see how your product or service can address their needs.
• Personal Connection : Featuring customer interviews or testimonials adds a personal touch. Prospective clients can connect with real people who have benefited from your offerings, making your brand more relatable.
• Versatility : Case study videos are highly versatile and can be used across various platforms , including your website, social media channels, email marketing , and presentations . This ensures your success stories reach a broad audience.
• Measurable Impact : Including data and metrics in your videos demonstrates the tangible results achieved by your clients. This evidence of ROI can be very persuasive for potential customers.
• Lead Generation : Well-crafted case study videos can generate leads by addressing problems similar to those your potential customers face, making them valuable assets in your video marketing strategy.
• Storytelling : Effective visual storytelling in case study videos helps forge an emotional connection with your audience, making your brand memorable and engaging.

After perfecting your case study video, it’s time to share it with your target audience. But where should you promote it?

• Your Website – Embed the video prominently on your website’s homepag e or a dedicated landing page to make it easily accessible. Consider having a section just for case studies, giving prospects a convenient reference point.
• Social Media – Share the video on your company’s social media platforms like Facebook , Twitter, LinkedIn, Instagram, and YouTube . Optimize it for each platform and actively engage with your audience through comments, likes, and shares to boost its visibility.
• Email Marketing – Include the video in your email marketing campaigns , especially targeting those interested in the topic. Adding the video to your email signature can also create a dynamic touchpoint.
• Sales and Marketing Presentations – Integrate the video into your sales pitches and marketing presentations . Real-world examples of success can be highly persuasive during client interactions.
• Content Marketing – Use the video in blog posts, articles, or other written content related to the case study’s topic. Create videos as teaser content from snippets to pique interest and direct viewers to the full video for more details.

Now that you have seen some examples of case study videos, you can now create your case study video. Case studies don’t always stick to a strict timeline or template, but some key steps are usually involved in creating a case study video. Follow these steps to create an engaging case study video that will resonate with your audience.

1. Identify the Right Story

The first step in crafting an attention-grabbing case study video is selecting the right story. You need a story that resonates with your target audience and showcases clear results.

For instance, if you run a software company like Zight, don’t just feature any client who used your software. Highlight businesses that experienced a boost in efficiency with your platform . Numbers like these provide concrete proof of your product’s effectiveness.

Your audience is looking for solutions, so your story should present a compelling example of how you’ve delivered just that. A thoughtfully chosen story sets the stage for a truly engaging case study video.

2. Ask Important Questions

The next key step is to craft the right questions. These will be the basis of your case study video.

• Start by setting the scene for your viewers : Ask about the customer’s initial problem. For example, “What issues were you dealing with before using our product?”
• Then, dive into the specifics : Analyze the customer’s decision-making process with questions like, “What pulled you to our product instead of others?”
• Finally, highlight the results : Ask questions such as, “How has our product made a difference in your operations?” or “Would you recommend our service to others?”

This thoughtful questioning will help create a well-rounded story, listing the problem, the solution, and the impact of your product or service.

3. Choose the Right Audience

You might have a great customer success story and perfectly crafted questions, but they won’t make an impact if they don’t resonate with your target audience’s needs and interests.

Imagine you’re showcasing Zight. Your audience could range from tech-savvy professionals to small business owners who aren’t as familiar with advanced tools. If your case study highlights a large corporation using Zight’s advanced features , it might not connect with a small business owner looking for simple and effective screen recording solutions .

Before diving in, do some audience research. What challenges are they facing? What solutions are they after? Tailor your case study video to address these, using language and examples that speak directly to their needs.

4. Plan Out the Storyline

To craft an engaging storyline for your case study video, you need to guide the viewer through a story that resonates. Start with a compelling introduction that highlights a common problem your audience faces, making it instantly relatable.

For instance, if you’re showcasing Zight, an issue could be the struggle businesses face with lengthy communication chains that slow down decision-making. Many teams feel this pain, making it an effective hook. Then, introduce Zight as the solution. This is where you spotlight its unique features—like screen recording and sharing capabilities—that streamline communication and boost productivity .

Support your claims with testimonials or expert opinions to add credibility. Hearing from satisfied users can make a significant impact.

Finally, wrap up by showcasing measurable results . Use statistics or before-and-after comparisons to emphasize how Zight made a difference. Conclude with a clear call-to-action, guiding the viewer on what steps to take next.

5. Conduct Background Interviews

Conducting background interviews is essential before you start filming. These pre-shoot conversations offer valuable insights that can enhance your storyline. They help you understand the full scope of the customer’s experience , adding richness and depth to your case study video.

These interviews also help you identify key talking points and decide who should be featured in the video. Whether it’s the CEO providing strategic insights or a front-line employee sharing day-to-day benefits, understanding this in advance ensures you capture the most relevant content, saving you time and effort during production.

6. Develop Your Script

The video script is the backbone to create engaging video content, pulling together visuals, dialogue, and pacing to create a cohesive story. Here’s how to craft one that leaves an impact:

• Start by outlining the key points you’ve gathered from background interviews and your storyline.
• Be clear and specific—rather than saying, “Our product is great,” highlight its strengths with something like, “Our software boosts productivity by 40%.”
• Keep the tone conversational yet professional to ensure your message resonates.
• Make sure the script flows smoothly, making complex ideas easy to understand.
• Consider using bullet points or numbered lists to emphasize key features or benefits.

Wrap up with a compelling call-to-action , guiding viewers on what to do next, whether that’s visiting your website or reaching out to your sales team.

7. Back it up with Data

Including data and statistics adds credibility to your case study video. While a compelling story captures attention, solid data reinforces your claims and makes your video campaigns more convincing.

Incorporate charts, graphs, or other visuals to present the data. Visual elements help make complex information more digestible and memorable. Ensure the data aligns with your storyline and addresses the needs or concerns of your audience.

8. Select the Right Location

The location you choose for your case study video adds depth and context to your story. Opt for a setting that complements the narrative and enhances its authenticity. For instance, if your case study involves educational software, filming in a classroom or school can make the story feel more genuine.

Your location should also resonate with your audience. Remember to consider practical aspects like lighting , sound, and permissions. The perfect location can fall flat if it has poor acoustics or requires difficult-to-obtain permits.

9. Create a Shot List

A carefully planned shot list is essential for a smooth filming experience. It details every shot you need, acting as a guide for your production team.

For example, if you’re capturing a customer testimonial, your shot list might include:

• Close-ups of the customer speaking
• Cutaways of the product in action
• Wide-angle shots to set the scene

Your shot list should specify the type of shots—wide, medium, or close-up—and any particular camera movements like pans or zooms. This ensures you capture all the crucial elements of your video marketing campaign from product details to emotional moments.

A shot list also helps you manage time and resources efficiently, allowing you to anticipate special equipment or lighting needs ahead of time, and preventing last-minute scrambling.

10. Shoot and Edit

This is where all your planning comes to life. Stick to your shot list and script during the shoot, but be open to capturing spontaneous moments that could enhance the story. High-quality equipment is necessary for clear audio and well-lit scenes —these technical details can elevate your final product.

Editing is where you shape the story , choosing the best shots to create a compelling narrative. Use cutaways, transitions, and background music to keep the pacing dynamic and the viewer engaged.

Pay attention to color grading, sound mixing, and special effects, ensuring they match the tone and message of your video. Avoid overdoing effects, as they can easily overshadow the content.

Now that you’ve seen how major brands craft their case study videos, let these examples spark ideas for your own. Use them to motivate your sales team , improve your video marketing strategy, and captivate your audience.

In addition, incorporating tools like Zight offers practical solutions such as screen recording and GIF creation, these videos not only tell a compelling story but also demonstrate how your product can deliver tangible results. What are you waiting for? Sign up and get started .

Create & share screenshots, screen recordings, and GIFs with Zight

Get Zight for iOS.

Wise-Answer

Find answers to all questions with us

What are the objectives of a case study?

Table of Contents

• 1 What are the objectives of a case study?
• 2 How do you write an objective for a case study?
• 3 What are the key facts of the case study?
• 4 What is a career objective example?
• 5 What are the steps to solve a case study?
• 6 What are the learning objectives of case 2?
• 7 How is interprofessional collaboration used in case studies?
• 8 What are case 1 and case 2 of COPD?

The general purpose of a case study is to: → describe an individual situation (case), e.g. a person, business, organisation, or institution, in detail; → identify the key issues of the case (your assignment question should tell you what to focus on); → analyse the case using relevant theoretical concepts from your unit …

How do you write an objective for a case study?

Writing your research objectives clearly helps to:

• Define the focus of your study.
• Clearly identify variables to be measured.
• Indicate the various steps to be involved.
• Establish the limits of the study.
• Avoid collection of any data that is not strictly necessary.

What are the parts of case study?

There are usually eight sections in a case study:

• Synopsis/Executive Summary. Outline the purpose of the case study.
• Findings. Identify the problems found in the case by:
• Discussion. Summarise the major problem/s.
• Conclusion.
• Recommendations.
• Implementation.
• References.
• Appendices (if any)

What are the key facts of the case study?

Key facts are those facts in the case that are critical to the outcome of the case. All lawsuits arise as a result of disputes involving facts. Our legal system revolves around resolving disputes through the application of rules of law to the facts of a case.

What is a career objective example?

General career objective examples To secure a challenging position in a reputable organization to expand my learnings, knowledge, and skills. Secure a responsible career opportunity to fully utilize my training and skills, while making a significant contribution to the success of the company.

What are the 4 most important parts of case study?

The 4 Essential Elements of a Great Case Study

• Showcase the Problems You Answered. The customer has come to you with a problem or need for you to solve and you knocked it out of the water!
• Tell The Story of Your Customers’ Experience.

What are the steps to solve a case study?

Steps for solving a case study in Nutshell

• Identify the possible alternatives to attaining the objective.
• Evaluate the cause and effect of each alternative i.e. think about the outcome of each action /alternative.
• Work in the classroom.
• The Syndicate approach.
• Report writing.
• The problem (to understand properly)

What are the learning objectives of case 2?

How are case studies used in health case studies?

How is interprofessional collaboration used in case studies?

What are case 1 and case 2 of copd.

Share this post

Privacy overview.

Pardon Our Interruption

As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:

• You've disabled JavaScript in your web browser.
• You're a power user moving through this website with super-human speed.
• You've disabled cookies in your web browser.
• A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this support article .

To regain access, please make sure that cookies and JavaScript are enabled before reloading the page.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

Article Versions Notes

Sicuaio, T.; Zhao, P.; Pilesjö, P.; Shindyapin, A.; Mansourian, A. A Multi-Objective Optimization Approach for Solar Farm Site Selection: Case Study in Maputo, Mozambique. Sustainability 2024 , 16 , 7333. https://doi.org/10.3390/su16177333

Sicuaio T, Zhao P, Pilesjö P, Shindyapin A, Mansourian A. A Multi-Objective Optimization Approach for Solar Farm Site Selection: Case Study in Maputo, Mozambique. Sustainability . 2024; 16(17):7333. https://doi.org/10.3390/su16177333

Sicuaio, Tomé, Pengxiang Zhao, Petter Pilesjö, Andrey Shindyapin, and Ali Mansourian. 2024. "A Multi-Objective Optimization Approach for Solar Farm Site Selection: Case Study in Maputo, Mozambique" Sustainability 16, no. 17: 7333. https://doi.org/10.3390/su16177333

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

This paper is in the following e-collection/theme issue:

Published on 26.8.2024 in Vol 26 (2024)

Evaluation of a Natural Language Processing Approach to Identify Diagnostic Errors and Analysis of Safety Learning System Case Review Data: Retrospective Cohort Study

Authors of this article:

Original Paper

• Azade Tabaie 1, 2 , PhD   ; 
• Alberta Tran 3 , RN, CCRN, PhD   ; 
• Tony Calabria 3 , MA, CPHQ, CSSBB   ; 
• Sonita S Bennett 1 , MSc   ; 
• Arianna Milicia 4 , BSc   ; 
• William Weintraub 5, 6 , MACC, MD   ; 
• William James Gallagher 6, 7 , MD   ; 
• John Yosaitis 6, 8 , MD   ; 
• Laura C Schubel 4 , MPH   ; 
• Mary A Hill 9, 10 , MS   ; 
• Kelly Michelle Smith 9, 10 , PhD   ; 
• Kristen Miller 4, 6 , MSPH, MSL, CPPS, DrPH  

1 Center for Biostatistics, Informatics, and Data Science, MedStar Health Research Institute, Washington, DC, United States

2 Department of Emergency Medicine, Georgetown University School of Medicine, Washington, DC, United States

3 Department of Quality and Safety, MedStar Health Research Institute, Washington, DC, United States

4 National Center for Human Factors in Healthcare, MedStar Health Research Institute, Washington, DC, United States

5 Population Health, MedStar Health Research Institute, Washington, DC, United States

6 Georgetown University School of Medicine, Washington, DC, United States

7 Family Medicine Residency Program, MedStar Health Georgetown-Washington Hospital Center, Washington, DC, United States

8 MedStar Simulation Training & Education Lab (SiTEL), MedStar Institute for Innovation, Washington, DC, United States

9 Institute of Health Policy, Management & Evaluation, University of Toronto, Toronto, ON, Canada

10 Michael Garron Hospital, Toronto, ON, Canada

Corresponding Author:

Azade Tabaie, PhD

Center for Biostatistics, Informatics, and Data Science

MedStar Health Research Institute

3007 Tilden Street NW

Washington, DC, 20008

United States

Phone: 1 202 244 9810

Email: [email protected]

Background: Diagnostic errors are an underappreciated cause of preventable mortality in hospitals and pose a risk for severe patient harm and increase hospital length of stay.

Objective: This study aims to explore the potential of machine learning and natural language processing techniques in improving diagnostic safety surveillance. We conducted a rigorous evaluation of the feasibility and potential to use electronic health records clinical notes and existing case review data.

Methods: Safety Learning System case review data from 1 large health system composed of 10 hospitals in the mid-Atlantic region of the United States from February 2016 to September 2021 were analyzed. The case review outcome included opportunities for improvement including diagnostic opportunities for improvement. To supplement case review data, electronic health record clinical notes were extracted and analyzed. A simple logistic regression model along with 3 forms of logistic regression models (ie, Least Absolute Shrinkage and Selection Operator, Ridge, and Elastic Net) with regularization functions was trained on this data to compare classification performances in classifying patients who experienced diagnostic errors during hospitalization. Further, statistical tests were conducted to find significant differences between female and male patients who experienced diagnostic errors.

Results: In total, 126 (7.4%) patients (of 1704) had been identified by case reviewers as having experienced at least 1 diagnostic error. Patients who had experienced diagnostic error were grouped by sex: 59 (7.1%) of the 830 women and 67 (7.7%) of the 874 men. Among the patients who experienced a diagnostic error, female patients were older (median 72, IQR 66-80 vs median 67, IQR 57-76; P =.02), had higher rates of being admitted through general or internal medicine (69.5% vs 47.8%; P =.01), lower rates of cardiovascular-related admitted diagnosis (11.9% vs 28.4%; P =.02), and lower rates of being admitted through neurology department (2.3% vs 13.4%; P =.04). The Ridge model achieved the highest area under the receiver operating characteristic curve (0.885), specificity (0.797), positive predictive value (PPV; 0.24), and F 1 -score (0.369) in classifying patients who were at higher risk of diagnostic errors among hospitalized patients.

Conclusions: Our findings demonstrate that natural language processing can be a potential solution to more effectively identifying and selecting potential diagnostic error cases for review and therefore reducing the case review burden.

Introduction

Diagnostic errors are an underappreciated cause of preventable mortality in hospitals, estimated to affect a quarter million hospital inpatients, and account for an estimated 40,000-80,000 deaths annually in the United States [ 1 ]. These errors pose a risk for severe patient harm [ 2 , 3 ], increase hospital length of stay [ 4 ], and made up 22% and accounted for US $5.7 billion of paid malpractice claims in hospitalized patients throughout a nearly 13-year period [ 5 ]. In their analysis of malpractice claims occurring in the US National Practitioner Database from 1999 to 2011, Gupta et al [ 5 ] found that diagnosis-related paid claims were most likely to be associated with death and cost (following surgery); among diagnosis-related paid claims, failure to diagnose was the most common subtype and was more likely than other types to be associated with mortality. Several factors have been proposed as contributors to inpatient diagnostic errors including time constraints related to the concurrent care of multiple patients, unpredictable workflows, distractions, and competing priorities for trainees. From their systematic review and meta-analysis, Gunderson et al [ 2 ] estimate that 250,000 diagnostic adverse events occur annually among hospitalized patients in the United States, and this is likely an underestimation of the problem due to several challenges in diagnostic error measurement [ 6 ].

Challenges in identifying and measuring diagnostic errors occur due to the evolving and iterative nature of the diagnostic process, making it difficult to determine when, if at all, a correct or more specific diagnosis could have been established by clinicians to start the appropriate treatment [ 6 ]. Since its landmark report, Improving Diagnosis in Health Care , the National Academies of Science, Engineering, and Medicine (NASEM) has produced a common understanding of diagnostic error that includes accuracy, timeliness, and communication of the explanation to the patient or patient’s family member [ 3 ]. Diagnostic errors often involve missed opportunities related to various aspects of the diagnostic process [ 7 - 9 ] and diagnostic adverse events resulting in harm [ 10 ]. However, many hospitals currently do not capture or include surveillance for diagnostic errors, despite having robust systems in place to report and analyze patient safety issues [ 6 , 11 , 12 ].

A crucial first step to improving diagnosis in hospitals is the creation of programs to identify, analyze, and learn from diagnostic errors. Ongoing efforts by the Agency for Health Care Research and Quality have supported pragmatic measurement approaches for health organizations to build a diagnostic safety program and identify and learn from diagnostic errors such as those described in the Measure Dx resource [ 9 ]. One proposed and promising solution for hospitals to improve diagnostic surveillance is to build on existing efforts to collect patient safety data, root cause analyses, or other forms of case reviews for quality improvement purposes. Cases that have already been reviewed or investigated in the organization for general patient safety and quality purposes may be able to inform or be rereviewed for information and learning opportunities specific to diagnostic safety. Widely used case-based learning methodologies in particular, such as the “Learning From Every Death” initiative developed at Mayo Clinic [ 13 ] used both nationally and worldwide, offer an excellent opportunity for hospitals to augment their existing quality and safety efforts and support diagnostic safety.

Clinical notes in electronic health records (EHRs) written by health providers in free-text format are rich sources of a patient’s diagnoses and care trajectory through hospitalization time. Approaches to processing free text, such as through natural language processing (NLP) and machine learning (ML), have demonstrated significant opportunities to improve quality and safety within health care organizations in diverse applications [ 14 - 16 ] such as cancer research [ 17 , 18 ] and infection prediction [ 19 ] to sleep issues [ 20 ] and neurological outcome prediction [ 21 ]. Besides its use in the diagnostic process, ML models proved to have added benefits when used in diagnostic error identification [ 22 , 23 ]. However, despite significant progress and evidence about the use of these ML and NLP approaches to improve patient safety, the use of ML and NLP approaches to diagnostic safety and surveillance has largely remained untapped. A 2022 study demonstrates how an academic medical center’s implementation of an NLP-based algorithm to flag safety event reports for manual review enabled early detection of emerging diagnostic risks from large volumes of safety reports, and was among the first to apply an NLP approach to safety event reports to facilitate identification of COVID-19 related diagnostic errors [ 24 ]. Meanwhile, progress in the use of data mining approaches to develop electronic trigger tools offers promising methods to detect potential diagnostic events, promote organizational learning, and support the monitoring of data prospectively to identify patients at high risk for future adverse events [ 25 ]. To our knowledge, however, NLP has not yet been applied to case review data to facilitate the identification of diagnostic errors and understand its features and sources.

While free-text formatted clinical notes provide unique opportunities to incorporate ML models, the lack of reliable labels to represent diagnostic errors often limits the use of clinical notes for diagnostic safety surveillance efforts. The opportunity to train ML and NLP algorithms to identify diagnostic errors and opportunities depends on the collation of EHR data with existing efforts to identify diagnostic errors such as through case review findings from the Safety Learning System (SLS). To further explore the potential for this approach to be used to improve diagnostic safety surveillance, a rigorous evaluation of the feasibility and potential of using EHR and existing case review data is needed.

We hypothesized that ML and NLP methods can be applied to train models based on available case review data to examine content potentially related to diagnostic errors within EHR clinical notes. These approaches automatically identify features or information from free text using controlled vocabularies, rule sets, reference dictionaries, or lexicons.

Data Sets and Case Review Approach

We analyzed SLS data from 1 large health system comprised of 10 hospitals in the mid-Atlantic region of the United States. The SLS is one example of a holistic case review methodology delivered by health care organizations in the United States and globally. Established in 2015, the SLS builds upon the Mayo Clinic Mortality Review System of Huddleston et al [ 13 ] to review and analyze EHR data from patient mortality cases to find safety issues that could be found and mitigated. This approach was designed to enhance current quality improvement projects done within health organizations, providing a perspective and strategy based on the Safety II lens and rooted in the belief that every death provides an opportunity to improve care. With a Safety II lens, participating organizations use a holistic case review methodology designed to identify vulnerabilities in systems and processes of care delivery. Reviewers identify and translate these into different categories and labels to (1) define and quantify types of process of care and system failures contributing to adverse outcomes (errors) and (2) identify the components of the process of care and system failures that when fixed will improve performance (opportunities for improvement [OFIs]).

To ensure a sufficient cross-sampling of patients across different specialties and areas, patients are selected for case reviews at this health system based on their primary provider service line category (eg, medicine, surgery, etc) and hospital length of stay; patients in primary and ambulatory care settings are not included for case review selection. The case review process occurs according to the standardized SLS methodology and recommendations [ 13 , 26 ], and between at least 1 physician and 1 nurse within the health system who have both received training in the SLS approach. The case review outcome and identification of OFIs, including diagnostic OFIs, relies on the reviewer’s consensus of any findings and through multiple multidisciplinary and multispecialty meetings that may involve a committee Chair member, clinical department leader, or escalation to other leadership.

We obtained SLS data from February 2016 to September 2021; data in later years were available but not included because of key changes to the case selection process made during and in response to the COVID-19 pandemic. All hospitalized adult patients older than 18 years were included for analysis, regardless of their hospitalization outcome (eg, mortality or discharge location). Pediatric and neonatal patients were excluded.

Ethical Considerations

The original data collection and study protocol was approved by the institutional review board (00001245) at MedStar Health Research Institute on August 26, 2019.

Data Extraction

Medical record number, encounter number, length of stay, age, date of birth, sex, diagnosis at the time of admission (ie, ICD-10 [ International Statistical Classification of Diseases, Tenth Revision ] diagnosis codes), mortality, OFI categories (eg, delayed or missed diagnosis and diagnostic opportunities), number of identified OFIs and diagnosis issues (eg, the accuracy of diagnosis and confirmation or fixation bias) were the features and patient identifiers which were extracted from SLS data [ 13 , 26 ].

Because chart reviews generally occur at a single point in time within the patient’s care trajectory, they often do not contain information or details of the patient’s full hospital course. However, clinical notes written by health care providers are rich sources of patient’s health status throughout their hospitalization period [ 27 - 29 ]. Therefore, to supplement these chart review data, we additionally extracted and included all clinical notes from the EHR for patients who could be matched by patient identifiers (eg, encounter number and date of birth).

Coding Diagnostic Errors

Case reviewers can select any number of labels to describe a diagnosis issue or an OFI identified and agreed upon by consensus. For this study, diagnostic errors were defined by the available features from chart review pertaining to diagnosis and impacting the timeliness, accuracy, or communication of a diagnosis. Our definition of diagnostic errors was limited to the categories identified during chart reviews and recorded within the SLS data set; therefore, our diagnostic error definition does not include all aspects of the definition developed by the NASEM report [ 3 ]. Table 1 describes the SLS categories and values that were labeled as diagnostic errors and used to train our classification models. Patients were coded as having experienced a diagnostic error if one or more of the conditions listed in Table 1 were identified in their SLS case review.

a OFI: opportunity for improvement.

NLP Approach

We used an NLP approach on critical incident reporting system data to explore the features and risk of diagnostic error among hospitalized patients.

Features From Free-Text Data

Descriptive statistical analyses were performed to identify any differences among age, length of stay, and mortality between the female and male patients who had experienced diagnostic errors.

All EHR clinical notes were transformed to lowercase. Extra white spaces, numbers, punctuations, and stop words were removed and words were stemmed. The term frequency-inverse document frequency (TF-IDF) matrix was calculated for each clinical note using the bag-of-words from the preprocessed EHR clinical notes [ 30 ]. TF-IDF is a statistical measure that evaluates how relevant a word is to a document in a collection of documents and is a popular method to translate free text to numerical features in training ML models. The TF-IDF of a word in a document is calculated by multiplying 2 metrics: the number of times a word appeared in a document and the inverse document frequency of the word across a set of documents. TF-IDF is computationally efficient and easy to interpret. We excluded the most frequent words that had appeared in more than 95% of the EHR clinical notes, as these frequent words do not provide information to help with the classification. Moreover, we excluded the rare words that appeared in less than 5% of the EHR clinical notes [ 31 ].

In a TF-IDF matrix, the number of rows corresponds to the unique patients, and the number of columns represents the unique words found in EHR clinical notes. There are numerous unique words used in EHR clinical notes; therefore, the TF-IDF approach provides a high-dimensional input matrix for the classification task. The high-dimensional input matrix can lead to training inaccurate classifiers. To overcome that issue, we used the chi-square statistical test to select the most relevant words to identify diagnostic errors; therefore, if P values associated with a word (also called a feature) are less than .05, that word is selected and included in the feature matrix to train ML classification models.

Classification Models

In lieu of an existing model with the same objective in the literature, a simple logistic regression model was trained as the baseline classifier to identify patients within SLS data who were at higher risk of diagnostic error. Moreover, 3 forms of logistic regression models with regularization functions were trained on this data to compare classification performances and identify the best-performing model [ 32 ]: Least Absolute Shrinkage and Selection Operator (LASSO), Ridge, and Elastic Net.

• LASSO: for a more accurate prediction, LASSO regularization is used with a logistic regression model. The LASSO procedure encourages simple, sparse models which has fewer parameters in a way that the estimated coefficient of features with less effect will be set to zero. This characteristic makes LASSO well-suited for models showing high levels of multicollinearity or variable selection and parameter elimination is needed. LASSO is also called L1 regularization.
• Ridge: also called L2 regularization, Ridge is a regularization method used for models suffering from multicollinearity or high-dimensional feature space. Ridge regularization keeps all the features regardless of their effect on the model. However, it pushes the estimated coefficient of features with less effect toward zero to minimize their effect on the classification outcome. This characteristic of Ridge makes it well-suited when most features impact the outcome variable.
• Elastic Net: a logistic regression model with Elastic Net regularization is a weighted combination of LASSO (L1) and Ridge (L2) regularizations [ 33 ]. Elastic Net can remove the effect of the insignificant features by setting their estimated coefficient to zero and lower the effect of the less significant features by pushing their estimated coefficient toward zero while adding more weights to the more important features. From implementation and interpretation aspects, the Elastic Net model is simple to use. Such characteristics make this model an accepted baseline in ML-based studies [ 34 ].

The hyperparameters of the 3 classification models were optimized through cross-validation. All the analyses were conducted using Python 3 (Python Software Foundation).

Classification Performance Metrics

We calculated 7 common performance metrics reported for binary classifiers to compare the performance of the 4 classification models: area under receiver operating characteristics curve (AUROC), sensitivity or recall or true positive rate, specificity or true negative rate, positive predictive value (PPV) or precision, negative predictive value (NPV), F 1 -score, and area under precision-recall curve (AUPRC). The 7 metrics take values between 0 and 1. Values closer to 1 indicate a well-performing classifier. Multimedia Appendix 1 presents the definition of the performance metrics used in this study. Figure 1 presents the summary of the methods used in this analysis.

Descriptive Summary

In total, there were 2184 unique patient records within SLS data from February 2016 to September 2021. EHR clinical notes were cross-matched, extracted, and included in analyses for 1704 (78%) of these SLS patient records. Of those patients with cross-matched EHR data, 126 (7.4%) patients had been identified by case reviewers as having experienced at least 1 diagnostic error. A total number of 20,848 EHR clinical notes associated with the 1704 unique patients were used in this study.

Patients who had experienced diagnostic errors were grouped by sex: 59 (7.1%) of the 830 women and 67 (7.7%) of the 874 men in the larger cross-matched sample had been found to have a diagnostic error. Table 2 presents the descriptive statistics between female and male patient groups. We applied the Wilcoxon rank sum test for numerical features (ie, age and length of stay), and the chi-square test for mortality rate, admission diagnosis, and admission department or specialty. Patients in the female group were older than the male group by a median of 72 (IQR 66-80) versus a median of 67 (IQR 57-76; P =.02). Compared to the male group, female patients who experienced diagnostic error had higher rates of being admitted through general or internal medicine (69.5% vs 47.8%; P =.01), lower rates of cardiovascular-related admitted diagnosis (11.9% vs 28.4%; P =.02), and lower rates of being admitted through neurology department (2.3% vs 13.4%; P =.04). We observed no differences between groups in mortality rates and length of stay.

Classification Models’ Performance

Clinical notes were preprocessed for TF-IDF feature calculation. The bag-of-words included 2227 words, and each word was considered a feature (see Table S1 in Multimedia Appendix 2 for the top 100 words). We found that abscess, ascend, abnormality, scant, pair, and prefer were the top 5 features with the highest positive estimated coefficient (0.42 to 0.28); post, select, gave, muscl, hours, and unrespons were the top 5 features with the highest negative coefficients (–0.35 to –0.26). After applying the chi-square test, 250 features with a P value less than .05 were selected for the modeling process. All 4 ML classifiers were trained using the 250 selected features.

Table 3 presents the performances of the simple logistic regression and 3 regularized logistic regression models (LASSO, Ridge, and Elastic Net). The Ridge model achieved the highest AUROC (0.885), specificity (0.797), PPV (0.24), NPV (0.981), and F 1 -score (0.369) in classifying patients who were at higher risk of diagnostic errors among hospitalized patients in SLS system. The simple logistic regression model obtained the highest AUPRC (0.537). The simple logistic regression model classified all patients as the ones with diagnostic errors; therefore, it achieved a sensitivity of 1, and specificity and NPV of 0.

Figures 2 and 3 present the receiver operating characteristics curves and precision-recall curves for the 4 classifiers in this study. Models that give ROC curves closer to the top-left corner indicate a better performance. The AUROC values represent the probability that a patient who experienced a diagnostic error, chosen at random, is ranked higher by the Ridge model than a randomly chosen patient who did not experience a diagnostic error. The higher value of AUPRC indicates that the Ridge model can identify patients who experienced diagnostic errors more precisely with fewer false positives compared to LASSO and Elastic Net models.

a LASSO: Least Absolute Shrinkage and Selection Operator.

b AUROC: area under receiver operating characteristics curve.

c AUPRC: area under precision-recall curve.

Principal Findings

Our contribution is 2-fold; first, we integrated 2 data sources that are currently used by and available to many organizations across the United States, SLS and EHR data, to explore the use of ML and NLP algorithms to help identify diagnostic errors among hospitalized patients. Although case review methodologies offer rich insights into systems errors and OFIs, the predefined pull-down menus and structured data labels typically do not capture all the necessary clinical and contextual details that are considered by reviewers. Therefore, a large portion of these case review data are stored in free-text narratives that typically record key information and judgments decided upon by the multidisciplinary reviewers. However, given persistent issues of staff shortage and lack of time in health care settings, it is becoming increasingly important to lower the burden of systematic EHR data reviews for health care providers while maintaining the review systems in place. Second, any developed ML and NLP approaches can potentially be incorporated to generate a diagnostic error risk score for each patient. The predicted risk score can be used in identifying and prioritizing patients for focused chart reviews, thus lowering the burden of systematic EHR data reviews for health care providers while maintaining the review systems in place.

To our knowledge, this study is the first attempt to apply and test several different ML classification models to identify diagnostic errors within routinely collected organizational case review data. Despite a substantial body of literature about the prevalence of diagnostic errors in hospital settings, current efforts to identify diagnostic errors generally rely on reviews of patient cases and data by clinical or quality teams that often are resource-intensive. ML classification models and NLP techniques offer an opportunity to generate diagnostic error risk scores to sort through large data sets and identify signals of potential diagnostic errors that can be flagged for further review. However, these classification models require a high number of observations (and identified diagnostic errors) to perform well, which might not be feasible for health organizations that are just beginning to identify diagnostic errors or may have limited personnel and efforts to perform high numbers of case reviews. In this study, we accessed nearly 2000 patient records (and of those, only 126 cases of diagnostic errors), which is considered to be a limited data sample size in the field of ML. However, techniques, such as feature selection and n-fold cross-validations, can potentially be approaches to address small sample size challenges [ 35 ].

Using the results of the simple logistic regression model as the baseline performance, we found that 3 regularization functions, namely LASSO, Ridge, and Elastic Net, boosted the performance of the baseline model. The Ridge model outperformed the rest of the models in terms of multiple performance metrics: AUROC of 0.885, specificity of 0.797, PPV of 0.24, NPV of 0.981, and F 1 -score of 0.369. The Ridge algorithm tries to keep all features in the model even the features with a slight effect on the classification outcome. Since the patterns pointing at a diagnostic error were subtle in the clinical notes, even a small effect of a feature on the model’s classification outcome could be important for the classification model to learn. On the other hand, the LASSO algorithm rigorously removes features that have a small effect on the classification outcome. The Elastic Net model is a weighted combination of LASSO and Ridge. The performance results presented in Table 3 show that the values achieved by the Elastic Net model lie between those of the LASSO and Ridge models.

Insights From Diagnostic Errors Within Free-Text Clinical Notes

We did not find the free text formatted clinical notes in the EHR to reflect any sort of direct language around diagnostic errors. Our analysis identified no use of the terms misdiagnosis, missed diagnosis, or diagnostic error within clinical notes, finding instead more subtle signals pointing at diagnostic errors such as “there may be a chance of misreading the test,” or “insufficient data to make a diagnosis.” Our findings demonstrate that NLP algorithms can be used to identify such patterns and find the associations between diagnostic errors and the subtle signals in the clinical notes. A natural extension of this work can focus on using other feature extraction methods, such as Bidirectional Encoder Representations from Transformers contextualized word embeddings, and explore the use of the pretrained language models for this objective.

We found that the presence of terms, such as abscess, abnormality, “cp” (chest pain) , and dialysis in a patient’s EHR clinical note were associated with reviewer-identified diagnostic errors ( Multimedia Appendix 2 ). Misinterpretation of chest pain, specifically among female patients, has the potential to cause a cardiovascular-related diagnosis error [ 36 ]. Patients with chronic kidney disease are at higher risk of cardiovascular complications [ 37 ]. Missing such risk for a patient who is on dialysis, adds to the risk of diagnostic error.

Clinical and System Implications Around Diagnostic Inequity

Diagnostic inequity is defined as “the presence of preventable unwarranted variations in diagnostic process among population groups that are socially, economically, demographically, or geographically disadvantaged” [ 38 ]. Despite persistent and well-documented disparities in health care access and outcomes across different population groups, few studies have examined the association between diagnostic errors and health care disparities [ 39 ]. Recent evidence supports the notion that variation in diagnostic error rates across demographic groups may exist, particularly across sex. A systematic review of diagnostic errors in the emergency department, for example, found that female sex and non-White race were often associated with increased risk for diagnostic errors across several clinical conditions in emergency settings [ 40 ]. In cardiovascular medicine, a national cohort study of acute myocardial infarctions found that women were nearly twice as likely as men to receive the wrong initial diagnosis following signs of a heart attack [ 41 ]. Despite efforts to understand and reduce disparities in diagnosis and treatment, women not only continue to be understudied, underdiagnosed, and undertreated in cardiovascular medicine [ 42 ] but also may experience longer lengths of time to diagnosis than men in most patterns of disease diagnosis [ 43 ].

The analysis of case review data and other system-based data (eg, patient safety events or incident reporting) by subsets offer an opportunity to identify events in vulnerable patient populations and help sensitize clinicians to potential biases within the diagnostic process. To explore sex differences in diagnostic errors within our case review data, we statistically compared demographic and clinical differences between female and male patients who had been identified in case reviews as having experienced diagnostic error or errors. We found that of those patients who had experienced diagnostic error or errors, the female group of patients were older, had higher rates of being admitted through general or internal medicine or hospitalist (vs specialty) departments, and had lower rates of having a cardiovascular diagnosis on admission. These preliminary results of this study revealed unexpected differences between male and female diagnostic error groups, offering novel insights that warrant further investigation to fully understand the mechanisms underlying these relationships and their implications for clinical decision-making and practice. Future uses of NLP can potentially support clinical and system-based approaches to capture and increase the evidence around structural biases or disparities in diagnoses. Individual cases from these types of data sources could be used as example narratives to engage clinicians and improve clinician learning, contributing to the development of tailored clinician and systemic interventions that can improve quality and equity throughout the diagnostic process.

Limitations

This study has several limitations. Our definition of diagnostic errors was limited to the categories and labels used within the SLS data set, reviewer interpretations of cases (subject to reviewer bias), and does not include all aspects of the definition developed by the NASEM report [ 3 ]. Despite several continued differences in definitions of diagnostic error in the peer-reviewed literature [ 8 ], we recommend that quality and safety teams within health systems use the NASEM definition for diagnostic error—including errors in communicating the diagnosis to the patient—to develop any definitions, categories, or labels used in their case review and surveillance initiatives. Although a time-consuming task, future studies could consider EHR data chart reviews to have the ground truth for the diagnostic error cases and add to the accuracy of the data set used for training the ML classifiers. Additionally, due to staffing challenges and shifting organizational priorities, case review selection varies by hospital and has changed over time, resulting in a relatively small sample size and also introducing the potential for bias. Our data came from a single health system and may reflect the specific language, culture, and practices occurring within the system and therefore may not be similar to that of other health systems. To enhance the external validity and generalizability of results, future efforts and research studies should consider the random selection of cases to evaluate both diagnostic and general quality issues within the organization; studies with larger sample sizes can build on our preliminary findings and test differences between clinical subgroups. Finally, our classification models were developed and evaluated based on a retrospective cohort from EHR; therefore, the performance may deteriorate when the method is applied to real-time data. Further work or future studies should be conducted to prospectively validate the models.

Conclusions

We performed an NLP approach and compared 4 techniques to classify patients who were at a higher risk of experiencing diagnostic error during hospitalization. Our findings demonstrate that NLP can be a potential solution to more effectively identifying and selecting potential diagnostic error cases for review, and therefore, reducing the case review burden.

Acknowledgments

This work was supported by the Agency for Health Care Research and Quality (grant 5R18HS027280-02).

Conflicts of Interest

None declared.

Binary classification performance metrics.

The Estimated Coefficient from the Ridge Model.

• Newman-Toker DE, Wang Z, Zhu Y, Nassery N, Saber Tehrani AS, Schaffer AC, et al. Rate of diagnostic errors and serious misdiagnosis-related harms for major vascular events, infections, and cancers: toward a national incidence estimate using the "Big Three". Diagnosis (Berl). Feb 23, 2021;8(1):67-84. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gunderson CG, Bilan VP, Holleck JL, Nickerson P, Cherry BM, Chui P, et al. Prevalence of harmful diagnostic errors in hospitalised adults: a systematic review and meta-analysis. BMJ Qual Saf. Dec 2020;29(12):1008-1018. [ CrossRef ] [ Medline ]
• Ball JR, Miller BT, Balogh EP. Improving Diagnosis in Health Care. Washington, DC. National Academies Press; 2015:472.
• Hautz WE, Kämmer JE, Hautz SC, Sauter TC, Zwaan L, Exadaktylos AK, et al. Diagnostic error increases mortality and length of hospital stay in patients presenting through the emergency room. Scand J Trauma Resusc Emerg Med. May 08, 2019;27(1):54. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gupta A, Snyder A, Kachalia A, Flanders S, Saint S, Chopra V. Malpractice claims related to diagnostic errors in the hospital. BMJ Qual Saf. Aug 09, 2017;27(1):53-60. [ CrossRef ] [ Medline ]
• Zwaan L, Singh H. Diagnostic error in hospitals: finding forests not just the big trees. BMJ Qual Saf. 2020;29(12):961-964. [ CrossRef ] [ Medline ]
• Singh H. Editorial: helping health care organizations to define diagnostic errors as missed opportunities in diagnosis. Jt Comm J Qual Patient Saf. 2014;40(3):99-101. [ CrossRef ] [ Medline ]
• Giardina TD, Hunte H, Hill MA, Heimlich SL, Singh H, Smith KM. Defining diagnostic error: a scoping review to assess the impact of the national academies' report improving diagnosis in health care. J Patient Saf. 2022;18(8):770-778. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Measure Dx: a resource to identify, analyze, and learn from diagnostic safety events. AHRQ: Agency for Healthcare Research and Quality. URL: https://www.ahrq.gov/patient-safety/settings/multiple/measure-dx.html [accessed 2023-06-22]
• Zwaan L, de Bruijne M, Wagner C, Thijs A, Smits M, van der Wal G, et al. Patient record review of the incidence, consequences, and causes of diagnostic adverse events. Arch Intern Med. 2010;170(12):1015-1021. [ CrossRef ] [ Medline ]
• Singh H. Diagnostic errors: moving beyond 'no respect' and getting ready for prime time. BMJ Qual Saf. 2013;22(10):789-792. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zwaan L, El-Kareh R, Meyer AND, Hooftman J, Singh H. Advancing diagnostic safety research: results of a systematic research priority setting exercise. J Gen Intern Med. 2021;36(10):2943-2951. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huddleston JM, Diedrich DA, Kinsey GC, Enzler MJ, Manning DM. Learning from every death. J Patient Saf. 2014;10(1):6-12. [ CrossRef ] [ Medline ]
• Arnaud E, Elbattah M, Gignon M, Dequen G. NLP-based prediction of medical specialties at hospital admission using triage notes. 2021. Presented at: IEEE 9th International Conference on Healthcare Informatics (ICHI); August 09-12, 2021; Victoria, British Columbia, Canada. [ CrossRef ]
• Bansal A, Saraswat B, Sharma B, Nayan S, Kathuria K. Data extraction and integration from unstructured electronic health records. 2023. Presented at: 2023 3rd International Conference on Technological Advancements in Computational Sciences (ICTACS); November 01-03, 2023; Tashkent, Uzbekistan. [ CrossRef ]
• Sezgin E, Hussain S, Rust S, Huang Y. Extracting medical information from free-text and unstructured patient-generated health data using natural language processing methods: feasibility study with real-world data. JMIR Form Res. 2023;7:e43014. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Asri H, Mousannif H, Moatassime HA, Noel T. Using machine learning algorithms for breast cancer risk prediction and diagnosis. Procedia Comput Sci. 2016;83:1064-1069. [ CrossRef ]
• Goldenberg SL, Nir G, Salcudean SE. A new era: artificial intelligence and machine learning in prostate cancer. Nat Rev Urol. 2019;16(7):391-403. [ CrossRef ] [ Medline ]
• Henry KE, Adams R, Parent C, Soleimani H, Sridharan A, Johnson L, et al. Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing. Nat Med. 2022;28(7):1447-1454. [ CrossRef ] [ Medline ]
• Nasiri S, Gari D. Attentive adversarial network for large-scale sleep staging. 2020. Presented at: Proceedings of the 5th Machine Learning for Healthcare Conference, PMLR; September, 2020:457-478; Atlanta, GA, USA. URL: https://proceedings.mlr.press/v126/nasiri20a.html
• Tabaie A, Nemati S, Allen JW, Chung C, Queiroga F, Kuk W, et al. Assessing contribution of higher order clinical risk factors to prediction of outcome in aneurysmal subarachnoid hemorrhage patients. AMIA Annu Symp Proc. 2019;2019:848-856. [ FREE Full text ] [ Medline ]
• Cheheltani R, King N, Lee S, North B, Kovarik D, Evans-Molina C, et al. Predicting misdiagnosed adult-onset type 1 diabetes using machine learning. Diabetes Res Clin Pract. 2022;191:110029. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hercus C, Hudaib A. Delirium misdiagnosis risk in psychiatry: a machine learning-logistic regression predictive algorithm. BMC Health Serv Res. 2020;20(1):151. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shen L, Levie A, Singh H, Murray K, Desai S. Harnessing event report data to identify diagnostic error during the COVID-19 pandemic. Jt Comm J Qual Patient Saf. 2022;48(2):71-80. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Murphy DR, Meyer AN, Sittig DF, Meeks DW, Thomas EJ, Singh H. Application of electronic trigger tools to identify targets for improving diagnostic safety. BMJ Qual Saf. 2019;28(2):151-159. [ FREE Full text ] [ CrossRef ] [ Medline ]
• The safety learning system®. Discover the Hidden Causes of Patient Harm. URL: http://hbhealthcaresafety.org/safety-learning-system/ [accessed 2023-06-22]
• Alsentzer E, Murphy J, Boag W, Weng W, Jindi D, Naumann T, et al. Publicly available clinical BERT embeddings. 2019. Presented at: Proceedings of the 2nd Clinical Natural Language Processing Workshop; June, 2019; Minneapolis, Minnesota, USA. [ CrossRef ]
• Huang K, Altosaar A, Ranganath R. ClinicalBERT: modeling clinical notes and predicting hospital readmission. ArXiv. Preprint posted online on April 10, 2019. 2019:190405342. [ CrossRef ]
• Zhang D, Thadajarassiri J, Sen C, Rundensteiner E. Time-aware transformer-based network for clinical notes series prediction. 2020. Presented at: Proceedings of the 5th Machine Learning for Healthcare Conference; September, 2020; Worcester, MA, USA. URL: https://proceedings.mlr.press/v126/zhang20c/zhang20c.pdf
• Jones KS. A statistical interpretation of term specificity and its application in retrieval. J Doc. 1972;28(1):11-21. [ CrossRef ]
• Tabaie A, Sengupta S, Pruitt ZM, Fong A. A natural language processing approach to categorise contributing factors from patient safety event reports. BMJ Health Care Inform. 2023;30(1):e100731. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Aheto JMK, Duah HO, Agbadi P, Nakua EK. A predictive model, and predictors of under-five child malaria prevalence in Ghana: how do LASSO, Ridge and Elastic Net regression approaches compare? Prev Med Rep. 2021;23:101475. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zou H, Trevor H. Regularization and variable selection via the Elastic Net. J R Stat Soc Series B Stat Methodol. 2005;67(2):301-320. [ CrossRef ]
• Lee SJ, Weinberg BD, Gore A, Banerjee I. A scalable natural language processing for inferring BT-RADS categorization from unstructured brain magnetic resonance reports. J Digit Imaging. 2020;33(6):1393-1400. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Masino AJ, Harris MC, Forsyth D, Ostapenko S, Srinivasan L, Bonafide CP, et al. Machine learning models for early sepsis recognition in the neonatal intensive care unit using readily available electronic health record data. PLoS One. 2019;14(2):e0212665. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mehta PK, Bess C, Elias-Smale S, Vaccarino V, Quyyumi A, Pepine CJ, et al. Gender in cardiovascular medicine: chest pain and coronary artery disease. Eur Heart J. 2019;40(47):3819-3826. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jankowski J, Floege J, Fliser D, Böhm M, Marx N. Cardiovascular disease in chronic kidney disease: pathophysiological insights and therapeutic options. Circulation. 2021;143(11):1157-1172. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Giardina TD, Woodard LD, Singh H. Advancing diagnostic equity through clinician engagement, community partnerships, and connected care. J Gen Intern Med. 2023;38(5):1293-1295. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ibrahim SA, Pronovost PJ. Diagnostic errors, health disparities, and artificial intelligence: a combination for health or harm? JAMA Health Forum. 2021;2(9):e212430. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Newman-Toker DE, Peterson SM, Badihian S, Hassoon A, Nassery N, Parizadeh D, et al. Diagnostic Errors in the Emergency Department: A Systematic Review [internet]. Rockville (MD). Agency for Healthcare Research and Quality; 2022.
• Wu J, Gale CP, Hall M, Dondo TB, Metcalfe E, Oliver G, et al. Editor's choice—impact of initial hospital diagnosis on mortality for acute myocardial infarction: a national cohort study. Eur Heart J Acute Cardiovasc Care. 2018;7(2):139-148. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Burgess S. Understudied, under-recognized, underdiagnosed, and undertreated: sex-based disparities in cardiovascular medicine. Circ Cardiovasc Interv. 2022;15(2):e011714. [ CrossRef ] [ Medline ]
• Sun TY, Hardin J, Nieva HR, Natarajan K, Cheng R, Ryan P, et al. Large-scale characterization of gender differences in diagnosis prevalence and time to diagnosis. MedRxiv. Preprint posted online on October 16, 2023. 2023. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by S Ma, T Leung; submitted 17.07.23; peer-reviewed by D Chrimes, M Elbattah; comments to author 18.01.24; revised version received 21.03.24; accepted 20.06.24; published 26.08.24.

©Azade Tabaie, Alberta Tran, Tony Calabria, Sonita S Bennett, Arianna Milicia, William Weintraub, William James Gallagher, John Yosaitis, Laura C Schubel, Mary A Hill, Kelly Michelle Smith, Kristen Miller. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 26.08.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Modeling and monitoring cotton production using remote sensing techniques and machine learning: a case study of Punjab, Pakistan

• Published: 23 August 2024

Cite this article

• Sher Shah Hasan   ORCID: orcid.org/0000-0002-9330-4120 1 ,
• Muhammad Arif Goheer 1 ,
• Muhammad Uzair 2 &
• Saba Fatima 2  

48 Accesses

Explore all metrics

Agriculture is the backbone of Pakistan’s economy and makes up 24 percent of the national GDP and half the labor force. This makes crop estimation studies extremely vital for a country’s economic growth and food security. Cotton is one of the most important cash crops in Pakistan contributing 2.4% to the total value addition in agriculture. Remote Sensing (RS) and Geographic Information Systems (GIS) techniques can be used to effectively estimate crop yields even before harvesting. The objective of this study was to utilize RS/GIS, and machine learning to create a model for predicting cotton production; as well as identifying the impacts of climate-related factors on the growth and yield of cotton. Data from MODIS product MOD13A1, with a 16-day temporal resolution from 2011 to 2021 was used to calculate eleven vegetation indices in cotton-dominated districts of Punjab. These indices, along with rainfall data, temperature data, and historical yield data served as input to the machine learning models. Automatic Linear Modeling (ALM) and Artificial Neural Networks (ANN) were used to forecast the yields. The study also created a correlation between climate factors (rainfall and temperature) and cotton seasonal production. Pearson correlation coefficient of − 0.319 indicated a significant influence of maximum temperature on observed yields, while the Automatic Linear Modeling showed both maximum temperature and participation as a predictor for yield. These results underscore the vulnerability of cotton to climate change, proving cotton’s sensitivity to temperature and rainfall. Comparison of both models based on their predictive yield placed ALM model’s accuracy above ANN’s at 44.1%, providing insights into the effectiveness of traditional linear modeling versus neural network approaches in predicting cotton yields.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Cotton cultivated area detection and yield monitoring combining remote sensing with field data in lower Indus River basin, Pakistan

Determination of cotton and wheat yield using the standard precipitation evaporation index in Pakistan

An accuracy assessment of satellite-based cotton yield estimation using panel data regression: a case study of Uzbekistan

Explore related subjects.

• Artificial Intelligence

AARI (Ayub Agricultural Research Institute). (2024). Cotton. https://aari.punjab.gov.pk/crop_varieties_cotton . Accessed on February 02, 2024.

Adak, S., Bandyopadhyay, K. K., Sahoo, R. N., Mridha, N., Shrivastava, M., & Purakayastha, T. J. (2021). Prediction of wheat yield using spectral reflectance indices under different tillage, residue and nitrogen management practices. Current Science , 121 (3), 402. https://doi.org/10.18520/cs/v121/i3/402-413

Agatonovic-Kustrin, S., & Beresford, R. (2000). Basic concepts of artificial neural network (ANN) modeling and its application in pharmaceutical research. Journal of Pharmaceutical and Biomedical Analysis, 22 (5), 717–727. https://doi.org/10.1016/s0731-7085(99)00272-1

Article   CAS   Google Scholar  

Akbar, H., & Gheewala, S. H. (2020). Effect of climate change on cash crops yield in Pakistan. Arabian Journal of Geosciences . https://doi.org/10.1007/s12517-020-05333-7

Article   Google Scholar  

Ali, S. M., Khalid, B., Akhter, A., Islam, A., & Adnan, S. (2020). Analyzing the occurrence of floods and droughts in connection with climate change in Punjab province, Pakistan. Natural Hazards, 103 , 2533–2559.

Amin, M., Khan, M. R., Hassan, S. S., Imran, M., Hanif, M., & Baig, I. A. (2023). Determining satellite-based evapotranspiration product and identifying relationship with other observed data in Punjab, Pakistan. Environment, Development and Sustainability, 1–17.

Bajwa, S. G., Mishra, A. R., & Norman, R. J. (2009). Canopy reflectance response to plant nitrogen accumulation in rice. Precision Agriculture, 11 (5), 488–506. https://doi.org/10.1007/s11119-009-9142-0

Bannari, A., Asalhi, H., & Teillet, P. M. (2002). Transformed difference vegetation index (TDVI) for vegetation cover mapping. In IEEE International geoscience and remote sensing symposium (Vol. 5, pp. 3053–3055). IEEE.

Binte Mostafiz, R., Noguchi, R., & Ahamed, T. (2021). Agricultural land suitability assessment using satellite remote sensing-derived soil-vegetation indices. Land, 10 (2), 223. https://doi.org/10.3390/land10020223

Christensen, R. (2020). Variable Selection. Plane Answers to Complex Questions: The Theory of Linear Models . https://doi.org/10.1007/978-3-030-32097-3_14

Comtrade, U. (2021). International Trade Statistics Yearbook Volume II-Trade by Commodity. New York: United Nations Statistics Division. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/ https://comtradeapi.un.org/files/v1/app/publicationfiles/2021/VolII2021.pdf . Accessed on February 25, 2024.

Córcoles, J. I., Ortega, J. F., Hernández, D., & Moreno, M. A. (2013). Estimation of leaf area index in onion ( Allium cepa L.) using an unmanned aerial vehicle. Biosystems Engineering, 115 (1), 31–42. https://doi.org/10.1016/j.biosystemseng.2013.02.002

Crop Reporting Service, Punjab. (2021). Kharif Crops Estimates. https://crs-agripunjab.punjab.gov.pk/system/files/Kharif%20Estimates%202020-2021%20%28PDF%29%20final_0.pdf . Accessed on February 05, 2024.

Farrell, M., Gili, A., & Noellemeyer, E. (2018). Spectral indices from aerial images and their relationship with properties of a corn crop. Precision Agriculture, 19 (6), 1127–1137. https://doi.org/10.1007/s11119-018-9570-9

Feng, Y., Gong, D., Zhang, Q., Jiang, S., Zhao, L., & Cui, N. (2019). Evaluation of temperature-based machine learning and empirical models for predicting daily global solar radiation. Energy Conversion and Management, 198 , 111780. https://doi.org/10.1016/j.enconman.2019.111780

Gitelson, A. A. (2004). wide dynamic range vegetation index for remote quantification of biophysical characteristics of vegetation. Journal of Plant Physiology, 161 (2), 165–173. https://doi.org/10.1078/0176-1617-01176

Grassini, P., van Bussel, L. G. J., Van Wart, J., Wolf, J., Claessens, L., Yang, H., Boogaard, H., de Groot, H., van Ittersum, M. K., & Cassman, K. G. (2015). How good is good enough? Data requirements for reliable crop yield simulations and yield-gap analysis. Field Crops Research, 177 , 49–63. https://doi.org/10.1016/j.fcr.2015.03.004

Hassan, S. S., & Goheer, M. A. (2021). Modeling and monitoring wheat crop yield using geospatial techniques: A case study of potohar region, Pakistan. Journal of the Indian Society of Remote Sensing, 49 (6), 1331–1342. https://doi.org/10.1007/s12524-020-01290-610.1136/bmj.e4483

Hatfield, J. L., Prueger, J. H., Sauer, T. J., Dold, C., O’Brien, P., & Wacha, K. (2019). Applications of vegetative indices from remote sensing to agriculture: Past and future. Inventions, 4 (4), 71. https://doi.org/10.3390/inventions4040071

Huete, A., Didan, K., van Leeuwen, W., Miura, T., & Glenn, E. (2010). MODIS vegetation indices. Land Remote Sensing and Global Environmental Change . https://doi.org/10.1007/978-1-4419-6749-7_26

IFPRI (International Food Policy Research Institute). (2022). Ifpri.org. https://www.ifpri.org/publication/cotton-crop-situational-analysis-pakistan .

Ihuoma, S. O., & Madramootoo, C. A. (2019). Sensitivity of spectral vegetation indices for monitoring water stress in tomato plants. Computers and Electronics in Agriculture, 163 , 104860. https://doi.org/10.1016/j.compag.2019.104860

Jafari, R., Lewis, M. M., & Ostendorf, B. (2007). Evaluation of vegetation indices for assessing vegetation cover in southern arid lands in South Australia. The Rangeland Journal, 29 (1), 39. https://doi.org/10.1071/rj06033

Kahimba, F., Bullock, P., Ranjan, R., & Cutforth, H. (2009).  Evaluation of the SolarCalc model for simulating hourly and daily incoming solar radiation in the Northern Great Plains of Canada . Retrieved August 24, 2022, from https://library.csbe-scgab.ca/docs/journal/51/c0818.pdf .

Khairunniza-Bejo, S., Mustaffha, S., & Ismail, W. I. W. (2014). Application of artificial neural network in predicting crop yield: A review. Journal of Food Science and Engineering, 4 (1), 1.

Google Scholar  

Killi, F., & Bolek, Y. (2006). Timing of planting is crucial for cotton yield. Acta Agriculturae Scandinavica, Section B—Plant Soil Science, 56 (2), 155–160. https://doi.org/10.1080/09064710510029178

Kumar, R., Singh, M. P., Kumar, P., & Singh, J. P. (2015). Crop Selection Method to maximize crop yield rate using machine learning technique. IEEE . https://doi.org/10.1109/ICSTM.2015.7225403

Malik, T. H., & Ahsan, M. Z. (2016). Review of the cotton market in Pakistan and its prospects. OCL, 23 (6), D606. https://doi.org/10.1051/ocl/2016043

Mirzaie, M., Darvishzadeh, R., Shakiba, A., Matkan, A. A., Atzberger, C., & Skidmore, A. K. (2014). Comparative analysis of different uni- and multi-variate methods for estimation of vegetation water content using hyper-spectral measurements. International Journal of Applied Earth Observation and Geoinformation, 26 , 1–11. https://doi.org/10.1016/j.jag.2013.04.004

Muruganantham, P., Wibowo, S., Grandhi, S., Samrat, N. H., & Islam, N. (2022). A systematic literature review on crop yield prediction with deep learning and remote sensing. Remote Sensing, 14 (9), 1990. https://doi.org/10.3390/rs14091990

Nagini, S., Kanth, T. R., & Kiranmayee, B. V. (2016). Agriculture yield prediction using predictive analytic techniques. In 2nd International Conference on Contemporary Computing and Informatics . https://doi.org/10.1109/IC3I.2016.7918789

Panchal, G., Ganatra, A., Kosta, Y. P., & Panchal, D. (2011). Behaviour analysis of multilayer perceptrons with multiple hidden neurons and hidden layers. International Journal of Computer Theory and Engineering, 3 (2), 332–337.

Panda, S. S., Ames, D. P., & Panigrahi, S. (2010). Application of vegetation indices for agricultural crop yield prediction using neural network techniques. Remote Sensing, 2 (3), 673–696. https://doi.org/10.3390/rs2030673

PBS (Pakistan Bureau of Statistics), Annual Analytical Report on External Trade Statistics of Pakistan FY 2020–21 . (2020). Pbs.gov.pk. https://www.pbs.gov.pk/publication/annual-analytical-report-external-trade-statistics-pakistan-fy-2020-21 .

Piekutowska, M., Niedbała, G., Piskier, T., Lenartowicz, T., Pilarski, K., Wojciechowski, T., Pilarska, A. A., & Czechowska-Kosacka, A. (2021). The application of multiple linear regression and artificial neural network models for yield prediction of very early potato cultivars before Harvest. Agronomy, 11 (5), 885. https://doi.org/10.3390/agronomy11050885

Rahman, M. H. U., Ahmad, I., Ghaffar, A., Haider, G., Ahmad, A., Ahmad, B., et al. (2020). Climate resilient cotton production system: a case study in Pakistan. Cotton Production and Uses: Agronomy, Crop Protection, and Postharvest Technologies , 447–484.

Rahman-ur, M. H., Ahmad, A., Wang, X., Wajid, A., Nasim, W., Hussain, M., Ahmad, B., Ahmad, I., Ali, Z., Ishaque, W., Awais, M., Shelia, V., Ahmad, S., Fahd, S., Alam, M., Ullah, H., & Hoogenboom, G. (2018). Multi-model projections of future climate and climate change impacts uncertainty assessment for cotton production in Pakistan. Agricultural and Forest Meteorology, 253–254 , 94–113. https://doi.org/10.1016/j.agrformet.2018.02.008

Ransom, C. J., Kitchen, N. R., Camberato, J. J., Carter, P. R., Ferguson, R. B., Fernández, F. G., Franzen, D. W., Laboski, C. A. M., Myers, D. B., Nafziger, E. D., Sawyer, J. E., & Shanahan, J. F. (2019). Statistical and machine learning methods evaluated for incorporating soil and weather into corn nitrogen recommendations. Computers and Electronics in Agriculture, 164 , 104872. https://doi.org/10.1016/j.compag.2019.104872

Rondeaux, G., Steven, M., & Baret, F. (1996). Optimization of soil-adjusted vegetation indices. Remote Sensing of Environment, 55 (2), 95–107. https://doi.org/10.1016/0034-4257(95)00186-7

Sedgwick, P. (2012). Pearson’s correlation coefficient. BMJ, 345 (1), e4483–e4483.

Sibley, A. M., Grassini, P., Thomas, N. E., Cassman, K. G., & Lobell, D. B. (2014). Testing remote sensing approaches for assessing yield variability among maize fields. Agronomy Journal, 106 (1), 24–32. https://doi.org/10.2134/agronj2013.0314

Singh, R. P., Prasad, P. V., Sunita, K., Giri, S. N., & Reddy, K. R. (2007). Influence of high temperature and breeding for heat tolerance in cotton: A review. Advances in Agronomy, 93 , 313–385.

Steven, M. D. (1998). The sensitivity of the OSAVI vegetation index to observational parameters. Remote Sensing of Environment, 63 (1), 49–60. https://doi.org/10.1016/S0034-4257(97)00114-4

Waha, K., Krummenauer, L., Adams, S., Aich, V., Baarsch, F., Coumou, D., Fader, M., Hoff, H., Jobbins, G., Marcus, R., Mengel, M., Otto, I. M., Perrette, M., Rocha, M., Robinson, A., & Schleussner, C.-F. (2017). Climate change impacts in the middle east and Northern Africa (MENA) region and their implications for vulnerable population groups. Regional Environmental Change, 17 (6), 1623–1638. https://doi.org/10.1007/s10113-017-1144-2

Wu, W. (2014). The generalized difference vegetation index (GDVI) for dryland characterization. Remote Sensing, 6 (2), 1211–1233. https://doi.org/10.3390/rs6021211

Xue, J., & Su, B. (2017). Significant remote sensing vegetation indices: A review of developments and applications. Journal of Sensors, 2017 , 1–17. https://doi.org/10.1155/2017/1353691

Yakubu, O. I., Oluremi, A., & Ekpo, E. I. (2018). Predicting heat stress index in Sasso hens using automatic linear modeling and artificial neural network. International Journal of Biometeorology .

Yang, H. (2013). The case for being automatic: Introducing the automatic LINEAR modeling (LINEAR) procedure in SPSS statistics. Multiple Linear Regression. Viewpoints, 39 (2), 27–37.

Zhen, Z., Chen, S., Yin, T., Chavanon, E., Lauret, N., Guilleux, J., Henke, M., Qin, W., Cao, L., Li, J., Lu, P., & Gastellu-Etchegorry, J.-P. (2021). Using the negative soil adjustment factor of soil adjusted vegetation index (SAVI) to resist saturation effects and estimate leaf area index (LAI) in dense vegetation areas. Sensors, 21 (6), 2115. https://doi.org/10.3390/s21062115

Zhong, W., Zhang, T., Zhu, Y., & Liu, J. S. (2012). Correlation pursuit: Forward stepwise variable selection for index models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74 (5), 849–870. https://doi.org/10.1111/j.1467-9868.2011.01026.x

Download references

Author information

Authors and affiliations.

Global Climate-Change Impact Studies Centre, MoCC & EC, Islamabad, Pakistan

Sher Shah Hasan & Muhammad Arif Goheer

University of Sciences & Technology (NUST), Islamabad, Pakistan

Muhammad Uzair & Saba Fatima

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sher Shah Hasan .

Ethics declarations

Conflict of interest.

The authors have no conflict of interest.

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on request.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Hasan, S.S., Goheer, M.A., Uzair, M. et al. Modeling and monitoring cotton production using remote sensing techniques and machine learning: a case study of Punjab, Pakistan. Environ Dev Sustain (2024). https://doi.org/10.1007/s10668-024-05331-9

Download citation

Received : 05 September 2022

Accepted : 17 August 2024

Published : 23 August 2024

DOI : https://doi.org/10.1007/s10668-024-05331-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• GIS & RS
• Vegetation indices
• Yield predication
• Find a journal
• Publish with us
• Track your research