netflix cloud case study

• Bahasa Indonesia
• Sign out of AWS Builder ID
• AWS Management Console
• Account Settings
• Billing & Cost Management
• Security Credentials
• AWS Personal Health Dashboard
• Support Center
• Expert Help
• Knowledge Center
• AWS Support Overview
• AWS re:Post

Netflix Case Study

Online content provider Netflix can support seamless global service by using Amazon Web Services (AWS). AWS enables Netflix to quickly deploy thousands of servers and terabytes of storage within minutes. Users can stream Netflix shows and movies from anywhere in the world, including on the web, on tablets, or on mobile devices such as iPhones.

Learn more about how Netflix continues to innovate on AWS .

More Netflix Stories

Netflix's cloud journey on aws.

Netflix is the world’s leading internet television network, with more than 100 million members in more than 190 countries enjoying 125 million hours of TV shows and movies each day. Netflix uses AWS for nearly all its computing and storage needs, including databases, analytics, recommendation engines, video transcoding, and more—hundreds of functions that in total use more than 100,000 server instances on AWS.

Learn more »

Netflix Uses NICE DCV on AWS to Build VFX Studio in the Cloud for Artists Globally

Netflix used NICE DCV and Amazon EC2 G4 Instances to build virtual workstations that attract top VFX and animation artists worldwide and enable seamless collaboration between global teams.

Netflix & AWS Lambda Case Study

Watch Neil Hunt, Netflix’s chief product officer, explain how the company can use event-based triggers to help automate the encoding process of media files, the validation of backup completions and instance deployments at scale, and the monitoring of AWS resources used by the organization.

Netflix & Amazon Kinesis Data Streams Case Study

Netflix uses AWS to analyze billions of messages across more than 100,000 application instances daily in real time, enabling it to optimize user experience, reduce costs, and improve application resilience.

Netflix & Amazon SES Case Study

Netflix needed an email solution that was flexible, affordable, highly scalable, had global reach, and promised excellent deliverability. "We believed Amazon SES could help us be elastic, that we could pay as we went along, and that we could stop worrying about optimizing settings for each ISP, and so we embarked upon a plan to test and migrate to Amazon SES," said Devika Chawla, Director of Engineering for Netflix's Messaging and Platform team.

Additional Resources

Get started.

Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today.

Ending Support for Internet Explorer

Case Study: How Netflix uses Cloud for Innovation, Agility and Scalability

Suggested articles, top 5 predictions for the cloud in 2019, how to streamline your data archival process using the cloud, cloudsine showcases weborion to protect cloud-based websites and web applications at 10-11 april aws summit 2019, cloudsine @ div0 startup quarter – on 23 may 2019, cloud security seminar – on 4 sept 19, the future of digital workforce with intelligent automation seminar , 21 nov 19, case study: cloudsine accelerates centre for evidence and implementation (cei)’s cloud adoption journey, aws cloudgoat and mitigation strategies: part 1, aws cloudgoat and mitigation strategies: part 2, aws cloudgoat and mitigation strategies: part 3, aws cloudgoat and mitigation strategies: part 4, aws cloudgoat and mitigation strategies: part 5, weborion® launches javascript malware detection engine (jme), cloudsine | weborion® supports the launch of community-focused ai security quarter in div0 on 31 mar 2021, cloudsine is excited to partner with sginnovate new frontier event to build up deep tech community, product announcement: enhanced email alerts for weborion defacement monitor, cloudsine and weborion signs technology alliance partnership with new net technologies, sutd x cloudsine – artificial intelligence award, sginnovate’s powerx programme with cloudsine, statement on apache log4j2 remote code execution (rce) vulnerability on weborion products and customers – cve-2021-44228, the cybersecurity implications for website owners from the russia-ukraine conflict, how are hacktivists shaping the cybersecurity posture of nation-states in the russia-ukraine conflict, weborion® adds smart image hash (sih) feature to improve monitoring of compressed images, weborion® introduces ai nlp for web defacement monitoring, cloudsine | weborion® – technology alliance partnership with netrust pte ltd, the serverless model for the uninitiated, seamless integration with the weborion® api, the weborion® defacement monitor cloud saas is now available on aws marketplace, cloudsine exhibited in govware 2022, 18th to 20th october, dns – a brief summary of an easily overlooked system, enumerate, secure and detect changes in dns records, port scanning – exposing your network’s points of entry, anyone can enumerate your web server using port scanning tools, weborion® anti-defacement and web security stack is now available on indonesia’s lkpp e-katalog, weborion® anti-defacement dan web security stack kini sudah hadir di e-katalog lkpp indonesia, what’s new in pci-dss v4.0: payment page javascript monitoring, what’s new in pci-dss v4.0: http header tamper detection, magecart and card skimming detection, what’s new in pci-dss v4.0: supply chain inventory of software, what’s new in pci-dss v4.0: ssl cert monitoring, preventing web defacement: a technical manager’s guide to securing web applications.

“Planning without taking action is the slowest route to victory. Taking action without planning is the noise before defeat.” - Sun Tzu, The Art of War Introduction to cloud computing It is said that the world evolves at the speed of technological evolution. Organizations are constantly looking for new technologies...

“The best way to predict the future is to invent it.” - Peter Drucker, American economist and corporate philosopher Cloud computing, the advancement of computing over a network of servers, was a key driver of the tech industry in 2018. Mergers and acquisitions between large and small companies led to...

How to Streamline Your Data Archival Process using the Cloud Data archiving is the process of moving data that is no longer essential to a separate data store for long-term retention. Archived data consists of older data that might serve some importance to the organization, possibly for future reference or...

Event Date: 10-11 April 2019 Last Updated: 27 May 2019 The AWS Summit Singapore brings together the cloud computing community to connect, collaborate and learn about AWS. Cloudsine was recognized as an AWS Technology Partner and an AWS Consulting Partner last year. During the Summit, we showcased WebOrion™, an all-in-one...

Event Date: 23 May 2019 Last Updated: 27 May 2019 Startup Quarter - Secure your cloud, was a meetup organized by Division Zero (Div0) on 23rd May 2019 at ACE. Div0 an open, inclusive, and completely volunteer-driven cybersecurity community with a mission of promoting a vibrant cybersecurity community and safer...

Event Date: 4 Sept 2019 Last Updated: 10 Sept 2019 Cloudsine is glad to host our very own seminar on the theme of Cloud Security: Myths, New Security Concerns and Mitigations on 4 Sept 2019. Glad to engage and interact with an audience of Singapore and Indonesia customers, resellers and...

Event Date: 21 Nov 2019 Last Updated: 25 Nov 2019 Cloudsine had our first RPA seminar in collaboration with Automation Anywhere on the theme of the “Future of Digital Workforce with Intelligent Automation” on 21 November 2019. It was our pleasure to have Ehunt Siow and Sundarraj Subrammani from Automation...

Cloudsine Accelerates Centre for Evidence and Implementation (CEI)’s Cloud Adoption Journey by Providing Data Migration, a Customized File Portal that Integrates with AWS S3 and Cloud Data Security Assurance. The Centre for Evidence and Implementation (CEI)is a global team of research, policy and practice experts based in Australia, Singapore…

Part 1 of our own series of articles on CloudGoat and mitigation strategies. This is a step by step breakdown on how to interpret and think like an attacker and also how to go about mitigating the attacks.

Part 2 of our own series of articles on CloudGoat and mitigation strategies. This is a step by step breakdown on how to interpret and think like an attacker and also how to go about mitigating the attacks.

In this third part, we will explore privilege escalation using EC2 instance profile attachment to obtain full admin privileges on the AWS account and also exploiting SSRF on EC2’s metadata service to get credentials.

This is part 4 of the series on AWS Cloudgoat Scenarios and the mitigation strategies series where we explore and see how remote command injection on a web application can be used to compromise the AWS environment.

WebOrion® is pleased to announce the launch of our new Javascript Malware Detection Engine(JME). The JME adds to the powerful capabilities of our WebOrion® Monitor to detect defacements, malicious scripts and other website threats. Today, practically every website uses JavaScript. The power and flexibility of a scripting language embedded within...

AI Security Quarter of Div0 was officially launched on 31 Mar 2021 over virtual Zoom and attended by >50 cybersecurity and AI enthusiasts in Singapore. Cybersecurity and AI are both critically important technologies for the digital future. Attackers are using more automation and AI to help them probe and attack...

Cloudsine is excited to partner with SGInnovate at the New Frontier event on 10 Apr 2021 to help build up the deeptech community. The New Frontier event is organized by SGInnovate with Guest of Honour, Lawrence Wong (Minister for Education), to promote the growth of the deeptech ecosystem in Singapore...

Email alerts are the primary method that WebOrion Defacement Monitor uses to inform our customers about the changes to their websites. Through these email alerts, users are informed if their website becomes unreachable, or if any of WebOrion’s various engines are triggered during webpage monitoring. The email alerts are important...

Cloudsine, the parent company of WebOrion, is excited to announce the technology alliance partnership with New Net Technologies (NNT). NNT is a Cybersecurity and Compliance software company based in UK and is widely deployed in many Enterprise and Government Organizations globally. Cloudsine provides cloud consulting services and offers web defacement...

Cloudsine is honoured to partner with SUTD to sponsor the artificial intelligence award to nurture interests and identify talents in this area. Students from SUTD who are interested may enroll in 50.021 Artificial Intelligence module offered by the Information Systems Technology and Design (ISTD) pillar. In this course, students will...

"The PowerX Programme has given young Cybersecurity companies like us a boost to identify, recruit and train cyber talents that are critical for our growth." Matthias Chin, Founder and CEO of Cloudsine. The PowerX Cybersecurity and Software & Product Development programmes are SGInnovate’s 12-month programmes including structured training and industry...

There was a high severity vulnerability (CVE-2021-44228) impacting multiple versions of the Apache Log4j which was disclosed publicly on December 9, 2021. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled. The vulnerability impacts...

Cyber Threat Activities from the Russia-Ukraine Cyberwar The Russian incursion into Ukraine has led to a conflict that involves both the physical and cyber domains, with hacking groups of differing allegiances launching cyberattacks on government, military, financial and telecommunication websites. Cybersecurity specialists worldwide have highlighted growing concerns that the intensifying...

The Resurgent Threat of Hacktivism As the Russia-Ukraine conflict intensifies, cyberwarfare continues to be waged between the two countries. Concerns remain that state-backed hacker groups may target organisations outside of Eastern Europe in retaliation for the global sanctions imposed on Russia, or as false-flag operations to further promote political narratives....

WebOrion® is glad to introduce a new feature into the existing Integrity Analytics engine – Smart Image Hash (SIH). SIH helps reduce false alerts regarding image changes by analysing them in a smarter way. Images can make websites look more attractive and have been widely adopted ever since the inclusion...

AI technologies have been widely applied to different fields, but have you ever heard of using AI technologies to monitor the defacement of webpages? WebOrion is glad to introduce a new engine to the WebOrion defacement monitoring platform – AI Natural Language Processing (NLP) Engine. This engine analyzes webpage-changes and...

Cloudsine, the parent company of WebOrion, is pleased to announce the technology alliance partnership with Netrust Pte Ltd. Netrust is an established company since 1997 and is Asia’s first Public Certification Authority (CA) andSingapore’s only commercial IMDA-accredited CA. Cloudsine provides cloud consulting services and offers web defacement detection and response...

“Serverless” is a buzzword that is thrown around especially in the cloud industry. For the inexperienced, it may seem intuitive - “server” and “less”. It does not mean having less servers, but it actually refers to lesser (or no) management of servers. Serverless services allow developers to build and run...

Discover what is the WebOrion® API and the benefits of integrating the API with various systems such as Content Management Systems like Wordpress, and in the coming days, SIEM and SOAR systems. Our simple and easy-to-follow demonstration will also show you how to seamlessly integrate the WebOrion® API with your...

For the past decade, Cloudsine has been working with Amazon Web Services (AWS) to serve the market. Cloudsine is a consulting and technology partner with AWS and has used cloud computing to build and run many secure applications to support enterprise and government customers across Asia Pacific countries including Singapore,...

Thanks to all who visited our booth at GovWare 2022, held on 18-20 October 2022, at Sands Expo and Convention Centre, Singapore. We sincerely hope that all who visited us were able to catch a glimpse of what we currently do in the cybersecurity space with WebOrion, as well as...

As a website owner, one would surely come across the Domain Name Service (DNS). DNS is an extremely critical system on the Internet, as it is a system that helps translate domain names (which are easily recognisable and remembered) into IP addresses. It is important for all website owners to...

In this video, we will be sharing with you why your DNS records are important. How an outsider can conduct DNS enumeration to determine the attack surface. What can you do to hide and secure your DNS records. What are some tools WebOrion provide that can detect changes to the...

In computer networking, ports are points of entry to your computer – virtual origins and/or destinations of network connections. Port number definition and standardisation is overseen by the Internet Assigned Numbers Authority (IANA). Based on the list maintained by IANA, there are three types of ports amongst the total number...

In this video, we will discuss how hackers can easily enumerate your web server and potentially find vulnerabilities that they can exploit. It is important to understand how these attacks work so that you can take steps to protect your server and your website. We will walk through the process...

Today, we are excited to announce WebOrion® Defacement Monitor and Restorer is listed as a partner of LKPP E-Katalog, https://e-katalog.lkpp.go.id/, for PEP Category, Software Security, and Antivirus License. This opens up a new channel for Indonesian public sector agencies to quickly start protecting and monitoring their websites from cyber attacks...

Hari ini, dengan gembira kami mengumumkan WebOrion® Defacement Monitor and Restorer terdaftar sebagai mitra LKPP E-Katalog, https://e-katalog.lkpp.go.id/, untuk Kategori Peralatan Elektronik dan Pendukungnya, Keamanan Perangkat Lunak, dan Lisensi Antivirus. Hal ini membuka saluran baru bagi lembaga sektor public dan Pemerintah di Indonesia untuk segera mulai melindungi dan memantau situs website...

PCI-DSS is a set of security standards designed to ensure that all companies that accept, process, store or transmit credit card information maintain a secure environment. This article is part of a series of articles under the “What’s New in PCI-DSS v4.0” series where we explore what has changed in...

What is Magecart? Magecart is a type of cybercriminal group that specializes in stealing credit card information from online stores (a.k.a card skimming). The group's attacks typically involve injecting malicious code into the checkout pages of e-commerce websites to steal payment card data from customers. The Magecart group is known...

While web defacements may not be the most prevalent cyber attack in recent years, the consequences of web defacements are real – reputations may be damaged, client-customer relationships may be broken, financial losses may occur, etc. Web defacements can come in various forms, visual or non-visual (script inclusions). Hackers may...

In this video, we'll be discussing the important topic of preventing web defacement - a type of cyber attack that involves unauthorized alteration of a website's content or appearance. As a technical manager or CTO, it's crucial to understand the methods and motivations behind web defacement attacks and take steps...

“Planning without taking action is the slowest route to victory. Taking action without planning is the noise before defeat.” – Sun Tzu, The Art of War

Introduction to cloud computing

It is said that the world evolves at the speed of technological evolution. Organizations are constantly looking for new technologies such as cloud computing to meet their goals strategically and to drive business value. This article will address how cloud computing can possibly help organizations adopt efficient technologies and also improve productivity.

Cloud computing refers to computing on a network of remote servers accessible over the web, in order to store, manage, and process data. It utilizes computing resources of cloud providers, such as their data centers, instead of having the organization build their own local infrastructure.

Regardless of whichever industry one’s company belongs to (finance, retail or real estate), it is always advisable to understand the technology that other corporations are adopting. This is to solidify a competitive advantage by examining the lessons learned and best practises developed along the way.  In order to understand this better, let us illustrate how Netflix utilized cloud services to reach its level of success today.

Amazon Web Services used by Netflix

The cloud is an enabling technology for AI to mine and analyze data for deeply embedded insights. Cloud computing contributed an innovative breakthrough of accelerators for AI software. An accelerator is a class of microprocessor or system that is designed to provide hardware acceleration for AI applications such as neural networks, computer vision and machine learning. Currently, it is rather difficult for on-premise hardware to match the processing power of the accelerator hardware residing in the worldwide data centers of cloud providers (Source: Gartner, The Google Guys). Furthermore, cloud providers possess one more advantage over non-cloud AI: their extensive global network of data centers are in the better position to process the massive amount of data being generated all over the world. This alone makes it substantially easier to train machine learning models and neural networks for data insights and pattern recognition.

Analyzing customer data creates customer insights for any organization. It helps management avoid making assumptions about customers, which may be misinterpreted by the customer as apathy. Data analytics and personalized customer assistance (PCA) features are actually the largest areas of innovation on the cloud for organizations, up till 2019 (Source: InsideBigData, Google Cloud, IBM).

What sort of cloud services aid in discovering data insights and building personalized assistance features for customers? For one, Netflix uses Amazon RDS and DynamoDB, which provide the structural organization that these are cloud services that helps to build, develop and deploy custom machine learning models for each organization based on its unique goals and work environments.

While deciding whether to produce “House of Cards”, whereby 26 episodes cost $100 million in production, Netflix decided that it was more intelligent to use data analytics to determine which fan bases its new drama should target. These data were captured on their database for analysis. Using machine learning, they were targeting its marketing appeal at the fans of the British House of Cards, as well as the long-time fans of actor Kevin Spacey and director David Fincher.

With cloud-based AI services, organizations can index their entire product/service catalogue based on each customer/user’s profile. Age, location, gender, and other profile data helped to determine which products should be ranked first for each individual customer. Customers with different likings and profile data would see a personalized set of recommended products specially curated for their viewing. These personalized services tend to make users feel important and valued by the enterprise, instead of just being a source of revenue and hence retains the organisation’s customer base.

Cloud agility refers to the rapid provisioning of computer-related resources. The Cloud environment can usually provide compute instances or storage in minutes. Before cloud providers took off with IaaS, one had to email infrastructure suppliers and wait for a few weeks before the supplier replied with the requested provisions. (Source: Netflix, Amazon Case Study on Netflix). The existing IaaS delivery is executed using the consoles of cloud providers, allowing a faster release of new features for users. The benefit of such services reduce the time taken to develop, test and deploy software applications.

Most successful companies share a common trait: they had people who started developing a product/service prototype way ahead of their peers. The reason for their success is rather obvious – the first-mover advantage. Cloud computing is a technology designed to help organizations obtain the first-mover advantage, as evident from their rich variety of service offerings.

How did Netflix utilize agility features of the cloud for the cloud migration of their operations? They rebuilt their app functions inside the native cloud development environments first, later including app development for business operations. The large, cumbersome Netflix service of 2008 was refactored into microservices and unstructured scalable databases.

Netflix’s cloud database usage followed a pay-as-you-use basis, which helped them save costs whenever they rolled out the AI based feature called top personalized recommendations. (the AI has to mine data from their database, and so the database has to be hosted properly and securely on Amazon’s cloud). This AI feature, top personalized recommendations, showed users niche titles that would not be available on traditional cable networks but were similar to content liked by the user. As such, users purchased these niche titles more, generating more revenue; Netflix no longer had to spend so much money on acquiring new content to sell to users. The costs saved were estimated to be $1 billion by Netflix’s Vice President and Chief Product Officer, in a research paper published by them.

With such a progressive implementation, the management became more strategic and informed about budget evaluations and approvals. The purchase of hardware and the progressive release schedule of the re-morphing Netflix became more streamlined day by day. Gradually, a large organization like Netflix was no longer constrained by physical compute-resources and grew to become the global Internet TV network everyone knows today.

Scalability

Scalability refers to a software-based product or service which retains its intended function with no quality compromise when moved to an environment with more incoming customers. The user’s needs must be met no matter what changes and the response time should not get longer. The elaborations below highlight the relevance of cloud scalability to your organization.

By using services from cloud providers like AWS and Open Connect (for streaming), Netflix expanded its network of servers (both physical and virtual) from North America to the rest of the world, including areas like Europe and India.

Netflix is one example of an organization using the cloud. By running on AWS, it provided billions hours of service to customers around the globe. Users can order its products/services from almost anywhere in the world, using PCs, tablets, or mobile devices. 10,000 customer orders were processed every second during Netflix’s last peak demand season. This is a stark contrast from the few thousand DVD orders Netflix could handle in its early days before streaming and migrating to the cloud. Having 86 million customers worldwide who consume 150 million hours of content daily, this is rather strong evidence about how the cloud has powered Netflix’s scalability of business operations.

Cloud providers like AWS provide technologies such as container auto-scaling and application level load-balancing, to support the customer service that Netflix provides. Cloud providers possess the resources to handle the gigantic operational loads of their client organizations. The compute resources they provide are globally available, enabling customers from around the world to place orders literally anytime they prefer. Organizations that face a small home-country market no longer have to worry about global expansion.

Conclusion and Final thoughts

The most important action taken by enterprise organizations in 2018 was to engage a professional cloud vendor experienced in providing step-by-step solutions and enterprise-level cybersecurity.

Enterprise innovation is now centered upon (but not limited to) cloud-native machine learning models and data analytics. These technologies offer a pleasant side benefit, assisting organizations in managing their vast amount of customer and operations-related data. In the area of enterprise agility, cloud providers and third-party resellers have created intelligent software so as to help enterprises make important decisions faster than ever. In the area of scalability, the cloud has empowered various organizations to serve their users and customers around the world with better availability and response times. Selling to more customers beyond the home-country is now easier.

It is important for organizations to understand global industrial changes. The future of the cloud computing will continue to be several billion dollar industries, such as AI innovation, blockchain and cloud security (Source: Forbes). Hence, most organizations now find their boardroom discussions increasingly centered upon the topic of technology  in business strategy.

“Victorious warriors win first and then go to war, while defeated warriors go to war first and then seek to win”

― Sun Tzu, The Art of War

The Curie, 83 Science Park Drive,#02-01C, 118258, Singapore

TechRepublic

Account information.

Share with Your Friends

5 lessons IT learned from the Netflix cloud journey

Your email has been sent

With all the talk of containers, cloud-native applications, and cluster management platforms, it’s assumed that the enterprise migration to the cloud is a foregone conclusion. I’ve wanted to provide a little perspective. Some perspective has come from the poster child for all cloud infrastructure, Netflix.

Netflix recently published a blog post championing their seven-year journey to the public cloud. Netflix has contributed significantly to the overall knowledge of operating a predominantly cloud-based service but, it’s only after years of preparation that they were able to migrate the streaming portion of their service completely to the public cloud. Here are five top takeaways from their journey.

1. It’s not about cost savings

One of the impressive attributes of the Netflix cloud use case is the clarity around the value of cloud. Netflix didn’t communicate cost reduction as an advantage. Instead, Netflix spoke of the advantages of scale and reliability in leveraging AWS. Netflix realized cost savings due to the elastic nature of their workloads. The company wasn’t burdened by the fixed cost of scaling their private data center for peak load. However, by building elasticity into their application, they were able to reap the additional benefit of cost savings.

SEE: Cloud Data Storage Policy Template (Tech Pro Research)

2. Change management

3. availability.

It’s tough to achieve five nines (99.999%) availability in a private data center. It’s even harder to achieve that level of availability in a cloud-based application. Netflix has finally reached a four nines availability. In the past, I’ve used the rule of thumb that applications with two to three nines are targets for cloud migration. Just as Netflix was selective in the services they migrated to the public cloud, it’s critical that organizations understand the impact of service availability when migrating to the public cloud.

4. Cloud-native

One of the reasons Netflix was able to save money on public cloud vs. the private data center is their application architecture. Netflix didn’t simply lift and shift monolithic applications from their private data center to an AWS VM. Taking the forklift approach ticks off the check box for cloud without any of the real benefits. Since scale and reliability were the primary factors in Netflix’s cloud decision, it required a re-architecture. Organizations considering cloud need to evaluate their current application architecture. Lift and shift to the cloud may just shift your existing problems to a new platform, sans the insight from managing your infrastructure.

SEE: Should you follow Netflix and run your business from the public cloud?

Netflix is held up as the gold standard for leveraging cloud computing. The median annual salary at Netflix is an eye-popping $180K. There are not very many organizations that have the resources or reputation to attract the talent needed to undertake a full migration to the public cloud. It’s likely that most organizations will seek outside assistance in their migration efforts. The Netflix journey highlights the need to investigate deeply the credentials of organizations looking to assist in your cloud journey.

Your thoughts?

What are some of the insights you observed from Netflix’s cloud journey? Share you thoughts in the comments section.

Subscribe to the Cloud Insider Newsletter

This is your go-to resource for the latest news and tips on the following topics and more, XaaS, AWS, Microsoft Azure, DevOps, virtualization, the hybrid cloud, and cloud security. Delivered Mondays and Wednesdays

• Cloud security: 10 things you need to know
• Netflix wraps up Amazon cloud migration, rolls credits on last video streaming datacenter
• AWS outage: How Netflix weathered the storm by preparing for the worst
• Cloud computing goes hybrid as the norm: AWS, VMware, Azure duke it out
• Netflix on how to build services that scale beyond millions of users

Create a TechRepublic Account

Get the web's best business technology news, tutorials, reviews, trends, and analysis—in your inbox. Let's start with the basics.

* - indicates required fields

Sign in to TechRepublic

Lost your password? Request a new password

Reset Password

Please enter your email adress. You will receive an email message with instructions on how to reset your password.

Check your email for a password reset link. If you didn't receive an email don't forgot to check your spam folder, otherwise contact support .

Welcome. Tell us a little bit about you.

This will help us provide you with customized content.

Want to receive more TechRepublic news?

You're all set.

Thanks for signing up! Keep an eye out for a confirmation email from our team. To ensure any newsletters you subscribed to hit your inbox, make sure to add [email protected] to your contacts list.

Jira Software

Project and issue tracking

Content collaboration

Jira Service Management

High-velocity ITSM

Visual project management

• View all products

Marketplace

Connect thousands of apps and integrations for all your Atlassian products

Developer Experience Platform

Jira Product Discovery

Prioritization and roadmapping

You might find helpful

Cloud Product Roadmap

Atlassian Migration Program

Work Management

Manage projects and align goals across all teams to achieve deliverables

IT Service Management

Enable dev, IT ops, and business teams to deliver great service at high velocity

Agile & DevOps

Run a world-class agile software organization from discovery to delivery and operations

BY TEAM SIZE

Small Business

BY TEAM FUNCTION

Software Development

BY INDUSTRY

Telecommunications

Professional Services

What's new

Atlassian together.

Get Atlassian work management products in one convenient package for enterprise teams.

Atlassian Trust & Security

Customer Case Studies

Atlassian University

Atlassian Playbook

Product Documentation

Developer Resources

Atlassian Community

Atlassian Support

Enterprise Services

Partner Support

Purchasing & Licensing

Work Life Blog

Atlassian Presents: Unleash

Product updates, hands-on training, and technical demos – catch all that and more at our biggest agile & DevOps event.

• Atlassian.com
• Enterprise Cloud
• The space between: Netflix's cloud migration story

The space between: Netflix's cloud migration story

Netflix isn’t the typical company and its cloud migration isn’t the typical story. The journey to Confluence Cloud started routinely but slowly went sideways. Layers of data cruft from a decades-old, on-prem install; gaps in the migration tools; and migration testing that felt more like a slow spiral to nowhere led to a crisis point. In this session you’ll hear how new tools were developed on the fly to close the spaces between and how that led to a migration low on drama and high on likes.

Senior Software Engineer, Netflix

• Product Engineering And Development Simform acts as a strategic software engineering partner to build products designed to cater the unique requirements of each client. From rapid prototyping to iterative development, we help you validate your idea and make it a reality.
• Performance Engineering and Testing Our service portfolio offers a full spectrum of world-class performance engineering services. We employ a dual-shift approach to help you plan capacity proactively for increased ROI and faster delivery.
• Digital Experience Design Work with cross-functional teams of smart designers and product visionaries to create incredible UX and CX experiences. Simform pairs human-centric design thinking methodologies with industry-led tech expertise to transform user journeys and create incredible digital experience designs.
• Application Management and Modernization Simform’s application modernization experts enable IT leaders to create a custom roadmap and help migrate to modern infrastructure using cloud technologies to generate better ROI and reduce cloud expenditure.
• Project Strategy At Simform, we don’t just build digital products, but we also define project strategies to improve your organization’s operations. We use Agile software development with DevOps acceleration, to improve the software delivery process and encourage reliable releases that bring exceptional end-user experience.
• Cloud Native App Development Build, test, deploy, and scale on the cloud
• Cloud Consulting Audit cloud infrastructure, optimize cost and maximize cloud ROI
• Microservice Architecture Remodel your app into independent and scalable microservices
• Kubernetes Consulting Container orchestration made simple
• Cloud Migration Consulting Assess, discover, design, migrate and optimize the cloud workloads
• Cloud Assessment Assess cloud spending, performance, and bottlenecks
• Serverless Seize the power of auto-scaling and reduced operational costs
• Cloud Architecture Design Optimize your architecture to scale effectively
• DevOps Consulting DevOps implementation strategies to accelerate software delivery
• Infrastructure Management and Monitoring Competently setup, configure, monitor and optimize the cloud infrastructure
• Containerization and Orchestration Reliably manage the lifecycle of containers in large and dynamic environments
• Infrastructure as a Code Manage and provision IT infrastructure though code
• CI/CD Implementation Automate and efficiently manage complex software development
• BI and Data Engineering Our Data and BI experts help you bridge the gap between your data sources and business goals to analyze and examine data, gather meaningful insights, and make actionable business decisions.
• Test Automation Reduce manual testing and focus on improving the turnaround time
• Microservice Testing Make your microservices more reliable with robust testing
• API Testing Build safer application and system integrations
• Performance Testing Identify performance bottlenecks and build a stable product
• Load Testing Achieve consistent performance under extreme load conditions
• Security Testing Uncover vulnerabilities and mitigate malicious threats
• Technology Partnerships Reap benefits of our partnerships with top infrastructure platforms
• Process Management Right processes to deliver competitive digital products
• SaaS Development Services Build competitive SaaS apps with best experts & tools
• Cloud Migration Scale your infrastructure with AWS cloud migration
• Cloud Solutions for SMB Make your business smarter with AWS SMB
• Data Engineering Collect, process, and analyze data with AWS data engineering
• Serverless and Orchestration Manage complex workflows and ensure optimal resource utilization
• Cloud Management Improve AWS efficiency, automation, and visibility for better cloud operations
• AWS DevOps Consulting Accelerate the development of scalable cloud-native applications
• Advertising and Marketing Technology Transform customer engagement with AWS Advertising expertise
• AWS Retail Services Improve customer engagement and address retail challenges efficiently.
• AWS Healthcare Services Improve patient care and streamline operations with AWS
• AWS Supply Chain Services Achieve supply chain efficiency and real-time visibility
• AWS Finance Services Accelerate financial innovation with AWS expertise
• Technology Comparisons
• How it works

How Netflix Became A Master of DevOps? An Exclusive Case Study

Find out how Netflix excelled at DevOps without even thinking about it and became a gold standard in the DevOps world.

Table of Contents

• Netflix's move to the cloud

Netflix’s Chaos Monkey and the Simian Army

Netflix’s container journey, netflix’s “operate what you build” culture, lessons we can learn from netflix’s devops strategy, how simform can help.

Even though Netflix is an entertainment company, it has left many top tech companies behind in terms of tech innovation. With its single video-streaming application, Netflix has significantly influenced the technology world with its world-class engineering efforts, culture, and product development over the years.

One such practice that Netflix is a fantastic example of is DevOps. Their DevOps culture has enabled them to innovate faster, leading to many business benefits. It also helped them achieve near-perfect uptime, push new features faster to the users, and increase their subscribers and streaming hours.

With nearly 214 million subscribers worldwide and streaming in over 190 countries , Netflix is globally the most used streaming service today. And much of this success is owed to its ability to adopt newer technologies and its DevOps culture that allows them to innovate quickly to meet consumer demands and enhance user experiences. But Netflix doesn’t think DevOps.

So how did they become the poster child of DevOps? In this case study, you’ll learn about how Netflix organically developed a DevOps culture with out-of-the-box ideas and how it benefited them.

Simform is a leading DevOps consulting and implementation company , helping businesses build innovative products that meet dynamic user demands efficiently. To grow your business with DevOps, contact us today!

Netflix’s move to the cloud

It all began with the worst outage in Netflix’s history when they faced a major database corruption in 2008 and couldn’t ship DVDs to their members for three days. At the time, Netflix had roughly 8.4 million customers and one-third of them were affected by the outage. It prompted Netflix to move to the cloud and give their infrastructure a complete makeover. Netflix chose AWS as its cloud partner and took nearly seven years to complete its cloud migration.

Netflix didn’t just forklift the systems and dump them into AWS. Instead, it chose to rewrite the entire application in the cloud to become truly cloud-native, which fundamentally changed the way the company operated. In the words of Yury Izrailevsky, Vice President, Cloud and Platform Engineering at Netflix:

“We realized that we had to move away from vertically scaled single points of failure, like relational databases in our datacenter, towards highly reliable, horizontally scalable, distributed systems in the cloud.”

As a significant part of their transformation, Netflix converted its monolithic, data center-based Java application into cloud-based Java microservices architecture. It brought about the following changes:

• Denormalized data model using NoSQL databases
• Enabled teams at Netflix to be loosely coupled
• Allowed teams to build and push changes at the speed that they were comfortable with
• Centralized release coordination
• Multi-week hardware provisioning cycles led to continuous delivery
• Engineering teams made independent decisions using self-service tools

As a result, it helped Netflix accelerate innovation and stumble upon the DevOps culture. Netflix also gained eight times as many subscribers as it had in 2008. And Netflix’s monthly streaming hours also grew a thousand times from Dec 2007 to Dec 2015.

After completing their cloud migration to AWS by 2016, Netflix had:

And it handled all of the above with 0 Network Ops Centers and some 70 operations engineers, who were all software engineers focusing on writing tools that enabled other software developers to focus on things they were good at.

Migrating to the cloud made Netflix resilient to the kind of outages it faced in 2008. But they wanted to be prepared for any unseen errors that could cause them equivalent or worse damage in the future.

Engineers at Netflix perceived that the best way to avoid failure was to fail constantly. And so they set out to make their cloud infrastructure more safe, secure, and available the DevOps way – by automating failure and continuous testing.

Chaos Monkey

Netflix created Chaos Monkey, a tool to constantly test its ability to survive unexpected outages without impacting the consumers. Chaos Monkey is a script that runs continuously in all Netflix environments, randomly killing production instances and services in the architecture. It helped developers:

• Identify weaknesses in the system
• Build automatic recovery mechanisms to deal with the weaknesses
• Test their code in unexpected failure conditions
• Build fault-tolerant systems on day to day basis

The Simian Army

After their success with Chaos Monkey, Netflix engineers wanted to test their resilience to all sorts of inevitable failures, detect abnormal conditions. So, they built the Simian Army , a virtual army of tools discussed below.

• Latency Monkey

It creates false delays in the RESTful client-server communication layers, simulating service degradation and checking if the upstream services respond correctly. Moreover, creating very large delays can simulate an entire service downtime without physically bringing it down and testing the ability to survive. The tool was particularly useful to test new services by simulating the failure of dependencies without affecting the rest of the system.

• Conformity Monkey

It looks for instances that do not adhere to the best practices and shuts them down, giving the service owner a chance to re-launch them properly.

• Doctor Monkey

It detects unhealthy instances by tapping into health checks running on each instance and also monitors other external health signs (such as CPU load). The unhealthy instances are removed from service and terminated after service owners identify the root cause of the problem.

• Janitor Monkey

It ensures the cloud environment runs without clutter and waste. It also searches for unused resources and discards them.

• Security Monkey

An extension of Conformity Monkey, it identifies security violations or vulnerabilities (e.g., improperly configured AWS security groups) and eliminates the offending instances. It also ensures the SSL (Secure Sockets Layer) and DRM (Digital Rights Management) certificates were valid and not due for renewal.

• 10-18 Monkey

Short for Localization-Internationalization, it identifies configuration and runtime issues in instances serving users in multiple geographic locations with different languages and character sets.

• Chaos Gorilla

Like Chaos Monkey, the Gorilla simulates an outage of a whole Amazon availability zone to verify if the services automatically re-balance to the functional availability zones without manual intervention or any visible impact on users.

Today, Netflix still uses Chaos Engineering and has a dedicated team for chaos experiments called the Resilience Engineering team (earlier called the Chaos team).

In a way, Simian Army incorporated DevOps principles of automation, quality assurance, and business needs prioritization. As a result, it helped Netflix develop the ability to deal with unexpected failures and minimize their impact on users. 

On 21st April 2011 , AWS experienced a large outage in the US East region, but Netflix’s streaming ran without any interruption. And on 24th December 2012 , AWS faced problems in Elastic Load Balancer(ELB) services, but Netflix didn’t experience an immediate blackout. Netflix’s website was up throughout the outage, supporting most of their services and streaming, although with higher latency on some devices.

Netflix had a cloud-native, microservices-driven VM architecture that was amazingly resilient, CI/CD enabled, and elastically scalable. It was more reliable, with no SPoFs (single points of failure) and small manageable software components. So why did they adopt container technology? The major factors that prompted Netflix’s investment in containers are:

• Container images used in local development are very similar to those run in production. This end-to-end packaging allows developers to build and test applications easily in production-like environments, reducing development overhead.
• Container images help build application-specific images easily.
• Containers are lightweight, allowing building and deploying them faster than VM infrastructure.
• Containers only have what a single application needs, are smaller and densely packed, which reduces overall infrastructure cost and footprint.
• Containers improve developer productivity, allowing them to develop, deploy, and innovate faster.

Moreover, Netflix teams had already started using containers and seen tangible benefits. But they faced some challenges such as migrating to containers without refactoring, ensuring seamless connectivity between VMs and containers, and more. As a result, Netflix designed a container management platform called Titus to meet its unique requirements.

Titus provided a scalable and reliable container execution solution to Netflix and seamlessly integrated with AWS. In addition, it enabled easy deployment of containerized batches and service applications.

Titus served as a standard deployment unit and a generic batch job scheduling system. It helped Netflix expand support to growing batch use cases. 

• Batch users could also put together sophisticated infrastructure quickly and pack larger instances across many workloads efficiently. Batch users could immediately schedule locally developed code for scaled execution on Titus.
• Beyond batch, service users benefited from Titus with simpler resource management and local test environments consistent with production deployment.
• Developers could also push new versions of applications faster than before.

Overall, Titus deployments were done in one or two minutes which took tens of minutes earlier. As a result, both batch and service users could experiment locally, test quickly and deploy with greater confidence than before.

“The theme that underlies all these improvements is developer innovation velocity.” 

-Netflix tech blog

This velocity enabled Netflix to deliver fast features to the customers, making containers extremely important for their business.

Netflix invests and experiments significantly in improving development and operations for the engineering teams. But before Netflix adopted the “Operate what you build” model, it had siloed teams. The Ops teams focused on deploy, operate and support parts of the software life cycle. And Developers handed off the code to the ops team for deployment and operation. So each stage in the SDLC was owned by a different person and looked like this:

The specialized roles created efficiencies within each segment but created inefficiencies across the entire SDLC. The issues that they faced were:

• Individual silos that slowed down end-to-end progress
• Added communication overhead, bottlenecks and hampered effectiveness of feedback loops
• Knowledge transfers between developers and ops/SREs were lossy
• Higher time-to-detect and time-to-resolve for deployment problems
• Longer gaps between code complete and deployment, with releases taking weeks

Operate what you build

To deal with the above challenges and drawing inspiration from DevOps principles, Netflix encouraged shared ownership of the full SDLC and broke down silos. The teams developing a system were responsible for operating and supporting it. Each team owned its own deployment issues, performance bugs, alerting gaps, capacity planning, partner support, and so on.

Moreover, they also introduced centralized tooling to simplify and automate dealing with common development problems of the teams. When additional tooling needs arise, the central team assesses if the needs are common across multiple development teams and built tools. In case of too team-specific problems, the development team decides if their need is important enough to solve on their own.

Full Cycle Developers

Combining the above ideas, Netflix built an even better model where dev teams are equipped with amazing productivity tools and are responsible for the entire SDLC, as shown below.

Netflix provided ongoing training and support in different forms (e.g., dev boot camps) to help new developers build up these skills. Easy-to-use tools for deployment pipelines also helped the developers, e.g., Spinnaker. It is a Continuous Delivery platform for releasing software changes with high velocity and confidence.

However, such models require a significant shift in the mindsets of teams/developers. To apply this model outside Netflix, you can start with evaluating what you need, count costs, and be mindful of bringing in the least amount of complexities necessary. And then attempt a mindset shift.

Netflix practices are unique to their work environment and needs and might not suit all organizations. But here are a few lessons to learn from their DevOps strategy and apply:

• Don’t build systems that say no to your developers

Netflix has no push schedules, push windows, or crucibles that developers must go through to push their code into production. Instead, every engineer at Netflix has full access to the production environment. And there are neither strict policies nor procedures that prevent them from accessing the production environment.

• Focus on giving freedom and responsibility to the engineers

Netflix aims to hire intelligent people and provide them with the freedom to solve problems in their own way that they see as best. So it doesn’t have to create artificial constraints and guardrails to predict what their developers need to do. But instead, hire people who can develop a balance of freedom and responsibility.

• Don’t think about uptime at all costs

Netflix servers their millions of users with a near-perfect uptime. But it didn’t think about uptime when they started chaos testing their environment to deal with unexpected failure.

• Prize the velocity of innovation

Netflix wants its engineers to do fun, exciting things and develop new features to delight its customers with reduced time-to-market.

• Eliminate a lot of processes and procedures

They limit an organization from moving fast. So instead, Netflix focuses on hiring people they can trust and have independent decision-making capabilities.

• Practice context over control

Netflix doesn’t control and contain too much. What they do focus on is context. Managers at Netflix ensure that their teams have a quality and constant flow of context of the business, rather than controlling them.

• Don’t do a lot of required standards, but focus on enablement

Teams at Netflix can work with their choice of programming languages, libraries, frameworks, or IDEs as they see best. In addition, they don’t have to go through any research or approval processes to rewrite a portion of the system.

• Don’t do silos, walls, and fences

Netflix teams know where they fit in the ecosystem, their workings with other teams, dependents, and dependencies. There are no operational fences over which developers can throw the code for production.

• Adopt “you build it, you run it” culture

Netflix focuses on making ownership easy. So it has the “operate what you build” culture but with the enablement idea that we learned about earlier.

• Focus on data

Netflix is a data-driven, decision-driven company. It doesn’t do guesses or fall victim to gut instincts and traditional thinking. It invests in algorithms and systems that combs enormous amounts of data quickly and notify when there’s an issue.

• Always put customer satisfaction first

The end goal of DevOps is to make customer-driven and focus on enhancing the user experience with every release.

• Don’t do DevOps, but focus on the culture

At Netflix, DevOps emerged as the wonderful result of their healthy culture, thinking and practices.

Get in Touch

Netflix has been a gold standard in the DevOps world for years, but copy-pasting their culture might not work for every organization. DevOps is a mindset that requires molding your processes and organizational structure to continuously improve the software quality and increase your business value. DevOps can be approached through many practices such as automation, continuous integration, delivery, deployment, continuous testing, monitoring, and more.

At Simform, our engineering teams will help you streamline the delivery and deployment pipelines with the right DevOps toolchain and skills. Our DevOps managed services will help accelerate the product life cycle, innovate faster and achieve maximum business efficiency by delivering high-quality software with reduced time-to-market.

Hiren Dhaduk

Hiren is VP of Technology at Simform with an extensive experience in helping enterprises and startups streamline their business performance through data-driven innovation.

Cancel reply

Your email address will not be published. Required fields are marked *

Your comment here*

Sign up for the free Newsletter

For exclusive strategies not found on the blog

Sign up today!

Related Posts

Kubernetes Architecture and Components with Diagram

11 Powerful Docker Alternatives to Revolutionize Containerization in 2024

DevOps, CI/CD and Containerization: 44 Images Explaining a Winning Trio

Get insights on Gen AI adoption and implementation in 2024. Download the survey report now!

Chris Stokel-Walker

Case studies in cloud migration: netflix, pinterest, and symantec.

In October 2008, Neil Hunt, chief product officer at Netflix, gathered a meeting of a dozen or so of his engineering staffers in The Towering Inferno, the secluded top-floor meeting room at Netflix’s Los Gatos, CA headquarters. The room, which Netflix CEO Reed Hastings occasionally commandeers as his personal office, is away from the main office hustle and bustle of the start-up company, up a flight of stairs and across an outdoor wooden walkway up on the building’s rooftop—the ideal place for big-picture thinking.

Big thoughts were needed, because Netflix had a problem: its backend client architecture was, to put none too fine a term on it, crumbling more than the Colosseum and leaning more than the Tower of Pisa.

“We kept having issues with connections and threads,” Hunt  recalled  at an industry conference in Las Vegas, NV, six years later. “At one point we upgraded the machine to a fantastic $5 million box and it crashed immediately because the extra capacity on the thread pools meant we ran out of connection pools more quickly.”

It was an unenviable position to be in for the firm, which had introduced online streaming of its vast video library the year before. Netflix had just partnered with Microsoft to get its app on the Xbox 360, and had agreed to terms with the manufacturers of Blu-ray players and TV set-top boxes to service their customers. Millions of potential users of a new, game-changing technology were about to encounter what we now know as the multi-billion dollar industry of online video streaming that would transform Netflix from a failing company that mailed DVDs to movie buffs into a television and movie studio that rivaling some of Hollywood’s biggest names.

But, back in 2008, with a backend that couldn’t cope, the public wasn’t about to encounter anything—unless Netflix made some changes.

There were two points of failure in the physical technology, Hunt explained to the conference audience in Las Vegas: The disk array that ran Netflix’s database—a single Oracle database on an array of Blade webservers—and the single box that talked to it.

“We knew we were approaching a point where we needed to make this redundant,” said Hunt. But Netflix hadn’t yet forked out the cash for second data center that would alleviate the problem. “We were vulnerable to those single points of failure.”

“Let’s rethink this completely, go back to first principles, and think about doing it in the cloud.”

That much became abundantly clear in 2008 when the company pushed a piece of firmware to the disk array. It corrupted Netflix’s database, and the company had to spend three days scrambling to recover. (One contemporary news story on the outage—and the customer outrage it sparked—noted that some customers even went back to Blockbuster, which Netflix had made seem decrepit, for their DVDs.) “That wasn’t a total catastrophe because most customers weren’t reliant on the system being up to get value from the service,” explained Hunt—but as Netflix’s DVD mailing arm wound down and its new streaming service caught on, it would become a problem.

“We thought: ‘Let’s rethink this completely, go back to first principles, and think about doing it in the cloud’,” said Hunt.

Over the course of several meetings in The Towering Inferno, Hunt and his team thrashed out a plan that would ensure that database corruption—and the many other issues with connections and threads that seemed to plague the company back in 2008—would never happen again. They’d move to the cloud.

Whether companies are looking to run their applications serving millions of users or to underpin the databases and file servers of multinational businesses, the cloud provides a low-cost, flexible way to ensure reliable IT resources. Firms don’t need to worry about the physical upkeep of their own private data centers storing information; they can build out capacity as and when it’s needed, lowering costs and increasing their adaptability—important features for a young startup with unpredictable (and potentially limitless) growth. It has been a recent boon, born out by technological innovation, that helps power hundreds of thousands of companies, big and small, across the globe.

For Netflix, the move to the cloud proved a prescient decision: between December 2007 and December 2015, the number of hours of content streamed on Netflix increased one thousand times, and the company had eight times as many people signed up to the service at the end of its cloud migration process as it did at the start. Cloud infrastructure was able to stretch to meet this expanding demand while traditional server racks in a data center were not able to (the number of requests per month called through Netflix’s API  outstripped  the capacity of its traditional data center near the end of 2010). It also proved to be a major cost-saving move.

But at the same time, the cloud was still an unproven, young technology. Amazon, the current leader in cloud computing, had only been offering their Amazon Web Services (AWS) infrastructure products since 2006. Caution was required. Netflix started small, moving over a single page onto AWS to make sure the new system worked. “It’s nicely symbolic,” said Hunt. “We recognised that along the way we probably need to hire some new skills, bring in some new talent, and rethink our organisation.” The company chose AWS over alternative public cloud suppliers because of its breadth of features and its scale, as well as the broader variants of APIs that AWS offered.

“When Netflix made the decision to go all-in on the cloud, most people were barely aware the cloud existed.”

Today, the cloud is many companies’ first choice when it comes to storing data and serving their customers. AWS is a $12 billion company, four times bigger than it was in 2013. It has—and has long had—a 40% market share in the public cloud sector, much more than the combined market share of Microsoft, Google and IBM’s cloud offerings combined,  according to data  collated by Synergy Research Group. Those that aren’t utilising the cloud often feel they want to, and are frustrated when they can’t: Four in 10 businesses have critical company data trapped in legacy systems that can’t be accessed or linked to cloud services, according to a survey by market research company Vanson Bourne for commercial software company Snaplogic, while three in four say that their organization misses out on opportunities because of disconnected data. Vendors’ revenue from the sales of infrastructure products—including server, storage and Ethernet switches—for cloud IT topped $8 billion in the first quarter of 2017, according to analysts IDC.

But none of that was the case when Netflix started its great migration, nor was it true when Ruslan Meshenberg started at Netflix in January 2011, two years into Netflix’s big move. As one of the first companies to move its services into the cloud, Netflix was literally writing the rulebook for many of the tasks it was undertaking. Meshenberg was thrown in at the deep end.

“That was the very first set of objectives I was given,” he explains. “A complete data-center-to-cloud migration for a core set of platform services. Day one.”

It involved a lot of outside the box thinking—and plenty of trailblazing. “When Netflix made the decision to go all-in on the cloud, most people were barely aware the cloud existed,” he explains. “We had to find solutions to a lot of problems, at a time when there were not a lot of standard, off-the-shelf solutions.”

And the problems, when tackling such an enormous task as the migration of a company the size of Netflix, were numerous—particularly for a team used to the mindset that their system operated in a physical data center.

“When you’re operating in a data center,” says Meshenberg, “you know all of your servers. Your applications are running only on a particular set of hardware units.” The goal for the company in a physical data center is a simple one: keep the hardware running at all times, at all costs. That’s not the case with the cloud. Your software runs on ephemeral instances that aren’t guaranteed to be up for any particular duration, or at any particular time. “You can either lament that ephemerality and try to counteract it, or you can try and embrace it and say: ‘I’m going to build a reliable, available system on top of something that is not.’”

Which is where Netflix’s famed  Simian Army  comes in. You have to build a system that can fail—in part—while keeping up as a whole. But in order to figure out if your systems have that ability baked into their design, you need to test it.

Netflix built a tool that would self-sabotage its system, and christened it Chaos Monkey. It would be unleashed on the cloud system, wreaking havoc, bringing down aspects of the system as it rampaged around. The notion might seem self-defeating, but it had a purpose. “We decided to simulate the conditions of a crash to make sure that our engineers can architect, write and test software that’s resilient in light of these failures,” explains Meshenberg.

In its early days, Chaos Monkey’s tantrums in the cloud were a dispiriting experience. “It was painful,” Meshenberg admits. “We didn’t have the best practices, and so many of our systems failed in production. But now, since our engineers have this built-in expectation that our systems will have to be tested by Chaos Monkey, in production they’re now writing their software using the best practices that can withstand such destructive testing.”

Even without Chaos Monkey, there were still early setbacks, including a significant outage across North America on Christmas Eve 2012 thanks to an AWS update to elastic load balancers that tipped Netflix offline—a chastening event. But the company adapted, and came through it. By 2015 all of Netflix’s systems—bar its customer and employee data management databases, and billing and payment components—had been migrated to AWS. It would take a little longer before Meshenberg’s team could celebrate a job complete, but the relatively bump-free path (and the easy scaling up of systems as Netflix’s customer base skyrocketed) vindicated the move.

“The crux of our decision to go into the cloud,” says Meshenberg, was a simple one: “It wasn’t core to our business to build and operate data centers. It’s not something our users get value from. Our users get value from enjoying their entertainment. We decided to focus on that and push the underlying infrastructure to a cloud provider like AWS.”

For Netflix, dipping their toe into the water of cloud computing wasn’t an option. They had to dive in headfirst.

That said, making the leap was a brave move—not least given that, particularly when Netflix began its migration in 2009 and even when Meshenberg joined the company in 2011, cloud storage was still a relatively unknown technology in the Valley, and an unknown term to the general public. (The Institute of Electrical and Electronics Engineers (IEEE)  held  just its fourth ever international conference on cloud computing in Washington DC in 2011; technology analysts Gartner were still able, back in 2011, to  publish  a $2,000 “Hype Cycle” report explaining a technology that was on the rise.) Though those in the know understood the benefits of migrating to the cloud, and had a hunch that the general consensus would follow them, early adopters were still just that—pioneers pushing out the boundaries for the technology.

Going all-in on the cloud required betting on the future—and hoping that others would follow. But for Netflix, dipping their toe into the water of cloud computing wasn’t an option. They had to dive in headfirst.

“We had little doubt that cloud was the future,” explains Meshenberg. “If it was, it didn’t make sense to hedge our bets and straddle both worlds, because that would mean we would lose the focus of getting something done completely to the end.”

There was another factor in the decision for Netflix, too: scalability. “Our business was growing a lot faster than we would be able to build the capacity ourselves,” Meshenberg recalls. “Every time you grow your business your traffic grows by an order of magnitude, you have to rewrite the rules. The thing that worked for you at a smaller scale may no longer work at the bigger scale. We made a bet that the cloud would be a sufficient means in terms of capacity and capability to support our business, and the rest was figuring out the technical details of how.”

For Raj Patel, considering anything but the cloud was never really an option. Head of Cloud Engineering at Pinterest from 2014–2016, Patel joined a company that still had to engineer another move: from Amazon Web Service’s legacy cloud to a next-generation cloud system. “It wasn’t any different, frankly, than moving from a data center to the public cloud,” explains Patel. “We did a migration inside of Amazon.”

The move was one that some at the startup were wary of, even despite its benefits. “A cloud migration, in many cases, doesn’t necessarily get them anything,” says Patel. “The appeal has to be why they should do this before the five other things they were thinking about doing for their own group.”

At a small, nimble startup like Pinterest, time and resources are scarce, and an engineering team’s to-do list is as long as the sum of their collective arms. Getting people on-side with the cloud migration required deftness, discussion—and categorically not a top-down edict. It also required going person-to-person, winning small victories in support of the larger battle.

“You have to intuitively appeal or influence the motivations of an individual engineer to achieve your goal,” says Patel. “What I found was that at the earlier stages of the program I explicitly looked for folks that are early adopters or have a vested interest in doing that program or project, and you really focus on making them really successful. Then if the others see it they’ll get on board.”

Certain groups at Pinterest had pent-up frustrations with the older generation of Amazon’s cloud service, particularly when it same to the elasticity of potential future expansion. Data engineering-intensive applications ran up against walls with the old cloud server. Patel saw an in.

“We focused on those who would benefit the most,” he says, selling them on the idea of migrating over to a new cloud server, better equipped to deal with the developments they wanted to introduce. Patel’s team provided those early adopters with the tools to help them smoothly migrate over to the new cloud. That included embedding a consultant or solution engineer (rebranded “site reliability engineers” so as not to ruffle any feathers within the groups they joined) with each application team, who was able to provide the relevant tools and know-how to help ease the transition over to AWS. What the site reliability engineers from Patel’s team didn’t do, though, was impose any ideas or tools on the teams they joined.

"We focused on those who would benefit the most,” selling them on the idea of migrating over to a new cloud server, better equipped to deal with the developments they wanted to introduce.

“Any time you do a cloud migration—especially with engineers—there’s always this notion of: ‘Here’s my way of doing it, here’s your way of doing it: What’s the right way of doing it?’,” explains Patel. “If you had an outside group tell you this is the only way you’re going to do it, you’re going to run into a lot of friction.”

Rather, the teams worked collaboratively, engendering a sense of common purpose. Pinterest was, in truth, always going to make the move, and the company could have become forceful with its ideas, but Patel wanted a more consensual approach. “Their success is embedded with that application team,” says Patel. “Even though they might be talking about a central tool or approach, they’re perceived from the perspective of that application team.”

Like a pyramid scheme, the early adopters found success, and became proselytisers for the move. “When they talk to others at lunch, they say the migration is going really well; the guys doing it are really helpful, and it’s going just fantastic,” says Patel. “The next time you talk to the sceptics, they say: ‘Let’s go and do it.’”

At the same time, those systems that had successfully made the cloud-to-cloud migration were crowed about internally. Data democracy was crucial, says Patel, in getting across the message that the migration was something to be welcomed, not shunned. “We had important metrics about the progress we were making and would send it out to the whole engineering team to let them see it,” he explains. “People like data—engineers especially. They resonate with that progress.”

Six months later, Pinterest had transferred its backend to the more modern cloud system. The team held a party to celebrate the successful move, but truthfully, it was just another success for a company that has plenty of them.

“Think about it,” says Patel. “This was a company that was doubling or tripling in size every year. When I joined the company it was making $0 in revenue and the first year it was $100 million or something, then the next year something like three times that amount. That was the norm across the entire company. In some ways, it was just business as usual.”

When Patel moved to Symantec in April 2016 to become vice president for cloud platform engineering, things were far from business as usual.

“The magnitude of challenges are, I’d say, 5× with Symantec,” he explains. “That’s one of the things I’ve come to realize: While it’s interesting to talk about companies like Pinterest, Facebook or Instagram, their problem is already solved. They have some of the brightest engineers in the world, their applications are already designed for these cloud-type elastic architectures. In some ways, the challenge is not that interesting. But when you’re dealing with a 30-plus year-old company like Symantec, the challenge is a lot more interesting.”

For decades, Symantec had provided stability and assurance to customers—important, given its role as a security service. Unlike Pinterest, which was born in the cloud seven years ago, Symantec was founded in 1982, when computers were massive, hulking bits of hardware, hardwired to the wall. The company had been in business before the world wide web appeared as long as Pinterest has been in business, period. A publicly listed company—accountable to shareholders, with $3.6 billion of turnover—comes with more levels of hierarchy than a nimble, community-focused startup born in the Valley.

“There are more business units with general managers, instead of application teams,” explains Patel. “All those barriers are a lot more rigid in a larger enterprise than they are in the more nimble, engineering organisation approach you find in a startup.” There are also people who have been working in the company longer than some of Pinterest’s brightest young engineers—individuals who have decades of experience, and rightly should be listened to when they pass comment on the merits of such a move into the cloud. “Frankly, there were a lot of sceptics, and real architectural challenges in applications that simply have not been designed for the cloud,” says Patel. His work would end up closing down 27 separate data centers around the world and moving everything into the public cloud. The scale seemed almost insurmountable.

Even the business case for convincing staff at Symantec was more difficult; it simply wasn’t as easy an argument to make, because if it ain’t broke, why fix it?

“Your influencing job is probably 5× harder,” says Patel. “Because of the cultural transformation, you have to be a lot more convincing. You’re telling people to work differently which is very difficult, and sometimes the organization has the appetite to do those things, and sometimes they don’t.”

Much like Ruslan Meshenberg felt the need to win over his staff members, and just as Patel had to leverage the enthusiasm of early adopters at Pinterest to convince those who were less keen on taking the leap into the cloud, at Symantec Patel had to undergo a similar “hearts and minds” campaign.

Guided by Patel’s boss, the executive vice president of the sector, his team decided to show, not tell, fellow Symantec staffers about the benefits of cloud migration. “We took all the major classes of application and did a proof of concept for each one of them,” he explains. Patel’s team broke down the challenge, piece by piece, drawing up a technical feasibility study for each application, working with each group’s architect, building a proof of concept that could convince them such a move would work— “as opposed to saying: ‘We’re just going to run off this cliff and it’s going to work.’”

The attitude was a simple one: “Let’s remove the risk, and show that.”

It worked. Conviction built around the move; the only thing left to discuss was how exactly to handle the migration.

Big legacy companies planning a move to the cloud are faced with one of two options: They can go down the lift-and-shift path, or the fix-and-shift route.

The lift-and-shift route is the (comparatively) easy option. You take your pre-existing application as it presently works in a private data center, and make the minimum possible changes before moving it into the cloud. “I understand there’s going to be benefits to moving to the cloud, and I’m probably not going to realise most of them, but we’ll fix it later,” says Patel of the lift-and-shift approach.

Fix-and-shift is harder, but potentially more beneficial. You’re not just going to do the bare minimum work to ensure your application—which worked fine in an offline data center—will work in the cloud. You’re buying into the concept of moving to the cloud, fixing your culture along the way, and making it more adaptable to the new norm.

“A lot of the time what you’ll find is that traditional IT organisations tend to do lift-and-shift,” says Patel. “They’re taking the same thing they had in their private data centres and, whether it’s a corporate mandate or whatever, they say: ‘Let’s just go and move it to the cloud.’ They’re looking for roughly the same technical or organisational approaches to operating in the cloud before the cloud,” he adds. “And in my view, that’s why a lot of those efforts fail.”

It was the same choice that Neil Hunt and his team had considered back in The Towering Inferno conference room. “We could take the existing app, forklift it, and shove it into AWS, then start to chip away at it,” he explained. “That was unappealing. It would be easy to do but we’d bring along a lot of bad architecture and a lot of bad habits.”

Netflix’s second choice was equally unappealing at first glance, simply because of the scale of the task. “We would run our existing infrastructure, and side by side run our AWS infrastructure, and migrate one piece at a time, from one system to another.” As the cloud migration occurred, Netflix totally transformed. Its application also changed from a hulking, single monolithic application to a clutch of small microservices, each of which can be developed independent of the others. It recast the way the company thought about everything, completely changing the shape and makeup of the firm.

Years after Netflix’s brave decision to undergo the wholesale application and infrastructure refactor, Symantec came to the same decision: They’re fixing, then they’re shifting. Patel still has a way to go before he can breathe easily: The process has taken—and will take—time, but he’s hopeful about reaching the finish line that lingers temptingly on the horizon.

“I’ll personally feel a lot more excitement when we’re done here at Symantec, just because we’ll have done so much more organisationally,” he explains.

Patel already knows the jubilation that’s felt when you move an entire company into the cloud, and can’t wait to feel that again. For Ruslan Meshenberg, who had helped guide Netflix into the cloud without any major hitches, there was only one way to celebrate the achievement. It’s what Silicon Valley does best: Hold an amazing party.

“We had some fun, and we shared some battle stories,” says Meshenberg. The team shared a sense of achievement—personally and as a group. “Cloud migration involves every single person in a company, whether they’re engineering or not,” he adds.

Meshenberg, who had only known cloud migration in his time with the company, could move on from the project he was handed on the first day of his job, to task number two. It must’ve seemed easy-going in comparison, you’d think. “Relatively speaking,” he agrees — “but probably not less challenging. The only constant is change itself. Nothing stands still. We have to constantly re-evaluate our assumptions and ensure that our ecosystem evolves as well.”

“Cloud migration involves every single person in a company, whether they’re engineering or not.”

But Meshenberg still holds with him that sense of pride that his team and colleagues pulled off a major cloud migration without much of a hitch—and that they confounded the critics along the way, remaining ahead of the technical curve.

“When we went into the cloud we faced a lot of external scepticism, people saying this will never work, or that it may work but not for us,” he says. “It might not be secure enough, scalable enough—you name it.”

There’s a brief pause, a moment as Meshenberg collects his thoughts. Eventually, he comes out with 10 short words: “It was good to be able to get it done.”

About the author

Chris Stokel-Walker is a UK-based features journalist for The Economist , Bloomberg , the BBC, and Wired UK . His first book, YouTubers , was published in 2019, and his second, TikTok Boom, was published in July 2021.

Ray Oranges / Machas

ray-oranges.com

Buy the print edition

Visit the Increment Store to purchase print issues.

Continue Reading

Reliability, case study: how akamai weathered a surge in capacity growth, containers for the future, programming languages, julia: the goldilocks language, documentation, let’s talk about docs, the mystery of steganography, internationalization, it’s probably never going to work in german, open source, voting for transparency, what broke the bank, the team that powers vlc, explore topics.

• Learn Something New
• Scaling & Growth
• Ask an Expert
• Interviews & Surveys
• Guides & Best Practices
• Essays & Opinion
• Workplace & Culture

Software Architecture

Energy & environment, development.

How Netflix Moved Operations to the Cloud and Saw Revenue Boom: A Digital Transformation Case Study

Remember the time you had to request mail-order DVDs to catch the latest flicks while munching popcorn on your couch?

Me neither.

It’s strange to think that about a decade ago, streaming giant Netflix had a business model built around direct mail.

Request a movie, put a few in your queue for next time, and let the anticipation build as you wait for your first DVD to arrive on your doorstep.

Now, our instant gratification bells ring daily as we pour through episode after episode of new material. And, we can barely remember the (dark) time where we waited days for entertainment instead of having it literally at our fingertips.

The shift from mail-in orders to a cloud streaming service improved customer satisfaction and made Netflix billions.

The company’s move to the cloud came with a hike in customer loyalty and a brand that competitors still fight tooth and nail to beat in the market.

Netflix serves as the ultimate digital transformation case study.

They transformed their entire business model and charted unprecedented waters. Here’s how to use their model as inspiration for your contact center’s digital transformation.

How to move your operations to the cloud, Netflix style: A digital transformation case study.

21 years after they started renting DVDs, Netflix now sits at a valuation of almost $145 Billion .

They came to market as a disruptor of traditional video stores like Blockbuster and Family Video.

Netflix founders Reed Hastings and Marc Randolph wanted to bring customer-centricity to the video rental market. At the time, renting videos was inconvenient and costly, with customers often plagued by expensive late fees.

They created an entirely new way to watch movies and consume content. And as time went on and subscribers grew, they continued to shift to keep pace with new consumer demands.

In 2007 , they took their first step into the world of streaming video. They offered customers a streaming subscription in addition to the more traditional DVD rental service, giving customers the option to chart their own path.

Since then, they’ve seen exponential growth in subscribers and revenue. Let’s take a look at their trends over time. We’ll skip over the first few years of the company’s infancy and jump to the year the company went public.

Here’s how Netflix has grown since 2002.

That incredible growth trajectory, and willingness to change, made Netflix stock skyrocket by 6,230% in a 10-year period.

And, they did it all without crazy price hikes, keeping customers top-of-mind.

While Netflix has adjusted prices over the years, they strike a balance by adding more value and services for the dollar. In 2019 , the Basic plan increased by $1 a month (adding up to $12 annually). While the Standard and Premier plans rose by $2 per month, (adding up to $24 annually, for each plan).

Meanwhile, the company is putting some $15 billion towards creating new content binge-watchers will love.

After this price change, Netflix saw a slight blip in subscriber growth, with growth in Q2 coming in low. But, analysts don’t think for a second it’s the beginning of a downward trend. In fact, a similar event happened back in 2010 when Netflix moved to a pricing model that broke out streaming and video rentals. And they clearly rebounded.

When you put the numbers into perspective, you see this is the first dip in subscriber growth in nearly a decade. That’s pretty remarkable. And, revenue still increased for the quarter. It’s clear the value of the digital innovator’s services still outweighs the cost for most.

Plus, if you can post positive revenue numbers for over a decade and become a multi-billion-dollar company in about 20 years, you’re doing alright.

Here’s what Netflix did to reach these lofty heights. And, how you study the same tactics to lead your contact center through a successful digital transformation.

Stay true to your vision.

Netflix started out with the idea to make it easier and less expensive for people to watch movies.

But they didn’t want to stay in the DVD game forever. They had the foresight to predict that consumer behaviors would continue to shift. And, they wanted to stay ahead of the competition.

Only, they didn’t sacrifice their vision when it came time for company-wide changes. Instead, they realigned their business strategies to fit their vision, even as consumers and trends shifted.

What you can do:

As you make digital shifts in your contact center and your company, keep your vision constant. While tons of other factors may orbit around you, your vision keeps you grounded.

Use your company vision to guide your decision-making. And, use data and trends to predict how your customer behavior will shift.

As you shift to keep pace with your customers’ needs, align your operations to your customer behaviors to realize your vision.

Reinvent the wheel if the old one doesn’t solve customer problems.

Netflix soared from seed idea to a $145-billion-dollar valuation in only 21 years. (Wow, they did that in less time than it took big tech vendors to break CSAT scores.)

And they didn’t get there by spinning up a new-and-improved version of Blockbuster.

Ted Sarandos, Head of Content at Netflix said when he came on board at the early stages of the company founder Reed Hastings used his vision to scale and innovate at Netflix.

“We never spent one minute trying to save the DVD business,” said Sarandos .

The company leaders didn’t stick to traditional best practices because they no longer worked for modern customers.

Instead of piggybacking off what other companies did, Netflix solved problems differently. And, they solved them better. The proof is in a bankrupt Blockbuster and dwindling Family Video stores.

Want to know what you’re missing when you only look at digital transformation best practices? Pop over to our article on the topic.

Tailor your path and contact center strategies to your specific business needs. Focus on listening and understanding your customers, with the help of better data and customer surveys .

Find out what’s causing your customers’ pain. See what common questions your customers have. Work with your sales team to find out why customers are fleeing competitors. Discover why they choose your products and services in the first place. Then, work with your contact center and company leaders to develop the methods to solve these pains.

Don’t get caught up in what your competition is doing. What they’re doing might work, but your actionable data and customer information can guide you to a way that works better.

If you’re going to be consumed by one thought, let it be this one: how might we better serve our customers?

Don’t force your customers down a single path.

In the early phases of Netflix, internet speeds weren’t built for streaming movies. People who tried to download and view movies online were only frustrated by the lengthy, often interrupted experience of watching a film online.

Netflix didn’t want to enter the streaming market until the right infrastructure was available to support a platform with high-quality and high-speed content. They didn’t want to taint their brand from day one, linking the Netflix name to all the baggage that came with poor streaming experiences.

At the same time, they were watching postage prices. The price of postage kept rising, and internet speeds were on the ups. By watching how the market and internet infrastructure changed, they identified the right moment to launch their first streaming service.

They tested their streaming service with lower-quality video, first. They wanted to gauge interest and customer experience without canceling their bread-and-butter DVD service.

Those who wanted access to the crisp DVD picture could still order movies to their doorstep. Others who wanted instant access could forgo the high-quality picture for convenience, instead.

Your contact center and customer experience will change. It has to. But as you make changes and shift your operations to the digital era, keep options open for your customers.

Just because chat and email are on the rise as popular customer service channels doesn’t mean every customer wants to use them. Use past data and communication history to learn more about your customers. Then, coach your agents to handle each interaction based on the customer’s preferences.

Bringing changes to your contact center has the potential to transform your customer experience for the better. But, without careful intention, it can also cause friction. Introduce changes to your customers slowly, and make sure your agents are always there to offer extra help through the process.

Use data and trends to personalize your customer experiences.

This one’s huge. It’s how Netflix keeps customers engaged with their platform, and how they coined the term binge-watching

As Netflix made changes in their operations, they watched their data like a hawk. They looked for trends on how people watched content, what kept them watching, and how personalization fueled content absorption. Then, they used an algorithm to serve up content tailored to their customers’ specific interests.

“Like a helpful video-store clerk, it recommended titles viewers might like based on others they’d seen.” – Twenty Years Ago, Netflix.com Launched. The Movie Business Has Never Been the Same , by Ashley Rodriguez for Quartz .

And, as their new cloud-based business let them scale globally, their data points multiplied.

Previously, Netflix could only mail DVDs to U.S. customers. Shipping DVDs overseas wouldn’t have been financially sustainable while keeping prices fair for all customers. Moving to an online business model allowed Netflix to target and reach new audiences without taking on the costs of shipping globally.

Doing this not only scaled their business, but it diversified their data and made their algorithm smarter. Enter, extreme personalization and binge-watching fever on a global scale.

Track and analyze data from your customer interactions. Create custom reports and dashboards to distill important findings from your data. Then, use the trends and patterns you find to personalize your customer service experiences.

From the way you send customer surveys to the tone your agents use, your interactions tell you what your customers want. Lean into your analytics for valuable insight into how to help your customers.

And, use the data to transform your contact center too. Customer data is a powerful tool to drive business change. If your metrics show customers aren’t happy, your company leaders want to know about it. And, they’ll want to fix it. There’s no better case for company transformation.

Netflix took risks to transform their business. But, there’s no bigger risk than stagnation. Staying the same doesn’t help you reach your contact center goals. Innovating and trying out your big ideas is what separates the leaders from the laggards.

Can your tech vendor survive in your digital transformation?

Learn how to choose vendors who make your transformation strategy possible.

Serverless Case Study – Netflix

A couple of days ago we published a case study on how Coca-Cola North America handles their  vending machine’s systems with serverless . Today we’re going to talk about another titan that turned to serverless. As you may have guessed from the title, we are going to be talking about Netflix.

Netflix is a streaming service founded in 1997 and, believe it or not, started out as a Blockbuster alternative for renting and selling DVDs through the mail. Yeah, it was such a long time ago. And while they are still renting about 3 million DVD’s a year they are also the number one video streaming platform for TV-shows and Movies.

Netflix delivers 10 billion hours of videos to 125 million customers every quarter and to serve that kind of audience they use a wide range of highly complex infrastructure that relies mostly on AWS. Imagine what the servers that run Netflix look like? Petabytes of data in hundreds of thousands of files changed daily, served millions of customers in 55 countries.

At the moment Netflix has moved completely to the AWS cloud infrastructure and while a full seven years to make the move from their own data center might seem a long time for most people, they wanted to make sure that the problems they were facing while using the self-managed data center would not get imported into the cloud so they ended up basically rewriting every aspect of their service to make Netflix a true cloud-native application. You can read more about the journey to the cloud in an article written by  Yury Izrailevsky , vice president of cloud platform engineering.

So how does Netflix make use of Serverless

Publishers upload thousands of files to Netflix on a daily basis and every bit of those files need to be encoded and sorted before they end up being streamed to the user. Once the files get uploaded to S3, Amazon triggers an event calling an AWS Lambda function that splits the video into 5-minute chunks that get encoded into 60 different parallel streams that Netflix needs. Once the last part of the video gets processed they get aggregated and deployed using a series of rules and events.

Another way that Netflix uses AWS Lambda is for their backup system. As thousands of files get changed and modified on a daily basis Lambdas are checking if the files need to be backed up, they check the validity and integrity of the files, and if anything fails they can backtrack to the source of the problem and restart the process.

In the space of security, Netflix has thousands of processes that stop and start instances all the time and they use Lambda to validate that each no instance is constructed and configured in accordance with the system’s rules and regulation. They also use Lambda to create alerts and shutdown in the event of unauthorized access.

Next came efficiency improvements using better production monitoring and dashboards. This information was based on the events system that Netflix built for Lambda, through which events trigger validations to ensure that the configuration fits real-world needs.

The last step was to remove the responsibility of the servers that manage all of Netflix’s media. When Lambda is responsible for the server deployment, compliance, and configuration, Netflix can be confident that provisioning processes and responding to new business needs are fully handled.

Amazon Kinesis Streams processes multiple terabytes of log data each day, yet events show up in our analytics in seconds. We can discover and respond to issues in real-time, ensuring high availability and a great customer experience. — John Bennett Senior Software Engineer, Netflix

To start reaping the true serverless benefits like Netflix today, sign up to Dashbird’s serverless observability platform .

• Clean and easy-to-understand user interface
• No latency added to the function execution time
• Great support staff
• Support for Java, Node.js, Python
• Start working with your data immediately
• Pre-configured error and threat alarms and custom alarms
• Aggregated real-time observability for AWS services
• Well-Architected insights and actionable suggestions for improving users’ architecture

Read our blog

ANNOUNCEMENT: new pricing and the end of free tier

Today we are announcing a new, updated pricing model and the end of free tier for Dashbird.

4 Tips for AWS Lambda Performance Optimization

In this article, we’re covering 4 tips for AWS Lambda optimization for production. Covering error handling, memory provisioning, monitoring, performance, and more.

AWS Lambda Free Tier: Where Are The Limits?

In this article we’ll go through the ins and outs of AWS Lambda pricing model, how it works, what additional charges you might be looking at and what’s in the fine print.

Made by developers for developers

Dashbird was born out of our own need for an enhanced serverless debugging and monitoring tool, and we take pride in being developers.

What our customers say

Dashbird gives us a simple and easy to use tool to have peace of mind and know that all of our Serverless functions are running correctly . We are instantly aware now if there’s a problem. We love the fact that we have enough information in the Slack notification itself to take appropriate action immediately and know exactly where the issue occurred.

Thanks to Dashbird the time to discover the occurrence of an issue reduced from 2-4 hours to a matter of seconds or minutes. It also means that hundreds of dollars are saved every month.

Great onboarding: it takes just a couple of minutes to connect an AWS account to an organization in Dashbird. The UI is clean and gives a good overview of what is happening with the Lambdas and API Gateways in the account.

I mean, it is just extremely time-saving . It’s so efficient! I don’t think it’s an exaggeration or dramatic to say that Dashbird has been a lifesaver for us .

Dashbird provides an easier interface to monitor and debug problems with our Lambdas. Relevant logs are simple to find and view. Dashbird’s support has been good , and they take product suggestions with grace.

Great UI. Easy to navigate through CloudWatch logs . Simple setup.

Dashbird helped us refine the size of our Lambdas, resulting in significantly reduced costs . We have Dashbird alert us in seconds via email when any of our functions behaves abnormally. Their app immediately makes the cause and severity of errors obvious.

End-to-end observability and real-time error tracking for AWS applications.

How Netflix increased developer productivity and defeated the thundering herd with gRPC

Netflix developed its own technology stack for interservice communication using HTTP/1.1. For several years, that stack supported the company’s stellar growth. But by 2015, there were pain points: Clients for interacting with remote services were often wrapped with handwritten code. Plus, when a team built a service that defined an API, there wasn’t a clear way to annotate and describe exactly how that API functioned, making it challenging to discover, audit, and understand what APIs were available in the ecosystem. Netflix needed a new solution that would allow service clients to work across languages, with an emphasis on Java and Node.js.

There were efforts to build an RPC (Remote Procedure Call) stack internally, but after a month-long evaluation of several technologies, the Runtime Platform team chose to implement and extend gRPC instead. gRPC is a high-performance RPC framework developed by Google and optimized for the large-scale, multi-platform nature of cloud native computing environments. It connects services across languages, clouds and data centers, and connects mobile devices to backend servers.

For each client, hundreds of lines of custom cache management code were replaced by just 2-3 lines of configuration in the proto. Creating a client, which could take up to 2-3 weeks, now takes a matter of minutes. As a result, the time to market has been reduced by orders of magnitude. Additionally, the fact that clients no longer contain handwritten code means that there are far fewer errors when interacting with remote services. Latency has also been improved.

Projects used

By the numbers

Time to market

Reduced by orders of magnitude

Creating a client

Went from 2-3 weeks to minutes

Hundreds of lines of custom cache management code replaced by 2-3 lines of configuration in proto per client

For most of the 130 million Netflix subscribers waiting for the next season of Stranger Things , remote procedure calls (RPC) probably don’t mean anything.

But for the Runtime Platform team responsible for improving developer productivity at the company, RPC was becoming a bottleneck to building and supporting the high-availability services that Netflix users expect.

Netflix had developed its own technology stack for interservice communication using HTTP/1.1, and “the glue for all service communication” covered about 98% of the total microservices that powered the Netflix product, says Tim Bozarth, Director of Platform Engineering. For several years, that stack supported the stellar growth of the company’s streaming media business.

But by 2015, Bozarth’s team realized that it had also “perpetuated a few architectural patterns that we were struggling with, and at scale were impacting engineering’s productivity,” he says. Clients for interacting with remote services are often wrapped with handwritten code, which was time-consuming and “created opportunities for issues to arise, for bugs to be introduced, and for additional complexity to breed,” he says. Plus, when a team built a service that defined an API, there wasn’t a clear way to annotate and describe exactly how that API functioned or see, audit, and understand what APIs exist for both the service and the ecosystem.

While others at the company considered building a new RPC system internally, Bozarth and his team embarked on a month-long evaluation of several technologies. In the end, they chose gRPC. “It was comfortably at the top in terms of encapsulating all of these responsibilities together in one easy-to-consume package,” says Bozarth. “The things that we cared about the most were the architectural understanding in the IDL (proto) that is packaged as a part of gRPC, and the code generation that is derived from that proto. Those were by far the most interesting for us because they addressed two really key problems that we’d been facing at scale: not having clear APIs as well as lots and lots of handwritten client code.”

In addition to addressing those productivity-oriented problems, the team wanted a solution that wasn’t specifically coupled with Java, since engineers at Netflix had started to use other languages as well, like Node.js, Python, and Ruby—and that promise of cross-language compatibility and code-generation existed with gRPC. The implementation with Java applications went smoothly, with the team spending the first eight months taking the pieces of customization that existed in the company’s own internal RPC stack and porting and transitioning those over into the gRPC environment.

Making gRPC work with other languages took a greater effort. “If you have a Java server and a Node.js client, the cross-language generation and communication work very well from a protocol standpoint,” says Bozarth. “What’s different are the mechanics that exist in the other languages for customization, in terms of actual feature completeness and the idioms. So with Node.js, we had to do a lot of enhancement and a fair amount of wrapping. It took us almost a year to get the interceptor mechanics merged, but we were able to contribute back the entire interceptor layer to JavaScript. That’s a huge win.” (There’s now substantial traffic between Node.js and Java being done over gRPC at Netflix.)

“There are a number of people who struggled with the complexity of their clients and the challenges of operations, that they chose to rewrite their applications in gRPC because the value proposition that it made was so substantial.” — TIM BOZARTH, DIRECTOR OF PLATFORM ENGINEERING AT NETFLIX

With gRPC in place, developer productivity, which was always the team’s biggest driver, got a big boost: For each client, for example, hundreds of lines of custom cache management code were eliminated. “We’ve turned a very tedious, error-prone process into what amounts to maybe two or three lines of annotation, extra definition in your proto file, and we just generate those interactions for you,” says Senior Software Engineer William Thurston. Creating a client went from 2-3 weeks to minutes. “You can get started with a running application in a matter of minutes and then get that application working in a matter of hours,” says Bozarth. Time to market, which was typically three weeks before gRPC, has been reduced by orders of magnitude.

Plus, the fact that clients no longer contain handwritten code means that means that a common source of application errors has been eliminated. “It’s virtually bug-free code because it’s heavily vetted and generated, and that increases productivity and lowers the operational burden,” says Thurston.

Latency has also been impacted. “We’ve seen an incredible reduction in P99s for gRPC-oriented services,” says Bozarth. “We’ve also seen a squishing and a narrowing of our latency windows consistently across the board.”

“When we picked gRPC, we were betting that it would get the adoption and a lot of other people building useful things in open source. I think largely that bet has paid off.” — WILLIAM THURSTON, SENIOR SOFTWARE ENGINEER AT NETFLIX

Today, a huge part of the internal service-to-service communication at Netflix runs on gRPC. “The adoption has been a success and continues to move forward, especially in the Java space,” says Bozarth. All new Java development starts with a gRPC-enabled application. And while there’s no push to rewrite existing applications, he says, “there are a number of people who struggled with the complexity of their clients and the challenges of operations, that they chose to rewrite their applications in gRPC because the value proposition that it made was so substantial.”

The team also found gRPC to be invaluable for a project involving adaptive concurrency limits —a critical issue for any business that requires the highest service availability—which they’ve open sourced. “We’ve been able to effectively defeat the thundering herd problem by changing how the server does concurrency limits adaptively over time using the gRPC mechanics,” says Bozarth. “gRPC made it architecturally simple. We were able to approach this effort in a way that we couldn’t have done before.”

As early adopters of gRPC, Bozarth and Thurston say they’ve benefited from the community as much as they’ve given back. “One of the reasons we picked gRPC is we were making a bet that it would get the adoption and there would be a lot of other people building useful things in open source, and I think largely that has paid off,” says Thurston.

For Netflix, this is the exact place they want to be. “As the industry’s changing and as new, powerful technologies are emerging, we’re fairly early in the adoption curve,” says Bozarth. “But if you’re trying to build a large distributed system, RPC is critical to its long-term success. We believe that gRPC is a really powerful and important foundation for us as we move forward.”

How cloud seeding can make it rain or prevent extreme weather

Technology can make water vapour in clouds stick together into droplets of rain or snow.

Social Sharing

Cloud seeding has been named by some media reports as a possible contributor to record-setting rain and flooding in Dubai in the United Arab Emirates . Here's a closer look at what cloud seeding is, how it's used and whether it could have made the flooding worse.

What is cloud seeding?

It's the process of making tiny drops of water vapour and ice crystals in clouds stick together into larger, heavier droplets or pellets that fall as rain or snow.

Often, this is done by spraying particles of salts such as silver iodide or table salt using special flares carried by airplanes or projectiles such as rockets, cannons or missiles. 

The U.A.E. has also used drones to zap clouds with electric charges for cloud seeding .

Is this a new technology?

No. It's actually been around since the 1940s . It's been used in dozens of countries, including Canada . Despite that, interestingly, it was only recently that scientists have been able to prove it works by distinguishing between natural and induced rain or snow.

Cloud seeding to prevent drought?

What is it used for.

It's often used to fight droughts by inducing rain or snow , including by  the U.A.E. to try to recharge its dwindling groundwater supplies . 

In the U.S., there have been efforts to use it to fight wildfires .

In Canada, it's often used to lessen the damage caused by hail , by getting moisture in the clouds to fall as rain or snow before it can build up into hail.

Cloud-seeding pilots see busy year

How effective is it.

"The results of about 70 years of research into the effectiveness of cloud seeding are mixed," wrote atmospheric scientist William R. Cotton of Colorado State University in a 2022 article published in The Conversation . He added that it requires the right kinds of clouds with enough moisture, and the right temperature and wind conditions, and produces only small increases in precipitation. 

A recent study measured the snow from three cloud seeding events  and calculated that they caused enough water to fill 282 Olympic-sized swimming pools to fall over an area of 80 by 80 kilometres — a tenth of a millimetre on average at any given spot.

"Regardless of the mixed evidence, many communities are counting on it to work," Cotton said.

• Analysis reveals how climate change is influencing extreme weather
• Cloud seeders attempt to rescue 'hailstorm alley' from storms by reducing size of hail

The U.A.E. National Centre of Meteorology says cloud seeding can boost rainfall from a specific cloud by 25 per cent under optimal conditions , and the technology "plays a crucial role in the broader context of climate change mitigation and building climate resilience."

Hail suppression is also considered effective enough that insurance companies in Canada invest millions a year to seed clouds in Alberta .

Could cloud seeding have played a role in the Dubai floods?

According to The Associated Press , several reports quoted meteorologists at the National Centre for Meteorology as saying they flew six or seven cloud-seeding flights before the rains. Flight-tracking data analyzed by AP showed one aircraft affiliated with the U.A.E.'s cloud seeding efforts flew around the country on Monday.

Dubai residents wade through aftermath of flash floods

The National, an English-language, state-linked newspaper in Abu Dhabi, quoted an anonymous official at the centre on Wednesday as saying no cloud seeding took place on Tuesday, without acknowledging any earlier flights.

A number of researchers from the University of Reading, some of whom worked with the U.A.E. to develop electric cloud seeding technology , issued statements saying cloud seeding could not have been a major contributor to such severe weather .

"The U.A.E. does have an operational cloud seeding programme to enhance the rainfall in this arid part of the world, however, there is no technology in existence that can create or even severely modify this kind of rainfall event," wrote Maarten Ambaum, a meteorologist at the University of Reading who has studied rainfall patterns in the Gulf region.

"In this particular case, there would have been no benefit to seed these clouds as they were predicted to produce substantial rain anyway."

Climate change raises risk of destructive combination of fire and floods: study

Other researchers agreed.

"It is very unlikely that cloud seeding would cause a flood," Roslyn Prinsley, the head of disaster solutions at the Australian National University Institute for Climate, Energy and Disaster Solutions, told TIME , describing such claims as "conspiracy theories."

On the other hand, both Ambaum and Prinsley, pointed to climate change as a likely factor in the extreme rainfall, as a warmer atmosphere can hold more water vapour. 

A 2022 analysis of weather events around the world in the past two decades showed that episodes of heavy rainfall are becoming more common and more intense as the climate warms .

ABOUT THE AUTHOR

Science, climate, environment reporter

Emily Chung covers science, the environment and climate for CBC News. She has previously worked as a digital journalist for CBC Ottawa and as an occasional producer at CBC's Quirks & Quarks. She has a PhD in chemistry from the University of British Columbia. In 2019, she was part of the team that won a Digital Publishing Award for best newsletter for "What on Earth." You can email story ideas to [email protected].

• Subscribe to the What on Earth newsletter

With files from the Associated Press

Related Stories

• U.A.E. hit with heaviest rain ever recorded in the country
• Climate change could cost municipalities $700M more a year to maintain pipes, sewers, report says
• We've been cloud seeding for decades, but now we finally know it works

Add some “good” to your morning and evening.

The environment is changing. This newsletter is your weekly guide to what we’re doing about it.

What caused Dubai floods? Experts cite climate change, not cloud seeding

• Medium Text

DID CLOUD SEEDING CAUSE THE STORM?

CAN'T CREATE CLOUDS FROM NOTHING

Coming soon: Get the latest news and expert analysis about the state of the global economy with Reuters Econ World. Sign up here.

Reporting by Alexander Cornwell; editing by Maha El Dahan and Alexandra Hudson

Our Standards: The Thomson Reuters Trust Principles. New Tab , opens new tab

World Chevron

Russia warns of direct clash with West over Ukraine

Russia said on Monday that U.S., British and French military support for Ukraine has pushed the world to the brink of a direct clash between the world's biggest nuclear powers that could end in catastrophe.

Watch CBS News

What is cloud seeding and did it play any role in the Dubai floods?

By Li Cohen , Tracy J. Wholf

Updated on: April 18, 2024 / 8:46 PM EDT / CBS News

Stranded airline passengers and a cat submerged in floodwaters clinging to a car door handle became notable moments this week in Dubai as the normally arid city was inundated with historic levels of rain. Claims have gone viral that the deluge was brought on by cloud seeding, a technique that aims to increase precipitation, that is heavily utilized in the United Arab Emirates. 

But is it really to blame? 

Daniel Swain , a climate scientist at the University of California, Los Angeles, said that getting to the bottom of the "record-shattering extreme rainfall" requires breaking down the science behind the event and the technique. 

"There's currently a disconnect in the online discourse between the kind of human activities that likely did affect it (greenhouse warming) versus those which have actually been the focus of the online conversation thus far (cloud seeding), and what this means for how we collectively understand our ability to actively affect the weather on different spatial and temporal scales," he said in an emailed statement. 

What is cloud seeding? 

Many have questioned since the downpour in Dubai whether cloud seeding was to blame. But what is cloud seeding and how does it work exactly?

Cloud seeding is a technique used to improve precipitation. According to the Desert Research Institute, scientists do this by putting tiny particles called nuclei into the atmosphere that attach to clouds.

"These nuclei provide a base for snowflakes to form. After cloud seeding takes place, the newly formed snowflakes quickly grow and fall from the clouds back to the surface of the Earth, increasing snowpack and streamflow," the institute says. 

In the Middle East, instead of precipitation in the form of snow, its cloud seeding program generates increased rain. 

Scientists typically go about cloud seeding in two ways – using either generators on the ground or distributing the nuclei via aircraft. 

What caused the rain in Dubai? 

But was the rain in Dubai from cloud seeding? 

"Did cloud seeding play a role? Likely no," Swain said. "But how about climate change? Likely yes!" 

The world is continuing to see month after month of record-breaking heat and 2023 was the hottest year globally ever recorded. Scientists have found that warmer temperatures increase evaporation, resulting in more frequent and intense storms, such as the one that occurred in Dubai. Those conditions also fuel other extreme weather events, including droughts , putting opposing forces at intense odds that will likely strain communities without adequate adaptation. 

Andrew Kruczkiewicz, senior researcher at Columbia Climate School, told CBS News he doesn't believe there's any current evidence at this time that cloud seeding pushed the downpour over the edge. 

"This event was forecast fairly well days in advance and I think it's unlikely that a cloud seeding operation would move forward given the well-forecast intense rainfall," he said.

The nation's  National Emergency Crisis and Management Authority issued weather warnings on Monday before the storm's arrival, urging people to comply with local instructions from authorities and asking them to stay at home and only leave in the case of an emergency. 

Meteorologist Ryan Maue, former chief scientist at the U.S. National Oceanic and Atmospheric Administration, gave the Associated Press a more definitive answer: "It's most certainly not cloud seeding." 

"If that occurred with cloud seeding , they'd have water all the time," he said. "...when it comes to controlling individual rain storms, we are not anywhere close to that. And if we were capable of doing that, I think we would be capable of solving many more difficult problems than creating a rain shower over Dubai ."

The deluge, he said, "speaks more to questions around what are the resilience measures that are integrated into the urban planning standard operating procedures." 

"Almost everywhere on Earth there is a risk of flash flooding," he said. "Yet, since it's not the most frequent type of extreme event, sometimes it's lower on the priority list when decisions need to be made around infrastructure or resilience, or just urban development more broadly."  

How significant was the flooding in Dubai?  

More than 5.59 inches of rain fell over Dubai within 24 hours. While a half-foot of rain may not seem like much numerically, that's more than what the city sees in an average year, and other parts of the UAE saw even higher levels. 

It was a "historic weather event," the state-run WAM news agency said, adding that it was beyond "anything documented since the start of data collection in 1949." 

Dubai is normally dry and with a downpour like this being so unprecedented, the city's infrastructure was not prepared. The drainage systems were overwhelmed and Dubai International Airport, one of the busiest in the world, had to temporarily halt operations. One plane passenger told Reuters many people were waiting more than 12 hours to be able to resume travel. Footage from the airport shows planes taxiing in eerie floodwaters. 

"Over a year's worth of rainfall was experienced in just a few hours," Kruczkiewicz told CBS News. "And why that's important to to understand is that when you see this amount of rainfall in semi-arid arid area, the soil isn't designed to filter the water as fast as in other areas. ... You don't need that much water falling or rainfall falling in a short period of time to cause major issues." 

Is cloud seeding effective? 

According to the Desert Research Institute, how effective cloud seeding is depends on the specific project in which it's being used. Citing several studies, the institute said it's helped increase overall snowpack in some areas by at least 10% per year. Another study found that a five-year project in New South Wales, Australia resulted in a 14% snowfall increase.  

The UAE's National Center of Meteorology launched the Research Program for Rain Enhancement Science to advance the technology, saying that for dryer regions across the world, cloud seeding "could offer a viable, cost-effective supplement to existing water supplies." Many regions even beyond the Middle East have been suffering from water scarcity issues, including Colombia , Mexico and Hawaii . 

• Science of Weather
• United Arab Emirates
• Climate Change
• Severe Weather

Li Cohen is a social media producer and trending content writer for CBS News.

More from CBS News

Poll: Big majority of Americans favor taking steps to reduce climate change

Trump cancels North Carolina rally due to severe weather

Israeli strikes in Rafah kill 18, mostly children, Palestinian officials say

Ukraine's Zelenskyy says "we are preparing" for a major Russian offensive