The way we build, deploy, and run applications has changed significantly over the past few years. Having realized the importance of continuous feedback loops, organizations needed to ship software updates as fast as possible, leading to a culture of collaboration, where everyone works as a team and not as a member of a silo.
This is fundamentally due to the DevOps movement, which is all about removing barriers, enabling better collaboration between software developers and operations engineers, and which focuses on automation as the primary way to make software delivery lean and measurable.
DevOps’ rapid adoption can be traced to four macro trends. Understand them and your organization’s DevOps transformation will be better served.
1. Software-defined hybrid clouds
In the past few years, data centers have gone through a massive transformation predominantly driven by cloud computing. In the pre-DevOps era, the barrier to entry for setting up a data center was high. Organizations struggled with simple things such as scaling infrastructure, since it took months to obtain new machines and configure them. Once the data center was set up, doing updates was an even bigger pain point, since the Ops team had to manually run patches and updates on every machine.
Amazon made it very easy to obtain on-demand compute in the form of virtual machines. Almost all public and private clouds today have APIs that allow IT to make changes very quickly, dropping the SLA from weeks to a few seconds for many tasks, such as provisioning and networking. Besides cloud-native tools and APIs, several tools, such as Terraform, Ansible, Chef, and Puppet, enable infrastructure as code (IaC). As a result, virtually the entire data center can be programmed as code, versioned, and deployed automatically to reflect changes.
2. Web-scale requirements
Every organization is now a software organization. Companies such as Amazon, Netflix, and Uber have disrupted traditional business models and made software as a service the de facto way of delivering software. As a result, applications have transformed from being systems of record to systems of engagement, such as mobile apps or interactive self-service portals.
Since consumers directly interact with systems of engagement, those systems need to have the following attributes:
- 24/7 availability, with no downtime
- A scale that can support hundreds or thousands of concurrent active workflows
- The ability to scale up or down to manage the demand ebb and flow both economically and efficiently
- High quality
- Complex infrastructure, with load balancers, firewalls, and security
- Frequent updates, with continuous feedback loops to reduce risk while innovating faster
Almost all of these attributes created more work for Dev and Ops teams, and it became critical for them to work in lockstep to avoid buggy releases or less-than-perfect deployments. Also, as the frequency of releases increased, all tasks in the software delivery process needed to be performed much more often. This created the necessity for efficiency, further fueling interest in DevOps principles.
3. Modern application architectures
The move to systems of engagement forced software teams to find innovative ways to handle scale and to be more agile in building and shipping software, while also being efficient in managing the cost of goods sold. This gave birth to a modern application architecture known as microservices. In this approach, you break down the application into small decoupled units called microservices that can be independently developed, deployed, and scaled. The application business logic is implemented by composing the relevant microservices into a workflow. Some benefits of microservices are improved speed, agility, quality, availability, and efficiency.
4. Containers
While containers have been around for some time, Docker gave them tremendous momentum. In just four years, container adoption has gone from negligible to ubiquitous. Docker provides much-needed decoupling between the application runtime and the host OS on the virtual machine. This makes containers lightweight and portable across clouds. There was already interest in leveraging multiple clouds within an organization for scenarios such as disaster recovery or cloud bursting. Docker has made these scenarios more plausible.
As organizations explore these new areas, it creates new requirements for development and operations teams, and also increases the complexity of software delivery workflows. Organizations invest in DevOps principles and automation in order to handle this added complexity and create synergy across cross-functional teams.
So what’s next?
While the DevOps movement is gaining rapid adoption, it is not easy to change many years of ingrained organizational processes and human behavior. Also, the DevOps toolchain is relatively immature, with a lot of scripting and automation tools that target a specific task or silo. While the initial focus of the movement encouraged a cultural change to Dev and Ops mindsets, organizations now need mature tooling in order to avoid the inefficiency caused by manual handoffs and an excessive number of custom scripts.