The Biggest Gap in Kubernetes Storage Architecture?
# Blogs # cloud # applications # as-a-service # anything as a service
Year over year, Kubernetes consistently ranks among the most-loved platforms on Stack Overflow's Developer Survey. It's not hard to understand why. It offers easy scaling, great stability and reliability, works across many different environments. In short, it offers all the features that developers love.
Where it starts to lose its luster, however, is in production. The majority of applications out in the world are to some extent stateful, requiring persistent storage across one session to the next. Containers, and container orchestrators by extension, don't tend to play well with persistent storage and stateful applications. Let's explore why Kubernetes' storage architecture is lacking and what users can do to host stateful applications in production.
The biggest gap in Kubernetes' storage architecture? The fact that it doesn't exist — there isn't anything that could be appropriately called a storage architecture in Kubernetes.
Instead, Kubernetes provides a storage framework. The container orchestrator was never intended to provide storage in the first place.
There are a few reasons why this is the case and why it's a good thing, even though it might make a production manager's job more difficult.
First, containers are inherently ephemeral. The benefit of containerization lies in the ability for containers to be rapidly created or destroyed. Pinning a stateful workload to a container would prevent that from happening. In turn, this would prevent the fast failover, easy updates, lightweight provisioning and other benefits we derive from containers' ephemeral nature.
This isn't an insurmountable challenge, however. The major contributors in the Kubernetes ecosystem — specifically Google and Red Hat — could relatively easily release some definitive, "official" Kubernetes storage architecture.
This would incorporate Google's and Red Hat's particular design bias into the system, and it would contribute to vendor lock-in, something that would be against the open source nature of the Kubernetes project. If Google developed its own storage system for Kubernetes, that system would effectively become the default choice for storage even if it didn't meet the needs of your particular application.
By providing a storage framework — the Container Storage Interface, or CSI — rather than a storage architecture, the Kubernetes project offers users freedom and flexibility at the cost of simplicity.
What Does This Mean for Building Stateful Applications?
Because Kubernetes offers this framework for connecting a third-party storage architecture rather than its own storage architecture, Kubernetes users are faced with two choices:
Use open source projects to build a bespoke storage solution for your application.
Use a commercial-off-the-shelf storage solution that best fits your application's needs.
The first approach, as we'll see, comes with significant downsides.
Why the Open Source Approach Is so Challenging
Let's not kid ourselves, there are plenty of benefits to open source software. It's free, it's flexible, it leverages the wisdom of the crowd to overcome challenges and invent new solutions and more.
The open source model isn't ideal for all applications and all industries. Although open source solutions are free initially, their cost comes in the form of higher risk, larger teams, slower development and similar hidden costs.
In the context of persistent storage for containerized applications, platform users would need to rely on a solution based on multiple open source projects to meet their needs. Each of these different projects is geared toward solving a specific problem, resulting in a patchy storage solution.
Furthermore, this stitching together all but guarantees something is going to break at some point, and when it does, the only experts you have to consult on how to fix the issue are the ones you've trained internally. Consider that the cost of downtime, excluding hard-to-quantify reputational damage, can exceed $5,600 per minute. Most organizations relying on an open source solution will want to train developers to specialize in the given application's storage solution. Since these kinds of stitched-together solutions invariably require customization for each project, you'll need to hire developers to support each project commensurately.
Why Commercial Solutions Are Simpler
Actually, commercial solutions aren't better than open source solutions — not inherently anyway. A commercial enterprise storage solution could still be a poor fit for your specific project, require internal expertise, require significant customization, break easily and come with all the drawbacks of an open source solution.
The difference is that where an open source solution is all but guaranteed to come with these headaches, a well-designed commercial enterprise storage solution won't.
It isn't a matter of commercial versus open source, rather it's good architecture versus bad architecture. Open source solutions aren't designed from the ground up, making it much more difficult to guarantee an architecture that performs well and ultimately saves money. Commercial storage solutions, however, are. This raises the odds that it will feature an architecture that meets enterprise requirements.
Ultimately, all this is to say that commercial storage solutions are a better fit for most Kubernetes users than open source ones, but that doesn't mean you can skip the evaluation process. Whether open source or commercial, it's important to determine whether any solution meets your performance requirements.
At Ondat, we commissioned a study comparing the performance of four leading persistent storage solutions. Download the study to see how they performed on a series of standardized tests.
Romuald Vandepoel
Romuald Vandepoel is a technologist, facilitator, advocate, and open source contributor responsible for modernizing and transforming customer and partner organizations at Red Hat, Inc. He embraces his role as a trusted advisor within architecture boards and CTO offices providing guidance on culture, process and technology adoption strategies. As a transformational leader, he supports transitions from IT legacy siloed monoliths to ecosystems of Business-Unit-as-a-Service to accelerate the organization's innovation and growth. Romuald participated as moderator and speaker for both in-person and virtual events for Red Hat, FOSDEM, DevOps.com , DevNetwork, Tech Field Day, and CNCF. He also maintains and contributes to two open source projects, Trousseau and Discoblocks, and is a member of the Ondat Advisory Board where he provides guidance on open source strategies. Romuald is based both in Eindhoven and San Jose and has held diverse roles in tech companies including Tyco, SonicWALL, NetApp, and Red Hat.
Learn how Ondat can help you scale persistent workloads on Kubernetes.
On-Premises vs. Cloud Native Storage
StorageOS (now Ondat) Named a 2021 Winner in the Prestigious TiE50 Awards Program
How to Solve Kubernetes Persistent Storage Challenges
Deploying Cloud Native Persistent Storage in the Age of Containers