Not All Kubernetes Services Are Equal. We Should Know.
Kubernetes promises the long sought-after capability to fully abstract underlying public cloud, private cloud, and edge infrastructure from the perspective of software applications that perform specific functions, or workloads. For B23, the value of Kubernetes means that all of the innovative and ground-breaking data engineering and applied machine learning workloads that we have developed and operated based on years of experience can be seamlessly deployed in almost any environment that runs Kubernetes.
B23 supports and operates a variety of Kubernetes solutions including “pure” Kubernetes that we deploy to any arbitrary set of supported server hosts. We also support public cloud managed Kubernetes services from Google, Amazon, Microsoft, and DigitalOcean. We support integration to a previously running Kubernetes system to address private cloud Kubernetes solutions. Most recently, we support Rancher’s K3S for edge computing solutions (more on that exciting news in a later blog).
We’ve done Kubernetes the “hard way” from scratch, and we’ve done it the “easy way” using cloud managed Kubernetes, or at least we thought managed Kubernetes would be easy. In some cases, the “easy way” was just not so easy. That’s why the “conceptual value” of Kubernetes varies from the “actual value” of Kubernetes. It depends heavily on your cloud service provider.
Here are some of the high-level differences we have found in our pursuit to achieve our ultimate goal of infrastructure agnostic workloads using Kubernetes. They fall into the following categories:
Default security features and versions vary by Kubernetes service provider
Non-existing or limited built-in support for Kubernetes auto-scaling capabilities across service providers
Some service providers require proprietary or provider-specific functionality leading to vendor lock-in
The workflow and lifecycle management of Kubernetes and hosted workloads vary in capability and complexity
The SDK ecosystem for programmatically operating managed Kubernetes solutions vary greatly in maturity
The “Developer Experience” to operate Kubernetes in the cloud vendor’s Console varies considerably
Security, Upgrades, and Version Compatibility Support
The recent and highly visible security vulnerability in Kubernetes (CVE-2018–1002105) was an eye-opening moment for most organizations currently running Kubernetes or looking to adopt Kubernetes. The ability to patch and/or upgrade existing and running Kubernetes clusters in the wake of that announcement was critical for organizations. Running workloads on different cloud managed Kubernetes solutions, running different versions, and with different processes to patch and upgrade is an important consideration. Not all cloud managed Kubernetes services are easy to patch and upgrade. At B23, we believe it’s not a question of if more security vulnerabilities will be identified (they undoubtedly will), but it’s a question of how fast we can respond and continue to operate our business-critical workloads without interruption.
Auto-Scale
One of the most fundamental advantages to moving from legacy data centers to the cloud was the idea of autoscaling compute, memory, and/or storage based on the ever-changing demands software applications required to service end-users. Kubernetes makes autoscaling even easier than the prior virtual machine auto-scale services (we could write an entire blog on why this is important). Almost every workload can benefit from autoscaling services — from basic web server/e-commerce solutions to data engineering and applied machine learning workloads like the ones B23 offers to our customers. Using Kubernetes to host workloads in the cloud without the ability to autoscale resources does not make sense.
Avoiding Vendor Lock-In
Once you are in the weeds managing a cloud managed Kubernetes solution, it becomes clear very quickly how vendors are creating their moat to lock-in customers. Each cloud vendor is going to recommend an approach to use their capabilities to operate and secure a Kubernetes solution — using their provisioning and monitoring tools, etc. That’s totally understandable, and in most cases, these capabilities add value (and differentiate) managed Kubernetes offerings. In our view, the line is crossed when vendor-proprietary capabilities are required as a “must have” versus a best-practice.
Lifecycle Management
From our observations, most Kubernetes sysadmins still use vendor-specific “Cloud Consoles” for the lifecycle (provision, operate/manage, and delete) management of their Kubernetes systems. Each cloud vendor has its own terminology and best-practice security recommendations for role-based access control. It becomes very difficult to manage underlying resources with web-based cloud consoles with only a handful of clusters, that’s why all of our solutions are managed programmatically.
SDK Ecosystem
At B23 we use Python for most of our development and programmatic automation, with command line SDK’s as a strong second. The Python SDK capabilities for the cloud service providers vary in maturity. The command line SDK’s are fairly capable in most cases, but differences do exist. In one particular instance, regarding one well-known vendor, we find it hard to believe we were not the first to use the Python SDK based on the number of bugs we have submitted and documentation updates we recommended. It was essentially unusable from a Python SDK perspective.
Developer Experience
Granted, this is subjective, but there is a general consensus that some managed Kubernetes services are easier to develop against and operate than others. Since we use them on an hourly and daily basis, we certainly have our favorites that are conceptually easy to use and operate. Often, small differences in terminology can be annoying, and dealing with cloud-proprietary requirements can border on plain frustrating. Without naming names, I think it’s fair to say that we generally agree with the people who pontificate in this area.
Is There a Best One?
Many people ask us if is there a “best” managed Kubernetes service out there.
The answer is “yes”…
…but it also depends on what customers are looking for. In the real world, enterprises still buy software and cloud services from vendors they know and like. Sometimes it’s hard to argue the technical merits of a solution when human emotion is involved. That’s why it’s even more critical to know what you are getting into when considering which Kubernetes solution is right for your organization, or how to quickly accommodate when the decision was made independently of the technical merit.
#Analytics #DataScience #AppliedMachineLearning #Kubernetes #CloudComputing