Kubectl cleanup plugin:

Kubectl cleanup plugin: https://github.com/b23llc/kubectl-config-cleanup At B23, much of our work involves spinning up new Kubernetes clusters on edge computing and storage devices to test our applied machine learning (“AML”) workloads. Using our B23 Data Platform (“BDP”), we can quickly configure and connect to a new Kubernetes cluster on an edge device, on-premise, or in any number of public vendor clouds. This lets us iterate and develop our data engineering and AML workloads quickly and at low cost for our customers. For most people who manage and operate multiple Kubernetes clusters, it can get overwhelming the number of configurations required to connect to those clusters. In my case, I have 37 different context entries in my ~/.kube/config. Only 3 of those entries are persistent clusters — in this case for dev, staging, and prod. The rest of the entries were for ephemeral clusters which barely lasted for a single work day. This workflow has become commonplace with the increased availability of managed Kubernetes services from cloud vendors like GCP, AWS, Azure, and Digital Ocean to name a few. Commands like gcloud container clusters get-credentials and az aks get-credentials are really convenient for obtaining credentials for a newly launched cluster and connecting right away, but a cluster a day quickly turns into this: Kubectl cleanup plugin: https://github.com/b23llc/kubectl-config-cleanup At B23, much of our work involves spinning up new Kubernetes clusters on edge computing and storage devices to test our applied machine learning (“AML”) workloads. Using our B23 Data Platform (“BDP”), we can quickly configure and connect to a new Kubernetes cluster on an edge device, on-premise, or in any number of public vendor clouds. This lets us iterate and develop our data engineering and AML workloads quickly and at low cost for our customers. For most people who manage and operate multiple Kubernetes clusters, it can get overwhelming the number of configurations required to connect to those clusters. In my case, I have 37...

Not All Kubernetes Services Are Equal. We Should Know.

Kubernetes promises the long sought-after capability to fully abstract underlying public cloud, private cloud, and edge infrastructure from the perspective of software applications that perform specific functions, or workloads. For B23, the value of Kubernetes means that all of the innovative and ground-breaking data engineering and applied machine learning workloads that we have developed and operated based on years of experience can be seamlessly deployed in almost any environment that runs Kubernetes. B23 supports and operates a variety of Kubernetes solutions including “pure” Kubernetes that we deploy to any arbitrary set of supported server hosts. We also support public cloud managed Kubernetes services from Google, Amazon, Microsoft, and DigitalOcean. We support integration to a previously running Kubernetes system to address private cloud Kubernetes solutions. Most recently, we support Rancher’s K3S for edge computing solutions (more on that exciting news in a later blog). We’ve done Kubernetes the “hard way” from scratch, and we’ve done it the “easy way” using cloud managed Kubernetes, or at least we thought managed Kubernetes would be easy. In some cases, the “easy way” was just not so easy. That’s why the “conceptual value” of Kubernetes varies from the “actual value” of Kubernetes. It depends heavily on your cloud service provider. Here are some of the high-level differences we have found in our pursuit to achieve our ultimate goal of infrastructure agnostic workloads using Kubernetes. They fall into the following categories: Default security features and versions vary by Kubernetes service provider Non-existing or limited built-in support for Kubernetes auto-scaling capabilities across service providers Some service providers require proprietary or provider-specific functionality leading to vendor lock-in The workflow and lifecycle management of Kubernetes and hosted workloads vary in capability and complexity The SDK ecosystem for programmatically operating managed Kubernetes solutions vary greatly in maturity The...

B23 Highlighted at Jeffries’s Battlefin Conference

Having just returned from yet another extremely productive Jefferies’s BattleFin conference in Miami, our team was reflecting on several themes we observed occurring in the financial services and hedge fund artificial intelligence (“AI”) market.  With 470 attendees, the Jefferies BattleFin conference is the premier event to observe, hear, and meet with experts related to alternative data and AI for hedge funds.  The conference itself continues to grow in size, and in the diversity of data providers and technology services relevant to the technology-driven investor.  We were excited to participate in a panel discussion yet again this year about a topic we have a high degree of conviction and experience, and it was extremely productive to meet with so many new and familiar industry experts.   An overview of these themes from this year’s event include: Increased acceptance of Data-Engineering-as-a-Service using qualified third-party technology partners like B23 Machine Learning at-scale is becoming more tightly coupled to Public Cloud infrastructure More pragmatism around the amount of alpha that alternative data can provide by itself Challenges to adopting new, innovative ideas with so much turnover and cross-pollination   Data-Engineering-as-a-Service for Hedge Funds An emerging theme that was very prevalent this year was the acceptance of outsourced data engineering or Data-Engineering-as-a-Service.  Many of the institutions we spoke with are quickly aligning themselves to this trend, which is consistent with our observations also occurring in non-financial services verticals as well.   It was obvious that more and more institutions continue to pursue a strategy to outsource the “undifferentiated heaving lifting” of data engineering in order for those same firms to focus on higher order outcomes with respect to quantitative investment analysis. Funds are increasing passing on building themselves cloud-based data lakes, or developing durable and performant extract, transform, and load (“ETL”) applications hosted on...