The Modern Data Economy — Part I of III: Status of the Data Economy
May 10th, 2016
Dave Hirko is a Managing Director and Co-Founder of B23. Prior to co-founding B23, Dave was an Account Executive at Amazon Web Services.
Industrial Revolution Meets Modern Data Economy
When we started B23 several years ago, our initial focus was to enable a variety of Hadoop distributions on the Cloud, specifically on AWS since that was our background and experience at the time. During this period, we met with many executives at the leading Hadoop companies that wished us luck, and inferred that running Hadoop on the Cloud was likely just a niche. They said bare metal was the only way to go. Fast forward several years (and over a billion dollars of investment in this area overall), and we estimate that a significant number of customers are running Hadoop, Spark, and other distributed processing applications in the Cloud. Entire “unicorn” software companies have bet their business model running these platforms on the Cloud. The mainstream economy is following suite. The Cloud combined with the appropriate level of automation is a more secure, agile, and competitive platform even for the most data sensitive organizations. The top players in the Data Economy understand this premise, and have embraced it totally. The path forward is definitely not for the feint of heart, and having a capable guide on this journey is important.
Scaling technology is hard and important, and finding great people is even harder and more important. Our company has been fortunate to have worked on some of most cutting-edge data modernization projects in the world. Our backgrounds prior to B23 share that commonality. We have continued to attract and retain many of the best software developers and data scientists in the country, and we continue to seek out the best. Our amazing team, a pragmatic realization of how software can enable the modern data enterprise, and understanding the pulse of our customers has allowed us to become market leaders in this new economy.
B23’s roots started with challenging preconceived notions of “viability,” and we will continue to do so with our belief that the modern Data Economy requires productizing in the similar way the Industrial Revolution changed our economy forever.
Stratification of the Modern Data Economy
At B23, we believe every business is a data business (whether they know it or not!). Within this new Data Economy, there exists a stratification of companies. At the top are the “Data Elite.” Companies in this category are the familiar ones such as Amazon.com, LinkedIn, Airbnb, and others who have harnessed the value of data, and use it for competitive advantage. Below them, we see companies that fall into two additional categories. In this middle tier are those companies who recognize their data has business value, but cannot technically harness it. The final category are those companies who have yet to understand the competitive advantage of data, and are further behind on their journey to join the modern Data Economy. B23 is an enabler for the middle-tier set of companies that recognize their data has monetary value, but technically have not achieved that elite status. B23 has helped several of our customers graduate to the top tier by “productizing” their data offerings. This has allowed them to get to market quicker and monetize their data faster than competitors. The financial return for companies who have leveraged B23 to graduate into this elite tier is enormous, resulting in significant revenue gains for larger firms, to lucrative acquisitions and exits for smaller firms. The proof is in the pudding as they say.
Productizing the Data Enterprise
A key aspect for enabling success of modern data enterprise is the process we refer to as “productization.” Almost all of our customers’ operation runs on the Cloud. It’s difficult enough to build and optimize a petabyte scale repository fed from so many diverse sources and not miss a single field of data. So as gluttons for punishment, we levy an additional requirement that all of our solutions should be able to be replicated from scratch in only a few minutes or hours depending on the physics of data transfer involved. We replicate entire, fully-configured environments with only a few mouse clicks, or through execution of a single program. We believe that distributed processing systems should be ephemeral, and can be turned “off” and “on” as needed, similarly to how servers are treated in the Cloud. Turning integrated solutions “on” and “off” is hard, and doing it at the scale of the enterprise is challenging. The Cloud allows us to do this, albeit with some really awesome and smart automation code that we have developed after years of hands-on experience.
The implications of productization data pipelines, storage systems, and analytics in a very simple manner are enormous. Common use cases include building line-of-business (“LOB”) specific data repositories with a diverse set of analytical applications; the ability to go-to-market with different customers with different scale and price points; the ability to replicate entire environments for development and testing purposes; and finally for disaster recovery just in case something goes wrong. What this really offers is choice, flexibility, and durability.
The go-to-market at a variable scale and price point was particularly important for one of our customers who needed an entire storage and analysis environment available in minutes. The scope of capabilities, the size of storage, and the need to adjust pricing were all just variables that could be tuned to meet specific sales and business needs. In essence, B23 built a software “factory” for stamping out their product in a tunable manner to meet customer demands. Soon after productizing their analytics solution, they began to offer various “tiers” of capabilities at different prices that led to quickly winning several new Fortune 500 companies within a six-month period. Their ability to acquire customers quickly and maneuver on pricing relative to their competition led to extremely positive financial returns, and ultimately a successful acquisition.
In Part II we will discuss the Tools of the Modern Data Economy.