Next Generation Cybersecurity Analytics – Part I
With many recent high-profile cyber breaches, an overarching cybersecurity program is a critical business requirement, particularly in the financial services sector where customer trust is of utmost concern. B23 recently helped a customer implement a Next Generation Cybersecurity Analytic capability and we would like to share our experiences through a 3-part series of blog posts — “Next Generation Cybersecurity Analytics”. This first blog post will include the background and high level solution that B23 implemented. Our second follow-up blog post (Next Generation Cybersecurity Analytics – Part II) provides a technical overview of the software components, and the third post makes the case why a Next Generation Cybersecurity Analytics solution is required.
B23 has unique experience implementing large and complex Hadoop implementations. For this reason, B23 was selected by a S&P 500 financial services firm (the “Bank”) to enable a distributed cybersecurity analytics platform capable of handling peak ingest rates of 1 Petabyte (“PB”) every 66 minutes.
To understand and proactively address the emerging cyber threat, the Bank performed an internal business risk assessment of its cybersecurity vulnerabilities. This assessment quantified a monetary risk value which formed the basis of its return-on-investment (“ROI”) analysis to deploy a big data threat analysis platform.
As a result of the the risk assessment the Bank chose B23 to implement the Hortonworks Data Platform (“HDP”) as the basis for its threat analytics platform to address cybersecurity risks. The Bank also selected OpenSOC open-source framework for its cyber analytics solution. Originally released by Cisco, the OpenSOC framework helps organizations make big data part of their technical security strategy by providing a platform for the application of anomaly detection and incident forensics to the data loss problem.
<img class=”progressiveMedia-noscript js-progressiveMedia-inner” src=”https://cdn-images-1.medium.com/max/800/1*hjoB4Zi_MjasSo6fd9QCXw.png”>
By integrating numerous elements of the big data ecosystem, OpenSOC provides a scalable platform incorporating capabilities such as full-fidelity packet capture, indexing, storage, data enrichment, stream processing, batch processing, real-time search, and telemetry aggregation. It also provides real-time dashboards to effectively enable security analysts to rapidly detect and respond to advanced security threats.
With the goal of capturing every network packet inside the Bank’s enterprise, the team installed HDP with Ambari on a high performance cluster on the the Bank’s private network. The initial cluster size was 1.2 Petabytes (“PB”).
The B23 team implemented the OpenSOC PCAP topology within a relatively quick period of time (on the scale of weeks). B23 configured the OpenSOC dashboards to highlight GeoIP-tagged network flows processed in real-time on a world topology map. Within several minutes of “turning on” the threat analytics platform, the dashboard alerted bank security analysts of suspicious network behavior.
Notional Kibana Visualization
The first minute of operation uncovered previously unidentified BitTorrent network traffic flowing to high risk countries outside of the United States. After several days of analysis, many subsequent potential threats were identified and remedied. Based on their initial business risk assessment, the Bank concluded that the outcomes derived from the use of the Hadoop-based threat analytics platform paid for itself within days of operation. The Bank plans to evolve the capabilities of its threat analytics platform by incorporating anomaly detection with threshold analytics and tailored machine learning algorithms trained on their network flows to identify even more sophisticated cybersecurity threats.
In the next post we will give a detailed explanation of the OpenSOC software capabilities and the results and conclusions of the implementation.
#ArtificialIntelligence #BigData #Cybersecurity #DataScience