The Evolution of Security Data Operations and the Rise of Modern SOC
Key Takeaways:
- The volume of security data has exploded, leading to data saturation in many SOCs.
- Modern SOCs rely on efficient data ingestion, normalization, and automation to improve threat detection and response.
- AI has potential in security operations but is still in its early stages of development.
- AWS Security Lake provides a more affordable solution for managing massive security data, but challenges remain in implementation.
- Securing critical infrastructure is a financial and technological challenge that requires greater focus and investment.
The convergence of security and data operations, commonly referred to as SecDataOps, is reshaping how organisations approach threat detection and response. With increasing data volumes, new security threats, and innovations like AI and data lakes, the need for modernized Security Operations Centres (SOCs) is more urgent than ever.
Our CEO & Founder, Simon Johnson, recently sat down with Neal Bridges, CISO at Query, to explore the key challenges and innovations shaping this field. Not only is Simon the CEO & Founder of HOOP Cyber, but he also has an impressive 25-year career in cyber security which spans across prominent organisations like Splunk, FireEye, Cisco, and Qualys, providing him with unique insights into the development and future of SecDataOps.
The Explosion of Security Data
One of the most significant changes Simon has witnessed is the explosion of security data. As organisations become increasingly digital and move to cloud-first strategies, the sheer volume of data they generate is overwhelming traditional security tools. SOC teams are drowning in data, creating a data saturation problem that makes it difficult for them to identify genuine threats amidst the noise.
Simon pointed out that many organizations ingest up to 30 terabytes of data daily for security monitoring, a number expected to grow. The problem lies not in gathering data but in making sense of it. Traditional tools struggle to efficiently process and analyse these massive datasets, leading to inefficiencies and potential security gaps.
The Need for a Modern SOC
This is where the concept of a modernized SOC comes into play. As Simon explained, a modern SOC is about more than just integrating the latest tools. It revolves around optimizing data ingestion, normalization, and analysis to enable faster, more accurate threat detection. The foundation of a modern SOC lies in three key pillars:
- Data Ingestion: Efficiently pulling data from various sources using pipeline tools such as AWS Firehose or solutions like Cribl or Tenzir. The goal is to ensure seamless data flow from all systems into one central platform.
- Data Normalization: Leveraging standards like the Open Cybersecurity Schema Framework (OCSF) to standardize data from disparate sources. This allows for more straightforward querying and analysis, making it easier to detect threats across the enterprise.
- Automation: Automating repetitive, low-value tasks is critical to improving SOC efficiency and alleviating the burnout many security teams face today. The introduction of automation platforms that use AI models, like Times and Torq, has made it easier for teams to create playbooks and automate responses without needing advanced coding knowledge.
The Role of AI in Security Operations
During the conversation with Neal, Simon naturally veered into the role AI will play in security operations. There’s a lot of hype around AI-powered “co-pilots” in the security space, with vendors promising that AI can revolutionize SOCs by helping teams detect and respond to threats faster. However, Simon expressed cautious optimism.
While AI shows promise, especially in assisting with data analysis and automating certain tasks, there’s still a long way to go before it becomes a game-changer. Many AI solutions today are not mature enough to handle the complexities of real-world threat detection, and their limitations need to be acknowledged. That said, Simon is confident that AI will become a core part of the modern SOC, especially in areas like detection-as-code and automating basic threat responses.
AWS Security Lake: The Future of Data Storage?
One of the most intriguing parts of the discussion was Simon’s deep dive into AWS Security Lake, Amazon’s new service that aims to simplify security data storage and processing. Security Lake offers a centralised place to store an organisation’s security data, utilizing the OCSF schema for standardized data formats.
The promise of AWS Security Lake is that it can reduce the costs associated with ingesting and querying vast amounts of data. Many organizations today are facing skyrocketing SIEM costs due to the amount of data they are processing. Security Lake aims to store this data more affordably while allowing organizations to choose how they query it.
However, Simon was quick to note that Security Lake is not without its challenges. It’s not a replacement for a SIEM, and organizations must still figure out how to query and analyse the data once it’s stored. Additionally, there are significant hurdles in ensuring that all data is ingested and normalized correctly, especially when dealing with non-AWS data sources.
The Realities of Critical Infrastructure Security
Another important topic Simon and Neal touched upon was the state of security in critical infrastructure. As we’ve seen in recent high-profile hacks of telcos and ISPs, the security of critical infrastructure is still severely lacking. One of the major issues is the legacy nature of much of this infrastructure, making it highly vulnerable to modern attacks.
Securing critical infrastructure is a complex challenge, not just from a technological standpoint but also from a financial one. Many critical infrastructure companies are for-profit, and cybersecurity is often seen as an additional expense rather than a necessity. This creates an environment where vulnerabilities persist, leaving essential services exposed to cyberattacks.
Conclusion: The Road Ahead for SecDataOps
As Simon and Neal wrapped up their conversation, one thing became clear: the evolution of SecDataOps is only just beginning. The challenges are immense, but so are the opportunities. With innovations like AWS Security Lake, OCSF, and AI automation, we’re seeing the potential to revolutionize security operations and build a more resilient future. However, these technologies need time to mature, and organisations must be prepared to invest in the skills and infrastructure required to fully realize their benefits.
As Simon noted, “the key to success lies in finding balance. Organisations need to balance innovation with practicality, data privacy with operational speed, and human expertise with automation. Only then will we be able to tackle the growing complexities of modern cyber security.”
For more information, visit https://www.query.ai/.
