Big data security analytics mantra: Collect and analyze everything

Sampling security data is no longer adequate or necessary – CISO mindsets are due for a change.

In a recent research survey, ESG asked security professionals to identify the most important type of data for use in malware detection and analysis (note: I am an employee of ESG). The responses were as follows:

  • 42% of security professionals said, "Firewall logs"
  • 28% of security professionals said, "IDS/IPS alerts"
  • 27% of security professionals said. "PC/laptop forensic data"
  • 23% of security professionals said, "IP packet capture"
  • 22% of security professionals said, "Server logs"

I understand this hierarchy from a historical perspective, but I contend that this list is no longer appropriate for several reasons. First of all, it is skewed toward the network perimeter which no longer makes sense in a mobile device/mobile user world. Second, it appears rooted in SIEM technology which was OK a few years ago, but we no longer want security technologies mandating what types of data we can and cannot collect and analyze.

Finally, this list has "old school" written all over it. We used to be limited by analytics platforms and the cost of storage, but this is no longer the case. Big data, cheap storage, and cloud-based storage services have altered the rules of the games from an analytics and economics perspective. The new mantra for security analytics should be, "collect and analyze everything."

What makes up "everything?" Meta data, security intelligence, identity information, transactions, emails, physical security systems everything!

Now, I know what you are thinking:

While it may seem like hype to our cynical cybersecurity community, big data is radically changing the way we look at the world we live in. For example, we no longer have to rely on data sampling and historical analysis, we can now collect and analyze volumes of data in real time. The sooner we incorporate this new reality into our cybersecurity strategies, the better.

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags softwareapplicationsdata miningbig data analyticsCisco Subnet

More about IPS

Show Comments