Keeping up with Hackers using Unsupervised Machine Learning
By Arie Fred, VP of Product, SecBI
When you read (or hear) statements like when “the longer detection systems are in place, the more effective they become” often points to the quote coming from someone at a threat detection company that relies upon alerts and anomalies. This may be a cyber professional who also uses machine learning, however only supervised machine learning. And why do I say that? Because they are referring to creating a baseline together with prolonged training of the algorithm to “predict” anomalies based on what your network looks like and how it behaves.
The word ‘baseline’ is the key give-away, as a baseline contradicts the paradigm in cybersecurity that we are facing today. Worse yet, using a baseline in the approach to threat detection doesn’t solve the overall challenge of staying ahead of sophisticated hackers. The better approach is to “assume compromise”, without using the term “compromise” as a doomsday scenario, as it has become our daily reality.
In organizations with tens of thousands of users, some might be infected with different sorts of malware/adware, some with crypto-mining (willingly or not). When you baseline, your training model includes these and all current behaviors, even before they actually have a name (crypto-Jacking was a behavior way before it had a name), and thus you might actually miss important findings.
In contrast, unsupervised machine learning with a cluster-based approach does not set nor relate to an already outdated baseline. The cluster analysis approach enables setting thresholds and analysis parameters to group unusual associated behaviors, leading to detection of attacks that typically remain uncovered when using a baseline.
Therefore, SecBI cyber experts may be heard saying, “hackers have already found a way around your detection systems, but now you have the optimal way to detect the malicious activity, and quickly, before any serious damage.”