One of the most frustrating processes in evaluating security products is the proof of concept (POC). Call it a POC, bakeoff, evaluation, proof of value (POV), trial, whatever, it’s resource intensive and if you are evaluating a new product against an existing one or multiple net new solutions at once or over time, it’s slow and it’s problematic to have an apples-to-apples comparison.
Once you’ve done your homework which might include talking to industry analysts, reading third-party reviews, producing RFIs/RFPs, talking with references and reading through websites, it might be time to buy or it might be time for a POC. Regardless of endpoint security controls, firewalls, IPS, SIEMs, DLPs and related solutions, quality security vendors will want their solutions squarely compared to incumbents or new competitors in a thorough but fair evaluation.
Wouldn’t it be nice to know exactly how a new control will work when compared to other controls, in your environment, under the assault of real attacks, while integrated with your security management solutions - all in as little as an hour?
There are many great security vendors that welcome the evaluation of their solutions. If your vendor doesn’t afford this option, that could be a warning sign. A POC gives you a feel for how easy or hard it will be to deploy, configure, integrate and use the product in your environment. It also shows you the reality of the product as sometimes, not always, but sometimes, marketing and sales stretch the art of the possible. But how do you make your testing and evaluation fair across the various vendors and more importantly do it quickly, easily and thoroughly?
Leveraging security solutions that allow for security control assurance testing can prove to be very helpful during POCs. Solutions like these, often called security instrumentation solutions, are more commonly known to be used when evaluating your existing security controls to address questions like:
Are my incident prevention controls preventing attacks? Are my incident detection controls detecting attacks? Are my SIEMs and log management solutions collecting and correlating on these alerts? Is my security team prepared to respond? Are my processes designed to be efficient and effective? More simply put is my security stuff working the way I hope, pray and assume it should?But this same level of scrutiny can be applied during POCs. For example, you can evaluate the capabilities of an endpoint control, firewall, etc. by safely executing attacks across the security controls to see if they block, alert, etc., and further, if the alerts show up in the solution’s management console, then further do those events show up in your SIEM.
You can run a variety of evaluations across endpoints and networks such as: malware execution, CLI attacks, PowerShell attacks, tunneling, data exfiltration, SQL Injection and C&C traffic. These attacks can be safely executed across everything from existing security controls within the production environment to security controls deployed in a lab environment.
The results of the testing will paint an apples-to-apples comparison of how these security controls preformed in the face of a number of identical attacks. Did they block the attack, did they detect the attack, were they able to log that information to a SIEM, if it was logged to the SIEM was the information valuable and usable, etc?
Now regardless of you evaluating the capabilities of your existing security controls against new controls, or multiple new controls against each other, you are armed with a valuable assurance testing solution that can yield results fast. These security instrumentation solutions allow you to quickly, easily and thoroughly evaluate security controls and with empiric evidence, know how those controls integrate with other security controls like SIEMs. Solid endpoint and network security vendors will welcome this level of analysis. Other vendors may not want you to know this option even exists.
Now how great is it to know exactly how a new control will work when compared to other controls, in your environment, under the assault of real attacks, while integrated with your security management solutions - all in as little as an hour?
This article is published as part of the IDG Contributor Network.Want to Join?