How New Relic Infrastructure Makes Zero-Day Vulnerabilities Suck Less

Zero-day vulnerabilities suck. Even worse, they suck for a lot of reasons. There is the obvious immediate risk to revenue , your customer s’ privacy , and your company’s reputation ―not to mention the high-pressure burden it places on the poor folks responsible for dealing with these issues.

Worst of all, the absolute suckiest part about zero-days: addressing them is a cumbersome, manual process straight out of 1995 .

How it begins

The situation often begins when you find out that a library or kernel version was affected and you’re forced to ask yourself, “Am I running that library, version, and/or kernel?” A simple enough question, but answering it can unfortunately require you to manually log into every single host to truly verify the results.

Sure, you can rely on configuration management tools like Chef , Puppet , and Ansible , or you can look at your sanctioned virtual machine images, but all that tells you is “What do I think I should be running?” Those tools can crash, they can miss hosts, and many large organization have “rogue” teams dancing around the sanctioned systems trying to run fast enough to keep up with industry trends and business needs.

Now consider that a new zero-day vulnerability is announced about once every week , so most end up getting ignored. Finally, remember that just one machine susceptible to the issue is all it takes to wreak some serious damage .

Well, we felt that pain here at New Relic, too. We were jolted into action when Heartbleed first dropped. And we slogged our way through POODLE , Shellshock , and Dirty COW . But now, thanks to the arrival of New Relic Infrastructure , we are happy to say that when the next zero-day vulnerability drops, we think things will be a whole lot easier for all of us in the New Relic family.

Inventory: Audit what is where in real time

The Infrastructure Inventory page is designed to provide a real-time, filterable, searchable view into each host’s configurations, packages, system modules, and more. You can use the Inventory page for practical tasks like ensuring a version update was applied successfully across all your hosts, auditing version discrepancies across your hosts, or the holy grail: quickly identifying which hosts require an update to fix a security vulnerability.

There. That’s the marketing story. But to see New Relic Infrastructure in action, let’s go back in time to April 2014 and pretend that we just found out about Heartbleed . Try to hold back the cold sweats as you reread the CVE details :

Overview

A vulnerability in OpenSSL could allow a remote attacker to expose sensitive data, possibly including user authentication credentials and secret keys, through incorrect memory handling in the TLS heartbeat extension.

Systems Affected OpenSSL 1.0.1 through 1.0.1f OpenSSL 1.0.2-beta

Now, imagine that you had installed New Relic Infrastructure. Instead of the clunky process of logging into all your hosts to figure out which ones are vulnerable, you simply log into New Relic Infrastructure, click on Inventory, and search ‘openssl’. You’ll immediately see a list of all versions of OpenSSL you have across your fleet and what hosts they’re running on. Update the infected hosts, go through the same process to verify the change actually took, and you’ve officially cauterized the Heartbleeding in a matter of minutes.

Searching openssl immediately displays all instances of this library broken down by version and hosts running them.

View the data in New Relic Insights

Also, remember that New Relic Infrastructure is designed to surface the data used to compose its interface inNew Relic Insights, so if you want to run custom queries against the data and export them to a CSV, it’s easy to do so. Here is a simple view of all hosts and their associated kernel versions:

To recap: the next time someone sends an urgent email about the latest zero-day vulnerability, take a breath and look for the ‘Versions Affected’ section in the CVE. Search your New Relic Infrastructure Inventory page or run a query in New Relic Insights, patch, and verify.

Then don’t forget to scribble a quick note to remind your manager how you saved the company millions in a few minutes come review season.

We certainly hope that your team gets a few more uninterrupted nights of sleep and a few less-stressful days the next time a zero-day vulnerability rears its ugly head!

See our Infrastructure Inventory documentation page .

Adam Larson contributed to this post.