A vulnerability means different things to different people. For IT channel companies that offer security services, vulnerabilities provide a chance to demonstrate clear and tangible value. Such was the case for Heartbleed, a serious vulnerability in the OpenSSL cryptographic software library that allowed an attacker to steal sensitive data. The OpenSSL vulnerability, which was publicly disclosed in early April, has given managed service providers (MSPs) an opportunity to prove their worth and gain their customers' loyalty.
"There were some channel providers who were, let's say, agile enough that when a major incident came through they were able to detect it, respond to it and use that as a vehicle to engage more closely with their customers. They could provide short-term value and demonstrate why they should be the provider of choice," said Ryan Morris, principal consultant of Boulder, Colorado-based Morris Management Partners.
The most effective response [to Heartbleed] needed to be at the most individual point of contact.
principal consultant, Morris Management Partners
"It was a question of the agility factor because when you live in the security business, there's always a threat, but the question is, Are you able to recognize that soon enough, respond quickly enough and really deliver intensive service around that particular threat as fast as it needs to be delivered," Morris said. "It's not a question of are you good at security but, from an operational perspective, Are you agile enough?"
The challenges associated with the Heartbleed OpenSSL vulnerability presented a true test of a provider's agility. "It was a fairly pervasive vulnerability. The code that was affected had been in production for a couple years, so it was widely adopted," said Jay Keating, vice president of Managed Services for Kittery, Maine-based LogicsOne, a virtualization and cloud consulting and integration company.
In addition, "Information was very unclear as to whether a particular system was vulnerable. We could run three or four different assessment tools. One or two would come back with no problem, and one would say we have an issue. You have to assume the worst and get the thing updated as soon as you could," Keating said.
Successful MSPs prioritized patching for customers that had to meet regulatory requirements and for Internet-facing systems. These were updated within a couple days. "There was a period of mass patching, especially on websites, that took place right away," said Gregory Morawietz, CEO for Single Point of Contact, a Palo Alto, California-based managed service provider. "When [our customers] tell auditors they are updated within 24 hours, then we have to come through for them and remediate those problems in the time they specify. It's an agreement they have that we are in charge of enforcing."
Some systems took longer to patch, either because the vendor hadn't yet released a patch for Heartbleed or because the clients' teams were distributed and the update required a little more coordination. According to Keating, it took VMware about a week to release a patch. "Vendors were inconsistent with their updates," he said. "The assessment and information flow was pretty inconsistent, but aside from that all the patching was very simple. Once you knew where an issue lied, the update sequence was pretty simple."
Morawietz also commented on the need to wait for vendors to respond to the OpenSSL vulnerability. "Some of the services, like LogMeIn, control their SSL or their website, so they issued patches or they informed you that you need to upgrade your current version to a supported version. It was a lot of work that occurred in a frenzy," he said.
Has Heartbleed been fully migrated at your customers' sites?
Weeks after Heartbleed was publicly disclosed, analysis firm Netcraft reported that only 14% of sites affected by Heartbleed had taken the required steps to mitigate the problem.
That doesn't mean that the MSPs sat idly by while waiting for a patch. As soon as the vulnerability was announced, Morawietz said, "We started to look at what products our customers use that use OpenSSL with the [affected] version. ... We created a list of users, customers, companies, products and went through and did a spot-check on each company and said, Do you have these apps in your environment? Yes? OK, we need to upgrade and here's the plan. We had to go through an intense analysis of what people had so we could upgrade it."
This, according to Morris, is exactly where the channel shines. "The most effective response needed to be at the most individual point of contact," he said. The antivirus vendors provide tools and resources, "but they don't have the capacity to scale down to a one-to-one interaction servicing millions of customers. That's why the channel exists in the security space."
MSPs that delivered that one-to-one interaction strengthened their customer relationships. "Our customers were generally glad to have our services. This is when a shared service model really shines. For a customer with a narrow team to find the vulnerability on a broad array of technologies would've been a daunting task," Keating said. "For us to call proactively say we know where the vulnerability is, it's in these systems and you only have to sign off on the updates, and you could hear the sigh of relief on the other side. That was a good testament of the shared services model with an MSP."
Morris agreed. "From an agility point of view, some channel companies [handled the Heartbleed OpenSSL vulnerability] exceptionally well, and the output was not just that they solved a security challenge but that they cemented a relationship by showing value to a customer," he said.
Crystal Bedell is a freelance writer specializing in B2B technology.