Keeping your customer’s physical environment secure is more straightforward than dealing with security in a virtual environment. There are a number of hidden risks and concerns that solution providers need to be prepared for before fielding customer questions about vSphere security.
Virtualization expert Eric Siebert breaks down what you need to know about securing your customer’s vSphere environment, including Payment Card Industry Data Security Standard (PCI DSS) concerns, anti-virus software and ESX firewalls. Siebert also explains which third-party virtualization security products and vendors can be useful to solution providers.
How does security in virtual environments differ from physical environments?
Most of the security-hardening techniques that solution providers would normally use in physical environments apply to virtual environments as well. These techniques are typically used at the guest operating system (OS) level, which is no different in virtual environments. There ar e, however, other security areas that you need to be concerned with inside virtual environments that don’t exist with traditional physical servers.
Solution providers need to recognize that the host opens up more attack vectors inside virtual environments, with the biggest being toward the ESX Service Console and the ESXi Management Console. These consoles run as privileged virtual machines (VMs) on the host and hold the keys to accessing any VM on the host. There are a variety of methods that can be used to access a host, including Secure Shell, vSphere Client, scripting application programming interfaces (APIs) and Web browser access. All of these access points need to be properly secured to protect the host and its VMs.
Networking is another area in which security is much different compared to physical servers. Normally with physical servers, the network interface cards (NICs) are the physical endpoint to the network on the server and traditional physical networking security techniques using firewalls, routers and switches can be used. When using virtualization, the physical network is extended into the host that has its own virtual switches and NICs. Physical networking and security tools are not as effective with virtual networks because not all traffic leaves the host. As a result, tools that are designed for virtualization are needed to provide additional security to the virtual networks of the host.
Storage is another area of concern for solution providers. Because VMs are encapsulated into a single virtual disk file that makes them easily transportable, you need to put security controls in place to limit access to virtual data stores. This will prevent virtual disk files from being copied off by an attacker and accessed somewhere else.
What special concerns should solution providers be aware of when using virtualization when it comes to PCI DSS Compliance?
The older PCI DSS 1.2 version ignored virtualization for the most part; it didn’t prohibit using virtualization, but provided no real guidance or restrictions on securing it. PCI DSS 2.0contains some general virtualization guidance and clarifications. A key clarification was made in section 2.2.1, which previously said that you could only implement one function per server. If taken literally, it may seem as though virtualization was not allowed. Now section 2.2.1 has been updated to say when using virtualization you can only implement one function per virtual component (i.e. VM).
Although PCI DSS 1.2 mainly ignored virtualization, PCI DSS 2.0 essentially brings the whole virtualization layer into the scope of PCI DSS. Prior to the new release, only servers that dealt with cardholder data were considered in scope. Now, even if only one VM deals with cardholder data, all your virtual components are in scope because VMs are mobile and can easily move from one host to another.
There is also a lot of sharing associated with virtualization: VMs share hosts, storage devices and networks. PCI DSS 2.0 also doesn’t allow dev & test VMs to mix with production VMs on the same host and you must also segregate functions and networks with different security levels.
The PCI DSS 2.0 requirements will affect how you architect virtual environments and providing the proper segregation between all virtualization components will be critical to meeting the PCI DSS requirements. Reducing the scope of PCI DSS audits can make things vastly easier, which means building an isolated environment solely for VMs that handle cardholder data so they are not mixed in with all your other VMs that don’t handle it. You can read the full summary of the changes in PCI DSS 2.0 at PCISecurityStandards.org and also look at the companion document called Navigating PCI DSS that helps explain the intent of the requirements.
If an attacker compromises a single VM can they compromise all the other VMs on a host as well?
This is possible through traditional means by exploiting weaknesses such as OS vulnerabilities, but a VM cannot go through the hypervisor to gain access to another VM on the host. This is typically referred to as “Escaping the Cave,” where each VM is in its own cave and the hypervisor is outside and does not let any VMs out of their caves to prevent access to other caves. While this type of vulnerability is theoretically possible, the security design of vSphere makes it highly unlikely.
VSphere has a very good track record when it comes to security and has never experienced a situation in which a VM breached the hypervisor to gain control over other VMs. This type of vulnerability is possible in hosted virtualization platforms (non-bare metal) such as VMware Workstation because of vulnerabilities in the hosted OS, which is not designed specifically for virtualization.
What special security products are available for vSphere?
VMware released a new security product called vShield Zones, which was acquired when they purchased Blue Lane Technologies. VShield Zones’ initial release provided some basic virtual networking security, but wasn’t very well integrated with vCenter Server and did not enhance the VMsafe APIs. VShield 4.1, released in August 2010, fully supports the VMsafe APIs so you no longer need to have a vShield agent VM running inline to protect VMs on an isolated vSwitch. Instead of protection at the vSwitch level, vShield can now protect at the vNIC level and is no longer required to be inline.
VShield 4.1 was divided into a family of products that focus on different aspects of security in vSphere: vShield Zones, App, Edge and Endpoint. Zones and App provide levels two, three and four virtual network firewalling and traffic flow information. Edge is designed to work at the edge of internal networks to provide security and gateway services such as network address translation (NAT), virtual private network, (VPN) dynamic host configured protocol (DHCP) and Web load balancing to isolated VMs. Endpoint allows anti-virus and anti-malware software that normally runs individually on the guest OS of every VM to be offloaded to a dedicated security virtual appliance.
Additionally, there are many excellent third-party security products specifically designed to provide better security for virtual environments. Vendors such as Altor Networks, Catbird, HyTrust and Reflex Systems offer mature virtualization security products that have been around much longer than vShield has.
Do solution providers need to run anti-virus software on my host management console?
According to VMware’s Security Hardening Guide, ESX is less susceptible, but not immune, to viruses and following proper security best practices advises you not to install AV software on the service console. It isn’t possible to install AV software on ESXi, which has a much smaller management console and not a full OS.. Although you could install AV software on an ESX host, doing so may negatively impact the performance of the ESX host and all of the VMs that reside on it because of the resources that AV software typically uses. If you are forced to install AV software on an ESX host because of security requirements in your environment, you should make sure that you run a version that is designed for the version of Linux that the Service Console uses. Also try to minimize configuration to minimize the impact on the server’s resources.
What special considerations should I be aware of when running anti-virus software on VMs?
Because AV software can be very resource intensive, it’s critical to avoid having multiple VMs doing the same types of AV resource activities simultaneously. This includes installing AV updates as well as performing tasks such as a whole VM’s full-scheduled scans. It’s best to schedule these scans during off hours and stagger the start times so every VM doesn’t perform the operation at the same time. You risk creating very intensive I/O on the host if you don’t stagger them, which can bring all the VMs to a standstill. This is even more important in desktop virtualization because it typically has higher consolidation ratios.
A better method for solution providers is to offload the AV resource usage from each individual VM to a dedicated appliance using vShield Endpoint. Instead of a resource-intensive AV agent running on the VM, a small vShield Endpoint driver that is provided by VMware is loaded inside the guest OS of each VM that will be protected. A hardened-security VM supplied by a VMware AV partner runs on a secure vSwitch and is linked to the vShield Endpoint driver by the vShield Endpoint Loadable Kernel Module (LKM) that is running on the hypervisor. Only Trend Micro currently has a shipping product that will work will vShield Endpoint; other vendors like McAfee and Symantec have announced products also but have not delivered them yet.
What is the firewall that is built into ESX hosts used for and why does ESXi not have one?
ESX has a built-in firewall based on iptables that can be configured either through the vSphere Client (Configuration, Security Profile) or through the command-line Interface (esxcfg-firewall). This firewall is designed to only protect the ESX Service Console and provides no protection for VMs even if they are on the same vSwitch and network as the Service Console. The firewall is enabled by default and in most cases the rule configuration does not need to be changed but there are some exceptions if you want to allow certain services such as Simple Network Management Protocol or want to connect ESX to Network File System storage devices.
If you need to stop and start the firewall for troubleshooting purposes, you can also use the following command to stop it: Service firewall stop. Then to restart it using the Service firewall start command. It’s not recommended to disable your firewall for any length of time as it provides critical protection for your ESX host.
Because ESXi has a much smaller management console without a full OS running inside it, there is no built-in firewall. ESXi runs a limited set of well-known services that are automatically allowed and prevents the addition of further services that can be enabled in ESX. As a result of this, there is much less risk and VMware felt that an ESXi firewall was unnecessary.
How can I make my customer’s vSphere environment more secure?
VSphere is fairly secure by default, but can easily be compromised through a change of its configuration settings. Solution providers can also improve vSphere’s security and are recommended to follow the security guidelines that VMware published to provide the maximum security for a customer’s vSphere environment. There are also several additional third-party security guidelines that can help secure vSphere, such as those published by the Center for Internet Security and a recent joint-vendor white paper focused on PCI DSS 2.0. Products such as vShield and other third-party security products can provide additional vSphere add-on security.
About the expert
Eric Siebert is a 25-year IT veteran whose primary focus is VMware virtualization and Windows server administration. He is one of the 300 vExperts named by VMware Inc. for 2009. He is the author of the book VI3 Implementation and Administration and a frequent TechTarget contributor. In addition, he maintains vSphere-land.com, a VMware information site.
This was first published in March 2011