Solution provider’s takeaway: When you come across the different compatibility issues that may occur during a vSphere upgrade, such as hardware or third-party applications, it’s important to be ready. Use this chapter excerpt to know the ins and outs of preparing for a vSphere upgrade.
If you have an existing VI3 environment, at some point you’ll probably want to upgrade it to vSphere. Before jumping right into the upgrade process, though, there are many considerations and requirements that you should be aware of. Once you are aware of everything you need to know, you should then put together a plan for how you are going to proceed. Upgrading to vSphere is fairly straightforward, but there are many gotchas that can make the process more difficult. To avoid surprises during the upgrade, you should properly prepare and know all the steps so that your upgrade is trouble-free and uneventful. In this chapter, we will cover considerations and steps for upgrading your existing virtual environment to vSphere.
There are many things to consider when upgrading your VI3 environment to vSphere, such as hardware and software compatibility and upgrade methods. You should spend some time researching this to ensure that you have all your bases covered beforehand. Finding out after you upgrade that some of your management tools are not compatible with vSphere can make things very difficult. Upgrading is a much simpler process than downgrading, so make sure you consider everything before beginning your upgrade.
Your server and storage hardware may be supported in VI3, but don’t assume that it’s supported in vSphere. Check VMware’s online Hardware Compatibility Guide to make sure all your hardware components are supported in vSphere. This includes servers, I/O adapters, and storage devices. You may be able to get away with using servers that are not listed in the guide, but it’s critical that your I/O adapters and storage are listed. Refer to the Importance of the Hardware Compatibility Guide section in Chapter 11 for more information on this. The other consideration that you need to be aware of in regard to hardware is the requirement for 64-bit hardware. See the section Selecting Physical Host Hardware to Use with vSphere in Chapter 2 for more information on this.
You should also be aware that the following features in vSphere require very specific hardware.
- Hardware iSCSI—Very few hardware iSCSI initiators are supported, and most of them are based on the QLogic adapters. Be sure to check the Hardware Compatibility Guide before using one, because if it is not listed, vSphere will see it as a network adapter instead of a storage adapter.
- Fault Tolerance (FT)—This requires very specific CPU families from Intel and AMD. See VMware Knowledge Base article 1008027 (http://kb.vmware.com/kb/1008027) for a list of supported CPUs. You can read more about the FT feature in Chapter 10.
- VMDirectPath—This requires specific chipset technology from Intel and
AMD that supports Intel VT-d or the AMD I/O Memory Management Unit (IOMMU). You can read more about VMDirectPath in Chapter 2. Intel VT-d has been available for some time, but AMD finally released IOMMU in the HP ProLiant G7 servers.
Software and database compatibility
When upgrading vCenter Server you should be aware that the requirements have changed in vSphere and some older operating systems and databases are no longer supported. vCenter Server 4.0 no longer supports SQL Server 2000 and requires either SQL Server 2005 or 2008. Consequently, if your vCenter Server 2.5 is using a SQL Server 2000 database, it will complicate the upgrade process.
Migrating the database to SQL Server 2005, which is supported with both vCenter Server 2.5 and 4.0, is the easiest method. If you plan to use SQL Server 2008, though, because vCenter Server 2.5 does not support SQL Server 2008 you will have to shut down the vCenter Server, migrate the database to SQL Server 2008, change the ODBC data sources, and then install vCenter Server 4.0. If you are using the built-in MSDE database with vCenter Server 2.x, it will automatically be upgraded to SQL Server 2005 Express. Optionally, you can migrate it to another supported database format before upgrading.
In addition, beginning with vSphere 4.1, vCenter Server is only supported on a 64-bit Windows operating system. So, if you do not have a 64-bit Windows OS, you must use vSphere 4.0 instead. See Chapter 11 for more information on this. Also be sure to look at VMware’s Compatibility Matrix (www.vmware.com/pdf/vsphere4/r40/vsp_compatibility_matrix.pdf) to learn more regarding compatibility of the various vSphere components.
Third-party application compatibility
If you are using any third-party applications (e.g., backup, management) with vSphere, make sure you check to see if they are supported in vSphere. A newer version may be available that is supported in vSphere. Also, if you are using vendor-supplied hardware management agents running inside your ESX Service Console, check for a newer version of them that supports vSphere. Using older versions can cause hard crashes of your ESX server.
VMware product compatibility
When vSphere first came out many of the VMware companion products did not support it yet. Products such as Lab Manager, View, and SRM only worked with VI3 and required newer releases before they supported vSphere. Most of those products now support vSphere, but it’s always best to check first, especially if you are using a newer version of vSphere, such as 4.1. VMware publishes a Software Compatibility Matrix (http://partnerweb.vmware.com/comp_guide/docs/vSphere_Comp_Matrix.pdf) that you can use as a reference.
Planning an upgrade
Careful planning will make your upgrade go much more smoothly; without a solid plan your upgrade could turn into a nightmare. There are several methods that you can use to upgrade your environment, and the one you use will depend on several factors, such as acceptable downtime and disruption to the environment, how much extra capacity you have, and whether you are using new hardware. The upgrade to vSphere has three main phases that are done in sequential order, so you need to keep this in mind when planning your upgrade.
There is a definite order to follow when upgrading your environment. Think of the upgrade process as a pyramid, with vCenter Server being at the top, ESX and ESXi hosts in the middle, and virtual machines (VMs) at the bottom, as shown in Figure 12.1.
Figure 12.1 Upgrade order of a virtual environment, starting with vCenter Server
As a precursor, though, you should ensure that you identify and plan on any hardware upgrades that you may need to make to support vSphere. This could include memory upgrades, adding supported NICs and storage controllers, and upgrading hardware firmware levels. Additionally, you should plan to upgrade any third-party tools and agents at whatever point makes sense depending on their backward-compatibility support. VI3 and vSphere are backward-compatible, but not forward-compatible. For example, a vSphere 4 vCenter Server can manage VI3 VMs, but a VI3 vCenter Server cannot manage vSphere 4 hosts. The same holds true with the client: The vSphere Client 4 can be used with the VI3 vCenter Server and host, but the VI3 Client cannot be used with vSphere 4. All of these compatibilities are listed in VMware’s Compatibility Matrix (www.vmware.com/pdf/vsphere4/r40/vsp_compatibility_matrix.pdf). Here is more information on each upgrade phase.
- Upgrade Phase 1, vCenter Server—This is where you should start your upgrade, as vCenter Server is at the top of the compatibility pyramid. Once you upgrade your vCenter Server to vSphere, you must upgrade your VI3 Clients to the new vSphere Client. If you try to access a vSphere host or vCenter Server with a VI3 Client, you will be prompted that you must upgrade it first. Once you upgrade your vCenter Server, you can proceed with upgrading the rest of your environment.
- Upgrade Phase 2, ESX and ESXi hosts—You will most likely not be upgrading all your hosts at once, unless you have a small environment. vSphere hosts and VI3 hosts can coexist in the same cluster, and features such as VMotion and High Availability (HA) will still work. But although they can coexist, you should try to minimize the amount of time in which you have a mixed environment, because there are more risks of issues due to differences between the versions.
- Upgrade Phase 3, Virtual Machines—Upgrading VMs consists of upgrading the virtual hardware from version 4 that is used in VI3 to version 7 that is used in vSphere. In addition, you should upgrade VMware Tools on each VM to whatever vSphere version you are running on your host. However, if you are running a mixed environment of hosts and there is a possibility of a VM moving from a vSphere host to a VI3 host due to a Distributed Resource Scheduler (DRS) or HA event, you should not upgrade the virtual hardware and VMware Tools to version 7. You should instead wait until all your hosts are upgraded to vSphere, because although a vSphere host can support either version, a VI3 host can support only the VI3 version of virtual hardware and VMware Tools.
There are several ways you can upgrade your environment. Which one you use will depend on the following factors:
- Downtime—How much VM downtime you are willing to experience
- Capacity—Whether you have enough extra host capacity to move VMs around on hosts so that you can shut hosts down
- Hardware—Whether you are using new hardware for your vSphere hosts or reusing your current VI3 hardware
These factors will help determine whether you should do a fresh install or an in-place or migration upgrade when upgrading to vSphere.
Upgrade or fresh install
One decision you will have to make is whether to upgrade your existing hosts and vCenter Server or start fresh with a new installation. There are pros and cons to each method. For instance, a fresh install ensures that your hosts and vCenter Server are cleaner, with no residual files that you may have collected from the previous version. However, a fresh install requires that you reconfigure settings and other things that may have been wiped out. For ESX and ESXi hosts, you must reconfigure such things as your virtual networking, local user accounts, and DNS, time, security, and advanced settings. If you have any scripts or agents in the ESX Service Console, you must also reinstall them.
Your VMFS datastores will not get overwritten unless you choose to do so, so all your VMs will remain intact once the host is upgraded. Optionally, if you have hosts with spare capacity, you can cold-migrate or VMotion the VMs to those hosts while you perform the upgrade. If you have simple virtual networks and are mostly using default settings on your hosts, a fresh install might make more sense.
For vCenter Server, if you perform a fresh install you will lose all the configuration settings that are unique to vCenter Server, such as clusters, DRS, HA, roles, and permissions, as well as all historical performance statistics for hosts and VMs. This information is stored in the vCenter Server database, which can get quite large over time. For this reason, many people like to start with a fresh database so that all the old data in the database does not carry over to the new server. As part of the upgrade, the vCenter Server database schema is modified and elements such as tables, views, and stored procedures are updated. Again, if your environment is smaller and you don’t mind losing your old performance data and reconfiguring things, a fresh install might be the way to go.
The process for this is fairly straightforward. You just remove your hosts from vCenter Server, shut down vCenter Server and install a fresh copy with a fresh database, and then add your hosts back into vCenter Server and reconfigure your settings. Optionally, you can also build a new vCenter Server and leave your existing vCenter Server in place, and then migrate hosts to it one by one.
The disadvantage of fresh installs is that you have more downtime and disruption in your virtual environment and you have to reconfigure all your settings. There are typically a lot more settings in vCenter Server than in ESX and ESXi, and many people do not want to lose their performance data, so many will do fresh installs for ESX and ESXi hosts but not for vCenter Server. If you’ve upgraded your hosts and vCenter Servers several times in the past, you might want to take advantage of fresh installs to get rid of all the crud that may have carried over each time and was never cleaned up. If you do choose to do a fresh install, make sure you document all your settings so that you know what to reconfigure afterward.
In-place upgrade or migration upgrade
There are two methods for upgrading hosts and vCenter Servers. You can choose to upgrade them in-place, or build a new environment and migrate your VMs to it. The decision here is highly dependent on whether you have extra hardware available, or if you have enough spare capacity on your hosts to hold your VMs while you shut down the hosts. If you run vCenter Server as a VM, you don’t have to worry about extra hardware for that, but you’ll need extra host hardware or enough spare capacity.
There are two ways to do a migration: with a new vCenter Server or using an existing vCenter Server. To migrate with a new vCenter Server, you set up a new vSphere environment with a new vCenter Server, and then configure clusters and other settings and move hosts from the old vCenter Server to the new one. You can do all of this while the host is running, without incurring any downtime for the VMs. To upgrade the hosts, you will need to move the VMs off of them or shut them down while you perform the upgrade. Here are some methods you can use to migrate to a new environment with a new vCenter Server. These methods all require shared storage that all hosts can access.
The following method moves 3.x hosts with VMs to vCenter Server 4.0.
1. Build a new vCenter Server 4.0.
2. Disable HA and DRS on the 2.x vCenter Server (unless you have plenty of
3. Configure clusters and settings on the new vCenter Server 4.0.
4. Disconnect a 3.x host from the vCenter Server 2.x.
5. Add the 3.x host to the new vCenter Server 4.0.
6. Continue the process until all 3.x hosts are moved to the new vCenter
7. Shut down the 3.x hosts and upgrade them to vSphere 4.0. If you have enough capacity, you can do this as a rolling upgrade to reduce or eliminate VM downtime.
The following method moves 3.x hosts without VMs to vCenter Server 4.0.
1. Build a new vCenter Server 4.0.
2. Disable HA and DRS on the vCenter Server 2.x (unless you have plenty of
3. Configure clusters and settings on the new vCenter Server 4.0.
4. Move all VMs from the 3.x host to other hosts in the cluster and disconnect
the 3.x host from the 2.x vCenter Server.
5. Rebuild the 3.x host with vSphere 4.0.
6. Add the new 4.0 host to the 4.0 vCenter Server.
7. Remove VMs from inventory (you don’t need to shut them down) on 3.x
hosts managed by the vCenter Server 2.x.
8. Add VMs to 4.0 hosts managed by the vCenter Server 4.0.
9. Repeat until all 3.x hosts have been upgraded.
These methods become easier if you have new or unused existing server hardware onto which you can install ESX or ESXi 4.0. You would then add the new hosts to the new vCenter Server 4.0 and just move the VMs to the new hosts without shutting down the 3.x hosts in the old environment. The other methods involve having only one environment with only one vCenter Server and are considered an in-place migration. These methods use a single vCenter Server that has been upgraded to vSphere 4.0.
The following method moves VMs to other hosts while upgrading.
1. Move VMs from a 3.x host to other 3.x hosts in the same cluster.
2. Shut down the vacated 3.x host and upgrade it to vSphere 4.0.
3. Power on the new 4.x host.
4. Move VMs from the 3.x host to the new 4.0 host.
5. Continue the process until all hosts are upgraded to vSphere 4.0.
The following method shuts down VMs on the hosts while upgrading.
1. Shut down the VMs on the 3.x host.
2. Shut down the 3.x host and upgrade it to vSphere 4.0.
3. Power on the new 4.0 host.
4. Continue the process until all the hosts are upgraded to vSphere 4.0.
To move VMs from one host to another while they are powered on, you can use the VMotion feature if the VMs are on shared storage. For VMs on local storage you can either shut them down and cold-migrate them, or use Storage VMotion (SVMotion) and VMotion together if you have shared storage to move them to other hosts. Using SVMotion to move VMs on local storage to other hosts is a multistep process, but it does avoid downtime. Here’s how to do it.
1. Use SVMotion to move the VM from local storage on Host A to shared
storage on Host A.
2. Use VMotion to move the VM from Host A to Host B on the same shared
3. Use SVMotion to move the VM from shared storage on Host B to local
storage on Host B (or keep it on shared storage).
Once you decide on a method that meets your requirements, you can begin the upgrade of your virtual environment to vSphere.
Once you are ready to begin upgrading your environment, you should try to test the upgrade process first so that you are comfortable with it. If you have a dedicated test virtual environment, that is the perfect place to start. If not, and you have some extra hardware, try downloading an evaluation copy of VI3, installing it, and then upgrading it to vSphere. Even if you don’t do this, you can install vCenter Server 2.x on a VM and practice upgrading it to vSphere. Otherwise, you might try the upgrade first on noncritical hosts that you can afford to have more downtime with in case you run into problems with the upgrade. Read the vSphere Upgrade Guide on VMware’s website for the version that you are using; for 4.0 Update 1 the guide is located at http://vmware.com/pdf/vsphere4/r40_u1/vsp_40_u1_upgrade_guide.pdf. Pay very careful attention to the database steps when upgrading vCenter Server as this can be the trickiest part of the upgrade.
Rolling back to previous versions
It is possible, but not easy, to roll back to previous versions once you have upgraded them. Therefore, before you upgrade a host, vCenter Server, or VM, be absolutely sure you are ready to do it.
Rolling Back vCenter Server
For vCenter Server, it is critical to back up the SQL database before you upgrade; otherwise, once you upgrade it, you cannot go back to the old database schema. Here are the steps for rolling back vCenter Server to a previous version.
1. Completely uninstall vCenter Server 4.0.
2. Restore the vCenter Server 2.x SQL database.
3. Install vCenter Server 2.x and tell it to use an existing database, and select
your restored 2.x SQL database.
4. Reconfigure your license server with your 3.x license files.
Rolling Back ESX Hosts
Rolling back to a previous ESX version can be tricky, but it is possible. In most cases, it is easier to just reinstall ESX 3.x. Otherwise, if you want to roll back, here are the steps.
1. In the ESX Service Console, run the /usr/sbin/rollback–to–esx3 command which reconfigures the bootloader to boot the previous ESX 3.x version. The –f parameter forces the rollback and suppresses the confirmation message. Once you run the command, you can no longer boot to ESX 4.0.
2. Reboot the host and it will boot to ESX 3.x.
3. Once the host boots to ESX 3.x, delete the ESX 4.0 Service Console VM folder (esxconsole–<UUID>) from the VMFS datastore.
Rolling Back ESXi Hosts
Rolling back to a previous ESXi version is much simpler because it is stored as a single image and ESXi always saves a copy of the previous version image whenever upgrading. Only one previous build is ever stored, and once you revert back to a previous version, it is irreversible; to go to a newer version you must reinstall it. The steps for reverting back to a previous ESXi version are as follows.
1. Reboot the ESXi 4.0 host.
2. When you see the page that displays the current boot build, press Shift-r to select the standby build.
3. Press Shift-y to confirm the selection, and press Enter.
Rolling Back VMs
For VMs, if you upgrade their virtual hardware from version 4 that is used by VI3 to version 7 that is used by vSphere, be aware that this is irreversible. If you snapshot the VM before upgrading it, it is possible to roll back if you revert to the snapshot. There is also a workaround to go back to version 4 using vCenter Converter, as outlined in the following steps.
1. Install vCenter Converter on a workstation and run it.
2. Run the Convert Machine Wizard. For the source, specify a VM type and a vCenter Server/host to connect to on which the VM is located.
3. For the destination type, select a VM and choose the same vCenter Server/host.
4. On the Host/Resource page, give the VM a different name and choose the VM hardware 4 version.
5. Once the process completes, power on the new VM, verify that it works, and then delete the original VM. Once you delete the original VM, rename the new VM to the original VM’s name.
Before you upgrade any part of your virtual environment there is a pre-upgrade checklist that you should use to ensure that you are ready and have covered all your bases. VMware has published the complete checklist on its website (www.vmware.com/files/pdf/vsphere-migration-prerequisites-checklist.pdf), but here are some of the most important items.
- Be sure all your other VMware products and third-party products are compatible with the vSphere version that you are installing.
- Be sure all your server hardware, I/O devices, and storage devices are listed in the Hardware Compatibility Guide for the version of vSphere that you are installing.
- Ensure that the physical or virtual hardware for vCenter Server is sufficient. Although vCenter Server 2.x could get away with one CPU and 1GB of RAM, vCenter 4.0 requires two CPUs and at least 2GB of RAM (preferably 3GB) as the Tomcat service in vCenter Server 4.0 uses much more RAM than in 2.x.
- Ensure that your database is supported; MSDE, SQL Server 2000, and Oracle 9i are no longer supported in vCenter Server 4.0. Upgrade the database to a supported version before upgrading vCenter Server.
- For vCenter Server 4.0, make sure you use a 32-bit ODBC data source; for vCenter Server 4.1, you must use a 64-bit data source.
- If the database is 64-bit Oracle, make sure the default installation path of C:\Program Files(x86) is changed to remove the parentheses ().
- Ensure that for the Microsoft SQL database, the system DSN is using the SQL Native Client driver. You may have to manually install this.
- Ensure that the Oracle and Microsoft SQL databases have the appropriate permissions (MS SQL requires the db_owner privilege on the MSDB and vCenter database).
- Confirm that the vCenter Server system name is no more than 15 characters long.
- Ensure that ports 80, 443, 389, and 686 are not used by any existing application on the vCenter Server system and that there are no firewalls (including Windows firewalls) preventing these ports from/to the vCenter Server system.
- Ensure that you have taken a complete backup of vCenter Server, the vCenter database, templates in the vCenter repository, license files, and SSL certificate files before the install or upgrade.
- Ensure that there is either a local VMFS or a shared VMFS volume with at least 10GB of free space to store the ESX 4 Service Console .vmdk file.
- Ensure that you have backed up your ESX host (service console files, .vmx files, custom scripts, host configuration files, and local VMFS filesystem).
- Ensure that you have backed up your ESXi host (using VI CLI and the vicfg–cfgbackup command).
- For ESX hosts, if you are using any hardware management agents inside the Service Console, make sure you upgrade to the latest version. Also ensure that any third-party agents, scripts, or software is upgraded to the latest version.
- Ensure that there are no suspend files for a VM in order to do the VM hardware upgrade.
- Ensure that the VM has a CD-ROM device configured in order for VMware Tools to mount the virtual ISO and install/upgrade VMware Tools.
- Ensure that it is okay to upgrade the VM hardware from version 4 to version 7. Once upgraded, you cannot revert back to an earlier VM format unless you have created a snapshot of the VM prior to the changes.
- Ensure that the VMs are backed up before upgrading them.
- Ensure that you have the necessary licenses for the required features of VMware vSphere. The evaluation license is valid for 60 days after you power on the ESX/ESXi host.
- Ensure that you have a backup copy of the existing VI3 License Server license files. Once you are sure you meet all the necessary prerequisites, you’re ready to begin upgrading your environment.
Eric Siebert is a 25-year IT veteran whose primary focus is VMware virtualization and Windows server administration. He is one of the 300 vExperts named by VMware Inc. for 2009. He is the author of the book VI3 Implementation and Administration and a frequent TechTarget contributor. In addition, he maintains vSphere-land.com, a VMware information site.
Printed with permission from Pearson Publishing. Copyright 2010. Maximum vSphere: Tips, How-Tos, and Best Practices for Working with VMware vSphere 4 by Eric Siebert. For more information about this title and other similar books, please visit http://www.pearsonhighered.com.
This was first published in April 2011