VMware ESX Server installation

To tackle a VMware ESX Server installation, you must first know Linux. Get VMware installation scenarios in this excerpt from Virtualization: From the Desktop to the Enterprise.


IT reseller takeaway: You must know Linux to take advantage of VMware's ESX Server. Once you've got that down, follow the instructions for preparing an ESX Server installation outlined in this excerpt from Virtualization: From the Desktop to the Enterprise.

We'll spend a bit more time on ESX Server than the other VM applications because it's a standalone product, which means an operating system isn't required to host the ESX Server virtualization application. ESX Server directly communicates with system hardware and is said to run on the metal. Additionally, we'll cover two configuration scenarios. The first scenario covers what might be involved with a troubled server installation, and the second scenario details what you'll experience with a typical configuration.

If you escaped having to use Linux operating systems, you'll have to learn them if you want to take advantage of the power and scalability ESX Server offers. If you're familiar with Red Hat Advanced Server 2.1, you'll feel right at home with VMware ESX Server 2.1. If you're not really familiar with Linux, we'll give you a crash course in survival commands later in this chapter. You will, however, need to get your hands on a good Linux book for further investigation, such as Beginning SUSE Linux: From Novice to Professional by Keir Thomas (Apress, 2005).

ESX Server can be expensive to implement and is designed for enterprise-class internetworking needs. Though you don't need to be in a Fortune 500 company to use ESX Server, you'll want to install as much RAM and as many processors as you can; in addition, you'll need SMP licensing to run guest VMs with multiple virtual processors. The host will need at least two RAID controllers, one for ESX Server and one for guest VMs. Preferably, you'll want to store your guest VMs on a SAN. The network throughput needs of your ESX Server will be no different from the number of guests you'll host plus the needs of ESX Server. You'll want to have at least two network adapters, but install as many as possible -- you'll want to take advantage of ESX Server's teaming features. This lofty rhetoric boils down to the following for best-practice minimums for ESX Server deployment:


  • Two SCSI RAID controllers (RAID 1 for the ESX Server host and RAID 5 for the ESX Server guests)
  • Four 1 GB NIC interfaces
  • Two 3 GHz processors
  • 4 GB of ECC DDR RAM
  • Five 15,000RPM 146 GB SCSI hard disks
  • 500–800 MHz FSB

Short of an extra RAID controller and a few NICs, you may notice that best-practice specifications aren't much different from what you'd specify for a new file server. At 60 percent utilization, you can expect to minimally run four to ten servers on the hardware listed. If you're bent on using IDE devices for ESX Server, be aware that you can store VMs only on IDE devices with VMFS. It isn't that ESX Server can't run on IDE devices; it's just that VMware forces you to host on SCSI devices. With SCSI devices, you're assured the best possible performance for your guest VMs. Though SATA is clearly as good as some SCSI drives, and though it doesn't have the heat problems associated with extremely high spindle speeds, you won't get the ESX Server VMkernel to start after an IDE-only install. If SCSI drives are cost prohibitive and you need to leverage existing IDE drives, such as test lab equipment, you can invest in a SCSIto- IDE bridge, such as the one available from Acard. It allows you to run IDE devices on a SCSI controller, so ESX Server will install just fine. Additionally, ESX readily installs on IDE Fiber Channel–attached storage devices.

Now that you know what you can expect from your hardware based on how it measures up to best-practice minimums, you can turn to the actual install. ESX Server has two installation modes, graphical and text. During the graphical mode installation of ESX Server, the setup wizard may utilize extremely low video resolution for your install. If this happens, you won't be able to view the contents of the entire install screen because online help is consuming a third of your viewable real estate. This will become a particular problem when it's time to enter your product serial numbers. You can close the help window by selecting Hide Help in the lower-left corner of your display. As a word of caution, don't close the help just to close it; if you keep it open, you'll be presented with helpful information that will answer many questions and refresh your memory on install topics.

VMware doesn't support running nested virtual machines, which are virtual machines running within virtual machines. The complexity introduced by nested virtual machines can cause complications with regard to the host's resources. Attempting to run a nested virtual machine will end in failure.

Though ESX Server is extremely quick to install and VMware recommends reading the entire installation guide, just skim the guide prior to installation. You'll find it especially helpful if you're upgrading an existing ESX Server, installing in text mode, or performing a scripted installation. We'll point out typical installation mistakes as we walk you through the setup process. Whether you use graphical or text mode installation, you'll have to learn to navigate the installation menus with a keyboard. The graphical install supports a mouse, but sometimes the install wizard won't correctly probe your hardware, only the keyboard functions. If you already jumped ahead and booted off the installation CD-ROM, you may have noticed the Red Hat logo during boot. If you're familiar with a typical Red Hat OS install, then you're already familiar with the navigation keys outlined in Table 5-1.

Table 5-1. Red Hat Linux Installation Navigation Keys

Key Function
Tab Navigates options
Space bar Selects current option
Enter Selects current option
Arrow keys Navigates options

With navigation out of the way, go ahead and boot from the CD-ROM. You may need to adjust your host machine's BIOS boot-device priority from a floppy disk or the hard drive to the CD-ROM. If you have newer hardware, you can often change the primary boot device by using the system's boot menu: generally it's a function key, such as F12.

Upon booting from the CD-ROM, you'll be presented with the first installation screen that requires you to choose a "noapic" and/or "text" installation. Normally, it's sufficient to just press Enter and use the default, APIC/graphical mode. Advanced Programmable Interrupt Controller (APIC) mode avoids interrupt conflicts at the time of installation. APIC mode is helpful to use because devices may fail to share interrupts gracefully. After installation, errors related to devices contending for the same interrupt are generally resolved because one of the devices is often dedicated to a virtual machine. If interrupt conflicts persist, you'll have to resolve the issues by reconfiguring the hardware IRQs, swapping cards between different bus slots, or removing the device altogether.

To install, follow these steps:

1. Once installation is underway, the installation wizard will present you with the welcome screen; click Next.

2. The Installation Options screen presents you with three choices:

  • Install Default: Complete ESX Server install, formatting installation hard disks
  • Install Custom: Complete ESX Server install, allowing for keyboard and mouse customization
  • Upgrade: Preserves existing data and upgrades the host software Since you're performing a new install and want to explore the available options, select Install Custom.

    3. The Keyboard customization screen allows you to select your exact keyboard (if listed), layout, and use of dead keys. When dead keys are pressed, no visible character output is generated; they're generally used to make special characters (characters requiring accent marks). If you're inclined to create passwords with nonprinting characters, you'll want to enable dead keys. For this install, choose the defaults.

    4. The Mouse customization screen allows you to select your mouse type and, if you want, emulate three buttons on a two-button mouse. Selecting the default is a good choice. You can always change these settings after installation.

    5. The License Agreement screen asks you to agree to the terms. Check the box; if you don't, you might as well turn off your computer now.

    6. To install guest operating systems, you'll have to obtain a serial number. You can request a demo license directly from VMware if you haven't purchased one. Enter your license number information in the fields provided. If you want to use virtual SMP in your guest systems, you'll have to obtain that license separately from the ESX Server license. Depending on your physical hardware, the SMP license will allow you to configure your guests with one to four processors. If you choose not to enter license information at the time of installation, that's fine. You'll be presented with a warning. You'll have to enter license information prior to installing guest virtual machines.

    7. On the Device Allocation screen, you need to dedicate and/or share your physical hardware between the ESX Server console and guest VMs. You'll also need to divvy up your SCSI controllers and network adapters between the host system and guest VMs. Remember that one network adapter must be dedicated to the Service Console. If you share this network adapter with VM guests, you won't be able to remotely administer the server.

    8. You'll be required to specify the amount of RAM to be reserved for the console. VMware makes suggestions on the amount of memory to be reserved based on the total number of guest VMs. You can select the minimum if you're going to host only a few VMs; however, to accommodate future needs and quell any issues regarding performance, select a memory reservation setting one or two steps higher than you need.

    If you didn't install two network adapters in the host computer, this is where your actions will catch up to you. To manage ESX Server through the Service Console, one Ethernet adapter must be dedicated for it to use. Once again, ESX Server minimally requires two network adapters: one for the Service Console and one (or more) for the guest VMs. Now is a good time to install any additional NICs you want to use.

    Tip: To gain remote command-line access to an ESX Server or a GSX Server Linux host machine, use secure shell (ssh). This is a lot like Telnet except that communications are encrypted. If you're running a Windows-based operating system, any number of ssh programs are available for use, such as PuTTY. PuTTY is a Telnet and ssh client written for Windows operating systems and supports ssh v2. You can download it at http://www.chiark.greenend.org.uk.

    If you insist on using one network adapter during your initial install, that's okay. ESX Server will automatically dedicate the only adapter to the Service Console. If you had multiple adapters, you'd be given a choice of how to assign each of the cards—to the console or to VMs. If you add more adapters after the install, don't reassign the original NIC dedicated to the console; ESX Server can experience trouble using the correct driver for the new Service Console network adapter.

    Though it isn't necessary to have multiple SCSI controllers, your VMs will experience better performance if you have one SCSI adapter dedicated to the Service Console and one (or many) dedicated to your VMs. If you have one SCSI adapter, you can share it with the console by dedicating it to the Virtual Machines option and selecting Shared with Service Console. The Service Console will be installed to the highlighted drive. It can be either an IDE drive or an SCSI drive. However, VMs must be installed on SCSI drive. The VMkernel will fail to start otherwise. When you're done allocating your hardware, click Next.

    9. If you receive a warning indicating that ESX Server has to initialize your hard drive, you'll lose all the data on the disk. Assuming you don't need any of the data, select Yes to initialize the drive. If you select No, that's the end of your install and the beginning of your data salvage project.

    10. On the Disk Partitioning Setup screen, you can choose Manual or Automatic partitioning. If you're experienced and comfortable with Linux-like operating system disk partitioning, select Manual and have at it. You'll be required to create a minimum of three partitions:

  • /boot partition: The mount point of the boot partition and location of the OS loader
  • swap partition: The space reserved for the memory swap file
  • / partition (read: root): The location of file system and Service Console files

    Assuming you're manually partitioning the system, you'll be limited to creating four primary partitions (three of which ESX Server will consume). In the fourth partition, you can create multiple logical partitions as your needs dictate.

    If you want a guaranteed way to have ESX Server running in no time, choose Automatic Partitioning. If you have a single SCSI device, sda, the autopartitioning process will create the partitions in Table 5-2.

    Table 5-2. The Six Partitions of the Autopartitioning Process

    Partition Label
    /dev/sda1 /boot
    /dev/sda2 /
    /dev/sda3 swap
    /dev/sda4 extended
    /dev/sda5 vmkcore
    /dev/sda6 vmfs2

    When selecting Automatic, you'll be presented with the option to save any existing VMFS partition. VMFS partitions are where VMs are stored. If you're reinstalling ESX Server and want to save preexisting VMs, select Remove All Partitions Except VMFS. Because all partitions will be removed during automatic partitioning, the ESX Server installation wizard will present you with a warning. Assuming you don't mind losing all the data stored on your hard drive(s), including manufacturer-installed utility/ management partitions, select Yes to continue.

    11. The Defining Disk Partitions screen follows the Disk Partitioning screen. You can either edit the choices made by the installation wizard or click Next to continue. Before you continue jumping through the install, let's take a moment to tweak a few partitions using best practices. Adjust your /boot partition from 50 MB to 100 MB. Then highlight /boot from the Mount Point column, click the Edit button, and enter 100 in the Size (MB) field. Leave all other options as they are, and click OK.

    The maximum any single swap partition can be is 2048 MB, so you may need to create multiple swap partitions to achieve your calculated partition size. In addition, don't create a partition less than 256 MB for performance reasons. Now, reconfigure the memory swap file and create more as needed: highlight Swap from the Type column, and select Edit. Change the size of the partition to two times that of your installed RAM. For instance, if you have 512MB installed, enter 1024 in the Size (MB) field. Leave all other options as they are, and select OK.

    12. With partitioning out of the way, the installation wizard provides you with the Network Configuration screen. We highly advise you to use a statically assigned IP address for your ESX Server. You really don't want to have to guess the IP address of the Service Console from week to week, so select Static IP and enter the required host information. Be sure to use an FQDN for the Hostname field (for example, www.apress.com). If you don't use an FQDN, you'll encounter host name resolution problems, and other systems will have difficulty finding your new ESX Server. After entering the requisite information, or if you're opting to use DHCP, click Next to move on with the installation.

    13. On the Time Zone Selection screen, select the correct city and location where the ESX Server will reside. Time synchronization between networked hosts is important for authentication and logging purposes. For instance, some two-factor authentication systems will fail to work if the two hosts involved have the incorrect time. If your system clock uses Universal Coordinated Time (UTC), select the System Clock Uses UTC check box.

    If you want your system to account for daylight saving time, click the UTC Offset tab. The Use Daylight Saving Time (U.S. Only) option is at the bottom-center of the screen; choose it if necessary, and then click Next.

    14. The Account Configuration screen is where the superuser/root (administrator) password is configured. Be sure to do the following:

    • Choose a password you can remember.
    • Use an alphanumeric sequence.
    • Create a password greater than the six-character minimum, with eight characters optimally.

    Security zealots may want to use nonprintable characters in passwords as well.

    Tip: If you have difficulty creating and remembering strong passwords, create a simple algorithm you can use on common words to generate solid passwords. For instance, you can substitute vowels for prime numbers and turn the word football into f12b3l. You can make your passwords even stronger by starting and ending them all the same way. For instance, you could start all passwords you create with a # and end them with a !; so football is now #f12b3l!, baseball is now #b1s2b3ll!, and password is now #p1ssw2rd!.

    The root user account allows for the complete administration of ESX Server. It's a powerful account and can do anything to ESX Server you want—unlike Windows OSs, the root administrative account can delete protected system files. Rather than using the root account for administration, you should use a separate account and employ the su command to gain root-level commands only when it's required. Though using the root account in a training or test environment to help avoid complications is conducive to learning, habitually operating ESX Server as the root user in a production environment will inevitable result in damage to the file system. This happens because the superuser account overrides system safeguards, and mistakes generally go unnoticed. During initial installation, create an additional account for general system usage by selecting Add on the Account Configuration screen. When you're done, click OK and then Next.

    15. The system is now prepared for installation. Click Next to begin the package delivery.

    16. The final installation screen points out some important information. It notes the location of the setup log files:

    • Install log: /tmp/install.log
    • Kickstart log (file containing preinstall choices): /root/anaconda-ks.cfg

    You can refer to the log files after the setup for troubleshooting or documentation purposes. Once the install begins, don't interrupt the process. If you do, you earned yourself a reinstall.

    Tip: Like the Blue Screen of Death, ESX Server has the Black Screen of Death. Any number of reasons can cause the system to black-screen, including incompatible hardware, interrupted installs, and bugs in the code itself. Make sure you do plenty of research before installing ESX Server to avoid black-screen headaches. For instance, if you're installing Citrix on ESX Server, you'll receive the Black Screen of Death if you don't disable the COM ports in the guest VM's BIOS. Moreover, you'll also experience the Purple Screen of Death in the event of serious hardware failures.

    If you hang out for the install, you can get an idea of what's being delivered to your system by scanning the package summary. In total, it takes only five to ten minutes to install and only about 700 MB of disk space. The Congratulations screen declares the installation is complete.

    Before selecting Next to reboot, let's take a look at what's going on behind the scenes. The installation process has several sessions running that you can view by pressing Ctrl+Alt while simultaneously pressing one of the 12 function keys (F1–F12). Cycle through the different sessions saving F12 for last, because it may reboot your system. If your system doesn't restart, press Ctrl+Alt+F7: this will return you to the Congratulations screen where you can click Next to reboot. On reboot, you'll see the LILO Boot Menu screen. If you do nothing, the menu timer will expire, and the system will start. If you're familiar with how Linux or Novell OSs boot, you'll feel right at home while ESX Server initializes all its necessary processes; it ends in the Welcome to VMware ESX Server screen.

    Now that the installation is complete, you're ready to configure ESX Server to host virtual machines. To configure the server, you'll need to have access to an additional networked computer with a Web browser. Is it beginning to feel a little like NetWare? VMware recommends that your Web browser be Internet Explorer 5.5 or 6.0, Netscape Navigator 7.0, or Mozilla 1.x.


    • Introduction: Installing and deploying VMs on enterprise servers
    • Installing Microsoft Virtual Server
    • Installing VMware GSX Server for Windows
    • Installing VMware GSX Server for Linux
    • Installing the VMware management interface
    • Working with the VMware virtual machine console
    • Changing GSX Server's remote console port number
    • Installing VMware ESX Server
    • Verifiying and viewing configuration files
    • Using Linux survival commands
    • Understanding MUI and SSL
    • Configuring the ESX Server installation: Troubleshooting an ESX Server installation
    • Configuring the ESX Server installation: A smooth ESX Server installation
    About the author   
    Chris Wolf is an instructor at ECPI Technical College, as well as a leading industry consultant in enterprise storage, virtualization solutions, and network infrastructure management. He has a master's degree in information technology from Rochester Institute of Technology, and his IT certification list includes MCSE, MCT, and CCNA. Wolf authored MCSE Supporting and Maintaining NT Server 4.0 Exam Cram, Windows 2000 Enterprise Storage Solutions and Troubleshooting Microsoft Technologies, and he contributes frequently to Redmond Magazine and Windows IT Pro Magazine. Wolf also speaks at computer conferences across the nation.

Dig Deeper on Server management, sales and installation