Solution provider’s takeaway: Knowing how to move Logical Domains between CMT Servers can be valuable to VARs who need to find the best fit for their customer’s environment. In this chapter excerpt, find out how to use the Logical Domains Physical to Virtual (P2V) tool and Configuration Assistant.
3.5 Domain Mobility
Domains can be installed on one CMT server (the source host) and migrated to a different, compatible CMT server (the target host) for planned workload migration. Doing so can free up memory and CPUs on a server, or vacate it for planned maintenance. At the time of this writing, Logical Domains supported two forms of domain migration, both invoked by the ldm migrate command. The following command migrates a domain to the host at the specified address. The command will prompt for the root password of the control domain on the target system.
# ldm migrate ldg1 email@example.com
The ldm migrate command can be issued with the option -n to request a dry run that tests whether the migration is possible but does not actually migrate the domain.
In cold migration, the domain must be stopped. It must also be in the bound or inactive state on the source machine. Cold migration consists of verifying access to the domain’s I/O resources. In particular, the virtual disks must be accessible to both the source and the target. This step also ensures there are sufficient resources to bind the domain after migration, and then transmits the domain’s description to the target machine. Once the domain is migrated, the domain on the source machine is unbound (if it is currently bound to resources) and removed from the source domain configuration. Cold migration applies only to domains that are not running, but can be very helpful for planned migration of workloads that can tolerate an outage with a shutdown and reboot. It is very fast, as only descriptive information is transmitted between servers.
In warm migration, a running domain is suspended on the source machine and resumed on the target machine without a reboot. Domain execution is suspended while it is being migrated. This process may take a number of seconds or a few minutes, depending on domain memory size and network speed, and on whether the control domain has access to a cryptographic accelerator.
First, the domain managers on both source and target machines verify that sufficient capacity to bind the domain on the target is available. Then, the target system creates and binds a single-CPU version of the guest domain, and the source machine removes all but one CPU from the running domain. Next, the source machine suspends the domain and removes its last CPU. The domain’s memory contents and state are then compressed, encrypted (using the cryptographic accelerator if one is associated with the control domain), and transmitted from the source to the target.
Processing is multithreaded and takes advantage of the CPU threads in the control domain; it also exploits the cryptographic accelerator to ensure better performance. Domain memory contents are always encrypted before transmission, as it would be a significant security exposure to transmit a domain’s memory contents (which might include passwords or private information) as cleartext. The domain is then restored on the target system and its remaining CPUs added. The Oracle Solaris instance continues execution on the target machine with the same disk storage, IP addresses, and MAC addresses as before, and applications resume processing. Finally, the domain is unbound and removed from the source host’s domain configuration.
Warm migration is suitable for planned migration of any workload that can be paused for a number of seconds.
A live migration capability is anticipated, in which a migrating domain will be unresponsive for only a very small interval.
3.6 Physical to Virtual Conversion
Logical Domains provides a Physical to Virtual (P2V) tool to automate conversion of physical Oracle Solaris systems into guest Logical Domains. This tool moves file system contents from a physical server to a Logical Domain on the CMT server, and if necessary, replaces packages geared toward the sun4u architecture platform (all UltraSPARC and SPARC64 systems) with packages for thesun4v architecture provided by the CMT platform. The physical system can be running Solaris 8 or later on a sun4u system, or Solaris 10 running outside of a domain.
P2V migration consists of three phases, which are carried out under the control of the ldmp2v command:
- Collection: Runs on the physical machine and collects a file system image and configuration data using ufsdumpM or flarcreate. The resulting file can be transmitted to the target system’s control domain or stored on an NFS server that is available to both the physical system and the control domain.
- Preparation: Runs on the control domain of the target platform. It creates a guest domain, and restores the contents of the collected file system into virtual disks.
- Conversion: Runs on the control domain of the target platform. It upgrades the guest domain to prepare it to run as a domain. This process removes packages, replacing sun4u packages with their corresponding sun4v versions.
If the physical system is running Solaris 8 or Solaris 9, the P2V process installs the system image in a Solaris Container using a Solaris 8 Container or Solaris 9 Container under the guest domain’s Solaris 10 kernel. That practice lets the migrated image appear to be the same Solaris version as on the physical machine. The P2V process can optionally preserve the physical system’s network identity by reusing its MAC address.
The P2V process requires Solaris 10 system images to be available in either Solaris installation DVD or network install format, with the package SUNWldmp2v being installed in the control domain. The file /etc/ldmp2v.conf must be populated with variables indicating the names of the default virtual switch and virtual disk servers, and the type of disk back-ends to use for virtual disks. The P2V command must be made available to the physical system, either by NFS mount or by copying the command to a local disk.
Once these setup tasks are complete, a set of commands can be executed like those shown in the following example. In this case, sparcules indicates the login session on the physical server, and primary indicates the control domain login session. The collect phase writes a collected image to an NFS-mounted directory that is also accessed by the control domain. The preparation phase creates a domain and imports the collected file system image into virtual disks. The conversion phase boots the guest domain with an Oracle Solaris installation image to upgrade the system image to a software level that is compatible with the Logical Domains technology.
Once this process is complete, the guest domain has the same system identity and applications as it did on the original physical server. The amount of time to complete the process depends on the size of the file systems being copied and the network bandwidth available for its transmission to the control domain. The amount of administrator effort is dramatically reduced, with only a few commands needed to carry out a complete system migration.
3.7 Ease-of-Use Enhancements
The preceding examples manage Logical Domains with a command-line interface. This strategy is traditional and scriptable—but also requires prerequisite skills, can be error-prone, and is intimidating for the occasional user. It also does not scale well when many machines are involved.
These limitations can be addressed by using scripts and by making appropriate use of standards and automation in an enterprise. They are also addressed by the Logical Domains Configuration Assistant, which provides a graphical user interface (GUI) for the initial installation and configuration of Logical Domains software and guest domains on a server. The Configuration Assistant is a Java application that is provided with the Logical Domains install software. To start it, issue the following command:
$ java -jar /path/Configurator.jar
This command launches a GUI that steps the user through a set of panels, such as the one shown in Figure 3.4.
The GUI steps through a series of panels that let the user specify the number of domains to be created and the paths to their virtual disks. The last step lets the user save a script to run at a later time on one or more servers or, if the appropriate host name and password are provided, to run on a target CMT server. The Logical Domains package also includes the ldmconfig command, a text-mode tool that prompts the administrator for information about the domains and then generates the commands to build those domains according to best-practice naming conventions and a template of where the virtual disk back-ends are stored.
A more comprehensive solution is based on the Oracle Enterprise Manager Ops Center licensed product, described at http://www.sun.com/software/ products/opscenter/. This product provides full life-cycle support for provisioning, and managing systems in a data center from a single user interface and screen, for both real and virtual systems. Ops Center can create, delete, configure, boot, and shut down Logical Domains. It also provides monitoring and management, showing utilization charts for CPU, memory, and file systems. In addition, the Ops Center manages resource pools, permitting dynamic reallocation of resources based on policies, and provides a graphical interface for controlling domain migration. Figure 3.5 shows several physical nodes that are visible in the assets tab of the Ops Center, with details illustrating resources used by several domains.
Figure 3.4: Logical Domains Configuration Assistant
Figure 3.5: Oracle Enterprise Manager Ops Center Display for Logical Domains
3.8 Comparison with Oracle Solaris Containers
Oracle Solaris Containers and Logical Domains provide alternative and complementary methods to virtualize Solaris systems, so it is natural to compare them.
Solaris Containers provide ultra-lightweight OS virtualization with excellent isolation, security, observability, fast deployment, resource granularity, and native performance. However, because many Containers run on a single Solaris instance, they do not permit multiple kernel patch levels to be used. Solaris Containers also have a few restrictions, such as the inability to run an NFS server.
In contrast, Logical Domains are more like “business as usual” for system administrators and application planners. They are individually installed, patched, used, and managed, much like physical servers. Oracle Solaris can be installed in a domain via a JumpStart or by booting from a DVD device, just like on a physical server. Like Containers, domains can also be cloned from previously installed and customized instances. However, unlike Containers, a running domain can be migrated from one server to another even without rebooting the domain.
Logical Domains are available only on CMT servers, whereas Containers are available on any platform that supports Solaris. Also, because domains host full OS instances on dedicated CPUs and RAM, they have a larger resource footprint than Containers. Many more virtualized instances can be hosted on the same platform by using Containers than by using domains.
Solaris Containers and Logical Domains are complementary virtualization technologies. They can be combined without adding overhead by running Containers within domains to achieve the highest degree of flexible virtualization. Separate OS instances can be configured when different OS kernel levels are required, with each OS instance hosting many lightweight, virtualized Container environments.
Logical Domains provide low-cost, efficient virtualization on UltraSPARC T1, T2, and T2 Plus processors. Each domain is a separate, independent virtual machine with its own OS instance. Hardware resources are dedicated to each domain through a highly granular and dynamic resource allocation scheme.
Compared to other virtualization technologies, Logical Domains provide an extremely efficient hypervisor implementation that avoids the overhead inherent in traditional hypervisors, and avoids the license costs of commercial virtualization products.
Logical Domains is an ideal technology for migrating from older platforms to current hardware in a consolidated, energy-efficient platform. Oracle Solaris systems can be easily moved to a domain using the Logical Domains P2V tool, thereby leveraging Solaris and SPARC binary compatibility. Workloads from other operating systems and platforms can also be migrated to provide an efficient virtualized solution that reduces both energy costs and server sprawl.
Oracle VM Server for SPARC
This excerpt is from the book, ‘Oracle Solaris 10 System Virtualization Essentials’ by Jeff Victor, Jeff Savit, Gary Combs, Simon Hayler, Bob Netherton, published by Pearson/Prentice Hall Professional, Sept. 2010, ISBN 0-13-708188-X, Copyright 2011; for a complete list of contents, please visit the publisher site: www.informit.com/title/013708188X.
This was first published in March 2011