Moving storage virtualization up the stack
Storage virtualization is a solved problem.
To be sure, there are still some messy details to clean up. The SNIA's Storage Management Initiative, with all the tremendous progress it has made, still has challenges lying before it. Using virtual storage isn't as easy as we'd like it to be. But as an industry, we fundamentally know how to virtualize storage. We can create virtual devices with an impressive variety of cost, performance, and availability characteristics, using the facilities of storage systems, intelligent storage network switches and host-based virtualization software. Using storage network facilities, we can make virtual devices available to some servers and block access to them from others and re-provision devices to other servers when requirements change. Over the last decade, we've brought Fibre Channel networks from birth to maturity and followed that up by bringing iSCSI to a stage of relative maturity. It's time for our customers to enjoy the benefits we've brought them and for us as an industry to rest on our laurels.
Ah, but there's the rub. Storage virtualization, and its cousin, storage networks, were supposed to improve utilization of our customers' storage assets by making it possible to reconfigure and redeploy them as needs changed in the data center and across the enterprise. And they do that—all of the major vendors have delivered impressive toolsets for virtualizing storage devices and provisioning them to consumers. More forward-looking ones, like Veritas, EMC and others, have realized that managing heterogeneous devices as a homogeneous whole is the name of the game, and their network tools support all major storage and network components. But it turns out that that's only half the battle.
It's sure handy to be able to dismantle a 100-gigabyte striped mirrored virtual device that a Solaris server no longer needs and reconstitute it as two 25 gigabyte ones for an AIX transaction database server and four smaller striped ones for a Linux Web server. And when we view the problem from a storage perspective, that's how we describe it—as a problem of reconfiguring storage resources and moving them from one server to another. But what if we looked at the problem from our user's perspective? A data center manager is much more likely to describe his or her problem as something like, "My legacy sales app that runs on Solaris is being replaced by a WebSphere app on a new AIX server, and I have to move this year's online transaction history to the new platform and find new storage for it as well. I'm also getting complaints from my CRM users that their Linux databases keep getting corrupted, and they need to be able to 'turn back the clock' for a couple of hours at any time."
Being an astute and perceptive person, our data center manager would notice that since he or she only has to keep this year's sales history online for the new AIX sales app, 150 gigabytes of physical storage is being freed up. Being up on the latest technology, our manager realizes that he or she can reconfigure this freed-up storage and deploy part of it to the new AIX server and part to the Linux servers, and use it to take hourly split mirror snapshots, which he or she will recycle every four hours. Being also a methodical person, our manager will lay out and execute a migration plan that looks something like this:
Back up the Solaris sales history for this year.
Delete the Solaris file systems and virtual devices.
Create new virtual devices and provision them to AIX and Linux.
Create file systems on AIX and restore the sales history data onto them.
Create scripts to take hourly snapshots of Linux CRM data and recycle the oldest one every four hours.
As major suppliers and "trusted advisors" to this data center manager, we should ask ourselves how we help him or her solve this problem. Our answers will depend on what we supply.
The backup vendor will say, "Well, I supply backup software that allows my customers to back up on one system and restore on another."
The disk array vendor will say, "Well, my customers can create and delete virtual devices."
The network vendor will say, "Well, my customers can redirect storage devices from any host in the SAN to any other."
The file system vendor will say, "Well, my customers can take snapshots of their data as often as they'd like."
And this, too, is true. We all do excellent jobs of storage management in our respective realms. But we're leaving the higher level problem of using information technology to achieve business goals up to our customers. Why did this data center manager have to figure all this out for him- or herself? Why does he or she have to back up and restore the data, when all he or she wants to do is "move" it from one set of disks to the same set of disks on a different platform? As an industry, we are empowering our customers with magical new capabilities. But at the same time, we're creating an enormous requirement for highly skilled administrators who can look out across a data center with hundreds of servers (not the three in our little example), continually juggle tens of thousands of variables, and make optimal storage and data management decisions. We're delivering drills and wrenches, when what our collective customers need is robots and assembly lines.
This is not to say that we've done anything bad as an industry. Indeed, if we compare the capabilities we've enabled by virtualizing storage and putting it on a network with what was available to our customers a decade ago, the difference is astounding. But the key word here is "enabled." We've made it possible for our customers to create environments that are orders of magnitude more complex than what they could have contemplated in the mid-1990s, and we've left it up to them to manage the complexity.
It's time for us as a storage industry to take a hard look at how our customers see things and try to make IT life as they see it (not as we see it) easier. Why aren't our file systems better integrated with our virtual storage so they can grow, shrink, and respond to failures appropriately? Indeed, why does every one of our platforms have a different file system format so that we're forced to copy data when all we want to do is move it from one platform to another? Why don't our mirrored virtual devices recognize that their component LUNs are in different buildings of a campus, and when they grow, they have to grow under that constraint? Why don't our application developers have APIs that let them say things like, "replicas of this database can never be more than three minutes out of date"? Why can't a virtual storage device recognize that the sales database is using 90% of available I/O bandwidth and throw more resources at the problem before it gets critical? The storage systems we deliver today collectively have the information and capabilities to automate these kinds of actions. Why aren't we doing it?
As storage developers, we've done a great job. Advanced storage technology is in some measure responsible for many things we take for granted today that didn't exist a decade ago—mobile communications, the active Internet, video on demand, reliable business-to-business transactions, and so forth. Now it's time for us to take the next step . . . to ratchet up the capabilities we deliver to our users and enable them to fully utilize the impressive array of technology we've made available to them.
- TABLE OF CONTENTS
- Introduction: Observations on storage virtualization
- Delivering a global storage solution
- Virtualizing servers, storage, and networks
- Moving storage virtualization up the stack
- Virtualization of data
- Benefiting from virtualization
- Future directions for virtualization: A foundation for the future
This is excerpt was written by Paul Massiglia, technical director foundation technical product management for VERITAS Software Corporation, from the book Storage Virtualization: Technologies for Simplifying Data Storage and Management, by Tom Clark and courtesy of Addison-Wesley. Click for the complete collection of viewpoints.
If you found this book excerpt helpful, purchase the book from Addison-Wesley.
This was first published in October 2006