In an all-flash storage system, the controller is optimized for SSD's higher performance. Legacy disk controllers were designed to support a storage medium (spinning disk drives) that could generally provide less than 200 IOPS each. High performance SAS HDDs can do almost twice that but it's still insignificant when compared with the 8000 IOPS provided by a typical solid state drive. Given the latency of spinning disk drives, HDD array controllers had processing cycles to spare, which most would use to handle storage services like snapshots, volume management, replication, and the like.
When this latency is removed, as it is when SSDs replace HDDs, the controller becomes the bottleneck and performance can suffer greatly. All-flash storage systems feature controllers that are designed to keep data flowing to this faster storage medium, whether in drive form-factor SSDs or in flash circuit card modules. They typically have a more distributed controller architecture that allows for more data streams into and out of the storage media and will often handle overhead for storage services with separate processors. Some also have onboard capabilities that reduce the impact of network protocol processing on performance.
The controller in an all-flash storage system also addresses endurance issues. Unlike magnetic disk drives, NAND flash can't be overwritten at the bit level. In order to accommodate changes, an entire block of flash must be erased ahead of new data writes. This requires a separate housekeeping process that consolidates data to be saved from blocks that are targeted for deletion, creating additional data copy steps ahead of that erasure. Known as "garbage collection," all of these additional overhead steps consume CPU cycles in the storage controller, cycles that can impact performance if they're stolen from the processor that also handles data I/O.
While not unique to these solutions, all-flash storage systems have flash controllers that leverage dedicated hardware to handle this flash-specific overhead so it doesn't degrade performance. Compared with software-based flash controllers, they do a more efficient job of running these NAND flash operations (reducing write amplification and improving wear leveling), which increases flash chip endurance as well. This capability can also make feasible the use of lower-cost MLC and enterprise MLC (eMLC) NAND flash chips, which helps to bring down the effective cost per gigabyte of all-flash storage systems.
Now let's talk about the types of all-flash storage systems available and how customers would use one.
Use case 1: Augment performance
The first use case for all-flash storage systems was probably as a high-performance storage appliance add-on to an existing disk-based infrastructure. The earliest adopters of new technologies are often those on the "bleeding edge," users that seem to need more performance at any cost. Now, these flash appliances have come down in cost but are still typically used to augment performance. In these environments all-flash systems can provide a temporary or part time block storage area for running online applications like gaming or high-transaction databases during their peak loading periods. EBay uses all-flash appliances to augment performance in this kind of environment.
In addition to acting as a flash tier, all-flash appliances can offer a networked high-performance cache for accelerating applications that are resident on hard disk arrays or for virtual server and VDI environments. Texas Memory Systems, Kaminario, Violin Memory, Whiptail and Astute Networks have put out products in this category, although many of these companies have flash-based products in other flash categories as well.
Use case 2: Replace HDD arrays
The second use case for all-flash storage systems is as a replacement for existing disk-based storage arrays, in the form of all-flash arrays. These solutions employ the same basic technology as the first group, with controllers and internal infrastructure designed for flash storage, but have added storage services as well. As a replacement for a traditional disk array, all-flash storage arrays need to meet market demands for features like volume management, snapshots and replication. Some offer multiprotocol functionality as well, with block- and file-based storage.
All-flash storage arrays typically include storage efficiency technologies such as deduplication, thin provisioning and compression since these processes work well in solid state storage. Also, data reduction can reduce the effective cost per gigabyte, which has historically been the primary impediment to wider adoption of flash storage devices. Even though there may be a performance impact by implementing technologies like deduplication for a primary storage array, the manufacturers are counting on the likelihood that all-flash systems have performance to spare. Manufacturers in this space include Pure Storage, Nimbus, Skyera, Greenbytes, EMC (with XtremeIO) and Solidfire. And as mentioned above, many of these companies have solutions that could be included in the all-flash appliance category and be used to augment performance as well.
Key benefits of all-flash systems
In general, all-flash systems offer several key benefits compared with other flash technologies. They can be shared in a SAN configuration, like an existing disk array -- something that's appealing to users with many potential server hosts to accelerate -- rather than using server-side SSDs or PCIe cards. Sharing is also appealing for server virtualization implementations to provide high-availability and to support VMware's Storage vMotion.
Compared with hybrid flash systems (appliances with both flash and spinning disk capacity), all-flash storage systems offer faster performance that is also more consistent and predictable. Unlike hybrid arrays, their controllers can skip the overhead of caching and tiering algorithms that decide which data sets should be put on flash and the cycles required to actually move that data back and forth between the two storage areas. With all data on solid state storage, all-flash systems aren't as susceptible to workload variability like caching or tiering systems can be in highly dynamic environments, such as server virtualization. There's never the risk of a cache or tier "miss," the disruptive scenario where the required data is not on flash as expected and applications are made to wait while the data is fetched from spinning disk.
Cost: The TCO question
As alluded to above, the cost per gigabyte of flash systems has been an inhibitor to the adoption of flash-based storage devices. But where deduplication and other data reduction technologies don't make economic sense for traditional disk arrays, they're especially well-suited for all-flash systems. First, these processes run particularly well on flash storage due to its already high IOPS performance, making them even more efficient than they are on HDDs. Also, the base cost per gigabyte of flash is higher, so each additional percentage point of data reduction that's achieved has a better marginal return compared with HDDs. For this reason, care must be taken to include reasonable data reduction ratios when running return-on-investment calculations for all-flash systems.
Flash has become a fixture in current storage technology projects. The question is no longer whether to use a flash-based system, but where to put the flash. All-flash storage systems, either implemented as performance-augmenting appliances or as replacement arrays for traditional HDD systems, are another alternative -- and one that's making better economic sense every day as the effective cost-per-gigabyte differential between high-performance disk drives and SSDs keeps shrinking. And aside from the potential all-flash project, VARs should become familiar with all-flash storage systems because this technology is part of the larger flash storage discussion. Along with server-side SSD/PCIe solutions and hybrid flash appliances, all-flash systems are an implementation alternative that their customers will be asking about.
Eric Slack is a senior analyst with Storage Switzerland.
This was first published in October 2012