Top five data storage compression methods

Service providers whose customers complain that they're spending too much on storage can use one of these five storage compression techniques to cut storage costs.

It's a fact: Most data stored on disk today has at least some statistical redundancy and is easily compacted. Compressing this information cuts the amount of physical storage required to as little as one-third the space. Your customers are looking for ways to leverage lossless and transparent storage compression technology to control the cost of storage. Let's review the top five methods for storage compression.

File system compression

File system compression takes a fairly straightforward approach to reducing the storage footprint of data by transparently compressing each file as it is written to disk. Early PCs had relatively expensive hard drives; with storage space at a premium, software compression tools such as DiskDoubler and SuperStor Pro were very popular, and, in fact, they pioneered mainstream file system compression. But file system compression has long been plagued by complaints of poor performance. The truth is that it should be deployed selectively on highly compressible files that are written once and then accessed infrequently. For example, databases make horrible file system compression candidates, whereas application logs are great candidates for storage compression.

Storage array compression

Storage vendors have been slow to implement compression directly into their products because it is technically difficult to implement it at the block level below the file system. But what if there were a file system in the storage array? Sun Microsystems did just that when it integrated the server, SAN fabric and storage into a single system, the Sun Fire x4500. This server comes with ZFS installed and has loads of internal storage. Deploy it as a NAS device with ZFS compression enabled and look out!

NAS compression appliances

Isn't capitalism great? When there is a need, there will be a product to fill it. If your customer uses NAS storage and complains that it doesn't support compression natively, look to the Storwize STN-6000p series. Install the appliance between the storage device and the network, and it will do on-the-fly lossless and transparent storage compression. Improving the NAS storage capacity by 300% should provide a quick ROI.

Backup storage compression

Almost as old as file system compression is tape hardware compression. It's so commonplace that tape media manufacturers list compressed capacities in their marketing materials. Tape hardware compression is good for a few reasons: It doesn't slow things down, is included on every modern drive type and usually provides compression ratios of better than 2:1. Virtual tape libraries (VTLs) also support compression, but not all VTL compression is created equal: Several VTL manufacturers still use software compression, which slows down the write speeds.

Data deduplication

We saved the holy grail of all compression techniques for last. At the time of this writing, a Google search produces more than 560,000 pages of information about deduplication. The concept is relatively new, and the demand is high. Data deduplication comes in two forms: source-based and target-based. Source-based deduplication is handled at the client by products such as Symantec Puredisk and EMC Avamar. Target-based deduplication happens at the storage device in VTLs and NAS storage arrays like Data Domain, EMC DL3D, Diligent ProtectTIER, Sepaton S2100 and Quantum DXi series. It has become so prevalent that I believe it will simply become an embedded feature in storage devices.

About the author:
Brian Peterson is an independent IT infrastructure analyst, with a background in enterprise storage and open systems computing platforms. A recognized expert in his field, Brian has held positions of responsibility on both the supplier and customer sides of IT.