Q: How do you know if data deduplication is a potential fit for your customer's data?
Over and above the solutions that are now providing deduplication in the archiving and backup arena, we are now seeing vendors that are doing deduplication on their storage arrays intended for Tier 1 and Tier 2 applications. First, you want to consider if the storage dedupes are at a file level, a block level or some sort of variable-length data segment; this will have a significant impact. Next, consider the type of data; in a shared file system environment, there is often a significant amount of data duplication in Microsoft Office documents, email and static media such as images and movies. There can be a significant impact in the amount of storage required with a solution that supports dedupe here. Also, VMware can be significantly impacted, considering that a typical server consolidation ratio is 10- or 20-to-1, and that most of those are Windows servers sharing the same binaries. You have to be careful about any potential performance impact, though. Make sure to work with your vendor to ensure that data has been laid out to maximize the potential deduplication benefit while ensuring any potential performance impact is minimized.
Data that is typically not a good fit for dedupe is encrypted or compressed data.