Deduplicating data has become so popular that storage manufacturers are jumping all over themselves to get into, or to get a bigger piece of, the market. For solution providers, that means a golden opportunity to add a hot new offering to their lineups.

More and more companies are adopting deduplication as they realize its advantages, says Paul Swain, senior tech support specialist at Ingram Micro. By storing only the data that's changed rather than full copies of the data each time, deduplication speeds backup, reduces storage needs and even lowers telecom costs because less data is sent over the wire. This can be ideal for backing up virtualized servers, which are highly redundant, often using the same operating system image, the same applications and much of the same data.

"There's no denying the fact that everyone in the industry understands that data deduplication is a huge opportunity right now," says Stephen Johnson, disk-based backup product manager at HP. "It's a gem of a market that is growing substantially."

Most storage manufacturers have jumped into the deduplication market in the last three years. EMC acquired Avamar in 2006 and forged an OEM deal with Quantum in 2008. IBM entered by acquiring Diligent Technology in April 2008. And Dell made the move in June 2009, introducing a deduplication product that uses software from CommVault. A symptom of the gold-rush mentality was a bidding war that started early this summer over Data Domain, which has dominated the market for several years. Two competitors -- EMC and NetApp -- were locked in a bidding war for Data Domain until EMC won out with a $2.1 billion bid in July.

In addition to the proliferation of vendors, deduplication products are increasing in performance and storage capacity. Data Domain, which has expanded further into the large enterprise market, now has systems that offer up to 2.7 TB/hr. of aggregate inline deduplication, says Shane Jackson, senior director of product marketing and channel marketing at Data Domain.

The increase in manufacturers and offerings makes deduplication choices more complex, and thus opens the door to more consultative storage solutions sales.

"Deduplication technology is often misunderstood," Swain says. "People think data deduplication will always make your data extremely small no matter what." But some types of data deduplication work better than others in certain situations. And not all data deduplicates well, he notes. Sizing -- determining how much of a customer's data will deduplicate and how efficiently -- is a tricky area that requires installing tools onto the network and doing real-world testing, he says.

Solution providers need to know which products are most appropriate for a particular customer's needs, Jackson says.

Steven Patterson, director of sales in the data center practice of Nexus IS, a solution provider headquartered in Valencia, Calif., agrees. "If there's virtualization in play, we know that inevitably there are going to be some challenges around backup, so we want to make sure we understand the environment and can talk about the right deduplication that we can put in place," he explains.

So while the plethora of vendors and technologies coming into the market complicates the sale, Patterson feels that plays to his company's strengths. "I love the fact that new technologies and new players are coming into the market," he says. "Customers have different needs, and we value all of the choices and options."