A funny thing happened on East Carolina University's journey to creating a data-retention strategy. As part of a compliance project launched one and a half years ago, Brent Zimmer, systems specialist at the university, was working with attorneys and archivists to determine which data was most important to keep and for how long. But it soon became clear that it was just as important to identify which data should be thrown away.

Zimmer was aware of the importance of being able to quickly produce required information during litigation, "but the thing we never thought about was keeping data too long," he says. The risk is keeping data that you wouldn't otherwise be required to produce, but as long as it's discoverable, it could be used as evidence against you.

Like many organizations, East Carolina had its share of data to purge. "We never made anyone throw away anything unless they ran out of space on their quota," Zimmer says. Some users, he says, had e-mail dating back to 1996.

East Carolina is not unusual; many organizations hang on to more data than they need, for much longer than they should, according to John Merryman, services director at GlassHouse Technologies, a storage services provider. One reason is fear. "Companies are really sensitive because there's a perceived underhandedness to purging data," he says. "People might wonder, 'Why aren't you keeping all your records?'"

Another is the low cost of storage. Organizations have historically preferred to buy more disks than spend time and resources sorting through what they do and don't need. "Many people would prefer to throw technology at the problem than address it at a business level by making changes in policies and processes," says Kevin Beaver, founder of Principle Logic.

But thanks to e-discovery risk and burgeoning data volumes - 20 percent to 50 percent compound annual growth rate for some companies - the tide is starting to turn, according to Merryman. The average cost companies incur for electronic data discovery ranges from $1 million to $3 million (about Rs 5 crore to Rs 15 crore) per terabyte of data, according to GlassHouse.

A recent report from Gartner concurs. It states that the current explosion of data is outpacing the decline in storage prices, even before the resource costs for maintaining data are taken into account.

Estimating that the average employee might generate 10GB per year, at a cost of $5 (about Rs 250) per gigabyte to back it up, Gartner says a 5,000-worker company would face annual costs of $1.25 million (about Rs 6 crore) for five years of storage. And considering that many companies maintain multiple copies of data, thanks to test data, operational data and disaster recovery copies, not to mention back ups, "there's an explosion of data in most companies," Merryman says.

Aside from the costs, keeping all those records indefinitely is a gold mine for attorneys looking for evidence, he adds.

One way to address this problem is to set retention policies that reduce exposure to legal problems. But don't try to boil the ocean, Merryman advises. Instead, create policies from the application or business level down, rather than looking across the whole data landscape and letting policy bubble up. Also, create black-and-white rules that are easy to deal with.

Related Articles

• Four Steps to A Business CIO
• Storage Virtualization: Boon or Bane?
• Are You Ready for a Disaster?
• Hole In Your Cloud
• Why CIOs Are Turning To Storage Virtualization

Latest Articles

• The Quiet Threat: Cyber Spies are Already in your Systems
• Responding to Security Questions Successfully
• Security Secrets the Bad Guys Don't Want You to Know
• Don't Wait for Adobe Sandboxing to Secure PDF Viewing
• The 4 Tiers of a Secure B2B Framework