One of the most complex issues in creating a strong automated storage tiering and data archiving system has little to do with technology and everything to do with meeting business needs. That issue is designing a data/document management strategy that protects valuable corporate data, maintains it at the storage tier that best meets business needs, meets compliance requirements including privacy, security, and longevity, and specifies when it should be destroyed.
This is a complex problem that must be driven by business, legal, records management, and security needs rather than technology. For instance, financial industry compliance requires preservation of large amounts of data for several years in a guaranteed unaltered state. This may require writing that data to write-once media for long-term archiving. And while these long-term archives may be off-line the data cannot be lost. That means that it has to be in a format that can still be read years later and its physical location known. Large financial companies have suffered major losses in court when they have been unable to produce the required data in court.
In healthcare, HIPAA and similar regulations include strict security and privacy requirements for all personally identifiable information, including special training for all individuals with access to this data. That includes dev and test staff if they work with regulated data.
Even in companies that are not specifically regulated, the loss of customer information damages the company reputation and brand and can cost it business. Addressing this exposure requires involvement of legal, records management, and security experts, either internal or external.
This is obviously a complex issue in which IT must take the role of implementer rather than designer. It must call in the experts. And implementation will require its own set of experts – data security and a vendor implementation team if the data is to be kept in house, the service provider(s) if part or all of the system is to be outsourced to the cloud or traditional service providers.
Action Item: Business, legal, records management, security and other experts must be involved early in the process of designing solutions for data management, including DR and archiving. IT cannot make those decisions on its own, and if the expertise is not available in-house, then it should look outside for help. IT should be the implementer, not the designer.