Today structured operational systems maintain records glued together by explicit and implicit metadata. The metadata in systems that store unstructured data is sparse and consists mostly of a few dates and maybe a file name; the exception is email where some standard metadata exists.
The requirement for creating a complete and accurate business record is to combine both structured and unstructured data. A simple example is providing a special price for a client order.There are numerous unstructured emails, excel spreadsheets, signoffs and other data elements that may be combined with structured data entries to form a more complete system of events. An effective record management system allows these elements to be connected together to provide an unambiguous record.
For records to be managed around key events in business processes, more extensive metadata must be created with a combination of automatic and user inputs. There are no perfect technologies that can retrofit all metadata into existing unstructured data, but it should not be assumed that none will be developed (see discussion tab). Email has some of the metadata required, but is still incomplete. The cost of manually adding metadata is prohibitive and unlikely to take place in most cases.
This means that the essential prerequisite technology is the ability to tag new unstructured data with metadata automatically when the data is created or susequently used. This will require extensions to applications that create and store unstructured data.
Many additional technologies will be required to create effective and high performance records systems. The ability to encapsulate both the data and the metadata together is one such interesting technology. However, this and other inventions are of little value until the metadata is first established.
Action Item: CTO's should make sure that executive management do not assume that metadata can be applied retroactively to existing unstructured data, and point out forcefully that the legal exposure is growing daily in proportion to the amount of data being stored. Implementing technologies that have the ability to create metadata at the time data is stored will be the key determinant of when this exposure can be brought under control.
Footnotes: Note: see the lively discussion on the discussion page!
