It is no secret that enterprise information managers are buried by data, and the prospects for relief in the future are not good – especially when it comes to the unfettered growth of unstructured data. Pick your poison. The need to store and manage business records such as documents, emails, images, video, Web content and other forms of electronically stored information (ESI) is fast becoming the top priority in many enterprises. They must rely on the “discoverability” capabilities of their information management solutions portfolio to improve access time to critical data for knowledge management, legal, regulatory compliance, or security purposes.
With the advent of new regulations including GLBA and SEC 17a-4 and updates to the FRCP mandating the need to electronically store and “quickly” produce business records, the last decade has seen a tremendous spike in the need for data archiving and related solutions such as classification, ediscovery and policy management. This has been matched by a raft of new data reduction technologies coming to compress, deduplicate, and otherwise reduce the size of data archives, whether it be of the active, passive, or deep variety, in-house, hosted, or architecturally hybrid in nature.
Generally accepted industry market share numbers for archive solutions adoption indicate a slight percentage edge for hosted and hybrid solutions over in-house solutions - perhaps 54% vs. 46% give or take a few percentage points, However, an informal Wikibon survey indicates that in-house archiving solutions dominate the high end of the market. Roughly 70% of companies with more than 4,000 mailboxes have chosen to implement in-house systems, while the percentage is even higher for enterprises with more than 10,000 employees. Typically larger firms have the expertise to deal with the myriad business and technology integration challenges involved. They also wield more clout with vendors and can afford to hire the best consultants to help navigate through a very complex and costly implementation.
This high end of the in-house active archive market is stacked with major technology players including Autonomy, Computer Associates, EMC, HP, IBM, Microsoft and its many partners, OpenText, Oracle, and Symantec, along with niche or up-and-coming companies such as AXS-One, C2C, Commvault, Messaging Architects, Mimosa, Quest Software, Waterford Technologies, and ZL Technologies. However, the clear market share leader with roughly 30% of the high-end market is Symantec with Enterprise Vault, along with the array of products, interfaces and partnerships it has built into its enterprise archiving solution.
As is the case with many of the enterprise data archiving solutions offered by the major players in the space, Enterprise Vault (EV) became part of Symantec by acquisition, in this case, through the Veritas deal in 2004. The original code was created in 1997 by a messaging engineering team at Digital Equipment Corporation (DEC). It was taken to the next level by a group of investors who greatly enhanced and productized the system as an in-house email archive solution, which became KVS in 1999. To its credit, Symantec recognized the market opportunity and the lack of availability of robust enterprise email archiving solutions. Over the past five-or-so years it has done a good job of holding the competition at bay with frequent updates, new releases, additional acquisitions and functionality, integration services, partnerships, and, of course, superior account control and marketing.
Approach
Driven by market forces and stiff competition, Symantec EV has morphed over the past 10 years from an email archive system into a comprehensive solution which incorporates scores of the critical features and functions necessary to support a multi-faceted, multi-departmental corporation. These include classification and ediscovery functions, indexing, mailbox management, policy management, security features, integration with Exchange, Notes and SharePoint, Blackberry and Windows Mobile interoperability and a reliable interface for their own enterprise search engine, which leverages elements of Alta Vista enterprise search.
EV archives information from messaging, file, and collaborative systems using its storage optimization, classification, and retention technologies. EV can automatically capture, categorize, index, and enforce retention policies and secure unstructured information. EV’s newest major release, 8.0, became available in early 2009 with a slew of enhancements and capabilities for compliance, finance, HR, legal, IT and records management departments including:
- Optimized Single - Instance Storage (OSIS) which reduces the archive and allows for sharing of commonly repeated data between archived items like email, SharePoint documents, file system documents, reducing the footprint down to a single copy regardless of source.
- Integration with de-duplication technology from vendors such as NetApp, which enables a combination of backup and archive data to further reduce the archive and backup data footprint. Open storage capabilities from Enterprise Vault, like Net Backup, allow integration with practically any storage device.
- An improved end-user experience in EV 8.3, which allows for transparent access to archived items from familiar interfaces like Outlook, Notes, or Windows Explorer without the need for shortcuts, meaning there is no separation between live and archived information.
- Simplified deployment, administration, and management, including new installation process and setup, which provides recommendations and default settings based on industry best practices.
- Enhanced policies to allow users to “re-file” archived messages for new retention requirements without IT support.
- Discovery Accelerator with guided review, bulk marking, and search within a search, which helps users explore search results by prioritizing or clustering the results to reduce discovery costs by culling search result sets .
- Ability to consolidate email and other ESI into a single repository.
Functionality
Recently, Wikibon facilitated a user group of information management professionals from a cross section of industries to help begin to define a future state for an overall functionality set for managing unstructured data across the enterprise. Symantec is the consensus market leader for active archiving with EV, as well as a major provider of services and solutions that are critical to enabling unstructured data management (UDM) in the enterprise, such as security. Therefore Wikibon thought it useful to map the leader in the space to those future state functional areas. Either EV includes the capabilities listed below or has partner relationships to enhance or provide the necessary functionality.
- Policy and Rules Management: EV addresses this area to include policies for retention, disposition, information assurance, use, and distribution. However, it's not apparent whether the EV policy engine can be federated to address similar policy requirements across other UDM repositories (e.g., files systems).
- Content Management: EV works with most leading content management solutions including EMC’s Documentum and can also act as an archive for collaboration solutions such as SharePoint. Compliance with CMIS provides some level of interoperability.
- Configuration Management: EV includes functions establishing ownership and custodial responsibilities.
- Declaration and Classification: EV and its companion product Auto-Classification Engine (ACE) enables policies for the declaration and classification of data, including the ability to distinguish between business records and non-business information.
- Crawling, Collecting: EV has the ability to collect from file shares and data stores throughout the enterprise. With Symantec’s Open Storage Technology (OST) and its technology partners, such as Data Domain, data can be managed across storage types and different media.
- Search, Discovery, Hold and Preservation: EV indexing has functions for locating and gathering unstructured data scattered across the information management environment as well as working with partners for specialized search. EV will ingest data from solutions such as Stored IQ or hand off data to eDiscovery solutions such as, Concordance, Summation and Ringtail.
- Creation and Copy Management: EV has functions that enable rules governing the creation of information, copies to be maintained, and the de-duplication or single instancing of information along with partner products that can enhance this capability.
- Information Assurance: EV, along with Symantec security products and technology partners, enable policies for identity management, information authentication, access management, privacy control, use management, and auditing. EV will work with critical non-Symantec products for classification, encryption and enterprise search
- Analytics and Reporting: EV includes functions for monitoring, alerting, and real-time reporting on key information management events such as policy updates, configuration changes, security anomalies, classification events, and legal hold requests.
Futures and Concerns
The Symantec EV team, out of necessity, has a good track record for integration of the various components that have either been acquired or developed internally over the years to clip on to the base system. These include ACE and Message Labs (for encrypting emails), elements of the Alta Vista search interface as well as various data leakage, spam filtering, ediscovery, and single instancing related products and features.
However, some very large users with more than 30,000 mailboxes have complained about scalability issues and huge integration costs. At this juncture, EV is unable to take advantage of the more advanced grid-like architectures offered by several other solution providers that would aid in scalability and improve performance. Legal teams with heavy ediscovery workloads tend to use point solutions such as Clearwell or Kazeon to supplement the collection or culling and analysis phase – especially when searching data from outside the EV archive - then centralize what has been collected with their central archive store. Also, users report mixed reviews with auto-classification and very large index builds, although this could also be due to poor planning and implementation practices from buyers.
Many users and EV competitors claim the various derivatives of the open source search framework Lucene are better suited to the all important “find” task, critical to indexing, classification and ultimately “discovering” an enterprise’s entire corpus of data. In addition, some storage administrators dislike EV’s use of XML, as it adds to the size of objects stored in the EV archive. However, many discovery users appreciate the ability to review files and email without the necessity of the original application. The need for multiple “vaults” when implementing a 10.000 user plus system can cut into data management times and pulling all objects into a single “logical” vault may also impede performance.
Unlike many of its major competitors, including Autonomy (Interwoven), EMC (Documentum), IBM (FileNet), OpenText (Vignette), and Oracle (Stellent), Symantec has yet to purchase or develop a major Enterprise Content Management (ECM) solution. This may leave Symantec vulnerable from a market share retention standpoint if not at a disadvantage from a product coverage standpoint. Users believe legacy archiving solutions do not meet all the necessary requirements for a robust ECM system, and it is clear that buyers will continue to lean on a broad solution set that includes archiving, ECM and eDiscovery tools to meet their UDM demands.
It remains to be seen if SharePoint and its cadre of partners and third-party developers, or open source community offerings such as Alfresco, can become sufficiently robust to compete with the major ECM players, and if Symantec can effectively mitigate the ECM/Archiving convergence strategy of its primary competition. Symantec and others argue that this convergence strategy has not necessarily worked. The ECM/Archiving vendors are still selling these products as standalone solutions, and many customers are wary of centralizing unstructured content in a single master ECM-like repository due to performance and functionality concerns. At this point, it appears Symantec, like other archiving vendors such as Mimosa, is betting on SharePoint and Microsoft as other archiving vendors. This strategy will work until Microsoft decides to enter the archiving space in earnest.
With all the buzz around Cloud Computing, there is the potential for hosted and hybrid solutions to gnaw away considerably at the lower end or EV’s sweet spot, while newer, better architected, more scalable grid capable solutions win more of the mammoth deals at the very high end.
Bottom line
Unlike the ERP space, the enterprise active archive solutions market has yet to mature. The enterprise role of the active email archive has expanded well beyond what it was originally conceived to accomplish. Merely storing unstructured data without thought of how to manage, find, and delete it is no longer an option. Convergence of Compliance, ECM, eDiscovery, Policy, Records Management and Security needs with archiving solutions capabilities to manage the unmitigated growth of data, particularly unstructured data, calls for better integrated solutions, updated architectures, innovation and thought leadership from solutions providers. Buyers need to better understand vendor product pitfalls, and vendors need to delve deeper to understand use requirements. Buyers also need to understand that they may be implementing archiving solutions that will not meet their needs in five years. Whether or not new and innovative solutions from Symantec and the others emerge, and how long that takes, remains to be seen
Action Item: Larger enterprises with more than 1,000 users looking to implement an in-house active archiving solution with most of the functionality to satisfy their compliance, IT, and legal departments, who have the expertise, resources and organizational will to take on a major solutions implementation, need to take a hard look at Symantec’s Enterprise Vault, as it is the market leader among a field of solutions providers who have yet to convince the bulk of buyers that they deserve to be considered on the same level.
Footnotes: