With contributions by Michael Versace
WikiTrend: By 2012, unified search platforms that combine both Enterprise and Internet search capabilities along with offering some traditional BI and analytics functionality will be widely adopted by knowledge workers seeking flexibility to tune information access solutions for diverse workloads across a variety of content sources and data types.
IT professionals have spent much of the last decade responding to the collaboration, compliance, knowledge management, litigation, and regulatory fire drills brought on by the exigencies of their respective organizations, the result of which has been the unprecedented growth of enterprise data along with the need to retain and manage it.
Users and regulators, aided and abetted by an explosion of tools to create and all but instantaneously distribute information around the globe, have done their part to drive storage requirements through the roof, Meanwhile, IT has struggled to support bloated, porous infrastructures with a plethora of disparate tools and solutions, many of which do not work well together or provide no interoperability at all.
Enterprise Search Today
In particular, e-discovery for litigation support is getting a lot of buzz. Legal departments are in a reactive buying mode, purchasing point solutions to solve tactical problems and alleviate data access and management bottlenecks. However, in their rush to meet the immediate demands of audit, compliance, customer service, finance, IT, legal, marketing, records management, or sales departments, organizations are inadvertently creating silos of data that too often can only be accessed by tools specific to an application or solution at a time when the ability to find, index, classify, search, and act on information gathered from multiple locations, sources, and dimensions is what is needed in today’s business environment.
The ability to view and contextually relate data across all enterprise structured and unstructured content sources as well as web content to uncover new business insight is business critical. IT professionals need more visibility into the application selection process to ensure the implementation of more strategic enterprise search deployments across departments and across all solution repositories, including CRM, DBMS, ECM, ERP, file shares, message archiving, and records management systems. Search alone can be a blunt instrument. The ability to build an index and thereby unify (virtually), classify, delete or archive and then intelligently search data becomes even more critical for “finding” and linking data among and between disparate repositories. Therefore an enterprise search or information access platform is fast becoming a strategic necessity.
Unified Search Components and Convergence
Consumers of search technologies are already seeing evidence from major players, innovators, and relative startups that the information access landscape is changing dramatically to meet business requirements. While no one company yet “owns” all of the pieces from end-to-end, the convergence of existing search capabilities and the inclusion of analytics and multidimensional queries typically associated with business intelligence (BI) tool suites are beginning to emerge in the next generation of unified search platforms.
Functionality and characteristics of unified search platforms include:
- The ability to index and search all major enterprise repositories including flat file and relational data (structured and unstructured data).
- The ability to include Web content, Internet news feeds, blogs and real-time data from Twitter, user clickstreams, and other sources.
- Ease of use with non-technical, intuitive user interfaces that eliminate the need for user programming or tweaking beyond search or access requests.
- A platform built on a common set of base components that can be tweaked or tuned for different search tasks and content or data types (ediscovery, emails, images, customer records, voice, etc.)
- Multiple access points from desktop or smart phone, Web browser, Lotus or Outlook clients.
- The ability to implement and deploy quickly in a single department and then scale and migrate across the enterprise.
- Semantic linking and portability across structured and unstructured content types between and within each repository (MIME, XHTML. XML)
Unified Search Players
Wikibon interviewed several enterprise search visionaries and thought leaders from a variety of companies that are likely to be an integral part of the information access landscape in the coming years. Highlights of these conversations follow:
Andrew McKay, senior vice president of Attivio along with at least two other members of their management team are former FAST Search and Transfer (now Microsoft) executives. “Search engines don’t understand relationships”, says McKay. "Attivio provides a hybrid index that combines the strengths and capabilities of both business intelligence and enterprise search, breaking down the silos between structured data and unstructured content, between information and action, between open source and commercial software, between customers' short-term requirements and our long-term technology roadmap.” Attivio’s Active Intelligence EngineTM (AIE) is “schema agnostic” allowing for the discovery of information from data and content silos without a lengthy predefinition phase. McKay adds that AIE allows for the integration of information from internal and external source systems to support multiple applications for consumer portals, site search, discovery and data analysis. McKay believes that the creation of a common index is one of the keys to providing the “agile content network” that users demand.
Coveo wants users to stop fighting movement of data around the enterprise from different repositories and leave it where it “lives”. “Repositories hold data not information,” says Executive Chairman and founder Louis Tetu. “Users don't care about technology, they care about access to information for insight. Enterprises need to leverage their knowledge assets to unlock the value of information by using information access tools.” Tetu is passionate about “connecting people to information and connecting people to people through information to increase return on knowledge capital.” As the former CEO and co-founder of Taleo, Tetu is well aware that people are at the heart of any enterprise information access strategy and the key to its successful implementation. "An integral part of Coveo’s strategy is to make information access ubiquitous and available through multiple, intuitive access interfaces, from an Outlook sidebar to a floating desktop search bar to a Blackberry or other smartphone interface," says Tetu. “Users need just-in-time access to information which is relevant and comprehensive, and enterprises need to leverage both their knowledge assets and their IT infrastructure to unlock the value of the information they hold. Tetu also believes that the initial entry into the information access world should be as painless and inexpensive as possible. To that end Coveo is now offering free downloads of its Expresso entry level search solution for up to 50 users, including mobile access. Coveo’s full, Enterprise Search Solution is also, according to Tetu, highly configurable and easy to implement and scales to billions of documents. “We are only at the beginning of what search is capable of delivering to enterprises.”
The team at Endeca believes that too often information management pundits and the shepherds of enterprise data make a false distinction between structured and unstructured data. According to Chief Strategist Paul Sonderegger, “at the core of the Endeca Information Access Platform (IAP) is the MDEX Engine, our innovative semi-structured database that provides advanced query capabilities for high-performance search and information access across the entire spectrum of structured and unstructured enterprise data.” Sonderegger, who spent several years working with end-users on enterprise search strategies while at Forrester Research, concludes that knowledge workers with no technical expertise are the prime targets for information access and search technologies. “The Endeca approach is simple: Users need to implement solutions fast with zero training upfront along with having the ability to explore data dimensionally. With our semi-structured database, each record has its own schema and describes itself, which offers a truly unified index. Meanwhile, IT needs a search platform to support different search applications depending on knowledge workers needs. Improving one individual decision at a time, millions of times over, is how discovery drives a hundred-million-dollar ROI.”
Paul O’Hagan, offering manager for the IBM Enterprise Search Division, came to Big Blue along with the DataMirror (data transformation) acquisition where he led product development. “Analysts estimate that 80% of enterprise information is unstructured content, yet business intelligence traditionally only examines the remaining 20% of information available in structured databases. Imagine what insights your organization is missing about your customers, products, processes, and compliance risks by not analyzing and reporting on ECM and other unstructured content.” O’Hagan is a strong proponent of “finding” data and determining what to keep before you search it and managing “data in motion”. To that end, earlier this year IBM announced an offering called InfoSphere Content Assessment specifically designed to address the problem of understanding what content matters to a business so decisions about what to keep and search can be made. IBM has offered its OmniFind enterprise search framework for several years and has announced that it will be delivering an additional search platform in 2010 based on the Lucene open source index library to meet additional customer needs. “IBM has defined a comprehensive path for customers to manage all data types including BI tools for exploratory analytics (Cognos), structured data management (Optim) and comprehensive content management (FileNet, ediscovery, classification and Master Content), along with the ability to ingest content from external sources and synchronize it across IBM products.”
Lucid Imagination seeks to dispel the notion that open-source search is not user friendly and needs enterprise IT technical support to implement effectively. VP of Marketing David Fishman, a tech veteran with stints at Sun and Mercury Interactive (now part of HP), believes Lucid can dramatically reduce the “time and money scavenger hunt” that usually accompanies a Lucene or open source search implementation. According to a recent press clip, LucidWorks Certified Distributions feature tested versions of Lucene/Solr open source software based on the most recent stable releases, with additional value-add components, tools, bug fixes, and performance enhancements. “The new Certified Distribution demonstrates our commitment to adding value where it is critical to unlocking the benefits of open source Lucene/Solr for enterprises,” said Eric Gries, CEO of Lucid Imagination. “While the cost-savings and pace of innovation have accelerated the adoption of open source, the stability, reliability, and support of the software remain critical requirements for enterprises IT. We hope that our new Certified Distribution, with its authoritative Reference Guide, serves as a key enabler to further expand the adoption of Lucene/Solr in enterprise search applications.” Lucid is venture backed and wants to emphasize that it already delivers services for mission and business critical search deployments to dozens of enterprise class customers.
According to Microsoft’s Principal Search Technology Evangelist Nathan Treloar, the Enterprise Search Group in Microsoft has, with the acquisition of FAST, become the largest enterprise search development team in the industry. Treloar joined FAST in 2003, well before the acquisition last year, and says “We believe search needs to go beyond the simple search box and ’10 blue links’ to address concrete business problems through workflow enablement that supports actual user tasks. To this end, Microsoft is releasing new search capabilities in SharePoint 2010, including tight integration with the FAST search platform, that combine search with other critical work streams in SharePoint such as collaboration and content management. Outside of SharePoint, Microsoft continues to develop FAST search for Internet applications and we are seeing innovation in the use of search that combines other Microsoft services like Bing with enterprise search.” Microsoft search products are especially optimized for the Microsoft user community, however Treloar believes their capabilities will be embraced by users for the “completeness of search offerings that extend from the desktop, through the enterprise and out to the Web.”
Other Search Players to Watch
While Google is relatively new to the enterprise search game and today has limited large enterprise level functionality and interoperability through its GSA and Postini offerings, it is no doubt watching the space and busy developing its long term strategy for unified search.
Even though it does not have its own enterprise search platform, as the world's largest technology firm, HP has a goal to be a major information management solutions and services provider. HP with Neoview and other BI tools – not to mention the EDS acquisition - has huge resources to bring to bear if and when it decides it needs to enter the space.
I doubt anyone believes Oracle will be content to take a back seat to any of the information management vendors in this space as it “owns” a huge percentage of structured customer data in the DBMSs and ERP solutions. Look for Oracle to make a major play to help customers manage their unstructured content.
SAP with its NetWeaver Business Warehouse and its acquisition of Business Objects along with its huge ERP install base should not be overlooked as it will need to address the content management needs of its customers to maintain their market share.
Bottom Line
Purpose built search, indexing and classification engines and capabilities will continue to be optimized for, embedded into, and sold with point solutions or suites. Enterprise search products that offer an enhanced user experience and mobility along with the expectation of a low cost of initial entry will continue to be deployed tactically to alleviate known solutions bottlenecks. Major search platform players from Google to IBM and Microsoft along with smaller innovators are rolling out or planning new offerings that will provide organizations with more comprehensive search platforms to choose from in the near future.
Action Item: Users, lead by IT, need to review enterprise search vendor roadmaps and capabilities, test point solutions and make plans to consider the strategic value of information access platforms and their potential impact on overall corporate productivity and information governance.
Footnotes:
