The following are raw notes from Attivio's briefing to Wikibon analysts on 7.6.11:
- Unstructured content/data and structured data live in separate worlds.
- Each has value – Analyzing structured data via BI tools tells us “what” happened (i.e. how many widgets we sold last quarter by region.)
- Analyzing unstructured data tells us “why” (a new entrant to the market caused widget sales to go down).
- Bringing the two together delivers new value – the what and the why in one place.
- This is what Attivio’s Active Intelligence Engine allows customers to do.
- AIE lets users unify information across silos whether inside or outside the enterprise, structured or unstructured data.
- AIE is a platform on which to build applications. Once information is in AIE, can be consumed via dashboards, reports, search interface, etc.
- Search interface in particular is popular with users used to Google and Bing.
- Attivio customers build the end-user applications to best meet their needs.
- AIE means users spend less time looking for data in different silos --i.e. SharePoint, Salesforce, on the desktop, etc. – and more time exploring it and gaining insights to make better business decisions.
AIE different from Google. Google doesn’t do structured data. Google can’t provide you with a list of IBM’s last 50 acquisitions, for example (unless someone has created a page that lists them.)
Any application or platform for analyzing unstructured data must also analyze structured data – otherwise you still have silos.
AIE designed holistically. Each piece of info goes through the same path.
- Starts with security layer – user authentication, client IP address validation;
- Connectivity layer – This get s data out of repositories and into Attivio. Different than ETL. Connectors designed specifically for various data sources (file servers, DWs, vertical apps, etc). These connectors are bi-directional.
- Workflow layer – 70 components to transform, analyze and otherwise enrich data.
- AIE Inverted Index -- Write queries in SQL or search-style. Organize unstructured and structured data into tables and maintain relationships between data elements.
Maintaining relationships – AIE can process email and related attachments in their native forms but not lose the connection/relationship between the two. Key for deep analysis later.
Workflow layer components:
- Entity extraction – Covers 20 languages via dictionaries, patterns, or discovery model.
- Key phrase extraction. -- Lets you drill down to understand in aggregate what is being said in a selection of text.
- Automatic classification – Tags new docs as the appropriate class as you define them. Allows more granular search and drilldown.
- Sentiment analysis. – Attivio supports two ways: Document level sentiment analysis; And entity level sentiment analysis give greater insight, identifies sentiment directed toward different entities in a single document.
- Relevancy – Find me the best, most relevant example of xyz. Users can adjust relevancy based on field, context, phrase, proximity, first occurrence, freshness, static. Can create different relevancy requirements for different users (marketers v. engineers). Gives different views of the same info. For JOIN queries, give separate relevancy profiles to parent and child records.
These features generally not made available to end-users but to developers that are designing the end-user applications built on top of AIE.
Attivio’s Core Intellectual Property – Although we work like a search engine, we can organize structured and unstructured data in tables and retain relationships between tables and use them in queries. Can do a search type query or a SQL type query.
AIE does facet finding – recommend on the fly best options to present to user. Users can click down. You don’t have to write a great query like BI tools require.
Can separate documents or pieces of docs and join them at query time. This makes for efficient storage. Much lighter weight.
Semantic web allows creation of triples of data. Can’t really do this if data is in silos.
Use Case Example1 Large financial services firm using AIE to connect 90 separate applications to streamline the user experience. Determine which types of users use which applications. Ontology support.
Use Case Example2 AIE allows Bank IT group to quickly bring together data and content from multiple source and log files to identify issues and what’s causing them (the why) and fix them. Reduced time spent on each issue from 27 minutes to 3 minutes. Admins have a single place to solve entire problem.
Use Case Example3 Financial firm using AIE to observe and react to regulatory changes. AIE monitors 200 regulators and 700 news websites, alerts appropriate workers when a regulation in their area changes and gives them a way to adjust policies. Previously ad hoc approach. Some could slip through cracks. Gives both targeted and big picture view of regulatory issues.
SQL and JDBC support lets us connect to other BI tools.
Content spotlighting lets you focus on key phrases.
UIA is all about analyzing and enriching unstructured data, retaining the relationships, tying it together with structured data. Ultimately leads to better decision-making.
What don’t we do? We don’t have domain expertise for specified analytics. We have the tool but not the domain expertise. We’re not a provider of deep vertical apps. We sell a platform.
We don’t specialize in front end delivery. Our customers build it themselves with different tools/technologies. Use different languages to do this. We deliver toolkits. Our service people can help you build the UIs.
Attivio has 30 to 35 customers. 5 or 6 were Google Search Appliance replacements. “I don’t think Google understands the enterprise.”
One engineer that understands Java “can make the system (AIE) dance pretty quickly.” We focus heavily on things that make deployments go quickly.
Attivio has a lot of FAST alumni that have experience in the trenches that help customers implement AIE. We do implementations in weeks, not months or years. At one customer, went 7 weeks from contract to production with only 1 Attivio guy working with them. AIE also has easy-to-use GUIs to develop platform.
The biggest challenge for customers is getting access to systems. Some cultural issues more difficult than technical issues.
SPIKE. One architect from our services group sits with our customers’ developers over four or five days and when he leaves the customer has at least one data source integrated and one front-end tool created.
We use open source model in the way we work with developers. Let customers share best practices.
We focus on building apps that will power strategic initiatives. We want to unify info, but it needs to be done step-wise. Doesn’t happen all at once. You do this through a series of projects that solve business problems.
Scalability –We scale like a search engine with a MPP approach. Use scalability waterfall model. When on server is filled up, move to the next one. Can scale across mutli-node system.
Offer active dashboard that pushes relevant information to users. Factors: function, assigned responsibilities, interests. Sales manager with assigned territory. We present him with a dashboard of relevant sales pipeline and projections. We can create a Facebook-like news feed to keep him updated on activity relevant to his customers/sales area.
Action Item: By most estimates, around 80% of all corporate data is unstructured – mostly text in documents, web pages and emails. Organizations that do not take into account unstructured data when analyzing data and making business decisions risk missing key information and making incorrect, shortsighted decisions. Attivio’s goal is to help customers derive insight from all data available to them, be it structured or unstructured. AIE is a platform – not an application or end-user tool – to build applications that incorporate unstructured and structured data.
Footnotes: