Introduction

On July 10, 2012, the Wikibon community participated in a Peer Incite entitled Combining Unstructured and Structured Data to Deliver Big Data Business Value. The community learned from the experiences of Brigham Hyde, Ph.D., managing director at Relay Technology Management, how leveraging both structured and unstructured data, including the full complement of patent documents from the U.S. Patent and Trade Office, can reduce decision-making risk for pharmaceutical companies and their investors. We were also joined by Sid Probstein, CTO of Attivio, a software company specializing in enterprise search solutions and unified information access.

Although the Peer Incite focused on health information, disease, and the mass of data that surrounds pharmaceutical company decisions for moving forward with particular kinds of research, the discussion has clear implications for mainstream CIOs looking at ways to use big data to enhance decision making — and the bottom line — in their own organizations.

Doubtless, many organizations are still living in an insular spreadsheet-centric world. In fact, I’ve seen them. These are the organizations that struggle to make heads or tails of their data, that struggle to develop information sources that can assist in decision-making and that struggle to make a decision stick. Their plight is defined by struggle, even if they don’t know it. The reasons: Lack of completeness in data analysis and, as a result, lack of true faith in the decisions that are derived from such. This results in decisions constantly being rehashed as new details are learned or as people uncomfortable with the incomplete information continue to bring forward concerns.

Although these organizations face difficulty in the short term, their CIOs have the opportunity to map a data vision that brings the organization to true insight derived from complete data powering decisions that are carried out with conviction based on the trust that the full data set engenders. This is how decisions should be carried out.

Two questions are important to consider in any decision:

1. How certain are we that the decision we’ve made is the right one based on the data at hand?
2. At what point are we certain we’re making the best possible decision?

Getting to a full and complete trust in data can take some time, but the benefits are vast. The vision that carries this forward should consist of three or four “phases” for lack of a better term. The fourth phase will depend on the nature of the organization.

CIOs are often perceived by business unit heads as trying to stay ahead of the data problems. However, they’re also seen as not being that in touch with the people using the data to make the decisions. This disconnect means that the CIO needs to engage broadly across the organization for a Big Data initiative to truly succeed.

Underlying premise

CIOs should start with the idea that the organization wants to begin using unified information access (UIA) on the company’s next project. This environment provides tools that enable broad levels of functionality in search, business intelligence, analytics, workflow, and dashboards.

This means the CIO needs to get the expertise on board, get the infrastructure in place, and merge together existing data silos into a cohesive whole. It also means that everybody should get some level of statistics training in order to be able to understand basic concepts underlying the data. This doesn’t mean going out and hiring a bunch of sophisticated Bayesian modelers. If you provide good tools out of the box, those with enough training will be able to understand what they’re doing.

Get the data structure right

Once you accept the reality of the data situation, you can begin to take real action. First, get the internal data house in order with a focus on improving the overall quality of the data (see Data Quality is too Important to Ignore). Don’t expect internal data cleanup efforts to be easy. They take a lot of work (see Leadership challenges of a data cleansing effort).

During this effort, bring together as many sources of internal data as possible and establish internal data governance structures and processes intended to keep data clean for the future.

Create a data access layer to make the data useful to internal end customers

Keep users engaged in the process by working with the organization to identify tools and services that can be used to access data in a seamless way.

Real business value begins to take place as the organization creates intelligence across what used to be disparate data silos. At this point, the organization should start seeing the value in the process that is taking place as the walls come down and as information sources become more cohesive and more complete, leading to greater levels of confidence in the decisions that are made with the data.

Integrate independent sources of data

But, no matter how much internal data an organization has there is a wider ecosystem from which to pull data. There are formal data sources and more informal ones, such as news articles and the like. The internal data may paint a picture, but the external data will add the frame and the fill in the corners of the canvas.

Every industry has industry-specific data sources that can be leveraged to help improve decisions across the entire industry. After all, if getting your own data house in order results in better support for decisions, what if you could harness the power of decisions made across the entire industry including that of industry analysts and even individual news articles that could have an impact on your direction?

Combine internal and external data sources and determine best ways to monetize data

With a fully integrated set of data elements and fully rationalized reporting, dashboards, business intelligence, and decision-making based on an absolutely complete data picture, decisions can be made with confidence with an understanding for all of the pros and cons that accompany that decision.

Depending on the organization, this could be the time when the initiative can become another line of business. After all, if the data has value to you, some form of the data may have value to others.

Action Item: In today’s world, speed is everything, and the faster that an organization can gather information for good decision making, the faster that organization can launch initiatives to support the best decision and get to market. This is what impacts the bottom line in a significant way.

CIOs need to embrace the "information" part of their titles and harness the power of the organization around the shared goal of improved decision-making using quality data. By partnering with the organization as a whole, the CIO can create a vision that improves internal data, integrates external data sources and eventually monetizes the investment.

With the right vision, the CIO can get the investment necessary to build a data system that inspires confidence and improves decision-making.

Footnotes: