#memeconnect #ddn
Big data is often thought of as the province of the big Internet-based companies – Yahoo!, Google, and Facebook in particular. Jean-Luc Chantelain, executive VP of strategy and technology for DataDirect Networks disagrees. He believes that all companies have very large amounts of data, much of it unstructured, that does not make it into their data warehouses or analytics. That data includes relational data on their customers available from Facebook and other Internet sources, multi-channel interactions between customers and company service desks and sales personnel, Tweets and even videos posted by users that mention or involve the company or its products, or provide clues to consumer or business customer needs and desires that represent unrecognized business opportunities.
The problem, he told SiliconAngle.com founder John Furrier on Extraction Point, Mr. Furrier's interview program on SiliconAngle.TV, is that, “We are only at the start of trying to understand the data.”
In many industries, executives are only beginning to realize that this huge pool of data exists, and they often do not yet understand its value. For years they have waited for the revelations promised by the data warehouses and analysis of transactional data. They have invested huge amounts of time, effort, and money with a largely disappointing result. The data warehouse “has not delivered on its promises because it's incomplete, it needs to get not only the transactional data but other kinds of data as well,” he says. Meanwhile, business analytics continues to focus on “rear-view mirror” views of what happened in the form of quarterly and annual reports and compliance audits, in part because enterprises cannot gather the data from across their often vast, worldwide operations, load it in a central data warehouse, and analyze it fast enough to show what is happening now as opposed to three months ago.
“We need more proofs that analytics are going to make a difference,” he said. Fortunately, those are beginning to appear from verticals including financial services, hospitality, and oil and gas, that are pioneering the use of near-real-time big data analytics “So what we’re going to see, I think, is that gradually, the proof will start showing up, and more traditional enterprises will see the value.”
Another problem, he says, is that executives often do not understand what questions they should ask. “They have to understand what moves their business, what they can tweak in the business, before they can figure out the right question.”
One problem is that this data is very rich in metadata. Different companies or even different executives in a single organization can ask very different questions from the same data and get valuable insights. Someone might post a video on Facebook that can provide some insight into how that person uses a specific product. But then five people may comment on that video while others “like” it. The VP of sales might analyze that data to identify the network of customers and prospects it reveals and to find the key influencer in that network who could help the company drive sales. The head of product design might analyze that same data to find clues as to how to improve the design of the product to better meet customer desires. And the head of service might analyze it for ways top improve service to increase customer satisfaction while controlling the service budget.
“I think the difference between purely structured data and this much larger data set may be how rich that data is and how many facets it has that it didn’t have when it was limited to transactional data,” he says. “That probably is what defines big data. There's a lot of it, and its different. It’s highly unstructured. And the good point about unstructured data is it’s massive on the value, but that value is very well hidden, and the magic is in how you’re going to extract that value.”
Mr. Furrier suggested that the situation is analogous to the Jody Foster movie “Contact” in which a blind radio astronomer discovers a hidden signal from the stars in the white noise the Aracebo radio telescope is receiving from outer space. “I found that analogy to be very close to what an enterprise or even a customer is going to have to do” with big data, Mr. Chatelain said.
And one of the most significant aspects of this much more inclusive data set is, “if you can capture and analyze it in real time, it’s really predictive,” he says. This makes it much more useful that the vast majority of today's reports, which are like “watching a football game two weeks after it happens.” When businesses try to predict the future based on traditional analysis they presume that the trends of the past will continue into the future, but that often is not a valid presumption. Unexpected events, new ideas, and changing customer needs often bring older trends to an end and start new ones. The collapse of the U.S. real estate market bubble in 2008, for instance, created a radical new economic reality that left many companies struggling to catch up. More recently, the iPad, has radically changed the consumer computing marketplace. The impact of the earthquake, tsunami, and consequent explosions at the Fukushima Dai-ichi nuclear plant on the world's economy have yet to be seen.
“So the most valuable insights from big data are going to come from the ability to analyze the data and act on that analysis quickly, ideally as the information is created,” he said.
Action Item: The full data set of unstructured, semi-structured, and structured data that describes the enterprise and its activities, employees, customers, and investors, holds extremely valuable information. But to get the most from that very large data set, enterprises must look beyond the traditional relational data warehouse to new technologies the can combine these different data types, capture all this data in near-real-time, and support analysis as the data arrives. This often means analyzing incomplete sets of data because by the time all the data is created it may become too late to make changes that will allow the company to realize full value from new opportunities. And executives must learn to ask new questions of the analysis engine to extract valuable knowledge from these huge masses of data.
Footnotes:
