Home

Wikitip

Big Data and Rise of Predictive Analytics

Back in mid 1990s, when I was a Ph.D. student, one of the professors asked me what my career goal was and I said: “To help clients serve their customers better through use of Information Technology and Analytics“.

After completing my Ph.D. in November of 1998, when the IT revolution in Enterprise Solutions was about to take off and large and medium companies had started investing in ERP or CRM systems in a major way, I thought it was just a matter of time before Predictive Analytics goes main stream.

Back then, Siebel ’98 and Vantive were the ‘hottest‘ new tools in the market and Dot Com boom was on its way. I expected that in a year or two (or may be three), predictive analytics would become part and partial of all enterprise information systems and use of statistical techniques such as Regression Analysis, Factor Analysis and Multi-Dimensional Scaling would be common while analyzing and reporting information collected using ERP or CRM systems.

Looking back, I think I was over optimistic as this did not happen around 2001-2002 time frame as I expected. Most of the ERP and CRM applications had bare bones reporting functionality with just frequency (%) and advanced analytics was not leveraged.

If an application manager wanted anything more than frequency or % information, he/she had to invest in Business Intelligence (BI) or Data Warehouse (DW) solution. But again, BI or DW solutions analyze past information and are not “predictive” in nature the way high end statistical tools can be.

Yes, one could invest in SAS based analytics solution, but that was expensive, time consuming and out of reach of most companies – even Fortune 500 ones. As a result, use of Predictive Analytics was limited to a handful of use cases such as fraud detection in banks or customer churn management in telecom companies for example, where one could justify the investment in terms of time, effort and costs. But for a majority of ERP or CRM applications, data collected was never analyzed using Predictive Analytics and as a result investments in ERP or CRM systems did not deliver expected return on investment (ROI).

Things started to change around 2008-2009 with advent of social media tools. Again, I thought that it was just a matter of 12-18 months before Predictive Analytics goes main stream as large organizations will be required to use analytics tool to engage their customers on social media channels. In a blog post titled Social Media: The New Front End of CRM System published three years ago, I wrote that “the best any marketer can do is to Listen and Learn from what customers are saying and Engage them in meaningful conversations. In other words, treat Social Media channels as the front-end of CRM system, capture all relevant information from Social Media channels in the database and use Predictive Analytics and Knowledge Management tools to derive insights and help in decision making”. Again it turned out to be a case of over optimism as advanced analytics did not go mainstream in 2009-2010 as I expected.

Finally, yes, I say FINALLY after waiting for fifteen years, I am happy to say that Predictive Analytics (or Advanced Analytics as some would prefer to say) is going main stream in 2013, thanks to Big Data. It is not too difficult to understand why given the three Vs of Big Data, namely Volume, Variety and Velocity. Only way one can derive full benefits out of Big Data is by using predictive analytics and this is forcing large and medium companies to make necessary investments in building analytics infrastructure and reporting capabilities. And this is excellent news for those in Analytics profession or technology companies/service providers who help clients derive insights from mountains of (big) data.

What do you think? Do you agree that Predictive Analytics is going main stream in 2013 or is it a case of over optimism? Would love to hear your thoughts:

Link to original post: http://hkotadia.com/archives/5358

About the author:

http://www.linkedin.com/in/hkotadia

View Another Wikitip

Featured Case Study

Financial giant goes green

The corporate IT group of a very large, worldwide financial organization with 100,000 employees, has initiated an ongoing “greening” process. This is focused largely on reducing energy use both to decrease the corporation's carbon footprint while creating a net savings in operational costs over the lifetime of new, more energy-efficient equipment, including new storage systems.

read more...

Featured How-To Note

Planning a Green Storage Initiative

Fluctuating energy prices have heightened electricity and energy consumption as a major issue within the technology community. IT is a significant consumer of energy and IT energy costs have been rising disproportionately because of continued investment in denser IT equipment. Estimates from the EPA and others indicate that IT will account for 3% of energy consumption by 2012.

read more...