The biggest week of the Big Data calendar is here. Kicking off tomorrow #theCUBE is live at #BigDataNYC 2014 for two days of continuous coverage. We’ll be interviewing Big Data practitioners, thought-leaders and technologies live from the Hilton Times Square in New York City. Watch live coverage at live.siliconangle.tv.
On Thursday (10/16) afternoon, tune in to catch my presentation to capital markets professionals on the state of Big Data in the enterprise, the implications for IT heavyweights (Oracle, SAP, Teradata, etc.) and what savvy investors need to consider when placing their bets on the Big Data market. Following my presentation, we’ll have a live panel discussion to continue the conversation. Panelists include myself former Nokia Big Data team lead (and current Cloudera Big Data Evangelist) Amy O’Connor, Big Data Guru and CEO of Tresata Abhi Mehta, and former Cowen and Company software analyst Peter Goldmacher.
HP and Hortonworks deepened their relationship last week, and the deal says a lot more about the former than it does the latter.
The news is that HP is investing $50 million in Hortonworks for about a 5% ownership stake in the company (Hortonworks’ Series D valuation is estimated at $1.1 billion) and a seat on Hortonworks’ board. HP will resell the Hortonworks Data Platform (HDP) and provide Tier 1 support to customers. The two companies will also work together to certify the HP Vertica analytic database on YARN.
Organizations are increasingly coming to the realization that data is a core strategic asset, the new source of competitive advantage. Success in today’s economy is predicated on how organizations leverage data as much as any other corporate function. The challenge organizations face, then, is how to leverage this asset to its maximum potential in ways that are as efficient as possible and also minimize risk.
This is no small feat. It involves data itself (identifying and managing sources of data), technology (tools and systems to ingest, process, store, analyze and share data), governance (ensuring data is used ethically and in compliance with relevant policies/regulations) and people (aligning various stakeholders and business objectives.)
Ed Walsh, the CEO whiz kid is back doing what he does best – running a product company. This time he’s landed at Catalogic Software, a firm many people have never heard of—but probably will over the next several months.
Walsh is a storage industry veteran who is best known for taking small, relatively unknown product-focused companies that need clearer strategy and execution chops, building a team, pointing them at a problem, gaining traction and then selling to a larger player who needs to fill a gap. Walsh touts an impressive list of successful exits as a CEO, including Avamar (EMC), Virtual Iron (Oracle) and Storwize (IBM).
The “big four” megatrends of cloud, mobile, social and Big Data are putting new pressures on IT departments. These high level forces are rippling through to create demands on infrastructure as follows:
Cloud – Amazon has turned the data center into an API. This trend is forcing CIOs, as best they can, to replicate the agility, cost structure and flexibility of external cloud service providers. Unlike outsourcing of the 1990’s, which had diseconomies of scale at volume, cloud services have marginal economics that track software (i.e. incremental costs go toward zero). This trend will put added pressure on IT to respond to the cloud.
Despite the apparent contradiction, Hadoop and other emerging Big Data approaches are at the same time complementary to and disruptive to established data warehousing and business intelligence practices in the enterprise. I recently spoke with my colleague Stu Miniman about this and other findings from Wikibon’s Q2 2014 Big Data Analytics Survey in the below Cube Conversation. The survey, one of two major Big Data surveys Wikibon will undertake this year, is part of Wikibon’s new Big Data research service. The new service is focused on primary data-driven research designed to uncover how Big Data is practically applied in today’s enterprise, explore the impact on existing modes of data management and analytics, and to understand its implications for existing and start-up Big Data vendors. To find out more about Wikibon’s new Big Data research service, please email
The big Internet companies like Google, Amazon, Yahoo and Facebook are having a profound impact on the IT ecosystem. Wikibon has been tracking the hyperscale infrastructure developments for the past few years (see much of this at Wikibon.org/SLI – our Software-led Infrastructure page) and how both the operational models and technologies used are impacting service providers and enterprise environments. The hyperscale (or web-scale) companies still only make up about 20% of IT revenue, but more signifiantly, most of the growth (see server revenue as a barometer of this activity). There are significant differences between the applications in hyperscale (custom written to optimize for massive scale) and enterprise (typically off-the shelf), so the theme is not that the enterprise of tomorrow will look like the web companies of today.
Tape is Dead, Not!
The combination of tape and flash will yield much better performance and substantially lower cost than spinning disk. This statement will prove true for long-term data retention use cases storing large data objects. The implications of this forecast are: 1) Tape is relevant in this age of Big Data; 2) Certain tape markets may actually show growth again; 3) Spinning disk is getting squeezed from the top by flash and from below by a disk/tape mashup we call “flape.”
Spinning Disk: Slow and Getting Slower
This week there are two important enterprise technology conferences taking place. One – SAPPHIRE 2014 – will see an old guard enterprise tech giant attempt to show it is capable of adapting to a technology landscape increasingly dominated by the cloud and Big Data. The other – Hadoop Summit 2014 – will see dozens of start-ups born in this new world out to prove to cautious CIOs that their technologies and platforms are ready for enterprise-level workloads.
It’s an interesting juxtaposition. SAP is determined to join the ranks of the “cool” cloud and Big Data companies (Salesforce.com, Hortonworks, Amazon Web Services), while those cool companies are equally determined to join the “enterprise-grade” club dominated by IBM, Oracle and, yes, SAP.
It isn’t the zombie apocalypse, but for too long, IT administrators have been shackled to infrastructure that was as friendly and stable as the stumbling undead. The coordination between application, infrastructure and physical data center was poor, leading to over 70% of resources being spent on adjusting configurations and trying to keep the lights on. Hyperscale cloud providers were built for scalability from day 1, so they had to be able to manage orders of magnitude more gear with a smaller IT staff. While cloud providers can customize new applications, enterprise users are burdened with a portfolio of legacy applications. The transformation to a scalable, agile and fast methodology isn’t simple, there are lessons and technologies that the enterprise can learn from the largest IT shops.