Posts Tagged Hadoop
Despite the apparent contradiction, Hadoop and other emerging Big Data approaches are at the same time complementary to and disruptive to established data warehousing and business intelligence practices in the enterprise. I recently spoke with my colleague Stu Miniman about this and other findings from Wikibon’s Q2 2014 Big Data Analytics Survey in the below Cube Conversation. The survey, one of two major Big Data surveys Wikibon will undertake this year, is part of Wikibon’s new Big Data research service. The new service is focused on primary data-driven research designed to uncover how Big Data is practically applied in today’s enterprise, explore the impact on existing modes of data management and analytics, and to understand its implications for existing and start-up Big Data vendors. To find out more about Wikibon’s new Big Data research service, please email
This week there are two important enterprise technology conferences taking place. One – SAPPHIRE 2014 – will see an old guard enterprise tech giant attempt to show it is capable of adapting to a technology landscape increasingly dominated by the cloud and Big Data. The other – Hadoop Summit 2014 – will see dozens of start-ups born in this new world out to prove to cautious CIOs that their technologies and platforms are ready for enterprise-level workloads.
It’s an interesting juxtaposition. SAP is determined to join the ranks of the “cool” cloud and Big Data companies (Salesforce.com, Hortonworks, Amazon Web Services), while those cool companies are equally determined to join the “enterprise-grade” club dominated by IBM, Oracle and, yes, SAP.
The Hadoop ecosystem is an eclectic mishmash of start-ups, mid-sized vendors and IT heavyweights with products and services up and down the Big Data stack. Inevitably the ecosystem will consolidate and thin itself out through mergers, acquisitions and – unfortunately for some of these start-ups – bankruptcies.
Consolidation is part of the natural evolution of any given technology market after an initial period of frenzied innovation, and the Big Data market is no exception. I believe we are witnessing the start of this consolidation today. It will take several years to play out, but the first phase of consolidation is manifesting itself in the form of strategic technical partnerships between vendors that play in different segments of the Hadoop market.
I’ve already laid out my predictions for Big Data in 2014, but I also wanted to let the Wikibon community know how my colleagues and I plan to cover Big Data in the year ahead. We’ve organized our research agenda into three major buckets.
Technology. Clearly the technologies and products that collectively make up Big Data – including Hadoop, NoSQL data stores, analytic databases, data visualization tools and more – are maturing at a rapid pace (much faster, for example, than relational databases did in the 1980s.) Big Data is also applicable across industries, meaning these technologies are inevitably and increasingly intersecting with adjacent technology movements, namely the cloud, mobile computing and social media. As we have for the last several years, Wikibon will devote significant coverage to these developments with an eye on putting technology innovations in context for enterprise Big Data practitioners (both technology practitioners and line-of-business practitioners.)
Tomorrow marks the kickoff of Strata Conference 2013. This year, SiliconANGLE Wikibon is expanding its coverage from two days to three full days of live broadcast from the show floor. Tune into theCUBE at SiliconANGLE.tv all week to catch it all, and log on to strataconf.com/live between 8:45 am and 10:00 am PST Wednesday and Thursday to watch the live keynotes.
We start things off Tuesday morning when we welcome Edd Dumbill, Co-Chair of the Strata Conference, to theCUBE. Edd and hosts Dave Vellante and John Furrier will preview the upcoming action and layout the themes we’ll be covering.
We all know there’s lots of excitement and buzz surrounding Hadoop, but talk to some CIOs in “non-web” industries about moving mission critical apps to the open source Big Data framework and you’re bound to hear a little fear in their voices.
They’re worried that Hadoop is not ready for primetime because it has a single point of failure. That is, if the NameNode in a cluster goes down, the entire cluster goes down. Spinning clusters back up into working order following a NameNode failure takes time and, by definition, mission critical applications can’t go down … ever. Until the SPOF is solved, more than a handful of Fortune 500 companies will continue paying Oracle through the nose rather than risk a disruption to critical apps.
With its Hadoop appliance announcement at HP Discover in Frankfurt today, HP is determined to bring its hardware and infrastructure management expertise to the open source Big Data framework. The AppSystem for Hadoop appliance is a single SKU box that bundles pre-tuned hardware, including HP network switches and ProLiant Gen8 servers, optimized with one of three Hadoop distributions from Cloudera, MapR or Hortonworks.
While the new appliance is hardly the first of its kind – EMC Greenplum, Teradata Aster, among others, already have Hadoop appliances on the market – HP’s version provides enterprises with solid cluster management capabilities that allow it to integrate with exiting infrastructure, a choice of Hadoop software, and includes connectivity to HP’s other notable Big Data assets Vertica and Autonomy.
The fear (or is it disdain?) is sometimes justified. No developer wants to get locked in to a platform that dictates which tools she can use, which data sources she can integrate, which hardware she must deploy or that makes switching to a competing platform too costly to justify.
Next week theCUBE is back in action, this time covering two Big Data conferences in one. Strata Conference + Hadoop World on theCUBE kicks off live Wednesday (10/24) morning at 10 am ET on SiliconANGLE.tv. We’re broadcasting all day Wednesday and all day Thursday (10/25) from New York City with virtually non-stop live interviews with the smartest nodes at the conference.
We’ve identified the most compelling news and trends that will be developing at the show and programed our coverage to flesh them out in great detail. Among other trending topics, you’ll get full coverage and analysis of the emerging Big Data application development market, the state of real-time analytics in Hadoop environments, and new ecosystem partnerships, as well as some great advice for Big Data practitioners from Big Data practitioners.
Our friend Matt Asay, who oversees business development for streaming Big Data analytics player Nodeable (read hear about Nodeable’s recent shift in business model), penned a column today sizing up the Hadoop distribution competition. Asay narrows the competitors to two – Hortonworks and Cloudera – and proceeds under the premise that only one of the two can and will survive.