With its Hadoop appliance announcement at HP Discover in Frankfurt today, HP is determined to bring its hardware and infrastructure management expertise to the open source Big Data framework. The AppSystem for Hadoop appliance is a single SKU box that bundles pre-tuned hardware, including HP network switches and ProLiant Gen8 servers, optimized with one of three Hadoop distributions from Cloudera, MapR or Hortonworks.
While the new appliance is hardly the first of its kind – EMC Greenplum, Teradata Aster, among others, already have Hadoop appliances on the market – HP’s version provides enterprises with solid cluster management capabilities that allow it to integrate with exiting infrastructure, a choice of Hadoop software, and includes connectivity to HP’s other notable Big Data assets Vertica and Autonomy.
On the management front, AppSystem for Hadoop includes HP Insight Cluster Management Utility software, a visualization-heavy management tool for managing and monitoring Hadoop environments. Cluster Manager integrates with HP’s existing AppManager infrastructure management tools, allowing administrators to manage the complete application and infrastructure environment via a single pane of glass. The goal is to provide transparency on when Hadoop jobs are running, which infrastructure is being used, impact on the rest of the data center, and, if there are performance issues, a simple way to pinpoint the location and cause of the problem, according to Paul Miller, Vice President of Converged Application Systems within HP’s Enterprise Group.
On connectivity side, in addition to choice of Hadoop distribution, Vertica and Autonomy customers can move data to and from the the new appliance via purpose-built connectors. Vertica can also interact and analyze data sitting within Hadoop via an external table framework. HP plans to develop similar custom-built connectors for non-HP data sources and make them available as downloadable services some time in 2013, as well as adding support for vanilla Apache Hadoop.
More generally, the appliance model removes the need to role-your-own Hadoop cluster — including procuring and deploying hardware, and tuning Hadoop to run optimally on said hardware – resulting in performance improvements and significantly faster time-to-insight than the do-it-yourself approach, which can take weeks or longer (sometimes significantly longer) and requires significant internal expertise.
HP also announced new, pre-tuned Vertica and Autonomy appliances in Frankfurt.
While not earth-shattering, the new Hadoop appliance is an important step for both HP specifically and Hadoop generally. For HP, which is in the midst of well-publicized turmoil brought about by the Autonomy write-down, AppSystem for Hadoop, as well as the Vertica and Autonomy appliances, demonstrates the company’s commitment to making enterprise software and Big Data analytics a core component of its turn-around strategy. For Hadoop, the new appliance serves as further validation from a stalwart tech giant that the open source framework is solidifying its position as a mainstay of enterprise Big Data practices.
HP customers will also benefit from the open nature of the appliance from a Hadoop distribution standpoint. Customers have their choice from among the Big 3 distribution vendors, which can’t be said for EMC Greenplum and Teradata Aster Hadoop appliance customers. In the spirit of coopetition, HP also has an edge on the Big 3 in terms of providing ad hoc, SQL-like query capabilities in conjunction with Hadoop. Vertica is a mature, high-performance relational analytic database, where as Cloudera, MapR and Hortonworks are still in the early stages of developing such functionality within the Hadoop framework itself.
That said, this current iteration of AppSystem for Hadoop is hopefully just the first of many to come from HP. Among other enhancements to the offering, the company must make good on its promise to delivering easily-consumable connectivity to non-HP data sources, including Oracle and SAP but also to NoSQL data stores such as MongoDB and Cassandra. Even better, HP should join Cloudera, Hadapt and others in working to bring SQL and relational capabilities into the Hadoop framework, rather than simply relying on data connectors that adequately facilitate analyzing Big Data but add significant time and effort to the process.