EMC has been touting its “Cloud Meets Big Data” messaging for nearly two years now, and today it took a major step in transforming that message into reality.
EMC announced that it is forming a new “virtual organization” focused on Big Data and application development in the cloud. EMC is calling the new organization the Pivotal Initiative and it will include 800 employees from EMC’s Greenplum and Pivotal Labs divisions, and 600 employees from VMware’s vFabric, Cloud Foundry, GemFire, SpringSource and Cetas organizations. EMC owns over 80% of VMware, where former EMC COO Pat Gelsinger joined as CEO earlier this fall.
According to EMC, Paul Maritz, former VMware CEO and current EMC Chief Strategist with a focus on Big Data and cloud apps, will run the Pivotal Initiative. But sources report that EMC may appoint a dedicated figurehead to serve as the face of the new virtual organization next year, allowing Maritz to orchestrate its strategic direction behind the scenes. (This would be in line with Maritz’s own words — see video at end of this post.)
The main goal of the new organization is fairly straight-forward. Much of what is considered Big Data – such as social media data, mobile device data and other third-party data sets – live outside enterprise data centers. EMC believes, therefor, that Big Data projects are a perfect fit for cloud deployments. With the Pivotal Initiative, EMC wants to capitalize on both these trends by delivering a technology framework and related tools to help developers build modern applications on top of Big Data platforms like Hadoop that are also optimized for the cloud.
The other impetus for the spin-out is likely that Big Data, analytics and application development do not fit neatly inside either EMC’s or VMwares’ hardware and infrastructure-focused org charts. Pulling the various divisions into one operational unit should allow for better collaboration among the two companies’ cloud, Big Data and application developers.
The Pivotal Initiative isn’t going to just waltz in and dominate the Cloud meets Big Data market, however. It faces stiff competition in the form of Amazon Web Services, which just last week beefed up its own Big-Data-as-a-Service portfolio with a new on-demand analytic database offering called Redshift and a data workflow tool called Data Pipeline. These are in addition to AWS Elastic MapReduce, Amazon’s Hadoop-as-a-Service offering, and MapR’s Hadoop distribution hosted in the Amazon cloud.