Wikibon has written extensively about new designs for applications together with a new mobile-cloud application model. The reason for these changes is not to decrease application operational costs or even to decrease development costs (though both of these may happen). Focusing on budget reduction misses the real business opportunity.
The fundamental reason for adopting these approaches is to provide new applications with significant increases in productivity and functionality for the organizations that adopt them. Fundamental improvements are not about a 2% increase in revenue or productivity. Fundamental improvements are about radically reducing the number of people required to provide an organizational service and developing new ways of doing business. The key is orders of magnitude more data (big data) from more sources delivered to anybody, anywhere, on any device, and where possible result in the complete automation of organization processes.
The discussion focus for this Wikibon Alert is the availability of platforms on which to develop mobile-cloud applications with integrated near real-time analytics and automated feedback into operational systems. The two platforms analyzed are Pivotal and AWS.
Contents |
Data Flows in a Cloud-Mobile Application Model
Figure 1 details many types of data from outside an organization that can be used to monitor business processes, from the We and other data centers, telemetry and other data sources and aggregators of data. Much of that data will be purchased, and much of the data inside of an organization will be for sale.
Figure 1 also shows a topology of enterprise systems and the interactions among them. These systems maybe within an on-premise private cloud, within an external outsourced private cloud, a public cloud, or a mixture of some or all of them.
The top of the chart shows data sources from external sources, such as data from the Internet of things, and stream data from the net or data aggregators. The left side of the triangle represents the operational systems-of-record, and the right side the planning and analytical control systems. At the bottom are the backup and archive recovery systems. The triangle includes different types of systems:
- Big Streams: Real-time data analyzed by real-time transactional and analytical systems to make near real-time recommendations or updates on (say) the price of goods and services to an individual customer.
- Big Systems: Systems-of-record that process changes in input from customers, suppliers, partners, etc.
- Big Data: Systems will organize and analyze lakes of structured and unstructured data, with much tighter integration between the data warehouse and the systems of record.
- Metadata & Big Flash: Metadata held on large all-flash repositories that allow systems to understand the data available to them, often also on flash rather than disk storage. Keeping this metadata on flash is important because of the disparity between the accessibility of data on magnetic media compared with flash.
Source: © Wikibon 2013, Based on Figure 6 of Flash and Hyperscale Changing Database and System Design Forever
Hauling vs. Back-hauling Data
The advent of Big Data streams from the Web, data aggregators, and the universe of things will mean that the cost of hauling data to and from an enterprise site will escalate, as well as the elapsed time to receive and distribute the data. One remedy is to physically move the enterprise systems to very large data centers that are likely to house cloud providers, data aggregators, and the systems of competing enterprises. The cost and elapsed time to "back-haul" data by throwing a cable between two systems within a mega data center is far lower than any other access method and will allow a rich ecosystem of support cloud systems to develop. A model for such an approach is the SuperNAP colocation facilities in Las Vegas, Nevada, as well as sharing of Equinix facilities with Amazon.
Pivotal
The EMC strategy reflects EMC's belief that new applications made with a mobile-cloud application model will replace today's client/server applications. The prime movers in this space (Amazon, Apple, Google, Facebook) had to develop their own platforms. These applications are disruptive to existing infrastructure in general and storage arrays in particular. For example, the application will be responsible for availability and recoverability using commodity hardware, as in new ISV offerings such as Microsoft Exchange 2012/2013 and the Aerospike database.
Pivotal as a part of the EMC federation is putting a horse into the race to be the new platform for mobile-cloud, and in doing so help other EMC companies, for instance by using VMware as a basis for the virtualization, EMC storage software for some of the storage components and RSA for some of the security. As with VMware, Pivotal has been spun out as a separate company, with GE investing $100m for a 10% stake. EMC has a 69% stake, and VMware a 31% stake.
Pivotal is at heart a collection of open-source projects, which it hopes to meld together with proprietary extensions to enable much faster development of mobile-cloud applications. Pivotal has traditionally targeted developers, building communities from the bottom. Pivotal Cloud Foundry is a cloud abstraction and application automation PaaS service. Pivotal Spring is a bunch of "Lego" blocks for extending Java applications with three million loyal developers. Other components such as RabbitMQ messaging services, the tcServer extension of the TOMCAT server, and the redis KV store database, have dedicated developer followings.
Pivotal One Announcement
On November 12 2013, Pivotal announced Pivotal One as a Platform-as-a-Service (PaaS), an integrated platform that includes a set of application and data services that run on top of Pivotal CF. This platform will help development teams to build applications on cloud infrastructure that can be dialed-up and dialed-down, allowing enterprises flexibility on resources. The services announced with Pivotal One are:
- Pivotal CF, a distribution of Cloud Foundry;
- Pivotal HD, a distribution of Apache Hadoop;
- Pivotal AX, providing automatic analytic instrumentation;
- RabbitMQ, an open source messaging system, the tcServer extension of the TOMCAT server;
- MySQL service.
Verizon’s announced that it would add Cloud Foundry on top of the Verizon IaaS pltaform, to provide PaaS services to the Verizon Cloud services. Verizon has signed the Cloud Foundry community contributor license agreement (CCLA), and the company has also committed engineering resources to the Cloud Foundry open source project.
Pivotal Challenges
One of Pivotal’s major challenges is melding these components into a combined enterprise Platform-as-a-Service (PaaS). The overall platform could constrain the value of each component unless the developer sees value in all the other components. Forcing adoption of enterprise platforms, particularly immature ones, top-down by corporate edict is difficult. There are a great number of alternative solutions, and the business people funding any initiative are only interested in getting their project done as quickly as possible. Even if there were a corporate edict to use Pivotal, individual programmers/small teams would take the path of least resistance to coding and testing in the shortest possible time, using Amazon (or other) cloud platforms, where the reliability, ease of integration and range of solutions are superior.
Pivotal has 1,600 employees, and almost all the revenue at the moment comes from Gemfire (~50% plan, less in actuality) and Greenplum (~50% plan, much greater in actuality). Gemfire is a data-in-memory database, and Greenplum is a scale-put platform for Hadoop Big Data. A key management question for Pivotal leadership is how important the Pivotal platform is to the success of these products in the marketplace. Both of these products have a minor market share in a crowded market. Unless the Pivotal platform can assist these products to be successful, being part of the Pivotal group will be a distraction to two startups that need to focus on their core value to their customers.
Mobile-cloud applications will need to integrate many technologies, including in-memory databases, flash and big data Hadoop. Pushing in-house solutions is likely to diminish the Pivotal brand. Gemfire & Greenplum selling the Pivotal platform is likely to diminish their brands. Achieving economies of scale for marketing and sales have usually proved illusive, unless the products are complementary and the target buyers are the same. Neither is likely to be true with Pivotal, Gemfire and Greenplum. The result in organizations that try it is “marketectures” that waste internal resources and selling time; these marketecture slideshows appear be in full swing inside Pivotal (See Figure 2).
Source: Pivotal Presentation 2013
Other challenge for Pivotal include:
- The strong influence of GE (10% stake), which will pull the platform away from mobile-cloud towards the Internet-of-things and machine sensors. The emphasis of these applications will be extraction from distributed data, with time-series databases. It is difficult to see that all the different verticals in the Internet-of-things will coalesce round a single platform.
- The strong influence of VMware (31% stake), which will want Pivotal to focus on integrating VMware as the virtualization and abstraction layers. VMware recently launched the vCloud Hybrid Service (vCHS), with little traction or scale to date.
AWS as a Development Platform
The biggest development platform player is Amazon. The majority of shadow IT on Amazon is develop & test, with limited production at the moment. Amazon allows developers the total freedom to spin-up and try out different open-source and proprietary technologies and learn how to use each of them appropriately. It is here now. It interfaces with the infrastructure with a wide and ever widening set of APIs. The Amazon APIs are effective, dynamic, and constantly improving. Most open-source developers are making extensive use of Amazon for rapid development and early deployment. If successful, the application can be brought in-house.
Some of the traditional concerns with the Amazon public cloud are:
- Amazon is Proprietary – Pivotal is Open. The proprietary Amazon APIs make it difficult to leave Amazon;
- The proprietary APIs are a strength and weakness for Amazon. The strength is that they allow different modules of applications using different technologies to be integrated at development time and deployment time. The weakness is that bringing an application in-house where the equivalent APIs do not exist is hard coding work.
- Pivotal is focused on Apache open-source projects; however, to integrate all the different technologies into a working platform will require some proprietary glue from Pivotal. Open-source does not mean free!
- The cost justification for using Amazon (as the initially faster and more encompassing platform) comes down to the relative benefits of the applications being designed. If a project is adding a feature to an existing application that will add 1% productivity to 10% of the users, cost to develop and even more cost to deploy will determine the ROI on the project. However, Wikibon has consistently discussed and shown that the potential benefits of IO-centric mobile-cloud applications with integrated analytics and operations interfacing to Big Data have the potential for off-the-charts increases in user productivity and business innovation. In these cases Speed to Deploy will determine the ROI of the project. Unless Pivotal is as fast to deploy (and part of that is as fast to fail and retry different technologies) as Amazon for true mobile-cloud applications, it will not succeed in the marketplace.
- Understanding and knowing what you will pay is difficult with Amazon;
- This is a legitimate concern often expressed by Wikibon members; Amazon needs to fix this in a way that guarantees no budget surprises for customers.
- Public clouds such as Amazon are less secure against other company subpoenas, theft from other organizations and governments, and general hacking;
- Wikibon interviewed Steven Schmidt (VP Security Engineering & Chief Information Security Officer at AWS), who put forward the thesis that AWS is more secure than on-premise systems. This is probably true for most enterprise data centers. However, convincing regulators and boards is a challenge for AWS.
- Other countries will not allow data or communication to flow through American owned assets (The Snowden effect).
- This is a likely outcome of the NSA debacle, which is felt very strongly in Europe and the rest of the world. This should be a major concern to all American cloud providers. At the very least, American cloud providers (including PaaS) will need to spend more money setting up data centers in many more countries, and dealing with local communications and other equipment companies that guarantee not to work with the NSA. There are also some early moves to limit chips from U.S. and Chinese sources in servers and communication devices (the Iran centrifuge success may have had unintended consequences).
- Amazon and EMC are both seasoned international organizations and are likely to be able to respond to these mandates if necessary for all companies in their portfolios.
- Amazon storage lacks the availability, reliability & recoverability of traditional arrays in traditional enterprise data centers.
- A year ago this would have been a true statement. With the availability of additional Amazon storage services, and the ability to use external storage services in the same Equinix data centers (Zadara VPSA and Zadara-NetApp, both on a multi-tenant Storage-as-a-Service pay by the hour consumption model) and the ability to replicate across AWS regions, AWS is very close to the same levels of storage availability, reliability and recovery functionality of an enterprise data center.
Other Mobile-Cloud Development Platforms
Many companies will be competing for mobile-cloud development platform business, including Adobe, Google, IBM, Microsoft Azure, SAP and many others.
Conclusions
The key to success of all platforms (Apple, Android, Windows Desktop, Windows Server, Linux, etc.) is the support of application developers. Amazon has very strong support from developers, who find the ease-of-use, ease of experimentation and API integration provides a near-ideal environment for speed to deploy. Their attitude to being "entrapped" in Amazon with difficulty of "escaping" would be similar to a bear trapped on a river overhang catching salmon or in a field of beehives - bring it on! Competitors will have great difficulty in matching the completeness of software, the speed of introduction of new function and the ability to scale that Amazon has demonstrated to date.
For Pivotal to compete effectively, it would do well to jettison everything that does not directly contribute in helping its core platform foundation attract developers and focus on mobile-cloud. Other products within Pivotal that do not directly contribute to attracting developers for mobile-cloud applications could be in a separate part of the organization.
Action Item: It is a strategic imperative for organizations to determine key areas where mobile-cloud applications can change fundamental business processes. Understanding what technologies are important and how they can be integrated is a key learning process in developing these applications. For most organizations, the most important business metrics are speed to failure and speed to deployment, not cost to deploy. AWS flexibility is likely to be a major contributor to mobile-cloud application development for most organizations. Wikibon recommends that CIOs take a wait-and-see approach to Pivotal.
Footnotes:
