At the November 27, 2012 Wikibon Peer Incite we heard Dag Liodden, Co-Founder and CTO of Tapad, describe how the ad tech company is exploiting extended-memory databases to drive business results. Specifically, rather than use a pure in-memory architecture, Tapad is leveraging technology from Aerospike to deliver ad units in real-time to potential buyers.
The key technological nuance is that the Aerospike database allows Tapad to address memory as a “single level store” by deploying flash as a memory extension. While there may be a slight tradeoff in performance with this approach (relative to an all DRAM in-memory system), the cost benefits are so substantial that they outweigh pure in-memory alternatives.
It is becoming increasingly apparent to a group of practitioners within the Wikibon community that data management will evolve from today’s norm – where transactional databases are isolated from analytics systems and feed them in a batch manner – to a new situation where transactional data is fed to analytics systems and relevant decisions can be made in real time (defined as before you lose the customer).
Technology vendors are going to market with two dominant approaches to this trend, including:
- Do the heavy transaction lifting on our “big iron” system then ship them to our expensive data warehouse, and we’ll do the “real-time” processing;
- Systems such as Aerospike that have the potential to unify transactional and analytic data to allow machines to make decisions.
We see the latter as the next generation approach with a much greater ability to deliver tangible business value at scale. There are caveats. In particular use of such systems will forego traditional rich SQL functionality, so practitioners should identify use cases where such capabilities are not needed. Often these are ad-serving or emerging social analytics applications as well as certain risk and fraud use cases in financial services. In addition, to the extent that flash is used as a memory extension, business cases that extend over three years or more should assume flash systems will need to be replaced due to wear out factors from these demanding workloads. Nonetheless, in many cases these added costs are inconsequential relative to overall business value.
Action Item: Innovative suppliers of database and data management technologies are finding new opportunities within so-called Big, Fast data markets-- i.e. those that feed transaction data into analytics systems in "real-time." Increasingly, this unified approach will drive greater levels of business value, and traditional vendors must find ways to compete without a "Big Iron" dogma. Specifically, suppliers must design next gen systems that leverage flash as an extension of memory to provide predictable and consistent performance at dramatically lower costs than pure in-memory database systems.
Footnotes:
