Introduction

On November 27, the Wikibon community was treated to a Peer Incite entitled Optimizing Infrastructure for Analytics-Driven Real-Time Decision Making. We were joined by Dag Liodden, Co-Founder and CTO of Tapad.

Tapad’s business model relies in the ability for its underlying technology to make millisecond decisions based on a variety of factors and then to deliver the ultimate results to a user’s browser session. If the company fails to meet stringent timing needs, it loses the opportunity to bid on placing ads for that particular user session. In other words, the wrong technology for Tapad would directly result in a bottom line issue.

Tapad’s needs are what most mainstream CIOs would consider very niche as most don’t require sub-millisecond response times, even for their most critical line-of-business apps. However, beyond Tapad’s technology was the thought process that led the company to eschew the use of RAM-based storage in favor of SSD-based storage. In that, I see three points of consideration for CIOs.

Predictable and consistent performance

We all hear about companies releasing marketing fluff that describes their products as being able to provide what amounts to insane levels of performance. For example, “Our new storage system stress test just pushed more than 1 million IOPS!” While it’s great that engineers were able to tweak things to achieve that level, what most CIOs are expecting is some semblance of predictability in how a system performs on an ongoing basis, not just in one specific use case.

Tapad had the same challenges. It needed to ensure that its advertising solution performed at very predictable levels. While there were solutions available that might have been able to push performance a bit higher, there was no guarantee regarding ongoing performance.

Sometimes, the most efficient is not the most effective

This has become something of another mantra of mine: Efficient does not always equal effective. I could design an incredibly efficient process, for example, but if the result is that a customer leaves dissatisfied every time, it’s not an effective process. The same goes for technology. Tapad needed database storage that was extremely fast. Although in-RAM solutions provide the best possible performance levels, they do so in ways that can’t always be predictable and that are certainly on the high end of the expense scale.

So, rather than investing in the most efficient solution, RAM-based storage, it instead relies on slower solid-state drives that provide it with the more effective, sustainable, and predictable outcomes based on its business needs.

A merging of operational and analytical databases is inevitable

Finally, it’s clear that Liodden's solution meets his very specific business use case and that the mainstream doesn’t yet have these kinds of needs. But, we’re getting there. Over time, what was once a niche solution can quickly become mainstream as the technology catches up and prices come down.

As CIOs continue to struggle under ever-increasing masses of data, it’s the kinds of technologies discussed in this Peer Incite that will enable the opportunity to merge today’s separated transactional and analytical applications. As CIOs look at the two kinds of databases today:

• Transactional databases are used to carry out ongoing business as usual, such as processing orders, updating customer information and so on.
• Analytics is used to monitor ongoing performance metrics, gauge current vs. historical trends, and, in some cases, to inform the transaction system of trends that might change how operations are handled.

With the capability of today’s hardware and software, the ability to combine these systems is starting to become reality, with the result being both the transactional system and the analytical applications will operate in real time and dynamically update one another so that the most current possible information is being served based on everything known about the current transaction as well as trends that might be related to parts of that transaction. This type of transition will require CIOs to consider new architectures and new databases which are additive to, not replacements for, existing databases.

Action Item: Potential action items to consider from this Peer Incite include:

• CIOs should always continue to focus on the most effective outcome that appropriately considers all elements of the process, not just on buying the fastest equipment out there.
• It’s easy to get impressed by vendor hype touting niche, massively-tweaked outcomes, but the focus should be on how solutions operate on an ongoing basis.
• CIOs might consider ways that they can begin creating deeper combinations of their siloed databases in order to ensure that the organization can make the best possible data-driven decisions.

Footnotes: