Dealing with Data System Complexity in Your Applications

1. Executive Summary

Organizations build applications with transactional/operational databases, analytical databases, and business intelligence.

Transactions happen at all hours and come from various sources. The speed of transactional data creation has soared, and these transactional workloads are processed by database engines designed and tuned for high transactional throughput.

Meanwhile, the big data generated by all the transactions require analytics platforms to load, store, and analyze volumes of data at high speed, providing timely insights to businesses. Data-driven organizations leverage this data, for example, for advanced analysis to market new promotions, to enable operational analytics to drive efficiency, or for predictive analytics to evaluate credit risk and detect fraud.

Thus, in conventional information architectures, this requires two different database technologies: online transactional processing (OLTP) database management systems (DBMS) to handle transactional workloads and online analytical processing (OLAP) DBMS to perform analytics and reporting. Data types also drive multiple technologies since many databases specialize in types like time series, geospatial, graph, JSON, etc.

This is only one of many areas of complexity in modern application development. If there is a single database that can be used to avoid the overhead, it is worthwhile to look into that database for complete application management. This report will explore some of the realities of modern enterprise application development, the stack for the response, and taking an architected approach with a consolidated database layer that meets required selection vectors.