The Ongoing OLAP Revolution
This is the first of several posts that will examine developments that continue to make OLAP (Online Analytical Processing) an exceptionally powerful, significantly valuable sector of the software solution market. As we will see, some of these developments have the potential of fulfilling the “promised” capabilities of OLAP as defined when the term was first used. And in that respect they truly are revolutionary. Of particular note is a technology that provides online capabilities via in-memory calculations in relational tables combined with dynamic relational database connectivity, which will be examined further in upcoming articles.
The topic under consideration here concerns the very first word of the OLAP acronym, Online. The word conveys the expectation of immediacy and dynamic continuity—something that we understand will happen “live”. We expect, for example, when we see someone “online” that we can send her an instant message and get an immediate, live response in real time.
What about Online Analytical Processing: has it lived up to the billing of “online-ness”? The short answer is No: not to the extent that, for example, transaction-processing is live, as when I pay my credit card bill online! And next receive an email nearly instantaneously.
Before considering this notion of “online-ness” further, and how OLAP technology vendors have tried to address it, we can briefly review the market and note how the expectation of being online arose in the first place…
Learn More: OLAP and Business Intelligence History
History Behind the Need for OLAP Technology
The term OLAP was coined 1993 by E.F. Codd, who listed 12 rules for defining what the technology should feature. Known as “the father of the relational database”—OLTP (Online Transactional Processing) systems—Codd was enumerating points related to what the newly described technology, OLAP, should do. Each of the 12 initial OLAP rules are worth examining to measure the definition versus the reality of the technology. [Further examination of these rules vs the reality in the field today will be the subjects of our upcoming posts] For the moment, we should note that Codd did not prescribe how OLAP technology should achieve online capability. In fact, the requirement to be online is not one of the 12 rules. But by bequeathing that OLAP acronym, he certainly raised expectations.
Over the ensuing 20+ years OLAP has developed into a core technology that, at minimum, provides the fastest possible access to data that can be “slice and diced” or viewed through a compelling front end, like a dashboard. Typically this is reporting data. On-the-fly analytics are enabled to some extent, though it’s a telling fact that so much reporting and analytical data still ends up in Excel spreadsheets.
The Paradigm Shift from OLAP “Processing” to Business Intelligence Decision Making
And so by now, OLAP and multidimensional have entered the popular technology lexicon, as have friendlier terms like Business Intelligence and Corporate Performance Management. The product market has grown to significant proportions as vendors highlight their product capabilities and benefits…and there are even some that define themselves as being the “antidote” to OLAP! But all speak to the business value and immediacy with which they deliver results, for the fastest possible decision support possible. “Online, in real time,” or approaching it, even if not in actual fact.
Let’s consider first to how vendors provide as much dynamism in report and dashboard manipulations as possible: in this case, they have most recently employed in-memory technologies to run all processing at run time. They make the argument that the exceptionally fast results available in selecting and reorganizing report data constitutes online analysis. At the same time it is generally acknowledged that even in-memory capabilities slow down with significant amounts of data. And the definition of analysis becomes awfully limited if we are talking only about new report creation and manipulations of existing dashboard elements. Put another way: if we require more expansive requirements of an OLAP system—one that allows ad hoc modeling and complex calculations, and even write-back for planning purposes—in memory capabilities alone don’t stand a chance of performing as well as advertised, much less in an online capacity.
Learn More: Types of OLAP Systems
Enhancing the Features of OLAP and Business Intelligence Technology Applications
Next, there are those technologies that provide far more extensive mathematical as well as planning, capabilities for “what if” applications, most often related to budgeting and forecasting. The way most vendors in this product category have met the challenges of delivering fast results is through the use of a proprietary OLAP (analytical) database. In this product paradigm, an ETL (extract, transform and load) process is required from the OLTP system, and data aggregations and calculations occur in the analytical database. The benefit of these systems is far greater responsiveness in building entirely new models, defining complex calculations and even expanding the definition of OLAP to include budgeting, forecasting and planning capabilities that allow for all kinds of “what if” and “write back” modeling.
The key online issue with these systems, however, is that the “write back” will occur to the analytical database, which must programmatically send it back (if necessary) to the OLTP system. Keeping in mind that an ETL process is required in the first instance (from the OLTP to the OLAP database), there is always some kind of timed update process required to bring data from “there” to “here,” or vice versa. Hardly the definition of a truly online system.
So the question still remains: is OLAP technology still alive? Yes! It’s finally evolving to completely encompass what OLAP was defined to be as proposed by Codd in the 1990s.
Stay tuned for our next article — Codd’s Paper: A Definition or a Wish List?