TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

October 19, 2006: TDWI FlashPoint - Synergy with SOA

In this issue, Evan Levy discusses service-oriented architecture.

• October 23, 2006

Welcome to TDWI FlashPoint. In this issue, Evan Levy discusses service-oriented architecture.

Synergy with SOA: What Data Warehouse Professionals Should Know

A 2006 InformationWeek survey of 120 business and technology professionals found that 82 percent of respondents hoped to achieve increased flexibility by moving to a service-oriented architecture (SOA) framework. With the pervasiveness and promise of SOA, it’s certain to affect not only operational but also analytical systems. Here’s a compendium of SOA basics that BI and data warehouse managers and practitioners should know.

SOA Defined

SOA is as much a business strategy as it is a technology framework. Simply put, SOA enables the use of loosely coupled business processes to a range of systems and applications across the business. These processes are treated as enterprise “services” that can be accessed by a range of applications without mandating specific knowledge of their underlying hardware, software, or operating systems. Via SOA, these services can interoperate: for instance, when a customer places an order (Service 1), the back-end systems can confirm inventory availability (Service 2), process payment (Service 3), and fulfill the order (Service 4). Each of these services is its own individual business process, and each can communicate with other business processes via SOA.

Since SOA isn’t operating system–specific, it removes the complexity of data translation and connectivity. Thus, a salesperson with a BlackBerry and a customer service representative using a Web-based terminal will see the same information with their respective applications—receiving critical customer information via SOA.

How SOA Works

SOA simplifies access to repetitive processes. Programming becomes easier with SOA, and business processes that are repeatable are provided as services to systems throughout the enterprise. Making a purchase, paying a bill, or shipping a product are all repeatable services.

How many times have you heard that—despite advances in data integration technologies—your systems “still don’t talk to each other”? In the context of the data warehouse, SOA makes data access much simpler, especially for operational queries that represent specific business events.

For instance, when a customer buys a tent from the online store of an outdoor gear retailer, the Web site might cross-sell another product in a “custom offer” message to the customer during the purchase transaction. While the data warehouse is rich with purchase details, the code necessary to identify the recommended product is very complex and requires significant data and development expertise. Prior to SOA, each application developer supporting order entry processing (whether via Web site or call center) needed to learn about the structures and data within the data warehouse, develop specialized queries, and construct application logic to process the query and answer set. With SOA, a single developer can construct a product recommendation “service” that contains all data warehouse, query, and data manipulation logic, and then provide access to all other applications.

Such repetitive business processes can be streamlined, tuned, and made available to multiple applications that might need them. For instance, SOA allows the company’s call center application and point-of-sale applications to access the same “get recommended product” service irrespective of their operational systems, geographic location, or product mix. Moreover, since this capability is a “service,” none of these systems is querying a database—they are making a request in a specific context. By providing applications with access to business data without exposing the underlying database directly to the calling application, the database is protected with an additional level of security.

Why SOA Matters to the Data Warehouse (and Vice Versa)

Although SOA simplifies access to frequently used data that reflects common business processes, it’s not intended to be used for ad hoc queries, which are usually unique and volatile. Ad hoc queries are situational, and are usually driven by a discrete business issue, as opposed to common business processes. While it’s technically feasible to create an ad hoc query “service,” it’s counter to the intent of SOA, which is meant to provide frequently used information to a client application quickly and easily.

The data warehouse needn’t change because of SOA. In most environments, SOA will allow access to data on the warehouse without the client application being aware of the particulars of the underlying database schema. Ultimately, the SOA function will work as middleware, residing on either a server or on the data warehouse, and converting the service request to a SQL query.

Supporting SOA allows the data warehouse to be leveraged more fully. More applications can access data from the warehouse via SOA. Plus, it simplifies development, because developers don’t need to understand the metadata or database schema that defines the data warehouse.

Code will need to be written to expose the data warehouse to other applications, but its complexity has been dramatically reduced through the inclusion of software tools and wizards provided by many of the database vendors.

Preparing the Data Warehouse for SOA

In supporting their company’s new SOA architecture and development efforts, BI and data warehouse teams should consider the following steps:

• Identify common operational queries. The most frequent or cross-functional operational queries are the ones aligned with the most important business processes. Reviewing these queries or canned reports can help you distinguish where the data warehouse can begin providing services.
• Examine the non-reporting applications that currently access the data warehouse. For instance, call center representatives will frequently request a customer’s “last 10 transactions.” Call center applications typically don’t store purchase history, but this is a perfect service for the data warehouse with historical purchase detail.
• Talk to your enterprise architects. Understand the status of the SOA within the company to determine the standard service set, and then map that to the data warehouse’s capabilities. In all likelihood, the architecture group has documented a list of data services, some of which surely pertain to the data warehouse.
• Support—or lead—data administration efforts. Data warehouses are often still the only enterprise systems that integrate cross-functional data and establish standard terminology. For an SOA environment to be successful, data standards are critical.

Many companies are still embracing SOA only in the context of development planning for OLTP applications. Because of the evolution of the data warehouse to support operational reporting, and the growing popularity of operational reporting within the business, the data warehouse as a key component within an SOA framework is inevitable. BI and data warehouse professionals should be prepared to expand their horizons in support of SOA.

Top