TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Q&A: Why You Should Separate Data from Business Logic

Looker's chief data evangelist, Daniel Mintz, explains the benefits of keeping data and its underlying business logic apart.

• By James E. Powell
• July 19, 2019

Traditionally, data and its logic have been stored together. Think of spreadsheets, where calculations and raw data are stored in the same file. Looker's chief data evangelist, Daniel Mintz, explains best practices and benefits for keeping data and business logic apart.

For Further Reading: Data Cataloging Comes of Age Modern Metadata Management Data Fabrics for Big Data

Upside: How has sharing business logic changed in the last 3-5 years? What's driving this push?

Daniel Mintz: Business logic has changed in recent years because data and logic no longer have to live together in the same space. Previously, you couldn't separate the logic from the data; they both had to live in the same place. Think of an Excel workbook: the data lives in the workbook, but so do the formulas. Now with the internet and the availability of incredibly powerful databases, we can keep the logic close to the people who need it and apply it to the data only when that data is needed.

Separating the logic from the data solves lots of issues surrounding sharing data in spreadsheets. Version controlling spreadsheets is a nightmare, collaborating is hard, and the data sprawl that comes from emailing spreadsheets inevitably leads to privacy and security concerns. Today's powerful data platforms separate the data and logic to give the users -- and, more important, the admins -- a single place to manage the business logic and the ability to leave data locked away until it's needed.

How is sharing business logic easier now than it has been?

The sharing of data logic is much easier now simply because logic is so much smaller when it's not bound up with the data. Data is heavy, logic is light. Separating the two can make a huge impact, especially for those looking to create data-driven cultures. This is because it empowers more people who traditionally would not have been able to make data-driven decisions.

But it's still not meeting business needs? How is this sharing working out?

Creating a flexible reusable business logic layer is a new concept that is in the process of being adopted by the business world. The amount of data in businesses of all sizes and backgrounds is exploding, but at the same time, people are still learning about data logic, giving it meaning and keeping everyone on the same page. It's not magic. There is still a lot of work to be done during this evolution, especially around data governance. However, the businesses that are embracing this new world of the business logic layer are seeing big benefits. We've made tremendous progress, but we still have a long way to go.

What workarounds are enterprises using to fulfill their needs now?

There are many businesses that still live in the world of workbook analytics. In fact, a recent Forrester report found that 66 percent of respondents reported that over half of their business intelligence (BI) content is in spreadsheets. This is a case in point that we still have quite a long way to go. (Source: Forrester's Business Technographics Global Data And Analytics Survey, 2016)

What's needed instead?

To truly change how we manage business logic, we need commitment from companies to govern the business logic. Tech alone never brings change; change requires technology and people. To create change, the business logic conversation has to go beyond the analyst. The commitment of being a data-driven company has to come from the top, and that trickles down to the workers who make it a reality.

Aren't there some serious downsides to sharing report logic? For example, by sharing logic, aren't you potentially giving away company secrets?

Sharing business logic is dangerous when it's delivered in archaic spreadsheet methods because that means data is floating around the enterprise with no checks. Having the logic and data that run the business all over people's laptops with no visibility into how it's being shared and modified is dangerous. Businesses that have a single enterprise-grade system drastically limit those dangers because they are actually able to govern the logic and keep it centralized, giving it that protection layer.

Who uses this shared logic? Is it suitable for end-user understanding or is it designed for IT?

The goal of shared logic is to get it out of the head of the analyst and into software so that anyone can access it. This frees data analysts from mundane tasks, allowing them to focus on more strategic campaigns while empowering employees from various departments to make data-driven decisions. The goal is for the nontechnical business user to be able to ask questions of the data using terms that they are familiar with such as "lifetime customer value," "forecast revenue," or "satisfied customers." Everyone should feel comfortable working in data -- that is the concept of business logic.

Is there a standard for sharing business logic or is it still the Wild West in terms of format, style, or layout?

Creating a standard for business logic is precisely why Looker was founded. LookML, our language for data which is the core of the Looker data platform, was invented specifically to solve this problem of data sprawl. The Looker data platform provides a single source of data truth for employees across all departments. With Looker, they are able to visualize their data, and they can work with it in a format they're comfortable with. When logic stays locked away in workbooks, it will never evolve past the Wild West.