A Closer Look at Looker
Looker claims that its take on BI self-service permits business analysts or other savvy user types to build their own dashboard or analytic views.
- By Stephen Swoyer
- February 24, 2015
As self-service analytic products go, Looker looks like nothing -- or like everything -- under the sun.
In the back-end, its data access component functions a little like a data federation engine -- or seems to, anyway. On the front-end, Looker exposes a browser-based visual self-service UI that looks a lot like something you might see in Tableau or Qlik Sense.
There are several other similarities. For example, Looker claims to be able to present a single, logical view of all relevant business data -- which is basically federation's stock-in-trade. Unlike federation, however, Looker does not shift the computation portion of the workload (i.e., data preparation and query processing) up to the source platforms. (In the federation model, data is processed in situ in order to minimize data movement and -- a feature, not a bug -- to eliminate the interposing data warehouse.)
Looker, like federation, proposes to eliminate the data warehouse citadel -- sort of. It can be installed alongside an operational application -- i.e., running against its transaction (OLTP) database -- or implemented in a more centralized scheme. For example, data can be extracted from multiple OLTP databases or other sources and fed (or replicated) into a traditional (R)DBMS or optimized analytic platform. It can be called a data warehouse, a data mart, or an operational data store. The key thing is that it doesn't have to be designed, built, and managed like a data warehouse. There's no data model; there are no traditional business views, nor traditional cubes.
If it's hard to avoid thinking of something like federation when looking at Looker, it's doubly hard to avoid thinking of self-service BI. Officials claim that Looker's take on self-service helps to distinguish it from traditional business intelligence (BI) tools that expose the equivalent of preconfigured data services or business views. Someone -- nominally, an IT-someone -- still has to configure, provision, and manage all of those services or views.
In the Looker paradigm, argues vice president of strategic alliances Keenan Rice, a critical portion of that work shifts to the business analyst or power user. In this regard, Rice claims, Looker's human-readable mark-up language, LookML, permits business analysts or other savvy user types to access and prepare data from source systems -- and also to build their own dashboard or analytic views, along with views for other, less-savvy business users. Finally, Rice argues, Looker's model helps to simplify and rationalize data management and BI.
“The cool thing about the LookML layer is that it doesn't need a traditional data warehouse. It can have a simple replica of a production database,” he says.
“This means you can finally decouple data management from the BI layer. You're no longer managing a data infrastructure in a BI layer and a data infrastructure that powers the BI layer.”
(No) Free Lunch?
To the extent that this is true, it runs smack against a long-standing problem: using a product like Looker -- much like using a Tableau or a Qlik -- isn't as simple as downloading it, installing it on your desktop, and going out and grabbing data from SAP, PeopleSoft, or other source systems.
Somehow, some way, you're going to have to lay some pipe -- i.e., build some connectivity plumbing. In Looker's case, this usually means extracting data from upstream sources and consolidating it in a central point: call it a data warehouse, call it a database, call it an operational data store -- regardless of what you call it, it still becomes a site of maintenance and ongoing support.
Looker improves on self-service a la Tableau because it acts as a kind of BI layer that sits on top of business data. This eliminates some of the biggest problems in Tableau environments: (1) the need to obtain appropriate credentials for access to data and, (2) the creation of multiple, point-to-point connections from source systems to the self-serving information consumer. (There's a third issue at stake, too: the classical self-service model used to give short shrift to data management priorities such as metadata management and data history. This, too, is starting to change.) As many Tableau-dependent organizations have discovered, this massively serial model can lead to upstream (source) performance problems.
Looker's model eliminates this issue. It's installed and managed by IT. In its simplest form, a Looker collection point can take the form of a MySQL RDBMS, says Rice. “Your engineering team can create a slave of your MySQL transactional data store, and that ... can be pretty sophisticated. You can build all of that sophistication up, almost like a semantic data warehouse,” he explains.
“The user is going to have things that are important to them: sales reports, customer value, lifetime orders, things they're going to be storing in the [slave] database. Those concepts are going to be things that have been derived from the [source] data.”
To create a production replica or slave is to create still another system that must be managed and maintained, however. In most cases, in fact, you're typically integrating data from several different production sources, which means it doesn't make sense to create discrete replicas of each of these. It's easier to replicate all of this stuff into a single database system.
In that case, you're getting very close to something like a data warehouse.
Nor does replication solve all of your problems. If access is hard, one of the most intractable bits of the self-service BI model has to do with how data is joined or manipulated.
This is where the Good Ship Self-Service tends to founder.
For example, some tools (such as Tableau and Qlik) expose back-end scripting facilities that are more than a little ETL-like, at least with respect to their technical complexity. At the very least, they require a degree of technical expertise if they're to be used effectively -- not necessarily development expertise, but technical expertise.
True, self-service tools have improved in this regard; arguably, both Tableau and Qlik manage to abstract a good deal of this complexity from users. (Qlik's 2012 purchase of data integration specialist Expressor Software helped it address precisely this problem.) Even so, they don't yet have the problem licked. Rice concedes that this aspect of the self-service model poses distinct challenges, but he disputes that it's intractable.
He cites Looker's LookML language, which, as a mark-up language, is by definition human-readable. LookML itself is based on YAML -- the recursively named “yet another mark-up language.” (This was YAML's original backronym. The YAML community now prefers the formulation “YAML Ain't a Mark-up Language.”) Looker automatically generates LookML when it connects to and scans or indexes a data source, Rice explains. An analyst can then manipulate data (in Looker's design environment) on the basis of human-readable LookML text. He says LookML functions as a modeling layer that gives developers or business analysts the ability to describe data and to create something like a business view: in the Looker model, however, the necessary transformations are performed at query time, not pre-defined and stored as persistent structures.
“We have what's called a 'generator.' You connect your database [or replica] to it and run our generator, which looks at the physical schema of that database and creates your LookML model,” he indicates. “That [LookML] model is going to be highly intelligible to a business analyst. They would have just an IDE [integrated development environment] in what we call the LookML section of the [Looker] app, where what they're doing is they're altering LookML or YAML text.”
No Excess Baggage
Rice not so selflessly positions Looker as a future-oriented BI offering. In other words, he claims, Looker isn't constrained by or freighted with “legacy” design or architectural considerations -- or, for that matter, by the expectations of a large and influential enterprise installed base.
At the same time, Rice concedes, Looker's customer base includes few, if any, traditional enterprise types. Instead, its users tend to be start-up (or what Rice calls “born-of-the-Web”) type companies. His earlier invocation of MySQL as a platform for a production OLTP database is a dead giveaway in this regard. Comparatively few enterprise IT organizations leverage MySQL as a primary OLTP repository for production systems. Its use in the Web world, however, is much more pervasive.
“We have this vision of what BI will be in the future and it's really coupled with what is happening right now in the data management landscape. That stuff is driving a lot of the way we innovate. We're building on that, but what we're looking at right now is the born-of-the-Web companies. We're looking at what are those data workloads [common in born-of-the-Web companies], and those are either online media, search engines, or e-commerce [workloads],” he explains.
“Right now, we're going into enterprise companies that have these types of newer workloads.”
One such traditional enterprise customer is a prominent provider of e-commerce and shipping and mailing products and services, Rice indicates. Another is a product group within a prominent multinational company that produces console video games.
“This group [collects] all of this data [from people who play their games]. These are workloads that would never make it into the EDW because the EDW isn't designed for them. A good bit of our customers are also using traditional BI, but they're modern companies that use Looker as a complement [to what they're doing with BI] -- or they don't have as much legacy baggage.”