What is your e-mail address?

My e-mail address is:

Do you have a password?

Forgot your password? Click here
close



Return to the Previous Page

Wayne Eckerson

Blog archive

Do We Really Need Semantic Layers?

It used to be that a semantic layer was the sine qua non of a sophisticated BI deployment and program. Today, I’m not so sure.

A semantic layer is a set of predefined business objects that represent corporate data in a form that is accessible to business users. These business objects, such as metrics, dimensions, and attributes, shield users from the data complexity of schema, tables, and columns in one or more back-end databases. But a semantic layer takes time to build and slows down deployment of an initial BI solution. Business Objects (now part of SAP) took its name from this notion of a semantic layer, which was the company’s chief differentiator at its inception in the early 1990s.

A semantic layer is critical for supporting ad hoc queries by non-IT professionals. As such, it’s a vital part of supporting self-service BI, which is all the rage today. So what’s my beef? Well, 80% of most BI users don’t need to create ad hoc queries. The self-service requirements of “casual users” are easily fulfilled using parameterized reports or interactive dashboards which do not require semantic layers to build or deploy.

Accordingly, most pureplay dashboard vendors don’t incorporate a semantic layer. Corda, iDashboards, Dundas, and others are fairly quick to install and deploy precisely because they have a lightweight architecture (i.e., no semantic layer). Granted, most are best used for departmental rather than enterprise deployments, but nonetheless, these low-cost, agile solutions often support sophisticated BI solutions.

Besides casual users, there are “power users” who constitute about 20% of total users. Most power users are business analysts who typically query a range of databases, including external sources. From my experience, most bonafide analysts feel constrained by a semantic layer, preferring to use SQL to examine and extract source data directly.

So is there a role for a semantic layer today? Yes, but not in the traditional sense of providing “BI to the masses” via ad hoc query and reporting tools. Since the “masses” don’t need such tools, the question becomes who does?

Super Users. The most important reason to build a semantic layer is to support a network of “super users.” Super users are technically savvy business people in each department who gravitate to BI tools and wind up building ad hoc reports on behalf of colleagues. Since super users aren’t IT professionals with formal SQL training, they need more assistance and guiderails than a typical application developer. A semantic layer ensures super users conform to standard data definitions and create accurate reports that align with enterprise standards.

Federation. Another reason a semantic layer might be warranted is when you have a federated BI architecture where power users regularly query the same sets of data from multiple sources to support a specific application. For example, a product analyst may query historical data from a warehouse, current data from sales and inventory applications, and market data from a syndicated data feed. If this usage is consistent, then the value of building a semantic layer outweighs its costs.

Distributed Development. Mature BI teams often get to the point where they become the bottleneck for development. To alleviate the backlog of projects, they distribute development tasks back out to IT professionals in each department who are capable of building data marts and complex reports and dashboards. To make distributed development work, the corporate BI team needs to establish standards for data and metric definitions, operational procedures, software development, project management, and technology. A semantic layer ensures that all developers use the same definitions for enterprise metrics, dimensions, and other business objects.

Semi-legitimate Power Users. You have inexperienced power users who don’t know how to form proper SQL and aren’t very familiar with the source systems they want to access. This type of power user is probably more akin to a super user than a business analyst and would be a good candidate for a semantic layer. However, before outfitting these users with ad hoc query tools, first determine whether a parameterized report, an interactive dashboard, or a visual analysis tool (e.g., Tableau) can meet their needs.

So there you have it. Semantic layers facilitate ad hoc query and reporting. But the only people who need ad hoc query and reporting tools these days are super users and distributed IT developers. However, if you are trying to deliver BI to the masses of casual users, then a semantic layer might not be worth the effort. Do you agree?

Posted by Wayne W. Eckerson on July 28, 2010


Comments

Mon, Aug 20, 2012 Scott FL

Providing adhoc query via a Semantic layer a.k.a. Business Objects Universe which guarantees proper metric calculations while allowing users to qeury "at will" is misleading at best. Key business metrics should be well defined in the business capability model and therefore be generated via a IT based set of processess and procedures that ensure that the metric is calculated correctly. These processes and procedures need to include source systems as well as the DW/BI layer. Key metric values shouldn't be allowed to be 're-constructed' via an adhoc query or it could loose its proper meaning. Therefore, there isn't a need for adhoc semantic layers that would 'regenerate' the value. The true benefit for adhoc is more aimed at data mining and does/should require an understanding of the data sources etc. Once data mining uncovers a potential new Key Metric then the EA process should kick-in and the metric generation etc be developed.

Mon, Aug 1, 2011 Tom

I think a semantic layer is still needed to help enforce some standards and analysis rules across an organisation. Its not so much about learning SQL. Learning SQL is no harder than learning a BI tool such as Business Objects, Cognos etc effectively. A good semantic layer in an organisation is in such the "source of truth" as it incorporates the companies business rules.

Mon, Oct 25, 2010 Lonnell Branch Columbia MD

Yes we still need the semantic layer. The semantic layer when implemented effectively provides all end users with the ability to access data in an efficient manner without becoming a data modeling expert and learning advance SQL. Today a significant number of the masses are being asked to analyze and adhoc report business process results and out comes. It is getting very hard to find users in the enterprise who don't need ad hoc reporting functionality. After reviewing some of the dashboard oriented BI tools I was amazed at the level of SQL knowlege the casual user needed just to get the base data for a report or visualization. The real solution to the problem is moving toward a distibuted meta data layer that provides more functionality than just a relational model and began to include the capabilities that an associative model can deliver.

Add a Comment

Your Name:(optional)
Your Email:(optional)
Your Location:(optional)
Comment:
Please type the letters/numbers you see above
Back to Top

Channels by Topic

  • Agile BI »
    Includes:
    • Agile
    • Scoping
    • Principles
    • Iterations
    • Scrum
    • Testing
  • Big Data Analytics »
    Includes:
    • Advanced Analytics
    • Diverse Data Types
    • Massive Volumes
    • Real-time/Streaming
    • Hadoop
    • MapReduce
  • Business Analytics »
    Includes:
    • Advanced Analytics
    • Predictive
    • Customer
    • Spatial
    • Text Mining
    • Big Data
  • Business Intelligence »
    Includes:
    • Agile
    • In-memory
    • Search
    • Real-time
    • SaaS
    • Open source
  • BI Leadership »
    Includes:
    • Latest Trends
    • Technologies
    • Thought Leadership
  • Data Analysis and Design »
    Includes:
    • Business Requirements
    • Metrics
    • KPIs
    • Rules
    • Models
    • Dimensions
    • Testing
  • Data Management »
    Includes:
    • Data Quality
    • Integration
    • Governance
    • Profiling
    • Monitoring
    • ETL
    • CDI
    • Master Data Management
    • Analytic/Operational
  • Data Warehousing »
    Includes:
    • Platforms
    • Architectures
    • Appliances
    • Spreadmarts
    • Databases
    • Services
  • Performance Management »
    Includes:
    • Dashboards, Scorecards
    • Measures
    • Objectives
    • Compliance
    • Profitability
    • Cost Management
  • Program Management »
    Includes:
    • Leadership
    • Planning
    • Team-Building
    • Staffing
    • Scoping
    • Road Maps
    • BPM, CRM, SCM
  • Master Data Management »
    Includes:
    • Business Definitions
    • Sharing
    • Integration
    • ETL, EAI, EII
    • Replication
    • Data Governance

Sponsored Links