View online: tdwi.org/flashpoint
September 11, 2014
ANNOUNCEMENTS
LAST CHANCE TO REGISTER TDWI World Conference
San Diego, CA
September 21-26, 2014
Save $275 as a Premium Member
NEW TDWI Checklist Report
Seven Keys to Data Blending
NEW TDWI Checklist Report
Demystifying Cloud BI: Six Issues to Consider
NEW TDWI Checklist Report
Preparing Your Organization for In-Memory Computing
NEW TDWI Checklist Report
Moving Beyond Spreadsheets: Six Steps for Enabling Small and Midsize Businesses to Gain Data Insights
CONTENTS
4 Simple Steps for Keeping Your BI Environment Fresh
Components Used Most Commonly Today
Mistake: Not Dealing with Governance
See what's
current in TDWI Education, Events, Webinars,
and Marketplace
4 Simple Steps for Keeping Your BI Environment Fresh
Patty Haines
Chimney Rock Information Solutions
Topics:
Business Intelligence
You may be in this situation: Your business intelligence/data warehouse (BI/DW) environment was designed and built a few years ago and has provided positive impact to your organization. It has been used over the years for operational and standard reporting, dashboards, scorecards, and predictive modeling. Some key intelligence has been identified by the business--both end users and managers alike--as a result. However, your business is changing at an increased pace. Outside influences--both negative and positive--are growing each year. Your BI/DW environment has not kept up and may need some enhancements, including tools, data, and processes.
Here are four simple steps you can take to ensure continued success with your BI/DW environment:
1. Stay connected with your business users and managers. It is important to stay involved with your business, both end users and management. This includes knowing and understanding both high- and low-level goals and metrics. Your BI/DW team needs to understand the ever-changing needs, risks, and exposure of your business so your BI/DW environment can stay in sync with organizational strategy and direction.
Regularly ask questions of your business users and management. Keep the channels of communication open; that way, when the business needs new data or different analytics, it is communicated quickly and concisely, allowing you to determine how to enhance your environment to meet the new needs.
When a BI/DW team does not stay in sync with the business, the business may go elsewhere to find the services it needs, often without asking. Being a part of their discussions and the solution for their information needs will keep your team in the loop.
No matter what project methodology you use, including waterfall, time-boxed, or agile, keep your business involved. This can include daily stand-up meetings, weekly updates, prototyping and demos, user group meetings and workshops, newsletters providing tips and techniques, and lunch-and-learn sessions. These activities will keep your team in touch with business users and managers and provide an avenue for sharing what different groups are doing with data, metrics, and tools.
2. Perform regular assessments for business value. Conduct periodic formal checks with your business users and management to evaluate the value they need to get from your BI/DW environment and whether it meets these needs. Complete assessments to better understand what’s needed, what’s working, and what direction the business would like to take. These can be documented as an assessment, information needs, or user stories, depending on your team’s methodology. The key is to create an evaluation process with the business, including those who do not use your BI/DW environment. These specific users may be able to point you in a new direction, allowing you to service more users with additional data or tools.
You should also evaluate the technical aspects of your BI/DW environment to identify possible issues with data, data delivery, missing data, current tools and their use, missed processing windows, and error logs. It is important to look at the statistics to see what is happening with your production processes and data. Some fine-tuning may be required.
3. Evaluate possible new sources of data. There are many new external data sources--government studies, outside reports, white papers, social media, and industry surveys--that may help supplement your organization’s data and provide insight about your customers or your organization’s products and services. Begin to evaluate these sources.
Review the assessment and road map from your original BI/DW projects, where you can find valuable information about what information needs were identified early. Some data may have been missed over the years or need enhancements.
Consider unstructured data, including social media, sales rep notes, survey notes, clinic trial notes, e-mails, and contracts. There are new methods and tools you can use to interpret and process unstructured data, so this can be merged with your structured data to provide even more value from your BI/DW environment.
4. Evaluate possible new tools. New tools and methods are regularly being introduced for processing, comparing, integrating, analyzing, and modeling data. Vendors are providing software in the cloud, with new business models provided as a service. New technologies are being used within BI/DW environments for big data, statistical modeling, data mining, federation, virtualization, and in-memory tools. These should be evaluated to determine whether they would be a good fit within your BI/DW environment.
Consider training sessions and presentations with experts from vendors and local user groups to discuss current best practices, new methods, and new technologies used in today’s BI/DW environments. Vendor and industry conferences are also useful for learning more about how other organizations are building and using their BI/DW environments.
Summary
A BI/DW environment is truly user focused and built for the business, with its methods and information needs in mind. The key for continued success is to stay up to date with your business. New data, tools, and processes exist that can easily fit into your current environment and bring immediate value to your business. Evaluating and identifying these new items can keep your BI/DW environment in sync with your business users.
Patty Haines is founder of Chimney Rock Information Solutions, a company specializing in data warehousing and data quality.
Integrating SQL and Hadoop
Jean-Pierre Dijcks and Martin Gubar
There is little doubt that big data is here to stay; over the last couple of years, it has grown in importance as a critical element to any data warehousing strategy. However, there are still many questions about the best way to implement a big data strategy and what tools are best for the job. One approach that has generated significant interest is integrating SQL and Hadoop.
This article highlights the benefits of integration, the pros and cons of various integration methods, and how adoption of this approach will change the data warehouse and the big data landscape.
Learn more: Read this article by downloading the Business Intelligence Journal, Vol. 19, No. 2
Highlight of key findings from TDWI's wide variety of research
Components Used Most Commonly Today
As no surprise, the “meat and potatoes” components are the most common. The list of top 15 components used today (see Figure 13) includes well-established hallmarks of the traditional data warehouse such as data stages, data marts, multi-dimensional schema, OLAP cubes, ODSs, tabular data, relational DBMSs, and metadata repositories. Conspicuously absent are newer platforms and features such as in-memory functions, Hadoop, clouds, columnar DBMSs, and appliances--but we’ll see these soon in the list of top components to be aggressively adopted in the next few years.
The data stage is the most common component of a data warehouse architecture (83%). Whether you have a true data warehouse or simply an ODS, row store, or a body of marts, you have to land, stage, and process (to some degree) the data coming into those environments for the purposes of reporting, analytics, data sync, archiving, and so on. At one extreme, some “warehouse” environments are almost exclusively data staging, because most data sets for reports and analyses are instantiated at run time, not beforehand. Comprehensive data warehouses typically have multiple staging platforms, one per data type or source. This explains why data stages can be slightly more common than core warehouses.
(Click for larger image)
Read the full report: Download Evolving Data Warehouse Architectures in the Age of Big Data (Q2 2014)
FlashPoint Rx prescribes a "Mistake to Avoid" for business intelligence and data warehousing professionals.
Mistake: Not Dealing with Governance
Fern Halper
Governance is about applying policies related to using services. Data governance deals with the roles, policies, and rules around data. It’s about defining the principles, roles and responsibilities, and rules with which an organization must conform. Governance is often combined with compliance and security issues across computing environments. Once you have explored your data and understand which data and analytics will be used for business purposes, it is important to understand that this data is now part of your corporate intellectual property and needs to be governed and secured appropriately.
Some organizations put off governance, stating that they are moving too fast and don’t have time for it. It feels too overwhelming to them. They fail to understand two concepts. First, governance can look different for different organizations. It doesn’t have to be a cumbersome process. Second, companies that deal with governance fairly early in big data efforts don’t run into issues such as data ownership, data definitions, or data provenance that can derail big data efforts. In other words, governance is a good thing. It is fine to use existing governance frameworks for your big data efforts--in fact, it makes sense to do so. If you don’t have a framework, you can start small, perhaps with a data steward. The important thing is to begin to put a plan in place so you’re not caught off guard as you grow.
Read the full issue: Download Ten Mistakes to Avoid in Big Data Analytics Projects (Q2 2014)
