Top Trends in BI and Self-Service Visual Analytics
Organizations need to seek the right balance of freedom and oversight that enable users to get their work done self-service style. Here are four trends that will help organizations achieve that goal in 2017.
- By David Stodder
- December 16, 2016
"Faster, better, cheaper" remains the mantra of software implementation, as it has been throughout the history of manufacturing of all things. However, in 2016 it became clear that to achieve "faster, better, cheaper" in the business intelligence (BI) realm, it takes more than just installing tools. Governance, guidance, and stewardship are proving to be just as important.
In many organizations, the rapid adoption of self-service visual analytics and discovery tools by nontechnical business users has resulted in a "Wild West" of data chaos. IT, which has been under pressure to give users the freedom to work with self-service tools, is now feeling heat because of the lack of oversight. Executives are worried about unsupervised data use potentially causing regulatory, policy, and security exposures; users are running into barriers they cannot overcome on their own and need more guidance.
Thus, heading into 2017, one of the over-arching themes in BI is the need for organizations to bring better governance, guidance, and stewardship into the picture even as they become increasingly self-service oriented. This does not mean that it's time for IT to "be the heavy" and clamp down on users; that will not work.
Organizations need to seek the right balance of freedom and oversight. Software vendors need to provide functionality that makes it easier to manage, govern, and steward data in an environment where users have more freedom to personalize their data access and interaction than they have with traditional enterprise BI. We expect that in 2017 such functionality will be a competitive differentiator.
Here is a look at two top trends we spotted in 2016 and four we anticipate will be strong in 2017.
2016 Top Trends
Self-Service Data Preparation
Easy, self-service data visualization is great, but what often slows users down the most is all the "plumbing" required to find, collect, integrate, and transform the data, whether it is coming from spreadsheets, operational systems, transactional business applications, or even less-structured data sources such as a data lake. To reduce delays and improve quality, users need tools that enable faster, smarter, and more repeatable data preparation without having to depend on IT for everything.
Self-service data preparation was a big story in 2016. Technologies for self-service data preparation can automate processes so that users have less need for manual work in finding the right data and cleansing, cataloging, and transforming it. In the research for our 2016 Best Practices Report, Improving Data Preparation for Business Analytics, we found strong interest in self-service capabilities, although not necessarily from standalone products. Just over a third (35 percent) of research participants said they are using data preparation features of their self-service BI, visual analytics, and data discovery tools; nearly a third (31 percent) are currently using self-service data preparation features of data integration, ETL, and data quality tools.
In the coming year, we expect to see a turbulent marketplace as BI and visual analytics tool vendors, data management platform vendors, and independent data preparation and cataloging vendors vie to win customers.
Cloud-Based BI and Analytics
With a growing share of ecommerce, customer relationship management, sales force management, and other types of business applications located in public or private clouds, it would make sense that BI, analytics, and data warehousing would not be far behind. However, many companies have been reticent about using cloud computing for these systems, primarily out of concern for data security but also about availability, functionality, and performance. Slow loading and replicating of data to and from the cloud is also a barrier.
In 2016, momentum toward using the cloud for BI, analytics, and data warehousing picked up. Research for our Best Practices Report, BI, Analytics, and the Cloud: Strategies for Business Agility, found that companies are still concerned about security and the other issues, but half (50 percent) of research are currently using a cloud deployment model for analytics and nearly a third (31 percent) are planning to do so in the next few years.
We expect that in the coming year as companies seek greater agility, more will reach for cloud computing -- often favoring private cloud deployment models -- for BI, analytics, big data lakes, and data warehousing. The attraction of being able to match business dynamism with the ability to spin up and down cloud services to perform analytics will grow stronger and companies will grow more confident in the cloud's security, availability, and performance.
2017 Anticipated Top Trends
Metadata for Self-Service BI and Analytics Will Improve
Everyone knows metadata is important; it is vital to building the knowledge bank about diverse data, including what it is, where it came from, and how it is being used. Accurate metadata is vital to data governance and stewardship, particularly for monitoring data quality and tracking the data's lineage. Collecting and managing metadata and master data has been hard enough with enterprise BI and data warehousing, but it is getting tougher in the era of big data and self-service analytics.
Organizations are looking for technologies that update how metadata is collected, managed, cataloged, and shared in a diverse data and user environment. The technologies have to be fast and easy because if metadata gathering and managing is too difficult, it tends to be ignored, which worsens the data chaos problem.
In 2017, we will see technology solutions maturing in how they automate the collection, cataloging, managing, and sharing of metadata. Data scientists, business analysts, and nontechnical users of self-service tools will have better information at their fingertips about data quality, regulatory policy adherence, and the data's fitness for purpose.
Natural Language Search Will Have a Bigger Role in BI and Analytics
The combined trends of BI and analytics democratization and the increasing diversity of data are moving natural language search functionality from nice-to-have to must-have for self-service tools. Search has long had a kind of bolted-on presence in BI suites for a while; now, however, users are demanding better integration of search functionality so that they can use it to find data quickly, write natural language queries, and do operations by just asking questions rather than writing SQL. Artificial intelligence and cognitive technologies inside tools will help make search more intuitive.
In 2017, we will see search become a bigger part of self-service tools. Users will be hungry for tips about how to make best use of search for interacting with data.
Embedded BI and Analytics Become Part of the Microservices Revolution
One of the biggest trends in software service development is the microservices architecture. Evolving out of the well-established concept of service-oriented architecture and implementation of DevOps and agile methods, the microservices architecture addresses today's requirements for application processes to function through continuous communication between independently deployed software services over networks.
To quote software development expert Martin Fowler , the microservices architectural style is "an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource application programming interface (API). These services are built around business capabilities and independently deployable by fully automated deployment machinery."
A major focus of this approach is to make it easier to change the mix of functions and add new ones. Traditional BI systems have been fairly monolithic; going forward, vendors will be implementing concepts of microservices and modern application programming interfaces (APIs) to make it easier to put together personalized solutions and embed components of BI and analytics such as dashboards and other visualizations into other application services.
The ability to embed BI and analytics is particularly important as organizations adopt more cloud-based business application services; embedding and API integration need to be formalized as well as easy and seamless. We expect to see this evolution become an important theme in 2017.
Agile Methods and Self-Service BI and Analytics Go Together
Agility -- the ability to sense change, adjust behavior, and take advantage of unexpected opportunities -- is a highly desirable quality for organizations. Decision makers need data insights to help them be aware of changes in their organizations' environments so they can adjust strategies and position resources to take advantage of events or circumstances rather than be left behind.
Many organizations are adopting agile methods for BI and data warehousing projects as an alternative to traditional, and slower "waterfall" development methods. Agile methods promote shorter development cycles and delivery of iterations or releases that users can test and implement sooner.
Self-service BI and visual analytics tools are turning out to be a great fit for organizations that are using agile methods. We are seeing in our research that the tools are helpful for users to explore data first to get a sharper sense of project requirements and then use the tools to test releases to see if features and functions fit their needs. We anticipate seeing more organizations using the combination of agile methods and self-service tools in 2017.
Faster, Better, Cheaper -- And Smarter
Easy-to-use self-service tools have excited users by giving them the means to work more effectively with data and personalize their experiences beyond what has been possible with traditional enterprise BI solutions. However, no organization wants a Wild West where anything can and will happen with the data. They need a modernized approach to governance and stewardship that is supported by intelligence and automation inside the tools and platforms.
The coming year should bring better tooling to support governance and management requirements and improved alignment between popular development methods and the technologies.