TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

AI and Analytics: The Rise of the Machines

We are headed toward a future populated with AI and machine learning, where machines take on more of the work people have traditionally done. What does that mean for the business analytics professional?

• By William McKnight
• June 20, 2017

Software has greatly enhanced the information management possibilities in the last few years. Consider the advancement of nonrelational systems such as Hadoop, NoSQL, and graph data stores, as well as streaming data, master data management, data virtualization, and data visualization. SSDs and in-memory (and soon GPUs) have changed the compute and storage game. The cloud permeates every decision these days and is the most invasive transformation in the technology business in our careers.

In my 19 years of information management consulting, the last few have seen the most change.

For Further Reading: AI and Cognitive Technologies Will Transform the Enterprise Artificial and Real Artificial Intelligence AI in the Crosshairs

We can no longer say platform capabilities are lagging behind our requirements as long as you're willing to explore, spend, and change. The Internet of Things (IoT) is connecting millions of formerly unconnected "things" to the Internet and game-changing insights are coming from advanced analytics and not from pursuing business-as-usual tactics or just democratizing access.

Despite the dramatic changes, another level of change looms. Our industry is rapidly being future-proofed by artificial intelligence and machine learning (AI/ML). Looming on the horizon is an injection of AI/ML into every piece of software, so keep those exploring glasses on! You're going to need them.

AI and DI

Consider the domain of data integration. If the machine can predict with high accuracy the steps you will execute in sourcing data to a destination, it will develop those steps and you can just make any minor modifications necessary based on hints provided. That knowledge is born of understanding not only your past processes but also the processes of hundreds or thousands of others around the globe.

Data integration jobs will not fail but instead will fix themselves. They should be able to examine the desired outcome and make themselves as efficient as possible. Even opening workbenches and consoles will be unnecessary. Data quality routines will also be point and click. The laborious process of mapping will be a thing of the past. We are headed towards a future populated with AI/ML, where machines take on more of the work people have traditionally done.

Machine learning is being built into databases so the data will be analyzed as it is loaded. This can be used to automatically call attention to variances and feed operational systems with actions. Predictive actions like this come from learned, nuanced analysis of granular data. They will come from AI/ML.

Although we will certainly deploy AI/ML in our future corporate applications, the split of the necessary AI/ML between the "edge" of corporate users and the software itself is still to be determined. The more we know about the AI/ML coming in our software, the more prepared we can be to make the most of the rise of the machines.