CEO Perspective: Future Trends in BI and Analytics
What's hot now and what's ahead in BI and analytics? Matthew Scullion, Matillion CEO, offers his perspective.
- By James E. Powell
- September 30, 2019
Getting data faster and getting it integrated faster than ever is key if enterprises are to get the greatest benefit from their richest asset. AI and ML are top of mind for Matillion CEO Matthew Scullion, but he's always aware of the need to transform data to make it useful and the challenges that poses. Scullion also shares the biggest trends he sees for the rest of this year and into 2020.
Upside: What technology or methodology must be part of an enterprise's data strategy if it wants to be competitive today? Why?
Matthew Scullion: Every part of the enterprise needs data and insights to make decisions. Where they run into issues is the setup of their data journey -- consolidating and joining together data in various formats and transforming that data into an analytics-ready state. The best way for enterprises to use those insights to be competitive is to get them faster so they can begin innovating quicker. This will happen inside the cloud and cloud data warehouses. Right now, the cloud is the only environment that offers the agility, power, and economics to keep up with the need for data insights in a practical way.
What one emerging technology are you most excited about and think has the greatest potential? What's so special about this technology?
There are lots of trends that have been out there where the tech came first and the search for potential applications came second. Things like IoT and real-time streaming have relevance but there wasn't always a business use case (and so the hype perhaps outweighed the value of those technologies in some cases). I am really excited about AI and ML, because AI and ML are applicable to most businesses and many different business processes. Almost every business can, over time, gain value through AI and ML.
The algorithms for machine learning haven't changed much in the last 40 years -- ML just needed an infrastructure. Suddenly with advances in other parts of the technology industry -- notably the cost effective and agile compute and storage capacity of the cloud, these AI/ML technologies are able to provide great ROI.
It is a similar phenomenon to what we find at Matillion with our "ELT" architecture. ELT, which is where the underlying data warehouse provides the horsepower for data transformation, has always been a pretty good idea, but a generation ago, people didn't have easy access to cost-effective data warehouse compute for ELT. Today. a powerful, scalable, cost-effective cloud data warehouse can be spun up in minutes, so the improvements in infrastructure delivered by the cloud have made this technology more relevant. The same is true of AI and ML.
Of course, we're excited about AI and ML for another reason. AI and ML use cases require lots of data to train the algorithms and to run over. That data needs to be in an analytics-ready state -- joined together, of high quality, and embellished with appropriate metrics the algorithms can use. Transforming data from siloed, source system state to analytics-ready state, at scale and quickly, is what Matillion's software does, so there's a good link between where we operate and this rising trend of AI and ML.
What is the single biggest challenge enterprises face today? How do most enterprises respond (and is it working)?
I think one of the single biggest challenges facing business today in terms of data analytics is the disconnect between creating/collecting data and usefully analyzing it. The expectation is that if you have the data, you can analyze it. The reality is that you need to transform source data -- which by its definition is siloed and raw -- into useful, joined together, embellished analytics-ready data. Data transformation is a key part of the process and a data warehouse isn't just the database technology, it's the way the data is modeled and transformed from how it was, to how it needs to be to gain insight.
Customers have a few options for transforming data in the cloud. They can use their legacy tools, but these were not built for the cloud. which causes headaches. They can hand code, but this is expensive, slow to create value, and hard to maintain. Data pipeline tools only move data, they don't transform it. This area is Matillion's focus -- data transformation for cloud data warehouses. We allow an organization to not only get their data into their cloud data warehouse (the easy bit) but also transform it into something useful and ready for analytics and insight.
Is there a new technology in data and analytics that is creating more challenges than most people realize? How should enterprises adjust their approach to it?
Right now the concerns around data governance, data privacy, and data security are creating challenges for businesses. News of data hacks and leaks are slowing down the innovation happening inside of enterprises and inhibiting them from using data analytics to compete in their space. While Matillion is not a direct participant in governance, privacy, and security, we are a data integration company and we take the necessary measures to ensure what we build works properly, is well architected, and is secure. But we are tangentially affected by these technology concerns because it slows the progress enterprises can make innovating with data and in doing so, improving their businesses.
In terms of adjusting approach, I think it's about a couple of things. Balancing actual fears and concerns against perceived ones, and balancing them against the opportunity cost of failing to compete using data. You can innovate with data in a manner that is appropriately secure, and you can implement good governance and privacy. Doing so and right sizing it to use case and stage is something that should be thought of as part of what's required to innovate with data, not a reason for not doing so.
What initiative is your organization spending the most time/resources on today? What internal projects is your enterprise focused on related so that you (not your customers) benefit from your own data or business analytics?
We use Matillion ETL in our own business to help provide understanding and predictability in our revenue pipeline, understand our go-to-market funnel, but perhaps most interesting, we also use it to understand our customer behavior in-product and help make our user experience superb. When our customers launch Matillion, we ask them to opt-in to our telemetry data service. This allows us to see how Matillion users interact with our product -- en masse and without compromising their individual privacy.
We have many hundreds of customers opted-in, providing many thousands of instances and users, so thousands of data points coming into our data warehouse every few minutes, giving us great insights into customer behavior and how they use our software and therefore how we can improve.
Where do you see analytics and data management headed in the rest of 2019 and beyond? What's just over the horizon that we haven't heard much about yet?
It's beginning to feel like cloud data analytics is going mainstream. The trend is making its way through the adoption curve, past the innovators, and into the early majority stage. We've seen from our large enterprise customers that innovative teams within their organizations are beginning to take on projects that incorporate cloud data analytics, and through their proof of concept it is becoming the standard for other parts of their organization to innovate, too. This is great for us to see because Matillion's product and delivery model is set up to support those projects inside small and midsize businesses all the way up to large enterprises.
Describe your product/solution and the problem it solves for enterprises.
Matillion is data transformation for cloud data warehouses. Our software empowers customers to extract data from a wide variety of sources, load it into their chosen cloud data warehouse, and transform that data from its siloed source state into useful, joined together, analytics-ready data -- ready to be consumed in analytics, machine learning, and artificial intelligence use cases. Purpose-built for the cloud, Matillion does this at scale, delivering fast time to value, and high performance with pay-as-you-go economics -- simplicity, speed, scale, and savings.