Q&A: The Big Deal about Big Data and Analytics
Doug Miles, the head of the Market Intelligence division at AIIM, discusses the developments facing the big data and analytics industry that are affecting enterprise markets today.
[Editor's note: This article was originally published by Enterprise Management 360° and is reprinted by permission. AIIM's reports can be downloaded for free at http://www.aiim.org.]
EM360°: We hear many solution providers with many analytics functions. From customer experience, to big data, advanced analytics, etc. Gartner's latest CIO survey concluded that analytics as well as business intelligence remain the no. 1 priorities for those decision makers in 2013. Are these types of solutions useful to enterprises, and second of all, is the sector somewhat convoluted?
Doug Miles: Well, I'm actually going to make it even more convoluted for you by raising the concept of "big content." AIIM is all about information management from a content point of view and so we cover enterprise content management (ECM), records management, and mobile content access. When we look at big data, we look at it perhaps more from the content analytics side of things, which has been around for quite a few years in terms of semantic analysis, but it has recently become quite exciting as people attach big data and big content analysis and see what they can do with that, using documents and social content, for instance.
In terms of the usefulness to the enterprise, I don't think there's any doubt that most people are pretty intrigued and excited about what they might be able to achieve with big data.
There is definitely a place for analytics to actually prosper, so where will current and popular analytic tools like Hadoop or even Google Analytics play in the future?
I think it's important not to get too deep into some of the technology. The first thing I would say is that Google Analytics is absolutely a fantastic tool and it's free, and people are using this particular tool in some very clever ways to model people's behavior as they come on to websites. You can add on to that some enhanced tools that will track how people work across the website and work out their patterns of activities before and whilst they are on the website.
The next step with some of the other tools is partly to do with the volume of the data that you need to deal with. What I find, both from the research that we did and with talking with end users, is that a lot of the applications, which we put under the category of "advanced business intelligence," are very clever ways of spotting patterns and trends. Hadoop and MapReduce, for example, can increase the speed [with] which you can pick this stuff up and analyze it. Instead of taking days, it can take minutes, but you have to take a step back and say, "Well, where is the core data that I want to analyze and is that core data in a format that I can use for analysis?"
If we look at the type of companies operating today from retailers, credit rating agencies, and account and tax departments, they have a number of spreadsheets containing a huge amount of their client's data. How can IT then cope with these growing and proliferating pressures residing within their infrastructures?
There are two views on life: keep everything because it may be useful, or delete everything because it takes up space -- and in a number of cases it might be incriminating. It's a very difficult compromise as there are some legal requirements that mean you can't keep details of people beyond three years after they've dealt with you, based on the Data Protection Act.
What we see happening frequently is that people have these big repositories of data and it's not been very well coded, especially when done by people rather than by machines. Unless you actually classify the data and tag it properly, then that task is going to become much more difficult later to both get the best out of your data and also to make sure that it isn't in a situation that might cause you problems.
Do you have any thoughts about where information management practices are actually heading towards, down the long-term road?
I don't know if you remember the Minority Report movie in 2002, but there are a couple of science fiction predictions in there. One of them was that you would have these precogs (pre-cognitions) where a woman would anticipate a crime or a murder that was about to happen and Tom Cruise would go rushing off to solve it.
Well, the data that some of the police departments or city authorities are collecting now, can actually predict when you might be likely to have a riot -- based on the temperature, the time of year, what is or isn't on sport or television, and perhaps what other current events are happening. The level of predictive intuition is still a little while away from being able to send around a policeman to stop it happening, but it is an exciting proposition.
There was one other thing in that film that was really interesting and that was that it used video billboards that implemented face recognition to target the advertising on the billboard to Tom Cruise as he walked around. We're nearly there with this technology now; we have electronic billboards that can change their adverts straight away, plus the ability to recognize faces and match them up to profile faces on social networks, for instance. We could then target those advertising billboards to the person who is walking by.
Some of that science fiction has come true in a much bigger way than we might have thought. There is no limit to what we might end up doing in terms of big data analytics as a concept.
In terms of information management as a whole, though, we have to examine the practices: how do we store this data, how we keep it legal and secure, and how do we manage it through its lifetime of usefulness to the business? AIIM does a lot of courses on these kinds of issues of information management, records management, and even social data management.
Doug Miles is head of the AIIM Market Intelligence Division. He has over 25 years' experience working with users and vendors across a broad spectrum of IT applications. He was an early pioneer of document management systems for business and engineering applications, and has most recently produced a number of AIIM survey reports on issues and drivers for ECM, capture, records management, sharepoint, cloud, mobile, and social business. Doug has also worked closely with other enterprise-level IT systems such as ERP, BI, and CRM. AIIM is a 65,000 strong worldwide community that provides education, research, and best practices to help organizations find, control, and optimize their information.