TDWI Blog: Data 360

Blog archive

Big Data Analytics: An Overview in 20 Tweets

By Philip Russom, TDWI

To raise an awareness of the new tool features, user techniques, and team structures of Big Data Analytics, I recently issued a series of twenty tweets via Twitter, over a two-week period. The tweets also helped promote a TDWI Webinar on Big Data Analytics. Most of these tweets triggered responses to me or retweets. So I seem to have reached the business intelligence (BI) and data warehouse (DW) audience I was looking for – or at least touched a nerve!

To help you better understand Big Data Analytics and why you should care about it, I’d like to share some of the thoughts from these tweets with you. I think you’ll find them interesting because they provide an overview of Big Data Analytics in a form that’s compact, yet amazingly comprehensive.

Every tweet I wrote was a short sound bite or stat bite drawn from TDWI’s recent report on Big Data Analytics, which I researched and wrote. Many of the tweets focus on a statistic cited in the report, while other tweets are definitions stated in the report.

I left in the arcane acronyms, abbreviations, and incomplete sentences typical of tweets, because I think that all of you already know them or can figure them out. Even so, I deleted a few tiny URLs and repetitive phrases. I issued the tweets in groups, on related topics; so I’ve added some headings to this blog to show that organization. Otherwise, these are raw tweets.

Defining Big Data, Advanced Analytics, and Big Data Analytics
1. #BigData #Analytics = where advanced analytics operate on big data sets. So, it’s about 2 things. Learn more in Webinar
2. Advanced #Analytics = data mining, statistics, extreme SQL, data viz, artificial intell, language processing.
3. Advanced #Analytics = database techs like MapReduce, in-database & in-memory analytics, column stores.
4. Advanced #Analytics = discovering unknown biz facts. Instead of advanced, should call it discovery analytics
5. #BigData = not just multi-terabyte datasets. Also about diverse data types & real-time or streaming data.
6. Bleeding edge of #BigData = data streaming from sensors, robotics, monitor devices, Web logs.

Benefits and Barriers for Big Data Analytics
7. #TDWI SURVEY SEZ: #BigData #Analytics benefits customer relations, BI, most pre-existing analytic apps.
8. #TDWI SURVEY SEZ: Bad skills, sponsors, & database software are leading barriers to #BigData #Analytics.

Organizational Issues and Big Data Analytics
9. #TDWI SURVEY SEZ: 30% consider #BigData a data mgt problem. 70% think it a biz opp when analyzed. Attend #TDWI Webinar
10. #TDWI SURVEY SEZ: #BigData #Analytics is owned by BI/DW team (41%), dep’ts (21%), IT/CIO (12%).
11. #TDWI SURVEY SEZ: Business analyst is most common job title for designer of #BigData #Analytics.

The State of Big Data Analytics
12. #TDWI SURVEY SEZ: 74% of orgs have some form of analytics today. But only 34% do #BigData #Analytics.
13. #TDWI SURVEY SEZ: 37% of orgs have 10Tb+ of #BigData just for #Analytics. More on #TDWI Webinar
14. #TDWI SURVEY SEZ: 20% of orgs expect to have 500Tb+ of #BigData just for #Analytics by 2013.
15. #TDWI SURVEY SEZ: 64% of orgs today manage #BigData for #Analytics in EDW, 38% outside EDW.
16. #TDWI SURVEY SEZ: 24% claim to have Hadoop today. #TDWI suspects most are experimental downloads. But still impressive
17. #TDWI SURVEY SEZ: #BigData is struc 92%, semi-struc 54%, hier 54%, events 45%, unstruc 35%, social 34%, Web 31%...

Future Trends in Big Data Analytics
18. #TDWI SURVEY SEZ: 33% will replace #Analytics platform within 3 yrs. Another 11% after that. 9% already replaced.
19. #TDWI SURVEY SEZ: Why replace #Analytics platform? Poor scale, loading, query speed, real time, SOA, self service, viz.
20. #TDWI SURVEY SEZ: #BigData #Analytics techs set to grow most: advanced analytics, data viz, in-memory DBs, unstruc data

Don’t miss my next TDWI Webinar on Hadoop. I’ll lead a panel of vendor representatives in a discussion of Hadoop and its value for BI, DW, and analytics. Register online, so you can join us December 14, 2011 at noon ET.

For a more detailed discussion of Big Data Analytics – in a traditional publication! – see the TDWI Best Practices Report, titled Big Data Analytics, which is available in a PDF file via a free download.

You can also register for and replay my TDWI Webinar, where I present the findings of the Big Data Analytics report.

Philip Russom is the research director for data management at The Data Warehousing Institute (TDWI). You can reach him at or follow him as @prussom on Twitter.

Posted by Philip Russom, Ph.D. on December 7, 2011


Average Rating

Add your Comment

Your Name:(optional)
Your Email:(optional)
Your Location:(optional)
Please type the letters/numbers you see above