Why NoSQL is Moving from "Need" to "Want"
In 2012, all of IT -- from developers to C-level IT executives and IT operations teams -- will start thinking differently about NoSQL.
By Billy Bosworth, CEO, DataStax
In early 2009 I was introduced to the world of NoSQL. I learned about Hadoop, Cassandra, and several other new Big Data technologies. I believed the problems of Big Data were real, and I knew from my background in relational databases that relational technology wouldn't be able to handle these new problems. The real question was: How big and disruptive could this be to the mainstream market?
When we talk about the mainstream database market, we're talking about nearly 30 years of ecosystem, education, and evolution. Trying to disrupt that market would take a hammer -- and a big one at that.
In 2011, a $16B hammer arrived on the scene to deliver the message that Big Data databases are here to stay. The combined revenue of Oracle, IBM, and Microsoft in the relational database market is approximately $16 billion, and in 2011, each of these companies helped tell the world why these new Big Data technologies were valuable. IBM jumped into fray with their beefed-up offering around Apache Hadoop called BigInsights. Microsoft (yes, the Windows company) announced not only a Hadoop connector for SQL Server, but their plans for a Hadoop-based distribution for Windows Server and Windows Azure.
The IBM announcement wasn't too surprising because IBM has such a broad range of technologies in their portfolio to address data needs of almost any kind. Microsoft, in contrast, caused a bit more eyebrow-raising because many people thought Hadoop and Windows just wasn't a natural fit. However, neither company carried the weight of the market leader.
The collective thud of jaws hitting the floor happened when Oracle, the $9B RBDMS gorilla, announced a Hadoop connector and a NoSQL database of their own as part of their Big Data Appliance.
What made the Oracle announcement particularly interesting was that just a few months earlier they published a white paper, Debunking the NoSQL Hype. Then, suddenly, they were educating thousands of people at Oracle Open World about the use cases where traditional relational databases just weren't a good fit.
The big question is: why the change?
There are two primal causes of technology disruption: Need and Want. In the early days, people who adopted NoSQL technologies (or even built them) did so because they simply had no other choice. The problems they needed to solve were so severe that they had to force their way into a new frontier of databases. They were the pioneers.
As we look back, 2011 was the year that Need turned to Want. It is the year that NoSQL and Hadoop entered the mainstream consciousness, thanks in no small part to the announcements by the Big Three. By "entering the mainstream consciousness," I mean that people started to think differently about their problems. They took to the whiteboard and started asking, "What if we could…" instead of saying "We can only…" That led to lots of prototyping in dark corners of IT and, in many cases, even rolling out new pieces of functionality for larger applications on NoSQL databases.
In 2012, imagination will become reality for many mainstream companies. This year:
- Developers will continue their surging adoption of these Big Data solutions as they struggle to keep pace with fast-changing application and data requirements.
- C-level IT executives will start thinking seriously about the architectural and cost benefits these systems can deliver -- in particular, by leveraging cloud resources (private or public). They will be held accountable to deliver cost savings while providing more and better services to their business.
- IT architects and operations teams will start to use these NoSQL databases to provide the long-awaited vision of "data as a service" inside their own organizations.
- Perhaps most important, business owners will demand solutions based on large amounts of seemingly never-ending data. They will want the core data from the application as well as all the data about how customers are using it. Basically, applications are going to start throwing off "data exhaust" and that will create even more exponential data growth.
All the while, "smart devices" are producing of a never-ending stream of data, and it seems everything from tennis shoes to appliances to automobiles are quickly being added to this category. That data will not be wasted – it will be coveted.
This year, possibilities will go from the whiteboard to the motherboard. Exciting times are surely ahead.
Billy Bosworth is the CEO of DataStax. He has 20 years of experience in the database industry in roles ranging from DBA to senior executive. Prior to DataStax, Billy spent 6 years at Quest Software, a provider of systems management software, where his most recent role was VP and GM of the database business unit. Under his leadership, the industry-leading Quest database business grew from supporting traditional relational databases to a portfolio that now includes tools for cloud, NoSQL, columnar, and Hadoop databases, as well as business intelligence offerings. Prior to Quest, Billy led product teams for Embarcadero Technologies' database productivity solutions. Billy holds a bachelor of science in computer science from the University of Louisville. You can contact the author at [email protected].