SQL's Staying Power
SQL might have lost some of its sexiness, but it's as essential as it's ever been.
- By Stephen Swoyer
- December 18, 2012
SQL may have lost some of its shine, but it is as essential as it's ever been. That's one nugget from Analytic Databases for Big Data, the latest entry in TDWI's Checklist Reports series. It also jibes with what many in the industry say they're seeing: NoSQL, Hadoop, and other non-traditional technologies are gaining ground, but not necessarily at the expense of SQL.
In fact, some argue, to the extent that these technologies have been able to gain ground, it's because they've borrowed from -- or co-opted -- SQL, typically as a front-end or interface.
According to Philip Russom, research director for data management with TDWI, most organizations are actually increasing their use of SQL in spite of increased adoption of NoSQL databases such as Mongo, Cassandra, and Couchbase.
Both trends should be at loggerheads. At best, writes Russom, author of TDWI's report, they're basically canceling one another out. "Most organizations are deepening the amount and sophistication of their SQL usage, while a few others are seeking alternatives to it, as seen in so-called 'NoSQL' databases," he allows.
"Many organizations rely heavily on SQL as the primary approach to advanced analytics. This makes sense because most BI professionals know SQL, and some know it well enough to hand code 'extreme SQL' applications."
Because of SQL's analytic heritage, it has an enormous built-in market. Thanks to the emergence of "extreme SQL" -- with its complex and performance-intensive query-types -- this market continues to grow, says Russom. "[A]lmost all tools for analytics, reporting, data integration, and data modeling support or generate SQL code that can be co-opted for analytics," he writes. "In fact, anecdotal evidence suggests that analytics is driving up the amount of SQL usage across the BI/DW community, whether the SQL is hand-coded, tool-generated, or a mix of both. Hence, SQL is more important than ever, as are advanced SQL skills."
This isn't surprising. Consider the recent "Big Data Survey" from JasperSoft Inc. It found that a clear majority of self-described "big data adopters" (62 percent) use conventional ETL tools to load data from Hadoop or other NoSQL sources into ... SQL databases. From there, it's destined to become grist for their existing analytic practices.
Don't Call It a Comeback
Michael Whitehead, CEO of data warehousing specialist WhereScape Inc., says that reports of SQL's demise have been greatly exaggerated.
"[L]ook at the number of NoSQL databases that have a SQL interface," argues Whitehead, whose company -- a champion of SQL-centric data warehouse systems -- admittedly has a stake in that language's long-term prospects. "Look at [the Apache] Sqoop [project]. If I am in discovery mode it's okay to leave the data in Hadoop; if I am in decisional mode, I need to put it with other data -- [which means that it] is persisted into a designed, extensible, maintainable platform called ... a data warehouse."
Whitehead also cites the Hive Query Language (HiveQL), which grafts a SQL-like -- but not completely SQL-compliant -- query interface onto Hadoop.
Seth Proctor, chief technologist with cloud database specialist NuoDB Inc., says his company's NuoDB database is -- at least in part -- a response to the SQL-shortcomings of the popular NoSQL database engines. "We're a key-value store/back-end store, which is like NoSQL. We can see the value of that. We even played around with different ideas on the client side, maybe not NoSQL, but simplified query languages that you might want," said Procter, in an interview at the TDWI World Conference in Boston.
"Theoretically, this system could speak any number of kinds of things, but SQL is what we speak. It's what we've really focused on. We've tried to be heavily standards-compliant. We've tried very hard to think about SQL-99."
Proctor describes poor SQL support as the Achilles' heel of NoSQL. "NoSQL adopters love it at first. After they've tried and failed [to solve a problem] with a conventional [SQL] database, it seems like just what they've been looking for," he explains. "After about six months, they start to get frustrated with it. It helped with some difficult problems, but it introduced new ones, too, primarily [as a result of] its non-compliance [with SQL standards]."
The point, says TDWI's Russom, is that NoSQL has its place. This place isn't going to -- or shouldn't -- come at the expense of a SQL-glot analytic database, however.
"[NoSQL] makes sense when the majority of data types analyzed are not relational and converting them to relational structures is not practical. In other situations, a NoSQL approach is useful when modeling and indexing data in relational schema would inhibit the discovery mission that many modern analytic projects are all about. For these situations, NoSQL databases or Hadoop are viable alternatives to more common analytic DBMSs that are relational with SQL support."