Imply Announces Major Open Source Contribution for Apache Druid
Company is introducing multistage query engine, improvements to Imply Polaris.
Note: TDWI’s editors carefully choose vendor-issued press releases about new or upgraded products and services. We have edited and/or condensed this release to highlight key features but make no claims as to the accuracy of the vendor's statements.
Imply, the company founded by the original creators of Apache Druid, have unveiled the second milestone in Project Shapeshift, the 12-month initiative designed to solve the most pressing issues developers face when building analytics applications. This milestone introduces the following:
- A multistage query engine for Apache Druid, representing the largest expansion of Druid’s architecture since its inception
- Imply’s Total Value Guarantee, a global guarantee for Apache Druid users that shows how the Imply subscription is effectively free for qualified participants
- Continued momentum for Imply Polaris, the cloud database service built from Apache Druid
Imply is uniquely built for the intersection of analytics and applications. Increasingly, companies are building modern analytics applications that combine analytics on big data with application traits including sub-second response, high concurrency, and operational use. The developers at leading companies, including Atlassian, Reddit, and TrueCar, are building these applications to underpin their digital businesses, and Apache Druid with Imply is their database of choice.
Imply introduced Project Shapeshift in 2021. Today it introduces three major announcements that help make Druid more capable, accelerate developer success, and enhance the Imply Polaris cloud service.
Apache Druid Expansion
A source of Apache Druid’s performance for interactive analytics at scale lies within its highly efficient and optimized single-stage query engine. A second engine—a multistage query engine, optimized for more complex data flow—was introduced in a private preview by Imply in March. The new engine was conceived to dramatically transform data ingestion and expand querying capabilities.
“We always thought of Druid as a shapeshifter when we originally built it to support analytics apps of any scale,” said Gian Merlino, CTO and co-founder of Imply and PMC chair for Apache Druid. “Now we’re excited to show the world just how nimble it can be with the addition of multistage queries and SQL-based ingestion.”
Imply announces its contribution to the multi-stage query engine in Apache Druid 24.0. In this release, the multistage stage query engine enables the following:
- Simplified and accelerated batch ingestion. Like every database, data must be ingested before it can be used. Now, ingestion in Druid uses common SQL queries that benefit from the new extensions to Druid, making it both easier to ingest data and up to 65 percent faster. With many Druid deployments ingesting hundreds of terabytes daily, this saves both time and expense.
- SQL-based in-database transformation. Formerly, Druid supported a limited set of in-database table transformation capabilities. Now, Druid supports any in-database transformation without tuning or expertise using SQL, enabling data enhancement, data enrichment, easy experimentation with aggregates, approximations (including hyperloglogs and theta sketches), and more with the same ease-of-use and performance enhancements as SQL-based ingestion.
- Foundation for expanded ecosystem support. With the addition of SQL-based ingestion and transformation, Druid has built a foundation for integration with a range of open source and commercial data tools, covering transformation (dbt), data integration (Informatica, FiveTran, Matillion, Nexla, Ascend.io), data quality (Great Expectations, Monte Carlo, Bigeye) and others.
Imply’s Total Value Guarantee for Apache Druid Users
Imply delivers a complete developer experience for Druid across its commercial distribution, cloud services, and expertise. With Imply, developers are able to save time from managing the database and save money on infrastructure—where both time and money contribute to total cost of ownership (TCO).
Now Imply is making it possible for Apache Druid users to get all the value of a partnership with Imply effectively for free—and lower their TCO for Druid. Imply is introducing the Total Value Guarantee for qualified participants that guarantees the total cost of ownership (TCO) to run Druid with Imply—measured across software, support, and infrastructure—will be less than the TCO when running Apache Druid on their own.
“At Imply, we love building and using open source software but having a partner can be helpful, too,” said Vadim Ogievetsky, CXO and co-founder of Imply. “Now with Imply’s Total Value Guarantee, developers can get a partner for Druid that will help them get all the advantages of Imply’s products and services and be there in the middle of the night if needed—with Imply effectively for free.”
Continued Imply Polaris Momentum
Imply Polaris, a database-as-a-service built from Apache Druid, was introduced in March. This cloud database was built to do more than cloudify Druid; it also optimizes data operations and delivers an end-to-end service from stream ingestion to data visualization. In five months, Polaris has added over 250 accounts, representing companies across a wide range of industries.
Imply is announcing a series of product updates to Polaris that enhance the developer experience, including:
- More flexible data ingestion. Polaris adds support for schemaless ingestion to accommodate nested columns, allowing for arbitrary nesting of typed data such as JSON or Avro. DataSketches are now supported at ingestion for faster sub-second approximate queries.
- Simpler Polaris operations. Polaris adds performance monitoring alerts to ensure consistent performance for ultra-low-latency queries and greater security with resource-based access control and row-level security. Finally, updates to Polaris’ built-in visualization enable faster slicing and dicing.
- More flexible pricing. Polaris adds new node types to flexibly meet any price/performance requirement at any scale. New comprehensive consumption and billing metrics were added for instant usage visibility.
Learn about the new enhancements in Apache Druid in this blog.