Dremio Announces Open and Forever-Free Lakehouse Platform, Dremio Cloud, In Partnership with AWS
New platform enables Git-like experience for data engineers.
Note: TDWI’s editors carefully choose vendor-issued press releases about new or upgraded products and services. We have edited and/or condensed this release to highlight key features but make no claims as to the accuracy of the vendor's statements.
Dremio, the lakehouse company, has released Dremio Cloud, a free data lakehouse platform. The company also announced the addition of two new services: Dremio Sonar, a lakehouse engine built for SQL, and Dremio Arctic, a metadata and data management service for Apache Iceberg that provides a unique Git-like experience for the lakehouse. Dremio Sonar is now generally available, and Dremio Arctic is in public preview.
Dremio Cloud is a lakehouse platform built from the ground up for SQL workloads, including mission-critical BI, according to Billy Bosworth, chief executive officer at Dremio. “In the past, companies had to weigh the pros and cons of data lakes versus data warehouses. We’ve eliminated this tradeoff by providing a frictionless and infinitely-scalable service that delivers the combined benefits of both.”
Dremio Cloud is Open and Forever-Free
With an open lakehouse architecture, unlike a cloud data warehouse, data itself takes center stage, no longer bound to specific vendor formats or systems. Companies can choose the right data processing/query engine for each use case, both today and in the future. There is no vendor lock-in with Dremio because the data is open by default, persisted in Apache Iceberg tables backed by Apache Parquet files in the company’s Amazon Simple Storage Service (Amazon S3) account. Iceberg is an Apache Software Foundation project with contributors from dozens of companies including Apple, Netflix, Salesforce, and Dremio. Dremio Sonar also supports additional table and file formats, as well as the ability to query data across a variety of data sources beyond Amazon S3, providing users with maximum flexibility.
Dremio is providing a forever-free edition of Dremio Sonar and Dremio Arctic on Dremio Cloud, supporting unlimited production use and infinite scale, with end-to-end security and SOC 2 Type 2 compliance. Dremio Sonar is over two times faster than Dremio’s existing query engine, which requires less than 50 percent of the cloud infrastructure previously needed, resulting in substantial cost savings. The Enterprise edition of Dremio Cloud adds advanced security controls, such as custom roles and enterprise identity providers (e.g., Okta), and enterprise support options.
Dremio Sonar Eliminates the Need for Cloud Data Warehouses
Powered by Apache Arrow, Dremio Sonar’s query performance empowers companies to run mission-critical BI, from ad hoc queries to low-latency dashboards, directly on the lakehouse. Sonar fully supports SQL Data Manipulation Language (DML) insert, update, and delete operations directly on the lakehouse. Because the data is automatically optimized in Amazon S3 as it is being changed, customers no longer need to fall back to a data warehouse when data is rapidly evolving.
“Companies can plug Dremio Sonar into any existing data lake and start querying in minutes, with support for multiple lakehouse metastores, including AWS Glue and Dremio Arctic, as well as the ability to query Amazon S3 directly,” said Tomer Shiran, founder and chief product officer at Dremio. “In addition, organizations can analyze data that resides outside the lakehouse (e.g., in relational databases) and perform live and accelerated joins across multiple sources.”
Dremio Sonar eliminates the need to physically export data into BI extracts/imports, cubes, data marts, and derived data warehouse tables, removing the need for a cloud data warehouse. Key features include:
- Dremio’s Reflections enables sub-second query response times by automatically and transparently rewriting query plans to utilize different aggregations or layouts of tables and views.
- Dremio Spaces creates a frictionless, personal, and collaborative user experience with an integrated semantic layer. With Dremio Spaces, data teams can deliver a consistent and secure view of the data to data consumers. In addition, analysts can curate new data sets in a self-service manner.
- Database-style updates and deletes (with SQL DML support), powered by Apache Iceberg, enable data warehousing workloads directly on the lake.
- Dremio SQL Runner and SQL Profiler are new visual tools that enable SQL-proficient users to run and troubleshoot queries and scripts directly from the Dremio interface.
- Dremio Database Connectors allow for joins across multiple data sources (Oracle, SQL Server, Redshift, etc.) beyond just the data lake.
- Native third-party integrations further simplify the experience with Dremio connectivity now built into many popular client tools, including Power BI, Tableau, dbt, Hex, Preset, and many others.
Dremio Arctic Delivers a Git-Like Experience for Data
Dremio Arctic is a forever-free service that delivers a Git-like experience for the data lakehouse, powered by the open source projects Iceberg and Nessie. Dremio Arctic’s Nessie-based metastore simplifies data engineering and analysis by making data workflows as powerful and intuitive as working with source code:
- Branches allow users to ingest, transform, and experiment in isolation, ensuring consistency across tables in the lakehouse
- Tags and Time Travel allow data analysts and data scientists to reproduce historic dashboards or machine learning models without maintaining data copies
- A simple user interface similar to GitHub provides observability and governance for data
- Table optimizations of Apache Iceberg tables are automatically scheduled and executed in the company’s AWS account, eliminating the need to manually optimize and maintain the data
Dremio Arctic is now in public preview with support for a wide variety of lakehouse engines, including Spark, Flink, Presto, Trino, and Dremio Sonar.
To learn more about Dremio Cloud, please visit dremio.com/dremio-cloud-now.