Introducing Supabase ETL

Source: Dev.to

Supabase ETL is a change‑data‑capture pipeline that replicates your Postgres tables to analytical destinations in near real time. It reads changes from your Postgres database using logical replication and writes them to external destinations. Setup takes minutes in the Supabase Dashboard.
The first supported destinations are Analytics Buckets (powered by Iceberg) and BigQuery.
Supabase ETL is open source – the code lives on GitHub at .

---

Why separate OLTP and OLAP?

Postgres excels at transactional workloads (e.g., reading a single user record or inserting an order). When you need to scan millions of rows for analytics, Postgres slows down.

Column‑oriented systems like BigQuery, or those built on open formats like Apache Iceberg, are designed for this. They can:

• Aggregate massive datasets orders of magnitude faster
• Compress data more efficiently
• Handle complex analytical queries that would choke a transactional database

Supabase ETL gives you the best of both worlds: keep your app fast on Postgres while unlocking powerful analytics on purpose‑built systems.

How it works

Supabase ETL captures every change in your Postgres database and delivers it to your analytics destination in near real time.

1. Create a Postgres publication that defines which tables to replicate.
2. Configure ETL to connect a publication to a destination.
3. ETL reads changes from the publication through a logical replication slot.
4. Changes are batched and written to your destination.
5. Your data is available for querying in the destination.

The pipeline starts with an initial copy of your selected tables, then switches to streaming mode. Your analytics stay fresh with latency measured in milliseconds to seconds.

Setting up ETL

You configure ETL entirely through the Supabase Dashboard. No code required.

Step 1: Create a publication

A publication defines which tables to replicate. Create it with SQL or via the UI:

-- Replicate specific tables
CREATE PUBLICATION analytics_pub
FOR TABLE events, orders, users;

-- Or replicate all tables in a schema
CREATE PUBLICATION analytics_pub
FOR TABLES IN SCHEMA public;

Step 2: Enable replication

Navigate to Database → Replication in your Supabase Dashboard and click Enable Replication.

Step 3: Add a destination

1. Click Add Destination and choose your destination type.
2. (Analytics Buckets only) First create an analytics bucket in the Storage section.
3. Configure the destination with your bucket credentials and select your publication.
4. Click Create and then Start to begin replication.

Step 4: Monitor your pipeline

The Dashboard shows pipeline status and lag. You can start, stop, restart, or delete pipelines from the actions menu.

Available destinations

Our goal with Supabase ETL is to let you connect your existing data systems to Supabase. We’re actively expanding the list of supported destinations. Currently, the official destinations are Analytics Buckets and BigQuery.

Analytics Buckets

Analytics Buckets are specialized storage buckets built on Apache Iceberg, an open table format designed for large analytical datasets. Your data is stored in Parquet files on S3.

When you replicate to Analytics Buckets, each table is created with a changelog structure. Each row includes a cdc_operation column indicating whether the change was an INSERT, UPDATE, or DELETE. This append‑only format preserves the complete history of all changes.

You can query Analytics Buckets from PyIceberg, Apache Spark, DuckDB, Amazon Athena, or any tool that supports the Iceberg REST Catalog API.

BigQuery

BigQuery is Google’s serverless data warehouse, built for large‑scale analytics. It handles petabytes of data and integrates well with existing BI tools and data pipelines.

When you replicate to BigQuery, Supabase ETL creates a view for each table and uses an underlying versioned table to support all operations efficiently. You query the view, and ETL handles the rest.

Adding and removing tables

You can modify which tables are replicated after your pipeline is running.

Add a table

ALTER PUBLICATION analytics_pub ADD TABLE products;

Remove a table

ALTER PUBLICATION analytics_pub DROP TABLE orders;

After changing your publication, restart the pipeline from the Dashboard actions menu for the changes to take effect.

Note: ETL does not remove data from your destination when you remove a table from a publication. This is intentional to prevent accidental data loss.

When to use ETL vs. read replicas

Read replicas help when you need to scale concurrent queries, but they’re still Postgres and don’t make analytics faster.

ETL moves your data to systems built for analytics, giving you:

• Faster queries on large datasets
• Lower storage costs through compression
• Complete separation between production workload and analytics

You can use both: read replicas for application read scaling, ETL for analytics.

Things to know

• Replication is managed through the Supabase Dashboard; no additional infrastructure is required.
• Latency is typically measured in milliseconds to seconds, depending on network and destination load.
• The initial snapshot may take longer for very large tables; subsequent changes are streamed continuously.
• Supabase ETL is open source – contributions and feedback are welcome.

ETL Constraints

• Tables must have primary keys – required for Postgres logical replication.
• Generated columns are not supported.
• Custom data types are replicated as strings.
• Schema changes are not automatically propagated to destinations.
• Data is replicated as‑is, without transformation.
• During the initial copy phase, changes accumulate in the WAL and are replayed once streaming begins.

We’re working on schema‑change support, additional destinations, and evaluating different streaming techniques to improve flexibility and performance.

Pricing

Supabase ETL is usage‑based:

Get Started

Supabase ETL is in private alpha.
To request access, contact your account manager or fill out the form in the Dashboard.

If you want to dive into the code, the ETL framework is open source and written in Rust. Check out the repository at .