2 years ago
Google BigQuery Destination Supporting Change Data Capture
Google BigQuery destination is now supported for change data capture (CDC).
Features
- Real-time data ingestion
- Support for objects larger than 10mb (for example from MongoDB)
Roadmap
In Planning
Change Event Stream (Thin)
improvement
Connector: HTTP Destination
improvement
Auto Setup Conectors
improvement
Alerts: Schema Changes
improvement
DBT Adapter
improvement
Easy data splitting
improvement
External Sharing with relevant permissions/views
improvement
Suggested Transforms
improvement
Netsuite Source
improvement
Salesforce Source
improvement
S3 Custom Directories and Keyless Support
improvement
Start a pipeline from a given point-in-time
improvement
Billing/Usage Alerts
new
Snowflake (source)
new
Data Selection
improvement
Secrets Store
improvement
Filtering - Remember Choices
improvement
Webhook as a source
improvement
Workspaces
new
Connector: JDBC as a Source
new
Connector: ClickHouse (Source)
new
In Progress
Sorting
improvement
Table/Column Renaming
improvement
S3 Source
improvement
Notifications - OpsGenie, PagerDuty, ServiceNow, Webhook
improvement
Dead Letter Queue (DLQ)
improvement
AWS Private Link (Self Serve)
improvementnew
Destination: Iceberg + 3 Catalog options
new
Alerting - Add datadog, servicenow, opsgenie, pagerduty, webhooks
new
Source: BigQuery
new
Incremental parallel thread snaphots (per topic)
improvementnew
Pub/Sub
new
Connector:Supabase
new
Connector: PlanetScale
new
On-Prem/Bare Metal Deployment
new
Connector: Elastic Search Destination
Connector: Timescale DB
new
Beta
Auto QA - Quality Checks
new
UI support for exclusions
improvement
Surface SMTs
new
Expose Kafka Endpoints (self service)
improvement
Connector: Neon
new
Self Service Transformations Input
improvement
Destination: Redpanda (Write)
new
Destination: Confluent (Write out)
new
Destination: Flink (Pull)
new
Connector: TinyBird
new
Connector: Materialize
new
Source: Redpanda (Pull)
new
Connector: Starburst (pull)
new
Source: Confluent (Pull)
new
Source: Flink (Pull)
new
Connector: Startree
new
PostgreSQL Snapshots without Signal Table
improvement
Connector: GCS Sink
new
BYOK (Bring Your Own Key)
new
Auto Un-Nesting
improvement
Done
Support SSH Tunnels in App
improvement
Metrics API
new
Multiple Environments
improvement
Usage Analysis
improvement
Destination: Azure Blob Storage
new
Destination: Kafka
new
Topics Page
improvement
BYOC - Azure
new
View Transforms Jobs
new
Usage by Topic
new
Conenctor: DuckDB/MotherDuck
new
Snowflake - Upsert Mode
improvement
Pipeline Stats Page
improvement
DynamoDB
new
BYOC AWS
new
New Region Support: East
new
Snowflake - Automate Dynamic Tables
improvement
AWS Ireland
new
Hosted Region Support - Sydney
new
Preview messages
new
Connector: Vitess
new
BYOC - GCP
new
Tags - Apply custom labels
new
2 years ago
MongoDB Source for Change Data Capture
MongoDB Source is now supported with change data capture (CDC).
Supports
- Snapshot/Backfilling before switching to streaming mode
- Schema evolution is handled with supported destinations
2 years ago
MySQL Source with Change Data Capture
MySQL Source is now supported with change data capture (CDC).
Supports
- Snapshot/Backfilling before switching to streaming mode
- Schema evolution is handled with supported destinations
3 years ago
Google BigQuery Destination Supporting Change Data Capture
Google BigQuery destination is now supported for change data capture (CDC).
Features
- Real-time data ingestion
- Support for objects larger than 10mb (for example from MongoDB)