2 years ago
PostgreSQL snapshotting is now faster
Historically, Streamkap's snapphotting queries have been generic across our sources.
We now use specific SQL suitable for PostgreSQL, which improves performance in loading historical data
Roadmap
In Planning
Change Event Stream (Thin)
improvement
Connector: HTTP Destination
improvement
Auto Setup Conectors
improvement
Alerts: Schema Changes
improvement
DBT Adapter
improvement
Easy data splitting
improvement
External Sharing with relevant permissions/views
improvement
Suggested Transforms
improvement
Netsuite Source
improvement
Salesforce Source
improvement
S3 Custom Directories and Keyless Support
improvement
Start a pipeline from a given point-in-time
improvement
Billing/Usage Alerts
new
Snowflake (source)
new
Data Selection
improvement
Secrets Store
improvement
Filtering - Remember Choices
improvement
Webhook as a source
improvement
Workspaces
new
Connector: JDBC as a Source
new
Connector: ClickHouse (Source)
new
In Progress
Sorting
improvement
Table/Column Renaming
improvement
S3 Source
improvement
Notifications - OpsGenie, PagerDuty, ServiceNow, Webhook
improvement
Dead Letter Queue (DLQ)
improvement
AWS Private Link (Self Serve)
improvementnew
Destination: Iceberg + 3 Catalog options
new
Alerting - Add datadog, servicenow, opsgenie, pagerduty, webhooks
new
Source: BigQuery
new
Incremental parallel thread snaphots (per topic)
improvementnew
Pub/Sub
new
Connector:Supabase
new
Connector: PlanetScale
new
On-Prem/Bare Metal Deployment
new
Connector: Elastic Search Destination
Connector: Timescale DB
new
Beta
Auto QA - Quality Checks
new
UI support for exclusions
improvement
Surface SMTs
new
Expose Kafka Endpoints (self service)
improvement
Connector: Neon
new
Self Service Transformations Input
improvement
Destination: Redpanda (Write)
new
Destination: Confluent (Write out)
new
Destination: Flink (Pull)
new
Connector: TinyBird
new
Connector: Materialize
new
Source: Redpanda (Pull)
new
Connector: Starburst (pull)
new
Source: Confluent (Pull)
new
Source: Flink (Pull)
new
Connector: Startree
new
PostgreSQL Snapshots without Signal Table
improvement
Connector: GCS Sink
new
BYOK (Bring Your Own Key)
new
Auto Un-Nesting
improvement
Done
Support SSH Tunnels in App
improvement
Metrics API
new
Multiple Environments
improvement
Usage Analysis
improvement
Destination: Azure Blob Storage
new
Destination: Kafka
new
Topics Page
improvement
BYOC - Azure
new
View Transforms Jobs
new
Usage by Topic
new
Conenctor: DuckDB/MotherDuck
new
Snowflake - Upsert Mode
improvement
Pipeline Stats Page
improvement
DynamoDB
new
BYOC AWS
new
New Region Support: East
new
Snowflake - Automate Dynamic Tables
improvement
AWS Ireland
new
Hosted Region Support - Sydney
new
Preview messages
new
Connector: Vitess
new
BYOC - GCP
new
Tags - Apply custom labels
new
2 years ago
Security enhancements
The following security enhancements have been added
- Ability to remember 2FA/MFA devices for 30 days
- 2FA/MFA methods supported now also include: Security keys, SMS and built in authenticators (i.e. fingerprint, face recognition) as well as auth apps
- Password repeat protection
- Exposed credential detection - email notification/error message
- Password strength meter
- Passwordless expiration time changed to 20 min
- Force Relogin changed to 90 days
- Maximum Concurrent Sessions 10
- Idle Session Timeout changed to 1 day
- 2FA/MFA custom name defaults to Streamkap
2 years ago
Starburst Destination for CDC (Change Data Capture)
Starburst destination is now supported for change data capture (CDC).
Features
- Real-time data ingestion
- Schema evolution support
- Parquet and Iceberg format
2 years ago
Added Slack for Sign-Up/Login
We've now added Slack to our social sign up and login options.
2 years ago
Added Start/Stop/Pause and Reset Options
Added Start/Stop/Pause and Reset Options to connectors
2 years ago
SQL Server Source for Change Data Capture
SQL Server Source is now supported for change data capture (CDC).
Features
- Snapshots for backfilling & then switch to streaming mode
- SQL Server upsert and insert modes to destination
- Schema evolution support
2 years ago
AWS PrivateLink
AWS PrivateLink
AWS PrivateLink is now supported.
AWS PrivateLink is more secure and lowers bandwidth costs
2 years ago
DocumentDB Source for Change Data Capture
DocumentDB Source is now supported for change data capture (CDC).
Features
- Snapshots for backfilling & streaming at same time
- PostgreSQL upsert and insert modes to destination
- Schema evolution for supported destinations
2 years ago
Personal Security Settings
Enhancing your personal configuration options
- Added password change, 2FA, audit and management of sessions
- Password change
- 2FA/Multi-factor authentication
- Audit of sessions
- Ability to terminate sessions