Announcing support for AWS DynamoDB

Read Docs

Excited to share support for Vitess

Vitess combines many important MySQL features with the scalability of a NoSQL database. Its built-in sharding features let you grow your database without adding sharding logic to your application.

Read Docs

Added support for IBM Db2 database as a source. 

IBM Db2 is a family of data management products, including RDBMS, for efficient data storage, retrieval, and analytics. It supports multiple platforms, including mainframes (z/OS) and distributed systems (LUW: Linux, UNIX, Windows).

Read Docs

We have updated our login options to improve security. The changes are as follows

Removed: Password login

Added: Magic Link. Magic link is a unique link that will be emailed to you and expires after 20 minutes. 

Therefore, if you currently login via username/password, the change in behavior will be that upon entering your email address, you will be emailed a link to login that will expire after 20 minutes. If you miss the 20min, you will need to request another magic link. 

We've done this to further strengthen and lock down our security posure to protect your data. As a reminder, we support other passkeys and other SSO options including Google, Microsoft, Slack, Github and enterprise SSO. You can also enforce 2FA at any time. 

Dashboard operational page is now available within the app 🚀

• Updates in real-time to the second
• Status of pipelines and transforms
• Overall trend/usage
• 10 Metrics for analysis over  1 day, 7 days, 30 days, 90 days, 365 days

Get started by visiting Dashboard

It's now possible to setup SSH tunnels in the app without liaising with support

When creating or editing a source, you will now see a toggle option to connect via SSH where you will br provided the key and the ability to input your jump host details.

In future, we will be adding support for tunnels on destinations as well as sources. 

SSH Tunnel Docs

Introducing a new metric to understand if pipelines are fully synced

Lag is the quantity of events which have been consumed by Streamkap but not yet delivered to the destination. 

While this should be 0, there are occasions where it can build up, such as:

• Sudden large influx of events and this could include snapshots
• The destination connector throughput capacity needs increasing
• The destination vendor service drops or is deterioated

Pipeline Summaries

Destination Connectors

Topic Level

Excited to announce that we now support additional services per account

What is a Service?

A service is the underlying architecture behind Streamkap. For example, this includes Apache Kafka, Apache Flink, Debezium, Connectors and so on, configured in such a way to be highly performant and scalable. 

At present when you login to Streamkap you are presented one service to work with.

What does this mean?

The problem with one service is that your development or staging pipelines are mixed in with your production pipelines and this can result in excess noise from alerts or the possibility of introducing mistakes. 

Equally, the development pipelines are sitting on top of a production grade service which is likely un-necessary. 

Going forward we now support additional services, allowing you to split out production from development/staging. 

Today, we support additional services in the same region. Soon, we will be supporting additional services of varying specifications and in other regions for increased flexibility

Alerts have now been moved to a dedicated page accessible on the sidebar. 

The gear icon is still used to configure the types of alerts and channels to receive on. 

We will be adding soon additional filters and a better UI to set your preferences

Snapshots (Also known as backfilling) previously happened when creating the source connector or adding new tables to the source connector. 

Going forward, this will now happen at the pipeline stage

• Created or edited source connectors will not snapshot upon creation
• When creating a new pipeline, a toggle will be default ticked to snapshot all the tables
• When editing a pipeline and adding new tables, a toggle will be default ticked to snapshot the new tables