Artie Transfer is a modern optimization stack that enables real-time data transfer between your databases and data warehouse, reducing the data latency from multiple hours down to seconds.

Ultimately, we increase the productivity of your data/engineering teams and allow companies to act on insights while the data is still relevant.

1.  - We leverage a variety of readers such as Debezium to record change data capture (CDC) events and publish them onto Kafka or Google Pub/Sub
2.  - Artie Transfer then consumes from the Kafka topic or Google Pub/Sub and directly writes the data into your data warehouse.

Artie Transfer handles all data mutations (inserts, updates and deletes) and provides automatic schema evolution detection. To understand more about how we’re able to do this, go to our blog post where we nerd out on the details!

Yes! When Artie initially connects to a new table, we will trigger snapshot mode such that we are able to capture the current state of your table and all the changes thereafter. Initial snapshots are included at no cost in our Cloud and Enterprise plans!

One bonus about our streaming architecture is that if you have a large table, instead of waiting days to weeks for the initial snapshot to load in your destination, you are able to see results in a matter of seconds.

To understand the list of sources and destinations that Artie Transfer supports, take a look at our documentation.

If you don’t see your sources or destinations listed, get in touch with us at [email protected]!

Artie Transfer does not store any of your data within its data processing layer and leverages an in-memory cache to optimize throughputs for columnar data stores.

Artie Transfer leverages Kafka and commits the offset upon a successful response from the destination. In the event of a crash, Artie Transfer will seek the previous offset and continue streaming. This means a full re-sync is not required and you should expect no data loss!

1.  - Reading directly from your Kafka topic if the CDC message is already available.
2.  - Deploying our readers such that we can publish the CDC messages onto Kafka.
3.  - Deploy the end-end solution all within your VPC.

Artie Transfer scales with you as you grow and will be able to maintain the same speed of transfer whether your table is 1 gigabyte or multiple terabytes. We are able to accomplish this because our architecture scales horizontally and leverages technologies like change data capture, Kafka, and Google Pub/Sub.

To understand more about how we’re able to do this, go to our blog post where we nerd out on the details!

Artie leverages change data capture (CDC) which means we only sync over changed data vs traditional data transfer methods of taking a snapshot of the entire source database. With CDC and a streaming based approach, we are able to achieve real-time data transfer without impacting performance on the source database. A side benefit of CDC includes lower network traffic and compute costs as less data is being processed.

Artie has built-in composite keys support and we also have the flexibility to support tables that do not have an explicit primary key.

Yes! Artie is able to handle TOAST columns, we do not require higher level replica identities for the table which may increase load on your source database. Artie knows how to disregard unchanged TOAST values by integrating with Debezium's unavailable value placeholder.