Hevo

Introduction

Hevo is a fully managed data integration platform that simplifies the process of collecting, transforming, and loading data from various sources into a centralized system. It streamlines the ELT (Extract, Load, Transform) workflow, enabling businesses to automate data pipelines without requiring extensive coding or manual intervention. Designed for efficiency, Hevo ensures real-time data synchronization while maintaining high reliability and scalability.

The platform follows a cloud-native, no-code approach, making data movement seamless and hassle-free. Key aspects of Hevo’s integration process include:

1. Connectivity: Hevo offers a diverse set of pre-configured connectors for databases, SaaS applications, cloud storage services, and other sources. Setting up a connection requires minimal effort, eliminating the need for custom development.
2. Data Extraction: Leveraging log-based Change Data Capture (CDC) and API-driven extraction, Hevo efficiently captures incremental data changes, ensuring optimized system performance and reduced load on source databases.
3. Data Loading: Once extracted, data is seamlessly loaded into destinations such as data warehouses, lakes, and BI tools. Hevo automatically manages schema changes, ensures data consistency, and handles errors without manual intervention.
4. Data Transformation: Hevo provides flexible transformation capabilities using Python and SQL-based processing, allowing users to clean, enrich, and structure data before analysis. Its intuitive interface simplifies data preparation, ensuring efficiency at scale.

With a focus on user-friendliness, Hevo enables quick setup - users select their data sources, configure pipelines, apply transformations, and automate the data flow with just a few clicks.

Connecting to Hevo

You can find instructions on how to connect to Hevo in this article.

What's imported

Imported metadata

Dataedo imports the following hevo items:

Pipelines as an ETL program, where each data flow (e.g., from one table to another) is treated as a separate object.

In lineage, the ETL program acts as a processor, connecting the Schema Mapper (represented as a Dataset in Dataedo) to the Destination.

Transformations that contain scripts,

Dataedo creates lineage between the Source object and the Transformation. However, at this stage, the output lineage to subsequent elements is not generated, as the scripts are not parsed.

Schema Mapper represented as a Dataset, it is an integral part of the lineage.

Sources and Destinations are the objects where a flow starts and ends in Hevo, representing entities from databases. Each element (e.g., a table) is treated as a separate object.

The Source maintains lineage between the original documentation and the Source object in Hevo (e.g., a table from a PostgreSQL database).

Precise matching is handled by Linked Source (more about linked sources here). Every Source object should have an assigned Linked Source, which can be verified in the Metadata and Settings tab.

Each Linked Source should be mapped to the correct documented database or another source. The matching process is automated, but if the program does not recognize the source, you can manually select the appropriate documentation.

Supported Dataedo features

*NA - not applicable

Automatic Data lineage

You can find information about automatic data lineage in this article.

Known limitations

Dataedo do not parse Python scripts or other transformations in this connector, which results in a lack of continuity in the lineage between all elements.