Connecting to Fivetran

Connection requirements

Dataedo connects to Fivetran using a REST API. To set up this connection, you'll need an API Key and an API Secret. These credentials are generated within the Fivetran platform and are used to authenticate requests to the API. You can find the instructions for generating an API Key and Secret here.

Required permissions

Recommended role is Account Reviewer. This role allows read-only access to all account information, including groups, connectors, and schemas.

Dataedo utilizes the following REST API endpoints to extract metadata from Fivetran:

• GET https://api.fivetran.com/v1/account/info - to verify the provided credentials
• GET https://api.fivetran.com/v1/groups - to select a group to document
• GET https://api.fivetran.com/v1/destinations/{destination_id} - to verify the selected group and get destination details (for linked sources)
• GET https://api.fivetran.com/v1/connectors/{connector_id} - to create linked sources
• GET https://api.fivetran.com/v1/connectors/{connector_id}/schemas - to create corresponding objects in Dataedo and object-level data lineage
• GET https://api.fivetran.com/v1/connectors/{connector_id}/schemas/{schema_name}/tables/{table_name}/columns - to create column-level data lineage

If you want to create a specific role for Dataedo, ensure it has permission to use the endpoints listed above.

Connecting to Fivetran

Connector selection

To import Fivetran, click the Add button in the upper left corner and choose New connection.

From the sources, choose Fivetran, and then click Next >

Connection details

Fill in the API Key and API Secret fields with the credentials generated in Fivetran.

Select the group (destination) to document. On the list, you will see the destination name with group ID in brackets, after selection, the field will be filled with just group ID.

Confirm the connection by clicking Connect.

Filtering and documentation title

Objects have schema and name as in the destination database. You can filter them by schema or name. After setting up filters, click Next >.

You can set up the documentation title. By default, it is the name of the destination. You can change it to a more descriptive one. After setting up the title, click Import.

Successful first import

The first import should be relatively quick. After it is done, click Finish.

Assigning linked sources

After the first import, you must assign linked sources to the corresponding Source Database in Dataedo. To do this, go to the Linked sources folder in Fivetran documentation, select a linked source to be assigned, expand Source Database and select the corresponding Source Database from the list.

Repeat this step for all linked sources that can be assigned. When linked sources are assigned their logo will change to a database icon.

Importing changes for data lineage

After assigning linked sources, you must import changes to import data lineage. To do this right-click on the Fivetran documentation and choose Import changes. Confirm the import by clicking Connect.

This import will take longer than the first one.

Results

After the second import, you will see the data lineage in the Fivetran documentation.

Known limitations

Dataedo only works with Fivetran sources that are applications (like Salesforce) or databases (like SQL Server) and are also supported by Dataedo. It doesn’t support other types of sources like events (such as Apache Kafka), files (like Amazon S3), functions (such as AWS Lambda), or sources without a corresponding connector in Dataedo (like Google Analytics).

Linked sources are not assigned automatically, this must be done manually.

Dataedo only imports information from the source connectors that are currently connected.

Transformations should be imported separately using dbt connector.