SQL Server Integration Services (SSIS) - Automatic Data Lineage

Michał Trybulec - Dataedo Team Michał Trybulec 3rd June, 2024

What to Expect

Dataedo imports automatically:

  1. Packages,
  2. Tasks (as processes in Dataedo),
  3. Sources and Destinations (derived from data transfer tasks),
  4. Column-level lineage dependent on the flow elements used (more about the elements here)

Data Transfer Tasks

The source or destination may be relational databases, files, odbc sources Image title

How is SSIS lineage shown in Dataedo?

Dataedo's documentation of a sample SSIS data flow is shown below.

Source

Image title

For Sourcs that use SQL queries to create them, there will be a lineage based on the parser

Image title

Process in lineage

Image title

Destination

Image title

FTP Tasks

Image title

Execute SQL Tasks - object level

Image title

For "Execute SQL Task" tasks that call a procedure, a lineage to that procedure will be created

Image title

Known Limitations

  • For some of the elements in the flow lineage is not created, a detailed description of the cases is here.
  • We don't have a proven lineage based on third-party vendor elements.
  • Lineage will not be created if variables are used as addresses

Troubleshooting

Lineage is created based on a matched Linked Source, check the Dataedo desktop to make sure it is matched properly. Image title