Automatic column-level lineage from imported Stored Procedures

Hubert Książek - Dataedo Team Hubert Książek 25th February, 2024
Applies to: Dataedo 23.x (current) versions, Article available also for: 24.x

Dataedo will create column-level data lineage for stored procedures based on the script. Just leave the "Automatic lineage" option selected during import or doing import changes, it is not possible yet to parse a single procedure or manual script beyond the mentioned operations.

Dataedo will divide the procedure into steps and represent them as processes, then create a data lineage for supported steps. Unsupported steps will be named after the first word from the process script and end with three dots.

Column-level data lineage from stored procedures

In the script tab, the processes contain fragments of the procedure script to which they refer.

Script of process definition from a stored procedure

Supported dialects

This function is available in the following dialects:

We are working on support for data lineage from stored procedures for other dialects.

Supported SQL syntax

We support the creation of an automatic lineage in all dialects for the following SQL statements:

  • INSERT (without inserting constant values),
  • UPDATE (without updating to constant values),

When temporary tables are created in a procedure (ones that do not normally exist in the documented database), Dataedo will try to combine the lineage that goes into them with the lineage that comes out of them, i.e. instead of SOURCE_TABLE -> #TEMP_TABLE -> DESTINATION_TABLE, will represent it as SOURCE_TABLE -> DESTINATION_TABLE.

Regrettably, due to the nature of ANTLR, the presence of unsupported elements within a procedure may result in the entire procedure being unable to be parsed. In the current stage, we treat procedures incorporating elements beyond those explicitly outlined as unsupported. Nevertheless, we do our best to make sure that the unsupported elements do not affect supported steps.

We do not support the creation of an automatic lineage for the following SQL syntax:

  • Common Table Expressions (WITH statements),
  • variables and parameters,
  • MERGE statements,
  • OUTPUT statements,
  • UPDATE when constants are set,
  • DELETE statements,
  • BULK INSERT statements,
  • COPY INTO statements,
  • syntax mentioned as unsupported in SELECT/CREATE VIEW
Found issue with this article? Comment below
Comments are only visible when the visitor has consented to statistics cookies. To see and add comments please accept statistics cookies.
There are no comments. Click here to write the first comment.