Azure SQL Database - Automatic Data Lineage

Hubert Książek - Dataedo Team Hubert Książek 11th October, 2024

What to Expect

Views

Dataedo analyzes SQL of database views with built in SQL Parser and builds column-level lineage from tables/views queried by the view script to the view object.

Learn more about Transact-SQL SQL parser

Views column-level lineage

Stored Procedures

Dataedo will create column-level data lineage for stored procedures based on the script. Script will be devided into steps represented as seperate processes. Data lineage will be created only for supported steps, the unsupported steps will be named after the first word from the process script and end with three dots. This is seen on the data lineage configuration tab in Desktop.

Stored Procedures object-level lineage

Stored Procedures column-level lineage

Polybase (external tables)

Dataedo will create object-level data lineage for Polybase connections. Dataedo supports following sources:

  • Oracle
  • MongoDB
  • Azure Data Lake Storage (ADLS) i Azure Blob Storage
  • SQL Server
  • S3-compatible object storage

Polybase lineage

Known Limitations

  1. Check the limitations for views lineage from SQL parser
  2. Check the limitations for stored procedures lineage from SQL parser

Troubleshooting

I don't see data lineage for views

  1. Make sure you have selected right SQL dialect - in this case Transact-SQL (SQL Dialect field at Data Source level).
  2. Rerun import of the source - maybe schema was imported in older version or configuration was incorrect.

I don't see data lineage for stored procedures

  1. Make sure Dataedo supports SQL syntax of procedure. Check Know Limitation above
  2. Rerun import of the source - maybe schema was imported in older version or configuration was incorrect.

I don't see data lineage for Polybase

  1. Make sure the source object has Linked Source with correctly assigned database
  2. Rerun import of the source - maybe schema was imported in older version or configuration was incorrect.

Cross database lineage is not built

  1. Make sure the source object has Linked Source with correctly assigned database
  2. Rerun import of the source - maybe schema was imported in older version or configuration was incorrect.