Dataedo 10 allows you to discover data stored in the database and review its contents and quality. Data Profiling module is a combination of useful metrics with a friendly User Interface. On top of it, Profiling in Dataedo allows you to peek into the most common, or random data from your tables. Learn more about Data Profiling.
Part of the improvements in the area of Data Profiling is the fact that we surfaced the ability to peak into Sample Data. Right now you can simply use a handy right-click option on a table to preview its values!
We introduced Data Lineage, in two parts. You can design flows using Dataedo Desktop and browse diagrams using Dataedo Web Catalog. Read more about Data Lineage in Dataedo.
In Dataedo 10.3 we also brought the ability to set column-level data lineage.
History of changes in documentation
We introduced the history of changes in descriptions, titles, and Custom Fields. Dataedo will store information about who, when, and how changed the values.
Read more about History of changes.
Online licenses and Dataedo Account
This biggest change in the 10 will present itself at the login screen to the Desktop App. You can now log in to Dataedo using your business email. Also, you can seamlessly switch through all your available licenses.
Permissions in the Web Catalog
From version 10.2, accessing any data via Dataedo Web Catalog requires correct permission. This allows detailed customization regarding who will be able to perform what action in the Dataedo Web Catalog.
Read more about Permissions in Dataedo Web Catalog.
Email Community notifications
We are introducing a feature of notifying users about comments and questions, regarding objects they are following.
Find out more about Email Notifications in Dataedo Web Catalog
Export and Import with DDL script
You can now export your data structure to data description language (DDL). In other words, you will be able to export the database to a set of statements like CREATE and ALTER. Read more about Export to DDL.
Starting from Dataedo 10.2 you can import your database schema using SQL dump from your database.
Read more about Importing from DDL script.
Import from interface tables
You can now import your database schema using interface tables, built-in Dataedo.
Read more about Importing from Interface Tables.
Redesigned Data Classification
We redesigned Data Classification and improved performance or algorithms. You can now classify only a portion of the Data Catalog and configure classification to match your needs.
We also added a few built-in classifications, so that the current list contains:
- CCPA - California Consumer Privacy Act
- FERPA - Family Educational Rights and Privacy Act
- GDPR - finds columns potentially holding personal data as defined by European GDPR
- HIPAA - Health Insurance Portability and Accountability Act
- PCI - Payment Card Industry
- PII - finds columns PII (Personally Identifiable Information)
Read more about Data Classification.
Report Catalog and Power BI
The report is a new type of object next to the Business Glossary and Database. Reporting is an object that contains data sets and visualizations. Check out more about existing object types in Dataedo.
In version 10.4 this feature is coming with the support of the Power BI connector so that you can automatically document your reports.
Read more about Power BI connector.
Reference Data Management
We created a new type of object – Lookup. Lookups allow you to describe values stored in objects’ columns. They are also known as reference data, lists of values, or dictionaries. You can manually create new lookups or create them automatically based on the values in your database.
Read more about that in our Reference Data Management section of the documentation.
Redesign of Web Catalog
We've made effort to make Web Catalog better and easier for you to use. The most important changes are:
- Navigation received UX improvement.
- New rich text editor. Read more about editing in Web Catalog
- Community is now easily accessible on a right-side panel
- We replaced our Grid (tables) engine and added a few features to it (like column sorting)
- We've added settings sections and an option to pin Custom metadata Fields
It's not a one-time thing - with the next releases we are going to continue improving Web Catalog UI and UX!
New metadata connectors
We added a set of new Metadata scanners:
- Azure Cosmos DB - Core/SQL API, Cassandra API, MongoDB API
- Amazon Keyspaces
- Power BI Premium Workspace/Dataset
- Microsoft Dataverse
- Tableau (BI data model)
- AWS Athena
- SAP ASE
- Hive Metastore
- SQL Server Integration Services (SSIS) (beta)
- dbt (beta)
- SAP HANA
- Astra DB
- Amazon DynamoDB
- Azure Blob Storage
- Azure Data Lake Storage (ADLS)
We introduced a way, to import files like JSON, CSV, or XLS directly from:
Learn more about supported databases by Dataedo.
Improvements in existing connectors
We made a few improvements in existing metadata connectors:
- Amazon Redshift and Azure SQL Database are now supported with Data Profiling.
- Azure SQL Database multi-factor authentication is now supported.
- PostgreSQL with better SSL support.
Dataedo Web Catalog
Dataedo 9 brings completely new way of interacting with data catalog. 9.1 adds option to edit descriptions.
Data Community in Dataedo Web Catalog
Dataedo 9.1 adds Community features to allow this discussion to occur naturally. Viewer users can now comment on objects and their documentation to point out any required changes in the database or issues with the documentation itself. This and future changes will empower your end-users to become true Data Citizens.
Editing Descriptions in Web Catalog
Sometimes, when browsing through descriptions in the Web Catalog you could encounter information that needs clarification or typos. Before, to correct these issues an editor needed to start Dataedo Desktop and make the changes there. In Dataedo 9.1 you can edit titles, descriptions, and custom field values directly from Dataedo Web.
Authentication with Okta/SAML in Web Catalog
From 9.2 Dataedo Web Catalog can be configured so that users can log in with Okta and possibly with other SAML providers. This simplifies deployment for admins and adoption in the company. It's enough for admins to enable access to Web Catalog in the SAML provider and users are able to log into the portal. 9.2 does not check available licenses. This will functionality be released at a later date.
Data Classification view in Web Catalog
New Data Classification module added to the Web Catalog showing a summary of column classifications grouped by classification, data domain and classification.
Primary/Unique keys (PK/UK) and Relationships (FK) editor
We are constantly expanding documentation editing capabilities for data stewards. Dataedo 9.2 adds option to define and edit primary and unique keys (PK/UK) and relationships (foreign keys) between tables and other data objects.
Support for Data Lake files
One of the most requested features was an option to document files commonly used in data lakes. In Dataedo 9.1, we've added option to import schema from files. We are constantly extending support for various formats and now the list of sources is the following:
- Apache Avro,
- Apache Parquet,
- Apache ORC,
- Delta Lake.
Support for NoSQL databases and new connectors
Dataedo 9 adds support for a number of new databases, including NoSQL databases, search engines, and data warehouses:
- MongoDB (most popular document database),
- Neo4J (most popular graph database),
- Elasticsearch (most popular search engine),
- Apache Cassandra (most popular wide column database),
- Google BigQuery,
- Vertica Analytics Platform.
Designer extension - manual views, procedures and functions
Till 9.2 objects designer only supported tables. From 9.3, you can design manual views, procedures and functions too.
Now, you can copy entire module with ERD and linked objects.
Option to paste multiple cells
We added ability to paste multiple cells at a time (with multiple values or the same), similarly to MS Excel.
Option to paste one value:
Option to paste and delete multiple values:
Business Glossary in (free) PDF export
From 9.2 free PDF export can also include a Business Glossary. This allows you to share your Business Glossaries to unlimited audience, without the need for the Enterprise Viewer licenses.
Azure Authentication support
This is a much-requested feature from Azure Active Directory users - now, you can connect to Dataedo repository and your Azure/SQL Server databases when importing schema with Azure Authentication.
Show/hide schema names
Till now, Dataedo decided whether to show schema name in the object name or not based on the DBMS. Many users pointed out that they would like to decide
Now, Dataedo has a smarter algorithm to show schema name – based on DBMS and number of schemas in your database, and it provides option to override this manually.
New sources and databases
In Dataedo 7 we added a number of new native connectors and a generic ODBC connection that works with majority of sources.
- IBM Db2
- Amazon Redshift
- Percona Server for MySQL
- MySQL 8.x
Business Glossary (Enterprise)
Dataedo Enterprise metadata repository has now the ability to define a common language and understanding of key terms across the organization - architects, developers and business users - in a Business Glossary, and link it to specific data assets (tables or fields).
Manual Data Dictionary
While we are constantly working on including more data sources in Dataedo, we understand that at any given time some will be unsupported.
To fix this, from Dataedo 7.0.2 you can add create your documentation and tables from within the app.
Later on, when we add support for your data source, you will be able to just click connect and import the actual structure from your database without losing your descriptions.
Manual tables in existing documentations
Additionally, you can now add tables not existing in your database to the documentation.
Just click the Add table button in the ribbon, or right click the documentation you want to create the table in and you can define your new table.
Schema Change Tracking (Enterprise)
Often, it's important to be able to see what and when changed in database schema. Dataedo 7 Enterprise has an option to enable tracking of schema changes.
Search in HTML export
Now, it is able to search tables and other objects by their names, aliases and column names (and aliases) in static HTML exports.
HTML export performance
We've completely redesigned our HTML export and now:
- Loading time is much faster
- HTML is more responsive
- HTML is lighter
- Time of export is much shorter
- No need to choose between formats
Custom field types (drop-downs and more)
Until now, all custom fields were text based. Many users asked for an option to add a drop down or checkbox type custom field and we added it in this release.
From Dataedo 7.2 you can set a type for any custom field.
Available types are:
- closed list with predefined values
- open list with predefined values and ability to add new elements while editing
- multiselect with predefined options
- number (integer)
- free text
Copy descriptions (beta feature)
Sometimes you have multiple databases with very similar structure and want to avoid documenting them multiple times, like Dev/UAT/Prod environments, or customizaions of databases for different customers. Now, you can have different documentations and copy descriptions between them to make the maintenance of separate documentations easier.
Sometimes a column is used in a standard way in the entire database, and a specific description fits it multiple times.
Now, by right clicking a column you can find all columns with the same name in documentation or repository and edit multiple descriptions at the same time.
Improvements in data dictionary
User defined columns in existing tables
You can also add user defined columns to an existing table. Just select your table and click the Design table button in the ribbon, or right click the table you want to edit and choose Design table.
Reordering columns in existing tables
In many DBMSes altering the order of columns in a table is not trivial. However logical column ordering can greatly improve understanding of their purpose.
From Dataedo 7.0.2 you can alter the order with which columns are displayed in your documentation.
Improved progress tracking
Many users requested a change in progress tracking to include only tables and columns. We added this as Tables & columns progress tracking option
Additionally, we've added an option to track progress of a specific custom field.
Dataedo 7.2 now suggests texts for columns descriptions, titles or text custom fields based on texts from other columns. Those suggestions are hinted by blue marker and available in context menu.
Cross documentation modules
Previously, objects could only be added to modules within their documentation.
To better support cross database environments, you can now add object to modules from any documentation in your repository.
You can add objects from different documentations as usual, by dragging them from the navigation tree, from object lists one by one or in groups or by choosing modules on the object's description page.
Filtering and quick search
When your table contains hundreds of columns, finding the one you want to edit can be cumbersome. In Dataedo 7.1 we've added an option to filter objects:
And an addition to our previous search feature - search within current tab:
This applies to almost all of our grid style tabs.
We've made multiple improvements to our ERDs.
Add related tables
You can now add all objects related to a node on an ERD by right clicking it and choosing Add related tables:
We redesigned and improved suggested entities list. It is now context aware and includes all available entities from repository and suggests entities assigned to module and related to selected entity/entities on the diagram.
Dataedo 7.1 now imports Oracle packages:
Importing Materialized Views
Dataedo 7 now imports materialized views from Oracle:
Until now, importing from Oracle required Oracle Client installation.
In Dataedo 7.1 you don't need to install Oracle Client to connect, you can now use direct connection option:
Repository on Azure
Until now, Dataedo supported file and SQL Server based repository. In version 7 we added ability to set up Dataedo repository in Azure SQL Database.
As of release 7.0.3 beta Azure repository is still under development. It is stable, but requires manual setup.
Much more to come...
This is a flagship feature of Dataedo 6 - the ability to add custom metadata fields to any database object/element. You can add additional descriptive fields to columns, tables, etc. This opens up a lot of new possibilities.
Those fields can be searched for, and used as filters during the search. Now you can find all confidential fields in your database easily.
Cross-database relationships and diagrams
Another key feature of 6.0 is the ability to create cross-database, cross-server, and cross-engine table relationships. It enables you to create relationships in Data Dictionary and use them to create ER diagram that shows multiple databases. This allows you to document how various systems are linked together. It's particularly useful in case of SQL Server and MySQL if your system has more databases as those engines don't support FKs across different databases.
One to one relations
Dataedo 6.1 introduced the ability to define cardinality of FK/relation to 1:1, both existing and user-defined relationships. In the data dictionary and ERDs.
User defined dependencies
You can now add a dependency to any object's dependencies tab. The new dependency can be created across databases and engines. This is particularly useful for databases such as MySQL where dependencies are not provided by database data dictionary, and you can document this information manually.
Just drag the objects you want to add the dependency on to either side of the dependencies tab.
Exporting descriptions back to the database
With Dataedo 6 you can save descriptions and custom fields from Dataedo repository to extended properties of your SQL Server databases or descriptions to comments of your Oracle databases.
New HTML layout and multiple databases
It is now possible to export multiple documentations to a single document. So if your system spans across multiple databases, you can hold them all in a single PDF file or a website.
We've altered the look of HTML export to make it even more intuitive.
Adding objects to ERD from navigation tree
Previously, to add an object to ERD you've had to use the object list on the right side of the screen. This could be cumbersome, especially when you've wanted to add objects from different documentations.
Now you can just drag any object you want to add from the navigation tree on the left side of the screen.
You can also drag entire branches to an ERD, eg. drag an entire module to the ERD to add all the tables and views included in it.
Adding objects from a different documentation to ERD
In Dataedo 6 it was possible to add objects from different documentaions to an ERD only if they had relations to existing objects.
Now you can add any object from any documentation to an ERD using the drag & drop feature described above.
Including multiple schemas in single Oracle import
Dataedo 6 can now import metadata from multiple Oracle schemas at once.
Scripts in exports
It is now possible to include view, function, stored procedure and trigger scripts in your documentation export.
You don't have to repeat information visible in script comments in descriptions anymore.
Improved FK View
Another small but pleasing change. We added table reference information next to a column. Now you can quickly see if a column is a foreign and to what table. This information is visible in UI and exported to HTML and PDF.
Improved manual keys view
From now on, all information from Keys tab (either imported from a database schema or added manually in Dataedo repository) will propagate to columns table, diagrams and exports.
Easier publishing on Confluence
We've added a special .js file to our HTML export making it even easier to include it on Confluence pages without issues with page sizing and formatting.
Easier to show/hide all columns on ER diagrams
We added a global option to show/hide columns on ER diagrams. You can now show all columns in all tables, hide all columns in all tables leaving only table header or show key columns.
SQL Server 2017 support
Dataedo 6.0 now officially supports new SQL Server 2017. Older versions should also work but were not tested.
Azure SQL Data Warehouse support
We've added support for Azure SQL Data Warehouse.