Introduction

A few days ago, I was talking to my son about writing this blog on data governance catalogs and he said, “I can help with that.” A few minutes later, he gave me the following lighthearted description of a data catalog. It’s humorous, but it also contains a great deal of truth. Let’s break it up and explore those truths.

“In the realm of data governance, the data governance catalog serves as a powerful enchanted tome, holding the secrets and knowledge of all your data treasures. Just like an ancient grimoire safeguarded by wizards, the catalog ensures that your data artifacts are organized, classified, and protected from the lurking shadows of Chaos.

“Think of the catalog as a magical compass that guides you through the perilous maze of the data landscape, helping you navigate through treacherous data lakes and perilous data dungeons. It empowers you to make informed decisions, unlocking hidden insights and allowing you to harness the mystical potential of your data assets.

“With the data governance catalog at your side, you can wield the arcane powers of data lineage, understanding the origins and transformations of each piece of information. It acts as a guardian against the malevolent forces of data breaches and misuse, shielding your kingdom from the curse of non-compliance and dark sorcery.

“Embrace the data governance catalog, for in doing so, you gain mastery over the elements of data, ensuring that the forces of Chaos remain vanquished, and your kingdom will thrive in the light of knowledge and wisdom.”

A Magical Data Governance Catalog

“The data governance catalog serves as a powerful enchanted tome, holding the secrets and knowledge of all your data treasures.”

Your data catalog serves as an inventory of metadata (secrets and knowledge) for your data systems (your data treasures). A primary requirement of an effective data catalog is that it’s comprehensive. Users can easily find all the information they need in one location and have confidence that the information returned is complete and trustworthy. In other words, it’s the “single source of truth” for your organization.

The data governance catalog is an inventory of your data assets, along with definitions and other metadata, that can help users learn what data exists and discover what the data means so they will be able to use and interpret effectively. Data users in your organization will be more accurate and efficient as their data literacy increases and the natural result is better data insights and decisions.

“Safeguarded by wizards, the catalog ensures that your data artifacts are organized, classified, and protected from the lurking shadows of Chaos.”

Every individual in your company who produces, manages, or uses data (this includes virtually everyone in your company) can and should become a “steward” of the data—all are responsible for “good data, used correctly.” The data catalog supports this mature concept of data stewardship because it allows producers, managers, and users of the data to see themselves as an integral part of data stewardship processes, including proper understanding of the data itself. If your data assets are not inventoried, documented, and actively managed, understanding and proper use will suffer. In other words, the “shadows of Chaos” will encroach into your decision-making processes.

“It empowers you to make informed decisions, unlocking hidden insights and allowing you to harness the mystical potential of your data assets.”

Over the relatively recent past, data has increasingly become recognized as a valuable business asset, but simply having data is not enough—it has to be used, and used correctly. Decisions based on “good” data are extremely powerful. The key points here are good data, correct understanding, and proper use of the data. The data governance catalog supports these efforts by documenting the data in a way that those in a support role can clean and manage the data, understand data concepts, and document the data that supports those concepts including any nuances needed for its proper use.

Furthermore, the data catalog by its very existence encourages collaboration between people, teams, and departments, first by supporting effective communication and second by informing them what data exists. In other words, the data governance catalog helps remove siloes and “demystifies” the data.

“With the data governance catalog at your side, you can wield the arcane powers of data lineage, understanding the origins and transformations of each piece of information.”

Knowing the lineage of your data gives you significant power. Looking at upstream lineage reveals to users the sources of the data as well as any filters or transformations applied to it during its journey. This lets them judge its usefulness for a given purpose and insights needed to interpret the data correctly.

On the other hand, knowing where data will go in its downstream journey supports impact analyses and change management activities. Ask an analyst or engineer how long it takes to manually comb through ETL jobs and identify where data goes, and they’ll tell you that the effort needed to create an accurate data lineage will pay for itself in short order.

“It acts as a guardian against the malevolent forces of data breaches and misuse, shielding your kingdom from the curse of non-compliance and dark sorcery.”

Your data governance catalog helps you protect sensitive or regulated data by documenting data systems and the processes that support them—specifically identifying your protected data assets and labeling them as such so they may be protected and used in ways appropriate to their sensitivity levels. The data catalog enables those responsible for data security and governance to ensure that proper security measures are in place and that they are functioning properly. It also tells where regulated data exist in your data systems, so it is simple to verify that appropriate measures are in place to protect that data and ensure that the data lifecycle is managed appropriately.

“Embrace the data governance catalog, for in doing so, you gain mastery over the elements of data, ensuring that the forces of Chaos remain vanquished.”

Chaos can take several forms in your data system, but one of particular concern involves communication. It’s important for people in your company to be able to talk to others in terms that both parties understand. The business glossary portion of your data catalog serves this purpose. It includes terms and definitions used in your organization and provides the basis for people to use precise terminology in their communications.

Further, these terms support standard use and understanding of data concepts in reporting. Without such a reference, analysts or data scientists may use the same term for different concepts or different terms for the same concept. Even if the concept is understood clearly, reports sometimes apply filters or aggregations in ways that lead to conflicting results. All these situations lead to data confusion and possibly to bad decisions. A business glossary serves as a powerful "ward" against such dangers.

“Your kingdom will thrive in the light of knowledge and wisdom.”

In the world of Knowledge Management, considerable emphasis is placed on the differences between information, knowledge, and wisdom. Some have said, “Information that has been analyzed and shared becomes knowledge and knowledge rightfully applied becomes wisdom.” Another way of stating this is that knowledge is information that has been interpreted and wisdom comes from the correct application of knowledge. Clearly both are desirable (interpretation and correct application). An effectual data governance catalog supports the understanding of your data (knowledge) and its proper use and interpretation (wisdom).

May your data kingdom thrive and vanquish the evil forces of Chaos!