---

How to Manage Data Governance with Unity Catalog and Privilege Models

As organizations scale their data platforms, managing data access, security, and compliance becomes increasingly complex. Databricks’ Unity Catalog offers a unified solution for managing data governance across workspaces and cloud platforms. In this blog, we will explore how to manage data governance using Unity Catalog, starting from foundational concepts to advanced privilege models.

---

🧱 1. Introduction to Data Governance in the Cloud Era

Data governance ensures that data is secure, compliant, accurate, and accessible to the right users. In the cloud, where data is distributed across multiple sources and tools, governance challenges include:

• Inconsistent access controls
• Lack of centralized metadata
• Difficulty auditing and tracing access
• Regulatory compliance (e.g., GDPR, HIPAA)

---

🔍 2. What is Unity Catalog?

Unity Catalog is Databricks’ unified governance layer for managing data assets, access policies, and lineage across all Databricks workspaces in an account.

Key Features:

• Centralized metadata management
• Fine-grained access control (RBAC)
• Audit logging and data lineage
• Native support for tables, files, notebooks, and ML models
• Multicloud support (AWS, Azure, GCP)

---

📚 3. Unity Catalog Core Concepts

Term	Description
Metastore	Top-level container for data governance. Each account has one metastore.
Catalog	Container of schemas (like a database).
Schema	Container for tables, views, and functions (similar to a schema in SQL).
Table/View	Structured data assets.
Volume	Storage for unstructured data/files (e.g., images, JSON).
Workspace	The Databricks environment where users run notebooks and jobs.

---

🛡️ 4. Setting Up Data Governance with Unity Catalog

Step-by-Step Setup:

1. Create Metastore – One per cloud region per account.
2. Assign Metastore to Workspace – Ensures all workspaces share a common governance layer.
3. Configure Storage Credential and External Location – Secure access to cloud storage (e.g., ADLS, S3).
4. Create Catalogs and Schemas – Define structure and ownership.
5. Ingest and Register Data – Create managed or external tables, files, and views.

---

🔐 5. Privilege Models: Unity Catalog Access Control

Unity Catalog uses Role-Based Access Control (RBAC). Privileges are granted to principals (users, groups, service principals).

Common Privileges:

Object Type	Key Privileges
Metastore	CREATE CATALOG, MANAGE GRANTS
Catalog	USE CATALOG, CREATE SCHEMA
Schema	USE SCHEMA, CREATE TABLE, MODIFY
Table/View	SELECT, INSERT, UPDATE, DELETE
Volume	READ FILES, WRITE FILES
External Location	READ FILES, WRITE FILES, USE LOCATION

---

🔄 6. Hierarchical Privilege Enforcement

Unity Catalog enforces access top-down:

1. USE CATALOG → Required to access schemas
2. USE SCHEMA → Required to access tables/views
3. Table-Level Privileges → SELECT, INSERT, etc.

Example:

To query a table sales.region_data:

• You need:
  USE CATALOG on sales
USE SCHEMA on region_data
SELECT on the table itself

---

🧑‍💼 7. Managing Users, Groups, and Service Principals

• Unity Catalog integrates with Identity Federation in Databricks.
• You can manage access via Azure Active Directory (AAD) or SCIM API.
• Best Practice: Assign privileges to groups, not individual users.

Examples:

GRANT SELECT ON TABLE sales.region_data TO `analyst_group`;
REVOKE ALL PRIVILEGES ON SCHEMA finance.reports FROM `interns`;

---

📜 8. Managing Grants with SQL & APIs

You can manage privileges using:

• SQL GRANT/REVOKE statements
• Unity Catalog REST APIs
• Terraform (for IaC-driven governance)

Sample Grant:

GRANT SELECT, INSERT ON TABLE marketing.ad_clicks TO `marketing_team`;

View Grants:

SELECT * FROM system.information_schema.table_privileges
WHERE grantee = 'data_scientists';

---

🧮 9. Fine-Grained Access Control (FGAC)

Advanced data governance requires row- and column-level access control:

Row Filter Policies (Preview):

CREATE ROW FILTER policy_sales_region
ON TABLE sales.transactions
USING (region = current_user_region());

Column Masking (Future roadmap):

Enables dynamic masking based on user attributes.

---

📈 10. Auditing and Lineage

Unity Catalog provides built-in audit logs and lineage:

• Audit Logs: Who accessed what, when, and how.
• Data Lineage: Trace data flow from source to consumption.

Tools:

• system.access.audit table
• Lineage tab in Databricks UI
• External SIEM integration (e.g., Splunk, Azure Monitor)

---

🔧 11. Automation and Governance at Scale

Use Terraform or Databricks CLI for reproducible and automated governance:

Example: Terraform Resource

resource "databricks_grants" "marketing_data" {
  table = "catalog.marketing.schema.ad_data"
  grant {
    principal = "marketing_team"
    privileges = ["SELECT"]
  }
}

Automate:

• User onboarding
• Grant provisioning
• Schema/table lifecycle management

---

🚨 12. Best Practices for Unity Catalog Governance

Best Practice	Description
Use Groups over Users	Simplifies access control management.
Define Naming Conventions	For catalogs, schemas, and groups.
Enable Audit Logs	Always enable audit trail for compliance and monitoring.
Periodically Review Grants	Clean up unused and risky permissions.
Use External Volumes for Files	Improve performance and decouple storage from compute.
Implement Least Privilege Access	Grant only what is necessary.
Separate Environments by Catalog	For prod, dev, test isolation.

---

🔮 13. What’s Next? Future of Governance in Databricks

Upcoming features in Unity Catalog:

• Attribute-based access control (ABAC)
• Policy-as-code integration
• Support for non-Databricks data sources (federated governance)

---

✅ Conclusion

Unity Catalog is a powerful platform for unified, scalable data governance in Databricks. By combining structured access controls with auditing, lineage, and automation capabilities, it enables organizations to confidently manage their data estates while remaining compliant and efficient.

Whether you’re just starting or managing enterprise-scale data, mastering Unity Catalog and its privilege models is key to a secure and modern data platform.

---