Introduction
Audit logs in Databricks Unity Catalog track data access, modifications, security changes, and governance actions. If audit logs are missing Unity Catalog actions, it could lead to compliance risks, security gaps, and lack of visibility into user activities.
🚨 Common issues with Unity Catalog audit logs:
- No Unity Catalog events in audit logs.
- Metadata changes (CREATE, ALTER, DROP) are not logged.
- Data access logs (SELECT, INSERT, DELETE) are missing.
- Databricks workspace logs general events but not Unity Catalog-specific actions.
This guide helps troubleshoot and enable full Unity Catalog audit logging in Databricks.
1. Verify That Audit Logging Is Enabled in Databricks
Symptoms:
- No audit logs are captured for Unity Catalog events.
- Audit logs are empty or only contain workspace events.
Causes:
- Audit logging is not enabled in the Databricks workspace.
- Logging is configured incorrectly (wrong storage account, missing permissions).
Fix:
✅ Check if audit logging is enabled:
databricks account settings get-audit-log-config
✅ Enable audit logging (Admin Required):
databricks account settings enable-audit-logs --storage-account <storage-name> --container <container-name>
✅ For AWS, ensure CloudTrail is configured to capture Databricks events:
aws cloudtrail describe-trails
✅ For Azure, ensure diagnostic settings include Unity Catalog logs:
- Go to Azure Portal → Monitor → Diagnostic Settings.
- Enable logs for Databricks and Unity Catalog.
2. Check If Unity Catalog Actions Are Included in Logging Configuration
Symptoms:
- Only workspace-level events are captured, but Unity Catalog actions are missing.
- No logs for catalog, schema, or table changes.
Causes:
- The audit log does not include Unity Catalog operations.
- Logging is capturing only cluster and job events.
Fix:
✅ Ensure that Unity Catalog-specific logs are enabled:
databricks logging enable --log-type UNITY_CATALOG
✅ Enable logging for metadata operations (CREATE, ALTER, DROP):
databricks audit-logs configure --include-metadata-changes true
✅ Enable fine-grained logging for SQL queries:
databricks logging enable --log-type SQL_QUERIES
✅ For AWS, add Unity Catalog logs to CloudTrail:
aws cloudtrail put-event-selectors --trail-name <trail-name> --event-selectors '[{"ReadWriteType": "All", "IncludeManagementEvents": true, "DataResources": [{"Type": "AWS::S3::Object", "Values": ["arn:aws:s3:::databricks-audit-logs/*"]}]}]'
3. Verify That Unity Catalog Logs Are Sent to Cloud Storage
Symptoms:
- Logs exist in Databricks UI but are not stored in cloud storage.
- Log files are missing in AWS S3, Azure ADLS, or GCS.
Causes:
- Logging storage path is incorrect or not accessible.
- IAM roles do not allow Databricks to write logs to cloud storage.
Fix:
✅ Check storage configuration for audit logs:
databricks account settings get-audit-log-config
✅ Ensure Databricks has permissions to write logs to S3/ADLS/GCS:
- AWS IAM policy for S3 logging:
{
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:GetObject"],
"Resource": "arn:aws:s3:::databricks-audit-logs/*"
}
- Azure ADLS permissions:
az role assignment create --assignee <databricks-service-principal> --role "Storage Blob Data Contributor" --scope /subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<storage-name>
✅ Verify logs are being written to storage:
aws s3 ls s3://databricks-audit-logs/
az storage blob list --container-name <container-name> --account-name <storage-name>
4. Check Unity Catalog Permission Auditing (GRANT, REVOKE)
Symptoms:
- No logs for permission changes (GRANT, REVOKE, ALTER).
- Security team cannot track access modifications.
Causes:
- Permission change events are not enabled in audit logs.
- Unity Catalog does not log permission changes by default unless configured.
Fix:
✅ Enable logging for GRANT and REVOKE actions:
databricks logging enable --log-type PERMISSION_CHANGES
✅ Manually verify permission changes using Unity Catalog SQL queries:
SHOW GRANTS ON CATALOG main;
SHOW GRANTS ON SCHEMA default;
✅ For AWS, enable permission tracking in CloudTrail:
aws cloudtrail put-event-selectors --trail-name <trail-name> --event-selectors '[{"ReadWriteType": "All", "IncludeManagementEvents": true}]'
5. Ensure Query History Logging for Data Access Audits
Symptoms:
- No logs for SELECT, INSERT, UPDATE, DELETE queries.
- Cannot track who accessed specific tables.
Causes:
- Query history logging is disabled.
- Unity Catalog SQL queries are not being logged.
Fix:
✅ Enable query logging:
databricks logging enable --log-type QUERY_HISTORY
✅ Manually check query logs:
SHOW HISTORY ON TABLE my_catalog.my_schema.my_table;
✅ For Databricks SQL warehouses, enable logging for queries:
- Go to SQL Warehouses → Edit → Enable Query Logging.
6. Troubleshooting Step-by-Step
Step 1: Verify That Audit Logs Are Enabled
databricks account settings get-audit-log-config
Step 2: Check If Unity Catalog Logging Is Included
databricks logging enable --log-type UNITY_CATALOG
Step 3: Ensure Logs Are Being Written to Storage
aws s3 ls s3://databricks-audit-logs/
az storage blob list --container-name <container-name> --account-name <storage-name>
Step 4: Verify Permissions for Writing Audit Logs
- AWS: Ensure S3 IAM permissions allow Databricks to write logs.
- Azure: Ensure Storage and Key Vault permissions are set correctly.
Step 5: Manually Check Query and Permission History
SHOW HISTORY ON TABLE my_catalog.my_schema.my_table;
SHOW GRANTS ON CATALOG main;
Best Practices for Capturing Unity Catalog Audit Logs
✅ Ensure Audit Logging Is Enabled for Unity Catalog
databricks account settings enable-audit-logs --storage-account <storage-name>
✅ Enable Logging for Metadata, Permissions, and Queries
databricks logging enable --log-type QUERY_HISTORY
databricks logging enable --log-type PERMISSION_CHANGES
✅ Verify That Logs Are Being Stored in Cloud Storage
aws s3 ls s3://databricks-audit-logs/
✅ Use CloudTrail (AWS) or Azure Monitor for Additional Logs
aws cloudtrail describe-trails
Conclusion
If Unity Catalog audit logs are missing or incomplete, check:
✅ Audit logging is enabled in Databricks settings.
✅ Unity Catalog-specific logs are turned on.
✅ Logs are correctly stored in AWS S3, Azure ADLS, or Google Cloud Storage.
✅ Permission changes and SQL queries are logged properly.
By following this guide, you can ensure that all Unity Catalog actions are audited for compliance, security, and governance.