,

User Access Logs Are Missing from Unity Catalog

Posted by

Introduction

Unity Catalog in Databricks logs all user access and activity for compliance, security, and auditing. However, if user access logs are missing, organizations may face challenges in tracking data usage, detecting unauthorized access, and ensuring regulatory compliance.

🚨 Common issues when user access logs are missing in Unity Catalog:

  • Audit logs do not appear in the system.access table.
  • User actions (SELECT, INSERT, DELETE) are not recorded in the logs.
  • Cannot track changes made to tables, schemas, or catalogs.
  • Missing events in Azure Monitor, AWS CloudTrail, or GCP Logging.

This guide walks through troubleshooting steps and best practices to restore and ensure proper logging of user activity in Unity Catalog.


1. Check If Audit Logging Is Enabled in Unity Catalog

Symptoms:

  • SELECT * FROM system.access returns no records.
  • Audit logs are missing for recent user actions.

Causes:

  • Audit logging is not enabled for Unity Catalog in the workspace.
  • The Databricks workspace is not linked to cloud logging services (AWS, Azure, GCP).
  • The log retention period has expired, removing old logs.

Fix:

Check if audit logging is enabled in Databricks workspace settings:

  • Go to Admin Console → Workspace Settings → Unity Catalog.
  • Ensure “Enable Audit Logging” is turned ON.

Query Unity Catalog’s system tables to verify logs:

SELECT * FROM system.access WHERE action_name = 'SELECT' ORDER BY event_time DESC;

If logs are missing, enable Unity Catalog logging manually:

ALTER SYSTEM SET AUDIT_LOGGING = TRUE;

Check the log retention period in Unity Catalog:

SHOW CONFIGS;
  • If retention is too short (e.g., 7 days), increase it.
ALTER SYSTEM SET AUDIT_LOG_RETENTION = '30d';

2. Ensure the Workspace Is Configured for Cloud Logging (AWS, Azure, GCP)

Symptoms:

  • Audit logs are missing in AWS CloudTrail, Azure Monitor, or GCP Logging.
  • Logs appear locally in Unity Catalog but not in external cloud storage.

Causes:

  • Cloud logging is not set up for Unity Catalog.
  • IAM roles do not have permissions to write logs to AWS S3, Azure Monitor, or GCP Logging.

Fix:

For AWS: Enable CloudTrail Logging

aws cloudtrail create-trail --name Databricks-Audit-Trail --s3-bucket-name my-audit-bucket
aws cloudtrail start-logging --name Databricks-Audit-Trail

For Azure: Enable Log Analytics Workspace Integration

az monitor diagnostic-settings create --name DatabricksAuditLogs \
--workspace my-log-analytics-workspace \
--logs "AuditEvent" "Administrative"

For GCP: Enable Logging for Databricks

gcloud logging sinks create databricks-logs \
--log-filter "resource.type=databricks_cluster" \
--destination "storage.googleapis.com/my-bucket"

3. Verify User Access Events in Unity Catalog System Tables

Symptoms:

  • Audit logs exist but do not show user-level actions like SELECT or DELETE.
  • Only admin or automated events (like cluster start/stop) are logged.

Causes:

  • Users lack required permissions to generate access logs.
  • Table and schema-level logging is not enabled in Unity Catalog.

Fix:

Check if the system.access table contains records:

SELECT * FROM system.access ORDER BY event_time DESC;

Grant logging permissions to all users:

GRANT USAGE ON SYSTEM TABLE system.access TO ALL;

Ensure table and schema logging is enabled:

ALTER SYSTEM SET TABLE_ACCESS_LOGGING = TRUE;

4. Verify Permissions for Unity Catalog Logging

Symptoms:

  • Users with correct permissions cannot see logs.
  • Only admins can access the logs.

Causes:

  • Lack of permission to access Unity Catalog system logs.
  • Users are querying logs from the wrong workspace or catalog.

Fix:

Grant permission to access logs:

GRANT SELECT ON TABLE system.access TO `user@example.com`;

Ensure correct workspace and catalog are used in queries:

USE CATALOG system;
SELECT * FROM access WHERE user = 'user@example.com';

5. Logs Are Delayed or Intermittently Missing

Symptoms:

  • Some events appear late or intermittently in logs.
  • Only partial logs appear when querying system.access.

Causes:

  • Cloud storage ingestion delays (AWS S3, Azure ADLS, GCS).
  • Too many concurrent queries causing log delays.

Fix:

Enable faster log delivery using IMMEDIATE LOGGING:

ALTER SYSTEM SET IMMEDIATE_LOGGING = TRUE;

Reduce query load on system tables to avoid delays:

SELECT * FROM system.access WHERE event_time >= NOW() - INTERVAL '1 HOUR';

Check cloud storage performance for log ingestion issues.


6. Logs Are Being Overwritten or Deleted Too Soon

Symptoms:

  • Older logs are missing before the expected retention period.
  • Time-travel queries do not retrieve past logs.

Causes:

  • Log retention period is set too low.
  • Manual or automated cleanup scripts are deleting logs.

Fix:

Increase log retention period:

ALTER SYSTEM SET AUDIT_LOG_RETENTION = '90d';

Disable automatic cleanup if enabled:

ALTER SYSTEM SET ENABLE_LOG_CLEANUP = FALSE;

Verify that no external scripts are deleting logs from cloud storage.


7. Logs Are Missing for External Table Access or API Calls

Symptoms:

  • API-based queries (e.g., REST API, JDBC) do not appear in logs.
  • External table access is not recorded.

Causes:

  • Logging for API and external tables is disabled.
  • External integrations do not support Unity Catalog logging.

Fix:

Enable logging for API calls and external queries:

ALTER SYSTEM SET API_ACCESS_LOGGING = TRUE;

Verify logs for API requests:

SELECT * FROM system.access WHERE action_name LIKE 'API%';

Step-by-Step Troubleshooting Guide

Step 1: Check If Audit Logging Is Enabled

SHOW CONFIGS;
  • If AUDIT_LOGGING is false, enable it.

Step 2: Verify Logs Exist in Unity Catalog

SELECT * FROM system.access ORDER BY event_time DESC;

Step 3: Ensure Cloud Logging Is Enabled for Your Platform

  • AWS → Check CloudTrail logs.
  • Azure → Check Log Analytics Workspace.
  • GCP → Check Cloud Logging.

Step 4: Verify User Permissions to Access Logs

GRANT SELECT ON TABLE system.access TO `user@example.com`;

Step 5: Increase Log Retention Period

ALTER SYSTEM SET AUDIT_LOG_RETENTION = '90d';

Best Practices for Managing Unity Catalog Audit Logs

Enable Cloud Logging for Long-Term Storage

  • Store audit logs in S3, ADLS, or GCS for extended retention.

Regularly Review Audit Logs for Security and Compliance

  • Set up automated monitoring and alerts for suspicious activity.

Restrict Log Access to Only Admins and Security Teams

  • Use RBAC controls to limit who can view audit logs.

Optimize Log Storage to Prevent Data Overload

  • Use log partitioning to efficiently query logs by date.

Conclusion

If user access logs are missing from Unity Catalog, check:
Audit logging is enabled in Databricks settings.
A proper log retention policy is set (e.g., 90 days).
Cloud logging (AWS, Azure, GCP) is correctly configured.
Users have permissions to access logs in Unity Catalog.
Table, API, and external logs are included in logging.

By following these steps, you can restore missing logs and ensure complete audit tracking in Databricks Unity Catalog.

guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x