1) Add users in Databricks

Best practice: sync from Microsoft Entra ID (SCIM). You can also add users manually.

UI (manual)

• Account Console → User management (People) → Add user (email) → assign to workspaces. (Microsoft Learn)

CLI (workspace-level)

# Requires CLI v0.205+ and auth
databricks users create --user-name "[email protected]" --display-name "Jane Doe"
databricks users list

This creates the user in the workspace (and adds to the account). (Databricks Documentation)

2) Create Service Principals in Databricks

You can create Databricks-managed service principals directly, or use Entra-managed ones (created in Azure). Databricks-managed SPs are simplest for Databricks automation. (Microsoft Learn)

UI

• Workspace: Settings → Identity & access → Service principals → Add. (Microsoft Learn)

CLI

# Create & list a Databricks-managed service principal
databricks service-principals create --display-name "spn-ci"
databricks service-principals list

(Databricks Documentation)

FYI: Service principals are API-only identities (no UI login). You grant permissions just like users, and you can run jobs “as” the SP. (Microsoft Learn)

3) Create a Service Principal in Azure (Microsoft Entra ID)

1. Entra admin center → App registrations → New registration.
2. Record Application (client) ID and Directory (tenant) ID.
3. Certificates & secrets → create a client secret (or use a certificate). (Microsoft Learn)

Then add that Entra service principal to Databricks (Account/Workspace) and grant roles/entitlements as needed. (Microsoft Learn)

4) Use a Service Principal in Databricks

Common ways to authenticate:

A) OAuth machine-to-machine (Databricks)

• Use the SP to obtain Databricks OAuth tokens (recommended for unattended access). (Microsoft Learn)

B) Microsoft Entra token (AAD) to call Databricks

• Get an Entra access token for the SP and use it with the Databricks REST API/CLI. (Microsoft Learn)

C) Personal Access Token (PAT) owned by the SP

• Create/manage tokens for the service principal and use them in automation. (Databricks Documentation)

After auth, assign the SP to workspaces and grant permissions (UC data, clusters, jobs, etc.). (Microsoft Learn)

5) Create & use Groups

Groups simplify permissioning (add users/SPs to groups, then grant the group access).

Prefer account/IdP-synced groups for Unity Catalog; workspace-local groups are legacy. (Databricks Documentation)

UI

• Account Console → Groups (recommended), or Workspace Settings → Identity & access → Groups. (Microsoft Learn)

CLI (workspace-level)

databricks groups create --display-name "data-engineers"
databricks groups list
# (Membership can be managed via SCIM or groups patch APIs.)

(Databricks Documentation)

6) What is SCIM in Databricks?

SCIM (System for Cross-domain Identity Management) lets your IdP (Microsoft Entra ID) auto-provision users & groups into your Databricks account. It keeps identities in sync and is the recommended approach. (Microsoft Learn)

7) Auto-provision users/groups from Microsoft Entra ID (SCIM)

High-level steps

1. In Databricks Account Console → Settings → User provisioning, click Set up and copy the Account SCIM URL and SCIM token. (Databricks Documentation)
2. In Entra → Enterprise applications, add the Databricks SCIM Provisioning Connector, set Provisioning mode = Automatic. Paste the Tenant URL (Databricks Account SCIM URL) and Secret token, Test connection, then Save. (Microsoft Learn)
3. Assign users/groups to the enterprise app, choose Sync only assigned users and groups, and Start provisioning. Initial sync starts immediately; subsequent syncs run roughly every 20–40 minutes. (You can monitor in Entra provisioning logs.) (Microsoft Learn)

Important notes

• Use account-level SCIM (recommended). If you previously synced directly to workspaces, disable those and migrate to account-level. (Microsoft Learn)
• Service principals are not synced by SCIM; manage them separately. (Microsoft Learn)

Tiny “starter” command set (CLI v0.205+)

# Add a user (workspace-level)
databricks users create --user-name "[email protected]" --display-name "Jane Doe"

# Create a Databricks-managed service principal
databricks service-principals create --display-name "spn-ci"

# Create a group
databricks groups create --display-name "data-engineers"

(Databricks Documentation)

Mohammad Gufran Jahangir