What Is a Service Principal in Databricks?

Posted by


✅ What Is a Service Principal in Databricks?

A service principal in Databricks represents a non-human identity — like an application, automation tool, or CI/CD pipeline — used to securely access Databricks resources without using a personal user account.


🔧 Common Use Cases with Examples

Use CaseExampleWhy Use a Service Principal?
🔁 Automated JobsA data pipeline in Azure Data Factory (ADF) triggering a Databricks jobTo allow ADF to authenticate to Databricks securely without hardcoding credentials
🚀 CI/CD DeploymentsGitHub Actions / Azure DevOps pushing notebooks, jobs, clusters using REST APIsTo allow CI/CD tools to deploy infrastructure to Databricks securely and reproducibly
📦 Data IngestionA Kafka consumer pushing real-time data into Delta LakeSo the ingestion tool (not a person) can write to Databricks securely
📊 Power BI / Reporting ToolsPower BI connecting to a SQL Warehouse using OAuth via a service principalFor secure and auditable access to query data without personal credentials
🔐 Unity Catalog PermissionsAssigning roles and access to tables, volumes, schemasSo that tools like Airflow, ADF, or APIs can read/write data under controlled permissions
🛡️ Auditing & SecurityKeeping audit logs clean by separating human vs automation identitiesImproves security visibility and compliance tracking


✅ How to Create and Use One (Summary)

  1. Register an app in Azure Active Directory.
  2. Create a secret or certificate.
  3. Assign that app as a service principal in Databricks via Admin Console.
  4. Grant it permissions on:
    • Clusters
    • Jobs
    • Catalog objects (via Unity Catalog)
  5. Use the token/secret to authenticate in automation (e.g., via REST API).

🔐 Bonus: Sample Python Usage with Token

import requests

token = "Bearer <sp-access-token>"
workspace_url = "https://<your-workspace>.azuredatabricks.net"

response = requests.get(
    f"{workspace_url}/api/2.0/clusters/list",
    headers={"Authorization": token}
)

print(response.json())

Leave a Reply

Your email address will not be published. Required fields are marked *

0
Would love your thoughts, please comment.x
()
x