Lakehouse Federation (Query Federation) in Databricks — What, Why, How
Lakehouse Federation lets you query external databases directly from Databricks—without copying the data into your lake. You point Unity Catalog at a source (PostgreSQL, SQL Server, Redshift,…
What is a streaming table in Databricks?
What is a streaming table in Databricks? A streaming table is a Delta table that Databricks keeps up-to-date automatically as new data arrives. It uses Structured Streaming…
Materialized view vs. (logical) view
Topic View Materialized View What it is A stored query; results are computed each time you query it A precomputed table of the query’s results, stored on…
Data Warehousing on Databricks with Databricks SQL & SQL Warehouses
What is Databricks SQL (DBSQL)? Databricks SQL is the lakehouse analytics experience for running SQL, building dashboards, alerts, and jobs on Delta tables. You connect to a…
Databricks Serverless Compute for Notebooks, Jobs, DLT, ML & Warehouses — Architecture, Setup, and Tips
What “serverless” means on Databricks Serverless lets you run Databricks workloads without creating or sizing clusters. Databricks provisions, scales, patches, and retires the compute for you, so…
Delta Sharing in Databricks — A Practical How-To (Provider & Recipient)
Delta Sharing lets you publish live, governed data to other teams or external partners—without copying files or building custom APIs. Recipients can query the latest data directly…
Delta Sharing in Databricks — What, Why, and How (Provider & Recipient)
TL;DR Delta Sharing lets you share live, governed data from Unity Catalog with other teams or external partners without copying it. You create a share, add data…
Workspace‑Catalog Binding in Unity Catalog (Limit Catalog Access by Workspace)
Why workspace‑catalog binding? By default, any workspace attached to the same Unity Catalog metastore can see and access catalogs (subject to object‑level grants). Workspace‑catalog binding lets you…
Column Level Masking in Unity Catalog — Simple, Practical Guide
What is column‑level masking? Column masks hide or transform sensitive values (PII/PCI/PHI) at query time. Every read of a masked column is replaced by the output of…
Row‑Level Filters & Column Masks in Unity Catalog
What you’ll learn Prereqs: Unity Catalog enabled; DBR 12.2 LTS+ for reads (see notes for dedicated compute), and a SQL warehouse or UC‑enabled cluster. Concepts in 30…
Functions in Unity Catalog with Databricks SQL (Scalar & Table UDFs)
What you’ll learn Prereqs: Unity Catalog enabled, a catalog/schema you can write to, and a SQL Warehouse or notebook attached to a UC‑enabled cluster. Quick definitions Create…
Databricks Secret Management & Secret Scopes — the Practical Guide
Secrets (passwords, keys, tokens) should never live in notebooks or job configs. Databricks gives you a built-in, governed place to put them—secret scopes—and safe ways to read…
Role Guide for Different Users in Databrick
Quick role guide (what each should do) Persona Typical actions Where they work UC privileges (high level) Workspace entitlements Admin / Data Steward Govern data, manage grants,…
Unity Catalog (how security works)
Unity Catalog (how security works) How to secure tables (and “hide” them) (You can also grant at catalog level to cascade.) (Databricks Documentation) What permissions exist? (most-used,…
Guide to Databricks Identity basics
1) Add users in Databricks Best practice: sync from Microsoft Entra ID (SCIM). You can also add users manually. UI (manual) CLI (workspace-level) This creates the user…
What is Databricks Secret Management?
What is Databricks Secret Management? A built-in way to store credentials (keys, passwords, tokens) safely and read them from notebooks/jobs without printing them. Values you fetch with…
What Auto Loader does (one line)
What Auto Loader does (one line) Watches a folder for new files and appends them exactly once to a Delta table, even if the job restarts. Minimal…
Delta Tables — Simple Guide (with easy examples)
This is the short, practical version of the big guide. Three tools you’ll use again and again: We’ll use one tiny table and show copy‑paste SQL for…
Mastering Delta Performance — Liquid Clustering, Deletion Vectors, and OPTIMIZE
This guide distills how to get the most out of Databricks Delta tables using three modern primitives: Use these together to build tables that are fast to…
Auto Loader in Databricks | Schema Evolution Modes | File Detection Modes (Deep‑Dive)
Databricks Auto Loader is the cloudFiles source for Structured Streaming that incrementally discovers and ingests new files from ADLS/S3/GCS into Delta Lake—with built‑in deduplication, schema management, and…