Here’s a simple line-by-line summary of the important points from the Serverless Compute release notes (09/24/2025):
Perfect 👍 Here’s a one-page cheat sheet table for the Serverless Compute release notes (09/24/2025):
Azure Databricks Serverless Compute – Release Notes Cheat Sheet
| Version | Date | Key New Features | Key Behavior Changes |
|---|---|---|---|
| 17.2 | Sept 25, 2025 | ST_ExteriorRing (polygon boundary); TEMPORARY metric views; Faster LokiFileSystem on S3; Auto Loader auto-detects partitions | DESCRIBE CONNECTION shows JDBC env; Option to truncate history in Uniform migration; split() fix; Better url_decode handling; Shared env for Python UDTFs |
| 17.1 | Aug 19, 2025 | Photon uses less memory on wide schemas | Invalid CHECK → error; Pulsar hides Bouncy Castle; Auto Loader uses file events by default; Teradata connector = case-sensitive |
| Env v4 | Aug 13, 2025 | Library & API upgrades | – |
| 17.0 | Jul 24, 2025 | SQL Procedures; Default collation for SQL funcs; Recursive CTEs; df.mergeInto API; SHOW SCHEMAS ALL CATALOGS; Better liquid clustering; MERGE allows random values | Auto Loader incremental listing deprecated; CREATE VIEW stricter errors |
| Perf Targets GA | Jun 10, 2025 | Choose Performance optimized (fast) or Standard (cheaper) | – |
| 16.4 | May 28, 2025 | Auto Loader clean processed files; Type widening (Preview); IDENTIFIER in catalog ops; listagg/string_agg; Python filter pushdown; Better UDF tracebacks | Cached reads respect options; MERGE must materialize source (if flag enabled) |
| Perf Mode Config | Apr 14, 2025 | Jobs can toggle Performance optimized vs Standard mode | – |
| 16.3 | Apr 9, 2025 | State reader GA; Delta downgrade GA; SQL scripting (Preview); Table/view default collation; H3 funcs; Multi-column ALTER TABLE | Kafka errors clearer |
| 16.2 | Mar 13, 2025 | timestampdiff/timestampadd in generated columns; DESCRIBE TABLE AS JSON; Blank-insensitive collations | Delta Sharing WITH HISTORY default; Credential mismatch → error |
| High Memory | Feb 7, 2025 | High-memory notebooks (Preview) | – |
| 16.1 | Feb 5, 2025 | Recursive Avro; Confluent Schema Registry; OPTIMIZE FULL; Delta identity cols; New SQL funcs (try_url_decode, randstr, etc.); VACUUM LITE; Built-in Oracle JDBC | RStudio removed; Type widening restricted; Regex parsing fixed |
| 15.4 | Oct 28, 2024 | UTF-8 funcs; Enable Iceberg on existing tables; try_url_decode; RELY on FKs; Faster COPY INTO | Views tolerate schema change; Removed ! (NOT) syntax outside boolean; Consistent Base64 errors |
| JDK 17 | Aug 15, 2024 | Upgrade from JDK 8 → JDK 17 | Regex parsing fixed |
| 15.1 | Jul 23, 2024 | Star (*) syntax in WHERE; Better JSON error recovery | – |
| 14.3 (Initial) | Apr 15, 2024 | First serverless release; input_file funcs deprecated | Spark config limits; UDF execution improved; CHAR fields padded |
General Info
- Serverless compute is automatically updated by Azure Databricks.
- Updates bring performance, security, and bug fixes without user code changes.
- Each environment version fixes Python versions and package sets for stability.
Version 17.2 (Sept 25, 2025)
New features
- New SQL function ST_ExteriorRing (get polygon outer boundary).
- Metric views can be TEMPORARY (exist only in the current session).
- LokiFileSystem on S3 now uses faster native I/O.
- Auto Loader can auto-detect partition columns in
singleVariantColumnmode.
Behavior changes
DESCRIBE CONNECTIONnow shows JDBC environment settings.- Option to truncate uniform history when migrating managed tables.
split()function fixed for empty regex + positive limit.url_decodeandtry_url_decodefixed in Photon (return NULL on invalid input).- Unity Catalog Python UDTFs share execution environment (STRICT ISOLATION optional).
Version 17.1 (Aug 19, 2025)
New features
- Photon engine uses less memory for wide schemas.
Behavior changes
- Invalid CHECK constraints now throw errors.
- Pulsar connector no longer exposes Bouncy Castle (install manually if needed).
- Auto Loader now defaults to using file events when available (faster).
- Teradata connector now defaults to case-sensitive string comparison.
Serverless Environment Version 4 (Aug 13, 2025)
- Includes library upgrades and API updates.
Version 17.0 (July 24, 2025)
New features
- SQL procedures supported (CREATE PROCEDURE / CALL).
- Can set a default collation for SQL functions.
- Recursive CTEs (rCTEs) supported.
- PySpark and Spark Connect support df.mergeInto API.
SHOW SCHEMAScan list schemas across ALL CATALOGS.- Liquid clustering optimizes deletion vectors better.
- MERGE now allows non-deterministic expressions (e.g.,
rand()). - Python MERGE APIs return DataFrames instead of Unit.
Behavior changes
- Deprecated Auto Loader option
cloudFiles.useIncrementalListingis now always false. CREATE VIEWwith materialized-view-only clauses now throws errors.
Serverless Performance Targets GA (June 10, 2025)
- Performance mode is GA:
- Performance optimized → faster.
- Standard → cheaper, slower start.
Version 16.4 (May 28, 2025)
New features
- Auto Loader can clean processed files (move/delete).
- Type widening supported in streaming and Delta Sharing (preview).
- IDENTIFIER supported for catalog operations.
- listagg / string_agg functions added.
- Filter pushdown support for Python data sources.
- Better Python UDF traceback messages.
Behavior changes
- Cached table reads respect options (not just first query).
- Source materialization in MERGE is now required (if flag enabled).
Performance Mode Configurable (Apr 14, 2025)
- Jobs can run in Performance optimized (fast) or Standard mode (cost-saver).
Version 16.3 (Apr 9, 2025)
New features
- State reader for Structured Streaming is GA.
- Delta protocol downgrade (DROP FEATURE) is GA.
- SQL scripting (procedural logic) in Public Preview.
- Default collation for tables and views.
- New H3 functions.
- Can alter multiple columns in one ALTER TABLE.
Behavior changes
- Better Kafka error messages.
Version 16.2 (Mar 13, 2025)
New features
- timestampdiff / timestampadd in Delta Lake generated columns.
DESCRIBE TABLE AS JSONreturns structured metadata.- Blank-insensitive collations supported.
Behavior changes
- Delta Sharing history enabled by default.
- Credential mismatch now returns errors.
Bug fixes
- Fix for incremental CLONE re-copying files.
High Memory Setting (Feb 7, 2025)
- Serverless notebooks can now request higher memory (Public Preview).
Version 16.1 (Feb 5, 2025)
New features (highlights only)
- Recursive Avro schema support.
- Confluent Schema Registry support.
- OPTIMIZE FULL for liquid clustering.
- Identity columns in Delta via Python APIs.
- New SQL functions: try_url_decode, zeroifnull, nullifzero, dayname, uniform, randstr.
- Automatic schema evolution in MERGE.
- Collations in Spark + Delta (Preview).
- VACUUM LITE mode.
- Built-in Oracle JDBC driver.
Behavior changes
- Hosted RStudio removed.
- No longer allowed: type widening for integer → decimal/double.
Version 15.4 (Oct 28, 2024)
New features
- UTF-8 validation functions.
- Enable Iceberg on existing tables.
- try_url_decode function.
- RELY keyword for foreign keys.
- Parallelized selective overwrites.
- Faster COPY INTO queries.
Behavior changes
- Views now tolerate schema changes.
- Removed support for undocumented
!(NOT) syntax outside boolean. - More consistent Base64 error handling.
JDK Upgrade (Aug 15, 2024)
- Serverless moved from JDK 8 → JDK 17.
- Regex parsing corrected.
Version 15.1 (July 23, 2024)
- Star (*) syntax allowed in WHERE clause.
- JSON parser recovers better from malformed JSON.
Version 14.3 (Apr 15, 2024 – first release)
- Initial serverless release.
- Removed support for many Spark configs.
- input_file functions deprecated.
- Various fixes (e.g., unhex, auto-aliases, padding CHAR fields).
- PySpark UDF execution now matches dedicated compute.
