Azure Databricks manage compute

Posted by

View compute

To view your compute, click 

compute icon

Compute in the workspace sidebar.

Pin a compute

  • 30 days after a compute is terminated, it is permanently deleted.
  • Admins can pin a compute from the compute list or the compute detail page by clicking the pin icon.

Edit a compute

You can edit a compute’s configuration from the compute details UI.

  • Notebooks and jobs remain attached to the compute after editing.
  • Libraries installed on the compute stay in place after edits.
  • Editing attributes other than size and permissions requires restarting the compute, potentially disrupting current users.
  • Only running or terminated computes can be edited.
  • Permissions for computes in any state can be updated via the compute details page.

Clone a compute

To clone an existing compute, select Clone from the compute’s 

Kebab menu

 kebab menu.

The following attributes are NOT included in the clone:

  • Compute permissions
  • Attached notebooks

Compute permissions

There are four permission levels for a compute: NO PERMISSIONS, CAN ATTACH TO, CAN RESTART, and CAN MANAGE. 

Workspace admins have the CAN MANAGE permission on all compute in their workspace. Users automatically have the CAN MANAGE permission on the compute they create.

Attach notebook to computexxx
View Spark UIxxx
View compute metricsxxx
View driver logs
Terminate computexx
Start and restart computexx
Edit computex
Attach library to computex
Resize computex
Modify permissionsx

Terminate a compute

To save compute resources, you can terminate a compute.  When the number of terminated compute exceeds 150, the oldest compute is deleted.

Automatic termination

You can also set auto termination for a compute. During compute creation, you can specify an inactivity period in minutes after which you want the compute to terminate.

Configure automatic termination

You can configure automatic termination in the new compute UI.  Ensure that the box is checked, and enter the number of minutes in the Terminate after ___ of minutes of inactivity setting.

Unexpected termination

Sometimes a compute is terminated unexpectedly, not as a result of a manual termination or a configured automatic termination.

Delete a compute

Deleting a compute terminates the compute and removes its configuration. To delete a compute, select Delete from the compute’s 

Kebab menu


Restart a compute

You can restart a previously terminated compute from the compute list, the compute detail page, or a notebook. 

Restart a compute to update it with the latest images

When you restart a compute, it gets the latest images for the compute resource containers and the VM hosts. It is important to schedule regular restarts for long-running compute such as those used for processing streaming data.

Inline Feedbacks
View all comments
Would love your thoughts, please comment.x