What are Shortcuts?

• What are Shortcuts:
  • Shortcuts are like pointers in OneLake that direct you to other storage places.
  • They can point to places inside or outside of OneLake.
• Shortcut Paths and Target Paths:
  • The place where a shortcut points to is called the target path.
  • The place where the shortcut itself is located is called the shortcut path.
• Appearance and Use:
  • Shortcuts look like folders in OneLake, and any tool or service that can access OneLake can use them.
• Behavior:
  • Shortcuts act like symbolic links, which means they’re separate from the target they point to.
  • Deleting a shortcut doesn’t affect the target.
  • If you move, rename, or delete the target, the shortcut might stop working.

Where all we can create shortcuts?

• Creating Shortcuts:
  • You can make shortcuts in both lakehouses and Kusto Query Language (KQL) databases.
  • These shortcuts can link to various destinations like other OneLake locations, Azure Data Lake Storage (ADLS) Gen2, Amazon S3 storage accounts, or Dataverse.
• Two Methods:
  • Interactive Creation:
    • Use the Fabric UI to create shortcuts easily by interacting with the user interface.
  • Programmatic Creation:
    • Alternatively, you can use the REST API to programmatically generate shortcuts.

Creation of Shortcuts in a Lakehouse

• Lakehouses have two main folders: Tables and Files.
• The Tables folder is for managed data, and the Files folder is for unmanaged data.
• Shortcuts can only be created at the top level of the Tables folder.
• If the shortcut points to data in Delta\Parquet format, the lakehouse automatically recognizes it as a table.
• In the Files folder, you can create shortcuts at any level, but table discovery doesn’t happen there.

Creation of Shortcuts in a KQL database

• Shortcut Creation in KQL Database:
  • When you make a shortcut in a KQL database, it shows up in the Shortcuts folder of the database.
  • KQL treats shortcuts similarly to external tables.
• Querying Shortcuts:
  • To retrieve data from the shortcut, you utilize the external_table function of the Kusto Query Language.

From Where can I access shortcuts?

Any Fabric or non-Fabric service that can access data in OneLake can use shortcuts. Shortcuts just appear as another folder in the lake. Spark, SQL, Real-Time Analytics, and Analysis Services can all use shortcuts when querying data.

• Spark
• SQL
• Real-Time Analytics
• Analysis Services
• Non-Fabric

Spark

Spark notebooks and Spark jobs can use shortcuts that you create in OneLake. Relative file paths can be used to directly read data from shortcuts. Additionally, if you create a shortcut in the Tables section of the lakehouse and it is in the Delta format, you can read it as a managed table using Spark SQL syntax.

df = spark.read.format("delta").load("Tables/MyShortcut")
display(df)
df = spark.sql("SELECT * FROM MyLakehouse.MyShortcut LIMIT 1000")
display(df)

Note :The Delta format doesn’t support tables with space characters in the name.

SQL

You can also read shortcuts in the Tables section of a lakehouse through the SQL analytics endpoint for the lakehouse. You can access the SQL analytics endpoint through the mode selector of the lakehouse or through SQL Server Management Studio (SSMS).

SELECT TOP (100) *
FROM [MyLakehouse].[dbo].[MyShortcut]

Real-Time Analytics

Shortcuts in KQL databases are recognized as external tables. To query the shortcut, use the external_table function of the Kusto Query Language.

external_table('MyShortcut')
| take 100

Analysis Services

You can create semantic models for lakehouses containing shortcuts in the Tables section of the lakehouse. When the semantic model runs in Direct Lake mode, Analysis Services can read data directly from the shortcut.

Non-Fabric

Applications and services outside of Fabric can also access shortcuts through the OneLake API.OneLake supports a subset of the ADLS Gen2 and Blob storage APIs.

https://onelake.dfs.fabric.microsoft.com/MyWorkspace/MyLakhouse/Tables/MyShortcut/MyFile.csv

Types of shortcuts

OneLake shortcuts support multiple filesystem data sources. These include internal OneLake locations, Azure Data Lake Storage (ADLS) Gen2, Amazon S3, and Dataverse.

Internal OneLake shortcuts

• Internal OneLake Shortcuts:
  • Allow you to reference data within different Fabric items like lakehouses, KQL databases, and data warehouses.
  • Shortcuts can point to data in the same item, across items in the same workspace, or even across items in different workspaces.
• Cross-Item Shortcuts:
  • You can create shortcuts between different types of items, like pointing from a lakehouse to data in a data warehouse.
• Access Authorization:
  • When a user accesses data through a shortcut, their identity is used to check if they have permission to read the data in the shortcut’s target location.
  • The user must have the necessary permissions in the target location to access the data.

ADLS shortcuts

• Creating Shortcuts to ADLS shortcuts
  • Shortcuts can also be created to ADLS Gen2 storage accounts.
  • When you create shortcuts to ADLS, the target path can point to any folder within the hierarchical namespace.
  • At a minimum, the target path must include a container name.
• Access
  • ADLS shortcuts must point to the DFS endpoint for the storage account. Example: https://accountname.dfs.core.windows.net/

• Authorization
  • Access to storage account endpoint can’t be blocked by storage firewall or VNET because storage firewalls are currently not supported.
  • The supported delegated types are Organizational account, Account Key, Shared Access Signature (SAS), and Service Principal.
    • Organizational account – must have Storage Blob Data Reader, Storage Blob Data Contributor, or Storage Blob Data Owner role on storage account
    • Shared Access Signature (SAS) – must include at least the following permissions: Read, List, and Execute
    • Service Principal – must have Storage Blob Data Reader, Storage Blob Data Contributor, or Storage Blob Data Owner role on storage account

S3 shortcuts

• Creating Shortcuts to Amazon S3:
  • You can make shortcuts to Amazon S3 accounts.
  • The shortcut’s target path needs to have at least a bucket name.
  • S3 doesn’t directly support folders, but you can use prefixes in the path to act like folders.
• Using Prefixes in Shortcut Path:
  • Adding prefixes in the shortcut path helps limit the data accessible through the shortcut.
  • Prefixes function like folders when you access data through the S3 shortcut.
• Access
  • S3 shortcuts must point to the https endpoint for the S3 bucket.
  • Example: https://bucketname.s3.region.amazonaws.com/
• Authorization
  • The supported delegated credential is a Key and Secret for an IAM user.
  • The IAM user must have the following permissions on the bucket that the shortcut is pointing to.
    • S3:GetObject
    • S3:GetBucketLocation
    • S3:ListBucket

Dataverse shortcuts

• Dataverse Direct Integration:
  • Dataverse directly connects with Microsoft Fabric, allowing organizations to expand their Dynamics 365 applications and workflows into Fabric.
• Integration Method:
  • Shortcuts facilitate this integration and can be created in two ways:
    • PowerApps Maker Portal: Users can create shortcuts through the PowerApps maker portal.
    • Fabric Directly: Shortcuts can also be generated directly within Fabric.

Some Limitations and considerations

• The maximum number of shortcuts per Fabric item is 100,000. In this context, the term item refers to: apps, lakehouses, warehouses, reports, and more.
• The maximum number of shortcuts in a single OneLake path is 10.
• The maximum number of direct shortcuts to shortcut links is 5.
• OneLake shortcut names, parent paths, and target paths can’t contain “%” or “+” characters.
• Shortcuts don’t support non-Latin characters.
• Copy Blob API not supported for ADLS or S3 shortcuts.
• Copy function doesn’t work on shortcuts that directly point to ADLS containers. It’s recommended to create ADLS shortcuts to a directory that is at least one level below a container.
• OneLake shortcuts pointing to ADLS or S3 shortcuts isn’t supported.
• Additional shortcuts can’t be created inside ADLS or S3 shortcuts.
• Lineage for shortcuts to Data Warehouses and Semantic Models is not currently available.

How to Create a OneLake shortcut?

If you don’t have a lakehouse, create one by following these steps:

Create a shortcut

1. Open a lakehouse.
2. Right-click on a directory within the Explorer pane of the lakehouse.
3. Select New shortcut.

Select a source

Under Internal sources, select Microsoft OneLake.

Select the data source you want to connect to, and then select Next.

Expand Files or Tables, and select one or more subfolders to connect to, then select Next.

Review your selected shortcut locations. Use the edit action to change the default shortcut name. Use the delete action to remove any undesired selections. Select Create to generate shortcuts.

Note :You can select up to 50 subfolders when creating OneLake shortcuts.

The lakehouse automatically refreshes. The shortcut(s) appears under the selected directory in the Explorer pane.

How to create an Azure Data Lake Storage Gen2 shortcut

Create a shortcut

1. Open a lakehouse.
2. Right-click on a directory within the Lake view of the lakehouse.
3. Select New shortcut.

Select a source

Under External sources, select Azure Data Lake Storage Gen2.

Enter the Connection settings according to the following table:

Browse to the target location for the shortcut.