Error when Connecting ADF(Azure data factory) pipeline to ADLS( Azure Data lake storage) for Data Copy – service principal/managed identity don’t have enough permission to access the data.

Posted by

Error

Operation on target iteratetables failed: Activity failed because an inner activity failed; Inner activity name: Copytabledata, Error: ErrorCode=AdlsGen2ForbiddenError,’Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ADLS Gen2 failed for forbidden: Storage operation ‘CreateFile’ on container ‘dese’ and path ‘Enterprise /xxx/cc/vv/ff/ff/hh/part-00000-065de8c7-0637-4e6e-ae15-6894d8fbbc40-c000.snappy.parquet’ get failed with ‘Operation returned an invalid status code ‘Forbidden”. Possible root causes: (1). It’s possible because the service principal or managed identity don’t have enough permission to access the data. (2). Please check storage network setting whether public network access is disabled. If disabled, use Managed Virtual Network IR and create Private Endpoint to access.

Details Description about Error

When trying to do data copy within ADLS (Azure data lake storage) using ADF (Azure pipeline ) than getting Error that the service principal or managed identity don’t have enough permission to access the data.

Solution

This is due to not proper access granted to ADF where pipeline is created to do data copy on ADLS

For Examples : ADF is EDM-LK-adf1-d having pipeline Datacopy_ADLS which is configured to run data copy for ADLS source ADLS name –eedmanhiadls1p to Sink ADLS name – eedmanhiadls1d

Please grant access to ADF on ADLS

Please go to storage account > IAM > Add role assignment, and add the special permissions for ADF

To source ADLS

Go to source ADLS – Access control (IAM), grant at least the Storage Blob Data Reader (Read permission for the files to copy) role to ADF

To Sink ADLS

Go to sink ADLS – Access control (IAM), grant at least the Storage Blob Data Contributor (Write permission for the sink folder) role to ADF

guest
0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x