{"action":"reopened","issue":{"url":"https://api.github.com/repos/Azure/azure-powershell/issues/7462","repository_url":"https://api.github.com/repos/Azure/azure-powershell","labels_url":"https://api.github.com/repos/Azure/azure-powershell/issues/7462/labels{/name}","comments_url":"https://api.github.com/repos/Azure/azure-powershell/issues/7462/comments","events_url":"https://api.github.com/repos/Azure/azure-powershell/issues/7462/events","html_url":"https://github.com/Azure/azure-powershell/issues/7462","id":366569459,"node_id":"MDU6SXNzdWUzNjY1Njk0NTk=","number":7462,"title":"MSI with ADLS gives invalid resource","user":{"login":"rahuldutta90","id":28450442,"node_id":"MDQ6VXNlcjI4NDUwNDQy","avatar_url":"https://avatars2.githubusercontent.com/u/28450442?v=4","gravatar_id":"","url":"https://api.github.com/users/rahuldutta90","html_url":"https://github.com/rahuldutta90","followers_url":"https://api.github.com/users/rahuldutta90/followers","following_url":"https://api.github.com/users/rahuldutta90/following{/other_user}","gists_url":"https://api.github.com/users/rahuldutta90/gists{/gist_id}","starred_url":"https://api.github.com/users/rahuldutta90/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rahuldutta90/subscriptions","organizations_url":"https://api.github.com/users/rahuldutta90/orgs","repos_url":"https://api.github.com/users/rahuldutta90/repos","events_url":"https://api.github.com/users/rahuldutta90/events{/privacy}","received_events_url":"https://api.github.com/users/rahuldutta90/received_events","type":"User","site_admin":false},"labels":[{"id":187436091,"node_id":"MDU6TGFiZWwxODc0MzYwOTE=","url":"https://api.github.com/repos/Azure/azure-powershell/labels/Azure%20PS%20Team%20:snail:","name":"Azure PS Team :snail:","color":"DDDDDD","default":false}],"state":"open","locked":false,"assignee":{"login":"cormacpayne","id":4462409,"node_id":"MDQ6VXNlcjQ0NjI0MDk=","avatar_url":"https://avatars2.githubusercontent.com/u/4462409?v=4","gravatar_id":"","url":"https://api.github.com/users/cormacpayne","html_url":"https://github.com/cormacpayne","followers_url":"https://api.github.com/users/cormacpayne/followers","following_url":"https://api.github.com/users/cormacpayne/following{/other_user}","gists_url":"https://api.github.com/users/cormacpayne/gists{/gist_id}","starred_url":"https://api.github.com/users/cormacpayne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cormacpayne/subscriptions","organizations_url":"https://api.github.com/users/cormacpayne/orgs","repos_url":"https://api.github.com/users/cormacpayne/repos","events_url":"https://api.github.com/users/cormacpayne/events{/privacy}","received_events_url":"https://api.github.com/users/cormacpayne/received_events","type":"User","site_admin":false},"assignees":[{"login":"cormacpayne","id":4462409,"node_id":"MDQ6VXNlcjQ0NjI0MDk=","avatar_url":"https://avatars2.githubusercontent.com/u/4462409?v=4","gravatar_id":"","url":"https://api.github.com/users/cormacpayne","html_url":"https://github.com/cormacpayne","followers_url":"https://api.github.com/users/cormacpayne/followers","following_url":"https://api.github.com/users/cormacpayne/following{/other_user}","gists_url":"https://api.github.com/users/cormacpayne/gists{/gist_id}","starred_url":"https://api.github.com/users/cormacpayne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cormacpayne/subscriptions","organizations_url":"https://api.github.com/users/cormacpayne/orgs","repos_url":"https://api.github.com/users/cormacpayne/repos","events_url":"https://api.github.com/users/cormacpayne/events{/privacy}","received_events_url":"https://api.github.com/users/cormacpayne/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/Azure/azure-powershell/milestones/58","html_url":"https://github.com/Azure/azure-powershell/milestone/58","labels_url":"https://api.github.com/repos/Azure/azure-powershell/milestones/58/labels","id":3683685,"node_id":"MDk6TWlsZXN0b25lMzY4MzY4NQ==","number":58,"title":"2018-11-06","description":"Reviews Complete: 10/22/2018; Code Complete: 10/29/2018; Release to CloudShell: 11/02/2018; Release to Customers: 11/06/2018","creator":{"login":"markcowl","id":1054056,"node_id":"MDQ6VXNlcjEwNTQwNTY=","avatar_url":"https://avatars1.githubusercontent.com/u/1054056?v=4","gravatar_id":"","url":"https://api.github.com/users/markcowl","html_url":"https://github.com/markcowl","followers_url":"https://api.github.com/users/markcowl/followers","following_url":"https://api.github.com/users/markcowl/following{/other_user}","gists_url":"https://api.github.com/users/markcowl/gists{/gist_id}","starred_url":"https://api.github.com/users/markcowl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/markcowl/subscriptions","organizations_url":"https://api.github.com/users/markcowl/orgs","repos_url":"https://api.github.com/users/markcowl/repos","events_url":"https://api.github.com/users/markcowl/events{/privacy}","received_events_url":"https://api.github.com/users/markcowl/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":18,"state":"closed","created_at":"2018-09-24T16:35:58Z","updated_at":"2018-11-27T19:37:50Z","due_on":"2018-10-29T07:00:00Z","closed_at":"2018-11-02T22:19:00Z"},"comments":2,"created_at":"2018-10-03T23:37:58Z","updated_at":"2018-11-27T19:37:50Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"<!--\r\n\r\nIf this issue is a bug report:\r\n- Upgrade to the latest version of AzureRM and verify you are able to reproduce the issue\r\n    - You can install the latest version of AzureRM from the PowerShell Gallery\r\n        - https://www.powershellgallery.com/packages/AzureRM\r\n    - You can also install the latest version from the Releases section\r\n        - https://github.com/Azure/azure-powershell/releases\r\n- Ensure that you repro the issue with $DebugPreference = \"Continue\" to receive the debug stream\r\n- If this bug involves an exception being thrown, please run Resolve-AzureRmError to receive extended information on the error\r\n- Fill out the below template\r\n\r\nIf this issue is not a bug report, please remove the below template\r\n\r\n-->\r\n\r\n### Description\r\n\r\nUsi MSI with ADLS gives an error with an invalid resource\r\n\r\n### Script/Steps for Reproduction\r\n\r\n<!-- Please provide the necessary script(s) that reproduce the issue -->\r\n1.\tGo to https://rc.portal.azure.com.\r\n2.\tClick the top right powershell-looking icon to launch Cloud Shell:\r\no\t \r\n3.\tIf you see a Bash window open at the bottom, toggle from Bash to PowerShell.\r\n4.\tTry running the following cmdlet: \r\no\t$files = Get-AdlStoreChildItem -Account sandboxadl -Path / -Debug\r\n5.\tObserve the error output\r\n\r\n```powershell\r\n\r\n```\r\n\r\n### Module Version\r\nCurrent version\r\n<!-- Please run (Get-Module -ListAvailable) to get the version(s) of all modules, including Azure installed on your machine -->\r\n\r\n```powershell\r\nGet-Module -ListAvailable\r\n```\r\n\r\n### Environment Data\r\n\r\n<!-- Please run $PSVersionTable to get the necessary environment data -->\r\n\r\n```powershell\r\n$PSVersionTable\r\n```\r\n\r\n### Debug Output\r\n\r\n```\r\nPS Azure:\\> $jobs = Get-AdlCatalogItem -Account sandbox -ItemType Database -Debug\r\nDEBUG: 8:43:29 PM - GetAzureDataLakeAnalyticsCatalogItem begin processing with ParameterSet '__AllParameterSets'.\r\n \r\nConfirm\r\nContinue with this operation?\r\n[Y] Yes  [A] Yes to All  [H] Halt Command  [S] Suspend  [?] Help (default is \"Y\"): A\r\nDEBUG: 8:43:32 PM - using account id 'MSI@50342'...\r\nDEBUG: [Common.Authentication]: Authenticating using Account: 'MSI@50342', environment: 'AzureCloud', tenant: '72f988bf-86f1-41af-91ab-2d7cd011db47'\r\nDEBUG: [HttpClientOperations]: Adding Header 'Metadata'\r\nDEBUG: [Common.Authentication]: Authenticating using Account: 'MSI@50342', environment: 'AzureCloud', tenant: '72f988bf-86f1-41af-91ab-2d7cd011db47'\r\nDEBUG: [HttpClientOperations]: Adding Header 'Metadata'\r\nDEBUG: [Common.Authentication]: Authenticating using Account: 'MSI@50342', environment: 'AzureCloud', tenant: '72f988bf-86f1-41af-91ab-2d7cd011db47'\r\nDEBUG: [HttpClientOperations]: Adding Header 'Metadata'\r\nDEBUG: Checking Cache request http://localhost:50342/oauth2/token?resource=https%3A%2F%2Fdatalake.azure.net&api-version=2018-02-01\r\nDEBUG: ============================ HTTP REQUEST ============================\r\n \r\nHTTP Method:\r\nGET\r\n \r\nAbsolute Uri:\r\nhttp://localhost:50342/oauth2/token?resource=https%3A%2F%2Fdatalake.azure.net&api-version=2018-02-01\r\n \r\nHeaders:\r\nMetadata                      : true\r\n \r\nBody:\r\n \r\n \r\n \r\nGet-AdlCatalogItem : The operation was canceled.\r\nAt line:1 char:9\r\n+ $jobs = Get-AdlCatalogItem -Account sandbox -ItemType Database -Debug\r\n+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n+ CategoryInfo          : CloseError: (:) [Get-AzDataLakeAnalyticsCatalogItem], TaskCanceledException\r\n+ FullyQualifiedErrorId : Microsoft.Azure.Commands.DataLakeAnalytics.GetAzureDataLakeAnalyticsCatalogItem\r\n```\r\n\r\n<!-- Please run the above script with $DebugPreference='Continue' and paste the resulting debug stream in the below code block -->\r\n\r\n### Investigation as copied from email\r\nThe ADLS/ADLA commandlet calls AuthenticationFactory (https://github.com/Azure/azure-powershell-common/blob/75e99f8571291238893d16eed3b30ce17f1c4788/src/Authentication/Factories/AuthenticationFactory.cs#L275 ) to get the ServiceClientCredential. \r\n\r\nBased on the targetendpoint which is “AzureEnvironment.Endpoint.AzureDataLakeStoreFileSystemEndpointSuffix” in case of ADLS or “AzureEnvironment.Endpoint.AzureDataLakeAnalyticsCatalogAndJobEndpointSuffix” in case of ADLA the token audience returned is of endpoint “AzureEnvironment.Endpoint.DataLakeEndpointResourceId” (https://github.com/Azure/azure-powershell-common/blob/75e99f8571291238893d16eed3b30ce17f1c4788/src/Authentication.Abstractions/Extensions/AzureEnvironmentExtensions.cs#L288 ).\r\n\r\nBased on the screenshot Anders shared. Value of “DataLakeEndpointResourceId” is https://datalake.azure.net without the trailing slash.  \r\n\r\nBase on the settings of AzureCloud, this is set to “\"https://datalake.azure.net\"” in AzureEnvironment (https://github.com/Azure/azure-powershell-common/blob/75e99f8571291238893d16eed3b30ce17f1c4788/src/Authentication.Abstractions/AzureEnvironment.cs#L50 )\r\n\r\nSo whatever Anders is seeing should be expected. But the problem is not there. Because in my powershell session, when I login using serviceprincipal I see the resource sent it has a trailing “/” added but for MSI that is not the case. \r\n\r\nNot sure but I think the reason is here:\r\n\r\nFor the serviceprincipal we get the resource as Uri which adds the trailing “/” to the resource (https://github.com/Azure/azure-powershell-common/blob/75e99f8571291238893d16eed3b30ce17f1c4788/src/Authentication/Factories/AuthenticationFactory.cs#L321 )\r\n\r\nFor MSI though we send the resource as a string here: https://github.com/Azure/azure-powershell-common/blob/75e99f8571291238893d16eed3b30ce17f1c4788/src/Authentication/Factories/AuthenticationFactory.cs#L337 thus no trailing “/” is added. Probable solution is to use Uri to get the normalized resource. Not sure why this was not hit before.\r\n\r\nAlso I think we should change wherever we are using the constant “http://datalake.azure.net” to “http://datalake.azure.net/” so that we do not hit this issue again. I think there are 3 places, AzureEnvironmentConstants.cs, AddAzureEnvironment.cs and SetAzureRmEnvironment.cs. \r\n\r\nAlso in SetAzureRMEnvironment.cs (https://github.com/Azure/azure-powershell/blob/edd6562ebfd16db5776320d0d0fb96303fa7b848/src/ResourceManager/Profile/Commands.Profile/Environment/SetAzureRMEnvironment.cs#L333 ) it would be better if we add trailing “/” at SetEndPointIfBound if the given endpoint does not have trailing “/”. This will prevent users passing wrong resourceid urls.\r\n\r\n"}}
{"actor":{"display_login":"cormacpayne"}}