title: Azure Data Factory connection to Databricks doesn't work when using Key Vault to retrieve token
body: <p>I have a Databricks instance which does some work. Jobs are triggered from Azure Data Factory. There is several environments and each one has its own Key Vault to store secrets.</p>
  <p>As long as I kept access token - let's say &quot;hardcoded&quot; - within a Databricks linked service configuration everything worked fine. But I need to comply with security standards, so keeping it in JSON which lays somewhere isn't an option - it was fine for the time being.</p>
  <p>Key Vault to the rescue - access token to the Databricks is created via API and stored in a Key Vault, now I wanted to use the Key Vault as linked service in Databricks linked service to populate access token, and the surprise comes here - it doesn't work.</p>
  <p><strong>I can't debug pipeline, I can't trigger it, I can't even test a connection, it always fails with 403 Invalid access token:</strong></p>
  <p><a href="https://i.stack.imgur.com/qfRcD.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/qfRcD.png" alt="enter image description here" /></a></p>
  <p>The JSON for this linked service:</p>
  <pre class="lang-json prettyprint-override"><code>{
      &quot;name&quot;: &quot;ls_databricks&quot;,
      &quot;type&quot;: &quot;Microsoft.DataFactory/factories/linkedservices&quot;,
      &quot;properties&quot;: {
          &quot;annotations&quot;: [],
          &quot;type&quot;: &quot;AzureDatabricks&quot;,
          &quot;typeProperties&quot;: {
              &quot;domain&quot;: &quot;https://**************.azuredatabricks.net&quot;,
              &quot;accessToken&quot;: {
                  &quot;type&quot;: &quot;AzureKeyVaultSecret&quot;,
                  &quot;store&quot;: {
                      &quot;referenceName&quot;: &quot;ls_keyVault&quot;,
                      &quot;type&quot;: &quot;LinkedServiceReference&quot;
                  },
                  &quot;secretName&quot;: &quot;DatabricksAccessToken&quot;
              },
              &quot;existingClusterId&quot;: &quot;*********&quot;
          }
      }
  }
  </code></pre>
  <hr />
  <p>While, using Postman I can easily access Databricks API using the same access token:
  <a href="https://i.stack.imgur.com/ViNN3.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ViNN3.png" alt="enter image description here" /></a></p>
  <hr />
  <p>Key Vault linked service itself works fine and connection test passes:
  <a href="https://i.stack.imgur.com/RYEec.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/RYEec.png" alt="enter image description here" /></a></p>
  <hr />
  <p>I have configured different linked service to connect to ADLS using Key Vault and it works as expected:
  <a href="https://i.stack.imgur.com/Nmxo1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Nmxo1.png" alt="enter image description here" /></a></p>
  <p>Does anybody have any ideas what's wrong here?
  It is just broken or I'm doing something wrong?</p>
  <p>p.s. Apologies for flooding you with all of these screenshots :)</p>
  <p>I'm using <a href="https://docs.databricks.com/dev-tools/api/latest/scim/scim-sp.html" rel="nofollow noreferrer">https://docs.databricks.com/dev-tools/api/latest/scim/scim-sp.html</a> SCIM API to entitle my service principal to a proper group on Databricks instance.</p>