title: Azure ML File Dataset mount() is slow & downloads data twice
body: <p>I have created a Fie Dataset using Azure ML python API. Data under question is bunch of parquet files (~10K parquet files each of size of 330 KB) residing in Azure Data Lake Gen 2 spread across multiple partitions. Then, I tried to mount the dataset in AML compute instance. During this mounting process, I have observed that each parquet file has been downloaded twice under the /tmp directory of the compute instance with the following message printed as the console logs:</p>
  <pre><code>Downloaded path: /tmp/tmp_3qwqu9u/c2c69fd1-9ded-4d69-b75a-c19e1694b7aa/&lt;blob_path&gt;/20211203.parquet is different from target path: /tmp/tmp_3qwqu9u/c2c69fd1-9ded-4d69-b75a-c19e1694b7aa/&lt;container_name&gt;/&lt;blob_path&gt;/20211203.parquet
  </code></pre>
  <p>This log message gets printed for each parquet file which is part of the dataset.</p>
  <p>Also, the process of mounting the dataset is very slow: 44 minutes for for ~10K parquet files each of size of 330 KB.</p>
  <p>&quot;%%time&quot; command in the Jupyter Lab shows most of the time has been used for IO process?</p>
  <pre><code>CPU times: user 4min 22s, sys: 51.5 s, total: 5min 13s
  Wall time: 44min 15s
  </code></pre>
  <p>Note: Both the Data Lake Gen 2 and Azure ML compute instance are under the same virtual network.</p>
  <p><strong>Here are my questions:</strong></p>
  <ol>
  <li><strong>How to avoid downloading the parquet file twice?</strong></li>
  <li><strong>How to make the mounting process faster?</strong></li>
  </ol>
  <p>I have gone through <a href="https://stackoverflow.com/questions/68556675/azureml-dataset-file-from-files-creation-extremely-slow-even-with-4-files">this thread</a>, but the discussion there didn't conclude</p>
  <p>The Python code I have used is as followed:</p>
  <pre><code>data = Dataset.File.from_files(path=list_of_blobs, validate=True)
  dataset = data.register(workspace=ws, name=dataset_name, create_new_version=create_new_version)
  mount_context = None
  try:
      mount_context = dataset.mount(path_to_mount)
      # Mount the file stream
      mount_context.start()
  except Exception as ex:
      raise(ex)
  
  df = pd.read_parquet(path_to_mount)
  </code></pre>