title: wandb: artifact.add_reference() option to add specific (not current) versionId or ETag to stop the need for re-upload to s3?
body: <p>I feel like this should be possible, but I looked through the wandb SDK code and I can't find an easy/logical way to do it. It <em>might</em> be possible to hack it by modifying the manifest entries at some point later (but maybe before the artifact is logged to wandb as then the manifest and the entries might be locked)? I saw things like this in the SDK code:</p>
  <pre class="lang-py prettyprint-override"><code>version = manifest_entry.extra.get(&quot;versionID&quot;)
  etag = manifest_entry.extra.get(&quot;etag&quot;)
  </code></pre>
  <p>So, I figure we can probably edit those?</p>
  <h2>UPDATE</h2>
  <p>So, I tried to hack it together with something like this and it works but it feels wrong:</p>
  <pre class="lang-py prettyprint-override"><code>import os
  import wandb
  import boto3
  from wandb.util import md5_file
  
  ENTITY = os.environ.get(&quot;WANDB_ENTITY&quot;)
  PROJECT = os.environ.get(&quot;WANDB_PROJECT&quot;)
  API_KEY = os.environ.get(&quot;WANDB_API_KEY&quot;)
  
  api = api = wandb.Api(overrides={&quot;entity&quot;: ENTITY, &quot;project&quot;: ENTITY})
  run = wandb.init(entity=ENTITY, project=PROJECT, job_type=&quot;test upload&quot;)
  file = &quot;admin2Codes.txt&quot;  # &quot;admin1CodesASCII.txt&quot; # (both already on s3 with a couple versions)
  artifact = wandb.Artifact(&quot;test_data&quot;, type=&quot;dataset&quot;)
  
  # modify one of the local files so it has a new md5hash etc.
  with open(file, &quot;a&quot;) as f:
      f.write(&quot;new_line_1\n&quot;)
  
  # upload local file to s3
  local_file_path = file
  s3_url = f&quot;s3://bucket/prefix/{file}&quot;
  s3_url_arr = s3_url.replace(&quot;s3://&quot;, &quot;&quot;).split(&quot;/&quot;)
  s3_bucket = s3_url_arr[0]
  key = &quot;/&quot;.join(s3_url_arr[1:])
  
  s3_client = boto3.client(&quot;s3&quot;)
  file_digest = md5_file(local_file_path)
  s3_client.upload_file(
      local_file_path,
      s3_bucket,
      key,
      # save the md5_digest in metadata,
      # can be used later to only upload new files to s3,
      # as AWS doesn't digest the file consistently in the E-tag
      ExtraArgs={&quot;Metadata&quot;: {&quot;md5_digest&quot;: file_digest}},
  )
  head_response = s3_client.head_object(Bucket=s3_bucket, Key=key)
  version_id: str = head_response[&quot;VersionId&quot;]
  print(version_id)
  
  # upload a link/ref to this s3 object in wandb:
  artifact.add_reference(s3_dir)
  # at this point we might be able to modify the artifact._manifest.entries and each entry.extra.get(&quot;etag&quot;) etc.?
  print([(name, entry.extra) for name, entry in artifact._manifest.entries.items()])
  # set these to an older version on s3 that we know we want (rather than latest) - do this via wandb public API:
  dataset_v2 = api.artifact(f&quot;{ENTITY}/{PROJECT}/test_data:v2&quot;, type=&quot;dataset&quot;)
  # artifact._manifest.add_entry(dataset_v2.manifest.entries[&quot;admin1CodesASCII.txt&quot;])
  artifact._manifest.entries[&quot;admin1CodesASCII.txt&quot;] = dataset_v2.manifest.entries[
      &quot;admin1CodesASCII.txt&quot;
  ]
  # verify that it did change:
  print([(name, entry.extra) for name, entry in artifact._manifest.entries.items()])
  
  run.log_artifact(artifact)  # at this point the manifest is locked I believe?
  artifact.wait()  # wait for upload to finish (blocking - but should be very quick given it is just an s3 link)
  print(artifact.name)
  run_id = run.id
  run.finish()
  curr_run = api.run(f&quot;{ENTITY}/{PROJECT}/{run_id}&quot;)
  used_artifacts = curr_run.used_artifacts()
  logged_artifacts = curr_run.logged_artifacts()
  </code></pre>
  <p>Am I on the right track here? I guess the other workaround is to make a copy on s3 (so that older version is the latest again) but I wanted to avoid this as the 1 file that I want to use an old version of is a large NLP model and the only files I want to change are small config.json files etc. (so seems very wasteful to upload all files again).</p>
  <p>I was also wondering if when I copy an old version of an object back into the same key in the bucket if that creates a real copy or just like a pointer to the same underlying object. Neither boto3 nor AWS documentation makes that clear - although it seems like it is a proper copy.</p>