title: Hash from object's content as object ID: fast alternatives for SHA256
body: <p>I'm working on design of <a href="https://en.wikipedia.org/wiki/Content-addressable_storage" rel="nofollow noreferrer">Content-addressable storage</a>, so I'm looking for a hash function to generate object identifiers. Every object should get short ID based on its content in that way: <code>object_id = hash(object_content)</code>.</p>
  
  <p>Prerequisites:</p>
  
  <ol>
  <li>Hash-function should be fast. </li>
  <li>Collision probability must be as low as possible. </li>
  <li>Optimal ID length is <code>32</code> bytes in order to address <code>256^32</code> objects at max (but this requirement may be relaxed).</li>
  </ol>
  
  <p>Taking into account these requirements, I picked up <code>SHA256</code> hash, but unfortunately it's not fast enough for my purposes. The fastest implementations of <code>SHA256</code> that I was able to benchmark were <code>openssl</code> and <code>boringssl</code>: on my desktop <code>Intel Core I5 6400</code> it gave about <code>420 MB/s</code> per core. Other implementations (like <code>crypto/rsa</code> in Go) are even slower. I would like to replace <code>SHA256</code> with other hash function that provides the same collision guarantees as <code>SHA256</code>, but gives betters throughput (at least <code>600 MB/s</code> per core).</p>
  
  <p>Please share your opinion about possible options to solve this problem.</p>
  
  <p>Also I would like to note that hardware update (like purchasing modern CPU with <code>AVX512</code> instruction set) is not possible. The main point is to find hash function that will provide better performance on commodity hardware.</p>