Randomising database for insert <p>Evening all, I've recently been reading the following blog post about sharding at Pinterest and I think there's some great stuff in there <a href="https://engineering.pinterest.com/blog/sharding-pinterest-how-we-scaled-our-mysql-fleet" rel="nofollow">https://engineering.pinterest.com/blog/sharding-pinterest-how-we-scaled-our-mysql-fleet</a></p> <p>What I'm unsure on though, is how best to decide where a brand new user should be inserted.</p> <p>So for those that don't know or have bothered to read the above article, Pinterest have a number of shards, each with a number of databases on. They generate IDs for objects based on a 64 bit shifting that determines a shard, the type of object (user,pin etc..) to determine a table and the local auto-increment id for the object in question. Now they try to put pins etc. on the same database as the 'board' they are on. But for a brand new object, what would be the best way of determining the 'shard' it lives on?</p> <p>For users that sign in via Facebook they use a modulus e.g</p> <pre><code>shard = md5(“1.2.3.4") % 4096 //4096 is the number of shards </code></pre> <p>But if I had a simple email/password registration form, do you think using a similar approach on email address would work for working out an initial shard? I'd assume it would have to be email in this case, otherwise they would have no way of knowing what database to validate the logging credentials against. Also I know that post is from 2015 so not too old and computing power moves quickly, but would there be a better option then using md5 here? I know the chance of a collision is minor - especially as we're just talking about hashing the email address here, but would it be worth using a different algorithm? I'm basically interested in the best way to determine a shard here and to work out how to get back to it (hence why I think it has to be email address)</p> <p>Hope this all makes sense!</p> <p>(p.s didn't take this with the Pinterest tag as it looks like that's just for api dev, but if someone thinks it might get better 'eyes' on the question then feel free to add it)</p>