title: Firestore distributed counters: How to know the ideal shards sum we need?
body: <p>TL;DR:  <strong>Is there any maths here to predict exactly how many shards should I use for my counters?</strong></p>
  
  <p>I'm a big fan of counters (yeah that's weird) and I'm writing a small polls app.</p>
  
  <p>I have <strong>counters</strong> collection where each document is a <strong>counter</strong> and each counter has a sub-collection of <strong>shards</strong>.</p>
  
  <pre><code>&lt;root_level&gt;
  ├── [counters]
  │   └── counterID
  │       ├── num_shards: 20
  │       └── [shards]
  │           ├── 1
  │           │   └── counts: 2
  │           ├── 2
  │           │   └── counts: 7
  │           └── 3...
  ├── [votes]
      └── voteId
          ...
  </code></pre>
  
  <p>Also, I've set up a Cloud Function to track any new added <strong>vote</strong> document inside a <strong>votes</strong> collection. This function get a random shard ID and increment it... just like the example in the documentation.</p>
  
  <pre><code>const counterRef = db.doc('counters/counter')
  
  export const onVote = functions.firestore
    .document('votes/{voteId}')
    .onCreate(async snapShot =&gt; {
      const counterDoc = (await counterRef.get())
  
      const { num_shards } = counterDoc.data()
      const shard_id = Math.floor(Math.random() * num_shards).toString()
      const shard_ref = counterRef.collection('shards').doc(shard_id)
  
      return db.runTransaction(async tsx =&gt; {
        const shardDoc = (await tsx.get(shard_ref))
        const oldCount = shardDoc.data().count
        return tsx.update(shard_ref, { count: oldCount + 1 })
      })
    })
  </code></pre>
  
  <p>And in my client, I have combined two observables: one for the counter and one for it's related shards sub-collection, and I do a basic reduce() to sum the total count.</p>
  
  <pre><code>combineLatest(counter$, shards$)
        .pipe(
          map(([counter, shards]) =&gt; {
            const count = shards.reduce(
              (acc: any, shards: any) =&gt; shards.count + acc,
              0
            )
  
            return { ...counter, count }
          })
        )
        .subscribe(counter =&gt; /* display counter */)
  </code></pre>
  
  <p>Well, I did several tests with 5, 10, 15 shards and 1000 votes and I'm getting data contention errors. But with 100 shards, I got no errors. But it seems like this is a coin-flip game. In fact, sometimes it just work with few shards, sometimes I need more shards.</p>
  
  <p>So my question is: <strong>Is there any maths here to predict exactly how many shards should I use for my counters?</strong> I'd like to have the best performance for the lowest cost.</p>
  
  <p><a href="https://i.stack.imgur.com/KdVBF.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/KdVBF.png" alt="Data contention errors"></a></p>