<p>My company is in the process of migrating data from one kubernetes cluster to another one.</p>

<p>Part of the migration is to move data from mongoDB.</p>

<p>The mongoDB installations came with some backup scripts, which I used as an entrypoint for my custom restore.</p>

<p>What I successfully did (at least as far as I can tell right now) is to run a <code>mongodump</code> on the old cluster and pipe it into a <code>mongorestore</code> in the new cluster.</p>

<p>It works, but it is really really slow. The dataset (<code>/data/db</code>) is around 65G big. The restore has been running for the last 6 hours or so and is barely moving forward.</p>

<p>Also, at some point the process was interrupted and instead of deleting all data I simply started to script again - thinking it would still apply everything and throw errors for duplicate keys which I can ignore.</p>

<p>This is what I precisely do</p>

<pre><code>kubectl --kubeconfig=old-cluster.conf exec -t $SOURCE_MONGO_POD -- \
  bash -c "mongodump --host $SOURCE_MONGO_REPLICASET \
  --username $SOURCE_USERNAME --password $SOURCE_PASSWORD \
  --authenticationDatabase admin --gzip --archive --oplog" |
kubectl exec -i $TARGET_MONGO_POD -- \
  bash -c "mongorestore --host $TARGET_MONGO_REPLICASET \
  --username $TARGET_USERNAME --password $TARGET_PASSWORD \
  --authenticationDatabase admin --gzip --archive --oplogReplay"
</code></pre>

<p>What is wrong with my approach. Why is my performance so bad?</p>

<p>Someone was suggesting to just copy over the <code>/data/db</code> folder, which might be faster and since I need a 1:1 migration, would be sufficient.</p>

Moritz Schmitz v. Hülst
<p>As we can read at <a href="https://docs.mongodb.com/manual/tutorial/backup-and-restore-tools/" rel="nofollow noreferrer">Back Up and Restore with MongoDB Tools</a>:</p>

<blockquote>
  <p>The  <a href="https://docs.mongodb.com/manual/reference/program/mongodump/#bin.mongodump" rel="nofollow noreferrer" title="bin.mongodump"><code>mongodump</code></a>  and  <a href="https://docs.mongodb.com/manual/reference/program/mongorestore/#bin.mongorestore" rel="nofollow noreferrer" title="bin.mongorestore"><code>mongorestore</code></a>  utilities work with  <a href="https://docs.mongodb.com/manual/reference/bson-types/" rel="nofollow noreferrer">BSON</a>  data dumps, and <strong>are useful for creating backups of small deployments</strong>. For resilient and non-disruptive backups, use a file system or block-level disk snapshot function, such as the methods described in the  <a href="https://docs.mongodb.com/manual/core/backups/" rel="nofollow noreferrer">MongoDB Backup Methods</a>  document.</p>
  
  <p>Because  <a href="https://docs.mongodb.com/manual/reference/program/mongodump/#bin.mongodump" rel="nofollow noreferrer" title="bin.mongodump"><code>mongodump</code></a>  and  <a href="https://docs.mongodb.com/manual/reference/program/mongorestore/#bin.mongorestore" rel="nofollow noreferrer" title="bin.mongorestore"><code>mongorestore</code></a>  operate by interacting with a running  <a href="https://docs.mongodb.com/manual/reference/program/mongod/#bin.mongod" rel="nofollow noreferrer" title="bin.mongod"><code>mongod</code></a>  instance, they can impact the performance of your running database. Not only do the tools create traffic for a running database instance, they also force the database to read all data through memory. When MongoDB reads infrequently used data, it can evict more frequently accessed data, causing a deterioration in performance for the database’s regular workload.</p>
</blockquote>

<p>You should consider using alternatives such as <a href="https://docs.mongodb.com/manual/tutorial/backup-with-filesystem-snapshots/" rel="nofollow noreferrer">Filesystem Snapshots</a> or <a href="https://docs.cloudmanager.mongodb.com/tutorial/nav/backup-use" rel="nofollow noreferrer">MongoDB Cloud Manager</a>.</p>

<p>Also have you considered adding a <a href="https://docs.mongodb.com/manual/replication/" rel="nofollow noreferrer">Replicaiton</a> to your current MongoDB?</p>
