title: How can I restore Cassandra snapshots?
body: <p>I'm building a backup and restore process for a Cassandra database so that it's ready when I need it, and so that I understand the details in order to build something that will work for production. I'm following Datastax's instructions here:</p>
  
  <p><a href="http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_backup_restore_c.html" rel="noreferrer">http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_backup_restore_c.html</a>.</p>
  
  <p>As a start, I'm seeding the database on a dev box then attempting to make the backup/restore work. Here's the backup script:</p>
  
  <pre><code>#!/bin/bash
  
  cd /opt/apache-cassandra-2.0.9
  ./bin/nodetool clearsnapshot -t after_seeding makeyourcase
  ./bin/nodetool snapshot -t after_seeding makeyourcase
  
  cd /var/lib/
  tar czf after_seeding.tgz cassandra/data/makeyourcase/*/snapshots/after_seeding
  </code></pre>
  
  <p>Yes, tar is not the most efficient way, perhaps, but I'm just trying to get something working right now. I've checked the tar, and all the files are there.</p>
  
  <p>Once the database is backed up, I shut down Cassandra and my app, then <code>rm -rf /var/lib/cassandra/</code> to simulate a complete loss.</p>
  
  <p>Now to restore the database. Restoration "Method 2" from <a href="http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_backup_snapshot_restore_t.html" rel="noreferrer">http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_backup_snapshot_restore_t.html</a> is more compatible with my schema-creation component than Method 1.</p>
  
  <p>So, Method 2/Step 1, "Recreate the schema": Restart Cassandra, then my app. The app is built to re-recreate the schema on startup when necessary. Once it's up, there's a working Cassandra node with a schema for the app, but no data.</p>
  
  <p>Method 2/Step 2 "Restore the snapshot": They give three alternatives, the first of which is to use sstableloader, documented at <a href="http://www.datastax.com/documentation/cassandra/2.0/cassandra/tools/toolsBulkloader_t.html" rel="noreferrer">http://www.datastax.com/documentation/cassandra/2.0/cassandra/tools/toolsBulkloader_t.html</a>. The folder structure that the loader requires is nothing like the folder structure created by the snapshot tool, so everything has to be moved into place. Before going to all that trouble, I'll just try it out on one table:</p>
  
  <pre><code>&gt;./bin/sstableloader makeyourcase/users
  Error: Could not find or load main class org.apache.cassandra.tools.BulkLoader
  </code></pre>
  
  <p>Hmmm, well, that's not going to work. BulkLoader is in ./lib/apache-cassandra-2.0.9.jar, but the loader doesn't seem to be set up to work out of the box. Rather than debug the tool, let's move on to the second alternative, copying the snapshot directory into the makeyourcase/users/snapshots/ directory. This should be easy, since we're throwing the snapshot directory right back where it came from, so <code>tar xzf after_seeding.tgz</code> should do the trick:</p>
  
  <pre><code>cd /var/lib/
  tar xzf after_seeding.tgz
  chmod -R u+rwx cassandra/data/makeyourcase
  </code></pre>
  
  <p>and that puts the snapshot directories back under their respective 'snapshots' directories, and a refresh should restore the data:</p>
  
  <pre><code>cd /opt/apache-cassandra-2.0.9
  ./bin/nodetool refresh -- makeyourcase users
  </code></pre>
  
  <p>This runs without complaint. Note that you have to run this for each and every table, so you have to generate the list of tables first. But, before we do that, note that there's something interesting in the Cassandra logs:</p>
  
  <pre><code>INFO 14:32:26,319 Loading new SSTables for makeyourcase/users...
  INFO 14:32:26,326 No new SSTables were found for makeyourcase/users
  </code></pre>
  
  <p>So, we put the snapshot back, but Cassandra didn't find it. I also tried moving the snapshot directory under the existing SSTables directory, and copying the old SSTable files into the existing directory, with the same error in the log. Cassandra doesn't log where it expects to find them, just that it can't find them. The docs say to put them into a directory named data/keyspace/table_name-UUID, but there is no such directory. There is one named data/makeyourcase/users/snapshots/1408820504987-users/, but putting the snapshot dir there, or the individual files, didn't work.</p>
  
  <p>The third alternative, the "Node restart method" doesn't look suitable for a multi-node production environment, so I didn't try that.</p>
  
  <p>Edit:</p>
  
  <p>Just to make this perfectly explicit for the next person, here are the preliminary, working backup and restore scripts that apply the accepted answer.</p>
  
  <p>myc_backup.sh:</p>
  
  <pre><code>#!/bin/bash
  
  cd ~/bootstrap/apache-cassandra-2.0.9
  ./bin/nodetool clearsnapshot -t after_seeding makeyourcase
  ./bin/nodetool snapshot -t after_seeding makeyourcase
  
  cd /var/lib/
  tar czf after_seeding.tgz cassandra/data/makeyourcase/*/snapshots/after_seeding
  </code></pre>
  
  <p>myc_restore.sh:</p>
  
  <pre><code>#!/bin/bash
  
  cd /var/lib/
  tar xzf after_seeding.tgz
  chmod -R u+rwx cassandra/data/makeyourcase
  
  cd ~/bootstrap/apache-cassandra-2.0.9
  TABLE_LIST=`./bin/nodetool cfstats makeyourcase | grep "Table: " | sed -e 's+^.*: ++'`
  for TABLE in $TABLE_LIST; do
      echo "Restore table ${TABLE}"
      cd /var/lib/cassandra/data/makeyourcase/${TABLE}
      if [ -d "snapshots/after_seeding" ]; then
          cp snapshots/after_seeding/* .
          cd ~/bootstrap/apache-cassandra-2.0.9
          ./bin/nodetool refresh -- makeyourcase ${TABLE}
          cd /var/lib/cassandra/data/makeyourcase/${TABLE}
          rm -rf snapshots/after_seeding
          echo "    Table ${TABLE} restored."
      else
          echo "    &gt;&gt;&gt; Nothing to restore."
      fi
  done
  </code></pre>