title: How to recover from Apache Cassandra CommitLogReplayException
body: <p>I had an unexpected shutdown of Mac OS X while running apache-cassandra 3.3.  When restarting cassandra in a new bootload, I got the following error:</p>
  
  <pre><code>INFO  15:25:15 Replaying /Users/eswenson/opt/apache-cassandra-3.5/bin/../data/commitlog/CommitLog-6-1461717351786.log, /Users/eswenson/opt/apache-cassandra-3.5/bin/../data/commitlog/CommitLog-6-1461717351787.log, /Users/eswenson/opt/apache-cassandra-3.5/bin/../data/commitlog/CommitLog-6-1461942807785.log, /Users/eswenson/opt/apache-cassandra-3.5/bin/../data/commitlog/CommitLog-6-1461943004975.log
  ERROR 15:25:15 Exiting due to error while processing commit log during initialization.
  org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: Encountered bad header at position 542295 of commit log /Users/eswenson/opt/apache-cassandra-3.5/bin/../data/commitlog/CommitLog-6-1461717351786.log, with bad position but valid CRC
          at org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:611) [apache-cassandra-3.5.jar:3.5]
          at org.apache.cassandra.db.commitlog.SegmentReader$SegmentIterator.computeNext(SegmentReader.java:105) [apache-cassandra-3.5.jar:3.5]
          at org.apache.cassandra.db.commitlog.SegmentReader$SegmentIterator.computeNext(SegmentReader.java:78) [apache-cassandra-3.5.jar:3.5]
          at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) [guava-18.0.jar:na]
          at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) [guava-18.0.jar:na]
          at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:399) [apache-cassandra-3.5.jar:3.5]
          at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:236) [apache-cassandra-3.5.jar:3.5]
          at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:192) [apache-cassandra-3.5.jar:3.5]
          at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:172) [apache-cassandra-3.5.jar:3.5]
          at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:283) [apache-cassandra-3.5.jar:3.5]
          at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551) [apache-cassandra-3.5.jar:3.5]
          at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:680) [apache-cassandra-3.5.jar:3.5]
  </code></pre>
  
  <p>I searched on stack overflow and google and did not find posts of this same problem.  I found issue <a href="https://issues.apache.org/jira/browse/CASSANDRA-9749" rel="nofollow">https://issues.apache.org/jira/browse/CASSANDRA-9749</a>, which is related, but this one is marked resolved and is related to continuing execution after an problem is detected rather than failing with an exception.</p>
  
  <p>What do I do in this situation?  Is there any recovery path (assuming no snapshots avaialble)?  Any way to truncate the commit log at the offending record?  Any tool available to fix the bad header (e.g. delete the record)?</p>
  
  <p>Thanks.</p>