title: Mysql slave out of sync after crash
body: <p>We have a "1 master, 1 slave" MySQL setup.  We had a sudden power outage that took down the slave.  After getting the machine back up, I found that the slave was out of sync with the master:</p>
  
  <pre><code>mysql&gt; show slave status\G
  *************************** 1. row ***************************
                 Slave_IO_State: Waiting for master to send event
                    Master_Host: 10.0.0.1
                    Master_User: slave
                    Master_Port: 3306
                  Connect_Retry: 60
                Master_Log_File: mysql-log.001576
            Read_Master_Log_Pos: 412565824
                 Relay_Log_File: mysqld-relay-bin.002671
                  Relay_Log_Pos: 6930
          Relay_Master_Log_File: mysql-log.001573
               Slave_IO_Running: Yes
              Slave_SQL_Running: No
                Replicate_Do_DB:
            Replicate_Ignore_DB:
             Replicate_Do_Table:
         Replicate_Ignore_Table: blah.table2
        Replicate_Wild_Do_Table:
    Replicate_Wild_Ignore_Table:
                     Last_Errno: 1032
                     Last_Error: Could not execute Update_rows event on table blah.info; Can't find record in 'info', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysql-log.001573, end_log_pos 689031225
                   Skip_Counter: 0
            Exec_Master_Log_Pos: 689030864
                Relay_Log_Space: 2944772417
                Until_Condition: None
                 Until_Log_File:
                  Until_Log_Pos: 0
             Master_SSL_Allowed: No
             Master_SSL_CA_File:
             Master_SSL_CA_Path:
                Master_SSL_Cert:
              Master_SSL_Cipher:
                 Master_SSL_Key:
          Seconds_Behind_Master: NULL
  Master_SSL_Verify_Server_Cert: No
                  Last_IO_Errno: 0
                  Last_IO_Error:
                 Last_SQL_Errno: 1032
                 Last_SQL_Error: Could not execute Update_rows event on table blah.info; Can't find record in 'info', Error_code: 1032; handler error HA_ERR_KEY_NOT_FOUND; the event's master log mysql-log.001573, end_log_pos 689031225
    Replicate_Ignore_Server_Ids:
               Master_Server_Id: 1
  1 row in set (0.00 sec)
  </code></pre>
  
  <p>We're using a binlog format of "ROW", so when I try to use mysqlbinlog to look at the offending row, I don't see anything of use.  I don't want to simply set the skip counter, because I think that would throw my table even further out of sync.  </p>
  
  <p>Is there anything I can do on the slave that would essentially "roll back" to a given point in time, where I could then reset the master log number, poition, etc?  If not, is there anything at all that I can do to get back in sync?</p>