title: AWS MySQL RDS instance becomes unresponsive and getting restarted automatically
body: <p>We have a AWS MySQL RDS instance which is about 1.7T in size. Sometimes it becomes unresponsive and no operations can be performed.</p>
  
  <ol>
  <li>CPU utilization, Write IOPS, read IOPS, queue depth, write throughput, write latency and read latency drops to zero. </li>
  <li>Number of connections get piled up.</li>
  <li>"Show engine innodb status" hangs</li>
  <li><p>Lot of queries (around 25 for each) by rdsadmin which are in hang state.</p>
  
  <pre><code>SELECT count(*) from mysql.rds_replication_status WHERE action = 'reset slave' and master_host is NULL and master_port is NULL GROUP BY action_timestamp,called_by_user,action,mysql_version,master_host,master_port ORDER BY action_timestamp LIMIT 1;
  
  SELECT NAME, VALUE FROM mysql.rds_configuration; 
  </code></pre></li>
  <li><p>After sometime, instance gets rebooted automatically with following error.</p>
  
  <p>MySQL restart initiated to address MySQL induced log backup issues. Note that as part of this resulution, a DB Snapshot will be performed after MySQL completes restarting.</p></li>
  </ol>
  
  <p><a href="https://i.stack.imgur.com/qsLDw.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/qsLDw.png" alt="enter image description here"></a></p>
  
  <p>What can be the issue? This happens quite often. Sometimes, for our surprise, this happens in off-peak times too. </p>