title: Solr RELOAD error "Error Cannot achieve consistency level QUORUM on attempt 1 out of 4 with CL QUORUM"
body: <p>We are trying to upgrade our cluster from DSE 4.0.0 to DSE 4.0.1</p>
  
  <p><strong>DC Cassandra</strong></p>
  
  <ul>
  <li>Cassandra node 1</li>
  </ul>
  
  <p><strong>DC Solr</strong></p>
  
  <ul>
  <li>Solr node 2</li>
  <li>Solr node 3</li>
  <li>Solr node 4</li>
  <li>Solr node 5 &lt;-- already uses DSE 4.0.1 because it is the version available in the repo when we provisioned the server</li>
  </ul>
  
  <p>We are aware that a <a href="http://www.datastax.com/documentation/upgrade/doc/upgrade/datastax_enterprise/upgradeFrom4_0_0.html?scroll=upgradeFrom4_0_0" rel="nofollow">special set of steps</a> is required because of a bug when upgrading from these specific versions. The problem is that in step 2, the RELOAD command returns the following response:</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;response&gt;
  &lt;lst name="responseHeader"&gt;&lt;int name="status"&gt;500&lt;/int&gt;&lt;int name="QTime"&gt;988&lt;/int&gt;&lt;/lst&gt;&lt;lst name="error"&gt;&lt;str name="msg"&gt;java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.&lt;/str&gt;&lt;str name="trace"&gt;org.apache.solr.common.SolrException: java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.backupResource(SolrCoreResourceManager.java:163)
          at com.datastax.bdp.search.solr.core.CassandraCoreContainer.create(CassandraCoreContainer.java:241)
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.reloadCore(SolrCoreResourceManager.java:380)
          at com.datastax.bdp.search.solr.handler.admin.CassandraCoreAdminHandler.handleReloadAction(CassandraCoreAdminHandler.java:141)
          at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:178)
          at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
          at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:669)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
          at com.datastax.bdp.search.solr.servlet.CassandraDispatchFilter.doFilter(CassandraDispatchFilter.java:96)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.cassandra.audit.SolrHttpAuditLogFilter.doFilter(SolrHttpAuditLogFilter.java:194)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.CassandraAuthorizationFilter.doFilter(CassandraAuthorizationFilter.java:100)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.DseAuthenticationFilter.doFilter(DseAuthenticationFilter.java:102)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
          at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
          at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
          at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
          at java.lang.Thread.run(Thread.java:744)
  &lt;/str&gt;&lt;int name="code"&gt;500&lt;/int&gt;&lt;/lst&gt;&lt;str name="params"&gt;deleteAll=false&amp;amp;name=mtg_keywords_v5.keyword_ad_results&amp;amp;action=RELOAD&amp;amp;reindex=false&lt;/str&gt;
  &lt;/response&gt;
  </code></pre>
  
  <p>In system.log, I can see the following:</p>
  
  <pre><code>WARN [http-8983-5] 2014-04-05 15:17:54,203 CassandraUtil.java (line 134) Error Cannot achieve consistency level QUORUM on attempt 4 out of 4 with CL QUORUM...
  ERROR [http-8983-5] 2014-04-05 15:17:54,303 SolrCoreResourceManager.java (line 162) java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
  java.io.IOException: java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.persistResource(SolrCoreResourceManager.java:668)
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.setResource(SolrCoreResourceManager.java:641)
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.backupResource(SolrCoreResourceManager.java:154)
          at com.datastax.bdp.search.solr.core.CassandraCoreContainer.create(CassandraCoreContainer.java:241)
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.reloadCore(SolrCoreResourceManager.java:380)
          at com.datastax.bdp.search.solr.handler.admin.CassandraCoreAdminHandler.handleReloadAction(CassandraCoreAdminHandler.java:141)
          at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:178)
          at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
          at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:669)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
          at com.datastax.bdp.search.solr.servlet.CassandraDispatchFilter.doFilter(CassandraDispatchFilter.java:96)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.cassandra.audit.SolrHttpAuditLogFilter.doFilter(SolrHttpAuditLogFilter.java:194)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.CassandraAuthorizationFilter.doFilter(CassandraAuthorizationFilter.java:100)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.DseAuthenticationFilter.doFilter(DseAuthenticationFilter.java:102)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
          at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
          at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
          at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
          at java.lang.Thread.run(Thread.java:744)
  Caused by: java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
          at com.datastax.bdp.util.CassandraUtil.robustCql3PreparedStatement(CassandraUtil.java:146)
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.persistResource(SolrCoreResourceManager.java:658)
          ... 32 more
  Caused by: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
          at org.apache.cassandra.db.ConsistencyLevel.assureSufficientLiveNodes(ConsistencyLevel.java:289)
          at org.apache.cassandra.service.AbstractWriteResponseHandler.assureSufficientLiveNodes(AbstractWriteResponseHandler.java:107)
          at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:680)
          at org.apache.cassandra.service.StorageProxy.mutate(StorageProxy.java:454)
          at org.apache.cassandra.service.StorageProxy.mutateWithTriggers(StorageProxy.java:524)
          at org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:423)
          at org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:407)
          at org.apache.cassandra.cql3.QueryProcessor.executeWithHooks(QueryProcessor.java:201)
          at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:188)
          at org.apache.cassandra.cql3.QueryProcessor.processPrepared(QueryProcessor.java:358)
          at com.datastax.bdp.util.CassandraUtil.robustCql3PreparedStatement(CassandraUtil.java:130)
          ... 33 more
  ERROR [http-8983-5] 2014-04-05 15:17:54,304 CoreContainer.java (line 958) Unable to create core: mtg_keywords_v5.keyword_ad_results
  org.apache.solr.common.SolrException: java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.backupResource(SolrCoreResourceManager.java:163)
          at com.datastax.bdp.search.solr.core.CassandraCoreContainer.create(CassandraCoreContainer.java:241)
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.reloadCore(SolrCoreResourceManager.java:380)
          at com.datastax.bdp.search.solr.handler.admin.CassandraCoreAdminHandler.handleReloadAction(CassandraCoreAdminHandler.java:141)
          at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:178)
          at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
          at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:669)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
          at com.datastax.bdp.search.solr.servlet.CassandraDispatchFilter.doFilter(CassandraDispatchFilter.java:96)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.cassandra.audit.SolrHttpAuditLogFilter.doFilter(SolrHttpAuditLogFilter.java:194)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.CassandraAuthorizationFilter.doFilter(CassandraAuthorizationFilter.java:100)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.DseAuthenticationFilter.doFilter(DseAuthenticationFilter.java:102)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
          at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
          at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
          at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
          at java.lang.Thread.run(Thread.java:744)
  ERROR [http-8983-5] 2014-04-05 15:17:54,305 SolrCoreResourceManager.java (line 433) java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
  org.apache.solr.common.SolrException: java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.backupResource(SolrCoreResourceManager.java:163)
          at com.datastax.bdp.search.solr.core.CassandraCoreContainer.create(CassandraCoreContainer.java:241)
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.reloadCore(SolrCoreResourceManager.java:380)
          at com.datastax.bdp.search.solr.handler.admin.CassandraCoreAdminHandler.handleReloadAction(CassandraCoreAdminHandler.java:141)
          at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:178)
          at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
          at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:669)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
          at com.datastax.bdp.search.solr.servlet.CassandraDispatchFilter.doFilter(CassandraDispatchFilter.java:96)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.cassandra.audit.SolrHttpAuditLogFilter.doFilter(SolrHttpAuditLogFilter.java:194)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.CassandraAuthorizationFilter.doFilter(CassandraAuthorizationFilter.java:100)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.DseAuthenticationFilter.doFilter(DseAuthenticationFilter.java:102)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
          at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
          at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
          at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
          at java.lang.Thread.run(Thread.java:744)
  ERROR [http-8983-5] 2014-04-05 15:17:54,307 SolrException.java (line 124) org.apache.solr.common.SolrException: java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.backupResource(SolrCoreResourceManager.java:163)
          at com.datastax.bdp.search.solr.core.CassandraCoreContainer.create(CassandraCoreContainer.java:241)
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.reloadCore(SolrCoreResourceManager.java:380)
          at com.datastax.bdp.search.solr.handler.admin.CassandraCoreAdminHandler.handleReloadAction(CassandraCoreAdminHandler.java:141)
          at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:178)
          at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
          at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:669)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
          at com.datastax.bdp.search.solr.servlet.CassandraDispatchFilter.doFilter(CassandraDispatchFilter.java:96)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.cassandra.audit.SolrHttpAuditLogFilter.doFilter(SolrHttpAuditLogFilter.java:194)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.CassandraAuthorizationFilter.doFilter(CassandraAuthorizationFilter.java:100)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.DseAuthenticationFilter.doFilter(DseAuthenticationFilter.java:102)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
          at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
          at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
          at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
          at java.lang.Thread.run(Thread.java:744)
  
   INFO [http-8983-5] 2014-04-05 15:17:54,307 SolrDispatchFilter.java (line 672) [admin] webapp=null path=/admin/cores params={deleteAll=false&amp;name=mtg_keywords_v5.keyword_ad_results&amp;action=RELOAD&amp;reindex=false} status=500 QTime=850
  ERROR [http-8983-5] 2014-04-05 15:17:54,308 SolrException.java (line 136) null:org.apache.solr.common.SolrException: java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.backupResource(SolrCoreResourceManager.java:163)
          at com.datastax.bdp.search.solr.core.CassandraCoreContainer.create(CassandraCoreContainer.java:241)
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.reloadCore(SolrCoreResourceManager.java:380)
          at com.datastax.bdp.search.solr.handler.admin.CassandraCoreAdminHandler.handleReloadAction(CassandraCoreAdminHandler.java:141)
          at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:178)
          at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
          at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:669)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
          at com.datastax.bdp.search.solr.servlet.CassandraDispatchFilter.doFilter(CassandraDispatchFilter.java:96)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.cassandra.audit.SolrHttpAuditLogFilter.doFilter(SolrHttpAuditLogFilter.java:194)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.CassandraAuthorizationFilter.doFilter(CassandraAuthorizationFilter.java:100)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.DseAuthenticationFilter.doFilter(DseAuthenticationFilter.java:102)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
          at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
          at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
          at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
          at java.lang.Thread.run(Thread.java:744)
  
  ERROR [http-8983-5] 2014-04-05 15:17:54,308 SolrDispatchFilter.java (line 696) Error request exception: java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
  org.apache.solr.common.SolrException: java.lang.RuntimeException: CQL3 prepared statement failed after 4 attempts, source exception follows.
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.backupResource(SolrCoreResourceManager.java:163)
          at com.datastax.bdp.search.solr.core.CassandraCoreContainer.create(CassandraCoreContainer.java:241)
          at com.datastax.bdp.search.solr.core.SolrCoreResourceManager.reloadCore(SolrCoreResourceManager.java:380)
          at com.datastax.bdp.search.solr.handler.admin.CassandraCoreAdminHandler.handleReloadAction(CassandraCoreAdminHandler.java:141)
          at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:178)
          at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
          at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:669)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197)
          at com.datastax.bdp.search.solr.servlet.CassandraDispatchFilter.doFilter(CassandraDispatchFilter.java:96)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.cassandra.audit.SolrHttpAuditLogFilter.doFilter(SolrHttpAuditLogFilter.java:194)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.CassandraAuthorizationFilter.doFilter(CassandraAuthorizationFilter.java:100)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at com.datastax.bdp.search.solr.auth.DseAuthenticationFilter.doFilter(DseAuthenticationFilter.java:102)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
          at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
          at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
          at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
          at java.lang.Thread.run(Thread.java:744)
  </code></pre>
  
  <p>How should I deal with this? Should I specify cl=ANY/ONE in the RELOAD command and retry? Or should I simply ignore the error and proceed to step 3?</p>
  
  <p>Take note that I have already drained all the nodes as instructed in step 1. For some reason, Cassandra node 1 is shown in OpsCenter as "Active - Drained"; Solr nodes 2, 4, and 5 are shown as "Search - Active - Drained"; <strong>but</strong> Solr node 3 is shown as "Search - <em>Unresponsive</em> - Drained"</p>