title: Why isn't curator recovering when zookeeper is back online?
body: <p>I have a <code>CuratorFramework</code> client (v5.1.0) running against a Zookeeper server (v3.7.0). If the
  Zookeeper server is shutdown while the client is connected to it I can see connection states
  (with a <code>ConnectionStateListener</code>) of <code>SUSPENDED</code> and then <code>LOST</code> and then nothing more even
  when the server comes back online.</p>
  <p>This feels like a very standard use case and that I must be missing something silly, but I can never
  get the client to connect again once the server is online.</p>
  <p>I have done some google searching and found nothing useful about how to handle recovery after a LOST state.</p>
  <p>I have a <a href="https://github.com/cjstehno/curator-recovery" rel="nofollow noreferrer">self-contained example</a> of what I am doing with
  the example code in the
  <a href="https://github.com/cjstehno/curator-recovery/blob/master/src/test/java/demo/CuratorRecoveryTest.java" rel="nofollow noreferrer">CuratorRecoveryTest</a> class
  (run in IDE rather than maven). The meat of it is (extracted from the test class):</p>
  <pre class="lang-java prettyprint-override"><code>// setup the server and client
  server = new TestingServer();
  
  client = newClient(server.getConnectString(), 60000, 15000, new RetryNTimes(1, 250));
  client.start();
  client.blockUntilConnected();
              
  // add the listener
  final var stateListener = new StateListener();
  stateListener.stateChanged(client, CONNECTED);
  
  // register the listener
  client.getConnectionStateListenable().addListener(stateListener);
  
  // verify connection
  assertTrue(client.getZookeeperClient().isConnected());
  
  // let things settle
  nap(3, &quot;initial settling&quot;);
  
  // stop zk
  stopServer();
  log.info(&quot;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt; STOPPED ZK SERVER&quot;);
  
  // let it bake
  nap(3, &quot;letting things bake&quot;);
  
  // ensure disconnected
  assertFalse(client.getZookeeperClient().isConnected());
  
  nap(3, &quot;disconnecting&quot;);
  
  // start zk
  server.start();
  log.info(&quot;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt; STARTED ZK SERVER&quot;);
  
  await().atMost(5, MINUTES).until(() -&gt; stateListener.getCurrentState() == CONNECTED || stateListener.getCurrentState() == RECONNECTED);
  
  // NOTE: it never gets here - no state changes after LOST
  
  assertTrue(client.getZookeeperClient().isConnected());
  </code></pre>
  <p>When this is run I get the following output:</p>
  <pre><code>[Thread-0] INFO org.apache.curator.test.TestingZooKeeperMain - Starting server
  [Thread-0] WARN org.apache.zookeeper.server.ServerCnxnFactory - maxCnxns is not configured, using default value 0.
  [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Starting
  [main] INFO org.apache.curator.framework.imps.CuratorFrameworkImpl - Default schema
  [main] WARN demo.CuratorRecoveryTest - CONNECTION-STATE-CHANGE: null --&gt; CONNECTED
  [main] DEBUG demo.CuratorRecoveryTest - Taking a 3s nap for initial settling...
  [main] DEBUG demo.CuratorRecoveryTest - Done napping for initial settling...
  [Curator-ConnectionStateManager-0] WARN demo.CuratorRecoveryTest - CONNECTION-STATE-CHANGE: CONNECTED --&gt; SUSPENDED
  [main] INFO demo.CuratorRecoveryTest - &gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt; STOPPED ZK SERVER
  [main] DEBUG demo.CuratorRecoveryTest - Taking a 3s nap for letting things bake...
  [main] DEBUG demo.CuratorRecoveryTest - Done napping for letting things bake...
  [main] DEBUG demo.CuratorRecoveryTest - Taking a 3s nap for disconnecting...
  [main] DEBUG demo.CuratorRecoveryTest - Done napping for disconnecting...
  [main] INFO demo.CuratorRecoveryTest - &gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt; STARTED ZK SERVER
  [Curator-ConnectionStateManager-0] WARN org.apache.curator.framework.state.ConnectionStateManager - Session timeout has elapsed while SUSPENDED. Injecting a session expiration. Elapsed ms: 20009. Adjusted session timeout ms: 20000
  [main-EventThread] WARN org.apache.curator.ConnectionState - Session expired event received
  [Curator-ConnectionStateManager-0] WARN demo.CuratorRecoveryTest - CONNECTION-STATE-CHANGE: SUSPENDED --&gt; LOST
  </code></pre>
  <p>which then fails when the waiting condition never happens.</p>
  <blockquote>
  <p>NOTE: This happens on an older version combination of Curator and Zookeeper as well, so this is not a &quot;bleeding edge&quot; issue.</p>
  </blockquote>
  <p>What am I missing?</p>