title: Bidirectional gRPC stream sometimes stops processing responses after stopping and starting
body: <h3>In short</h3>
  
  <p>We have a mobile app that streams fairly high volumes of data to and from a server through various bidirectional streams. The streams need to be closed on occasion (for example when the app is backgrounded). They are then reopened as needed. Sometimes when this happens, something goes wrong:</p>
  
  <ul>
  <li>From what I can tell, the stream is up and running on the device's side (the status of both the GRPCProtocall and the GRXWriter involved is either started or paused) </li>
  <li>The device sends data on the stream fine (the server receives the data)</li>
  <li>The server seems to send data back to the device fine (the server's Stream.Send calls return as successful)</li>
  <li>On the device, the result handler for data received on the stream is never called</li>
  </ul>
  
  <h3>More detail</h3>
  
  <p>Our code is heavily simplified below, but this should hopefully provide enough detail to indicate what we're doing. A bidirection stream is managed by a <code>Switch</code> class:</p>
  
  <pre><code>class Switch {
      /** The protocall over which we send and receive data */
      var protocall: GRPCProtoCall?
  
      /** The writer object that writes data to the protocall. */
      var writer: GRXBufferedPipe?
  
      /** A static GRPCProtoService as per the .proto */
      static let service = APPDataService(host: Settings.grpcHost)
  
      /** A response handler. APPData is the datatype defined by the .proto. */
      func rpcResponse(done: Bool, response: APPData?, error: Error?) {
          NSLog("Response received")
          // Handle response...
      }
  
      func start() {
          // Create a (new) instance of the writer
          // (A writer cannot be used on multiple protocalls)
          self.writer = GRXBufferedPipe()
  
          // Setup the protocall
          self.protocall = Switch.service.rpcToStream(withRequestWriter: self.writer!, eventHandler: self.rpcRespose(done:response:error:))
  
          // Start the stream
          self.protocall.start()
      }
  
      func stop() {
          // Stop the writer if it is started.
          if self.writer.state == .started || self.writer.state == .paused {
              self.writer.finishWithError(nil)
          }
  
          // Stop the proto call if it is started
          if self.protocall?.state == .started || self.protocall?.state == .paused {
              protocall?.cancel()
          }
          self.protocall = nil
      }
  
      private var needsRestart: Bool {
          if let protocall = self.protocall {
              if protocall.state == .notStarted || protocall.state == .finished {
                  // protocall exists, but isn't running.
                  return true
              } else if writer.state == .notStarted || writer.state == .finished {
                  // writer isn't running
                  return true
              } else {
                  // protocall and writer are running
                  return false
              }
          } else {
              // protocall doesn't exist.
              return true
          }
      }
  
      func restartIfNeeded() {
          guard self.needsRestart else { return }
          self.stop()
          self.start()
      }
  
      func write(data: APPData) {
          self.writer.writeValue(data)
      }
  }
  </code></pre>
  
  <p>Like I said, heavily simplified, but it shows how we start, stop, and restart streams, and how we check whether a stream is healthy.</p>
  
  <p>When the app is backgrounded, we call <code>stop()</code>. When it is foregrounded and we need the stream again, we call <code>start()</code>. And we periodically call <code>restartIfNeeded()</code>, eg. when screens that use the stream come into view.</p>
  
  <p>As I mentioned above, what happens occasionally is that our response handler (<code>rpcResponse</code>) stops getting called when server writes data to the stream. The stream appears to be healthy (server receives the data we write to it, and <code>protocall.state</code> is neither .notStarted nor .finished). But not even the log on the first line of the response handler is executed.</p>
  
  <p>First question: Are we managing the streams correctly, or is our way of stopping and restarting streams prone to errors? If so, what is the correct way of doing something like this?</p>
  
  <p>Second question: How do we debug this? Everything we could think of that we can query for a status tells us that the stream is up and running, but it feels like the objc gRPC library keeps a lot of its mechanics hidden from us. Is there a way to see whether responses from server may <em>do</em> reach us, but fail to trigger our response handler?</p>
  
  <p>Third question: As per the code above, we use the GRXBufferedPipe provided by the library. Its documentation advises against using it in production because it doesn't have a push-back mechanism. To our understanding, the writer is only used to feed data to the gRPC core in a synchronised, one-at-a-time fashion, and since server receives data from us fine, we don't think this is an issue. Are we wrong though? Is the writer also involved in feeding data received from server to our response handler? I.e. if the writer broke due to overload, could that manifest as a problem reading data from the stream, rather than writing to it?</p>
  
  <p>UPDATE: Over a year after asking this, we have finally found a deadlock bug in our server-side code that was causing this behaviour on client-side. The streams appeared to hang because no communication sent by the client was handled by server, and vice-versa, but the streams were actually alive and well. The accepted answer provides good advice for how to manage these bi-directional streams, which I believe is still valuable (it helped us a lot!). But the issue was actually due to a programming error.</p>
  
  <p>Also, for anyone running into this type of issue, it might be worth investigating whether you're experiencing <a href="https://github.com/grpc/grpc/blob/v1.19.0/src/objective-c/NetworkTransitionBehavior.md" rel="nofollow noreferrer">this known issue</a> where a channel gets silently dropped when iOS changes its network. <a href="https://github.com/grpc/grpc/blob/v1.19.0/src/objective-c/NetworkTransitionBehavior.md" rel="nofollow noreferrer">This readme</a> provides instructions for using Apple's CFStream API rather than TCP sockets as a possible fix for that issue.</p>