title: Proper error handling for kafka batch listener
body: <p>I am working on error handling implementation and have a question.<br />
  Let me explain the problem:<br />
  I receive a batch of messages, I do the DB lookup of each one in the for loop, then I need to collect all the looked up objects in the list and call bulk insert stored proc and bulk update stored proc with this list of objects.</p>
  <p>Now let's assume there is some exception happened during lookup. I want to retry this message. For this case I tried to use <code>DefaultErrorHandler</code>. But there is a problem, according to the doc, when BatchListenerFailedException with element index is thrown it commits the offsets of the records before the index. But as I said I need to execute bulk insert and update after lookup so I don't want to commit offsets before the index, these records weren't inserted/updated to the DB yet.</p>
  <p>Does it mean that my only option is to use <code>RetryingBatchErrorHandler</code> that retries the whole batch every time? Can I somehow continue processing messages that don't produce an error?</p>
  <p>Also if <code>RetryingBatchErrorHandler</code> is the only option, how could I be sure that in case of long backoff period (exponential backoff), kafka won't kill my consumer and won't initiate rebalance?</p>
  <p>My current implementation:</p>
  <pre><code>RetryingBatchErrorHandler retryingBatchErrorHandler =
                  new RetryingBatchErrorHandler(backoff,
                          (consumerRecord, e) -&gt;
                                  log.error(&quot;Backoff attempts exhausted for the record with offset={}, partition={}, value={}, offset committed.&quot;,
                  consumerRecord.offset(), consumerRecord.partition(), consumerRecord.value()));
  
  factory.setBatchErrorHandler(retryingBatchErrorHandler);
  </code></pre>
  <p>UPDATE: see comments in the Artem's answer.<br />
  This is how lookup step can be wrapped into <code>retryTemplate</code></p>
  <pre><code>LookedUpRequest lookedUpRequest = retryTemplate.execute(ctx -&gt; {
     //Lookup step
     return lookup.process(request);
  });
  </code></pre>
  <p>and if it fails then it will throw the exception further for batch error handler where <code>RetryingBatchErrorHandler</code> retry the batch according to its policy</p>