title: How to use two Kerberos keytabs (for Kafka and Hadoop HDFS) from a Flink job on a Flink standalone cluster?
body: <h1>Question</h1>
  
  <p>On a Flink standalone cluster, running on a server, I am developing a Flink streaming job in Scala. The job consumes data from more than 1 Kafka topics, (do some formatting,) and write results to HDFS.</p>
  
  <p>One of the Kafka topic, and HDFS, they both require separate Kerberos authentications (because they belong to completely different clusters).</p>
  
  <p>My questions are:</p>
  
  <ol>
  <li>Is it possible (if yes, how?) to use two Kerberos keytabs (one for Kafka, the other for HDFS) from a Flink job on a Flink cluster, running on a server? (so the Flink job can consume from Kafka topic and write to HDFS at the same time)</li>
  <li>If not possible, what is a reasonable workaround, for the Kafka-Flink-HDFS data streaming when Kafka and HDFS are both Kerberos protected?</li>
  </ol>
  
  <h1>Note</h1>
  
  <ul>
  <li>I am quite new to the most of the technologies mentioned here.</li>
  <li>The Flink job can write to HDFS if it doesn't need to consume the Kerberos-requiring topic. In this case, I specified the information of HDFS to<code>security.kerberos.login.keytab</code> and <code>security.kerberos.login.principal</code> in <code>flink-conf.yaml</code></li>
  <li>I am using <a href="https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/connectors/filesystem_sink.html" rel="noreferrer">HDFS Connector provided from Flink</a> to write to HDFS.</li>
  <li><p>Manually switching the Kerberos authentication between the two principals was possible. In [realm] section in <code>krb5.conf</code> file, I specified two realms, one for Kafka, the other for HDFS.</p>
  
  <p><code>kinit -kt path/to/hdfs.keytab   [principal: xxx@XXX.XXX...]</code></p>
  
  <p><code>kinit -kt path/to/kafka.keytab  [principal: yyy@YYY.YYY...]</code></p></li>
  </ul>
  
  <h1>Environment</h1>
  
  <ul>
  <li>Flink (v1.4.2) <a href="https://ci.apache.org/projects/flink/flink-docs-stable/" rel="noreferrer">https://ci.apache.org/projects/flink/flink-docs-stable/</a></li>
  <li>Kafka client (v0.10.X)</li>
  <li>HDFS (Hadoop cluster HDP 2.6.X)</li>
  </ul>
  
  <p>Thanks for your attentions and feedbacks!</p>