title: How to continuously send data to kafka?
body: <p>I'm trying to continuously send data (sniffed packets with tshark) to the kafka broker/consumer.</p>
  
  <p>Here are the steps I followed:</p>
  
  <p><strong>1.</strong> Started <em>zookeeper:</em></p>
  
  <pre><code>kafka/bin/zookeeper-server-start.sh ../kafka//config/zookeeper.properties
  </code></pre>
  
  <p><strong>2.</strong> Started <em>kafka server</em>:</p>
  
  <pre><code>kafka/bin/kafka-server-start.sh ../kafka/config/server.properties
  </code></pre>
  
  <p><strong>3.</strong> Started <em>kafka consumer</em>:</p>
  
  <pre><code>kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic \
                                             'my-topic' --from-beginning
  </code></pre>
  
  <p><strong>4.</strong> Wrote the following python script to send sniffed data to consumer:</p>
  
  <pre><code>from kafka import KafkaProducer
  import subprocess
  producer = KafkaProducer(bootstrap_servers='localhost:9092')
  producer.send('my-topic', subprocess.check_output(['tshark','-i','wlan0']))
  </code></pre>
  
  <p>but this is stays on the procuder terminal and outputs:</p>
  
  <pre><code>Capturing on 'wlan0'
  605
  ^C
  </code></pre>
  
  <p>nothing gets transferred to the consumer.</p>
  
  <p>I know I can use <code>pyshark</code> to implement tshark on python:</p>
  
  <pre><code>import pyshark
  capture = pyshark.LiveCapture(interface='eth0')
  capture.sniff(timeout=5)
  capture1=capture[0]
  print capture1
  </code></pre>
  
  <p>But I don't know how to continuously send the captured packets from the producer to the consumer. Any advice?</p>
  
  <p>Thank you!</p>