title: Flume agent - using tail -F
body: <p>I am new to Apache Flume.
  I have created my agent like:</p>
  
  <pre><code>agent.sources=exec-source
  agent.sinks=hdfs-sink
  agent.channels=ch1
  
  agent.sources.exec-source.type=exec
  agent.sources.exec-source.command=tail -F /var/log/apache2/access.log
  
  agent.sinks.hdfs-sink.type=hdfs
  agent.sinks.hdfs-sink.hdfs.path=hdfs://&lt;Host-Name of name node&gt;/
  agent.sinks.hdfs-sink.hdfs.filePrefix=apacheaccess
  
  agent.channels.ch1.type=memory
  agent.channels.ch1.capacity=1000
  
  agent.sources.exec-source.channels=ch1
  agent.sinks.hdfs-sink.channel=ch1
  </code></pre>
  
  <p>And the ouput i am getting is :</p>
  
  <pre><code>13/01/22 17:31:48 INFO lifecycle.LifecycleSupervisor: Starting lifecycle supervisor 1
  13/01/22 17:31:48 INFO node.FlumeNode: Flume node starting - agent
  13/01/22 17:31:48 INFO properties.PropertiesFileConfigurationProvider: Configuration provider starting
  13/01/22 17:31:48 INFO nodemanager.DefaultLogicalNodeManager: Node manager starting
  13/01/22 17:31:48 INFO lifecycle.LifecycleSupervisor: Starting lifecycle supervisor 9
  13/01/22 17:31:48 INFO properties.PropertiesFileConfigurationProvider: Reloading configuration file:conf/flume_exec.conf
  13/01/22 17:31:48 INFO conf.FlumeConfiguration: Added sinks: hdfs-sink Agent: agent
  13/01/22 17:31:48 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  13/01/22 17:31:48 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  13/01/22 17:31:48 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  13/01/22 17:31:48 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  13/01/22 17:31:48 INFO conf.FlumeConfiguration: Post-validation flume configuration contains configuration  for agents: [agent]
  13/01/22 17:31:48 INFO properties.PropertiesFileConfigurationProvider: Creating channels
  13/01/22 17:31:48 INFO properties.PropertiesFileConfigurationProvider: created channel ch1
  13/01/22 17:31:48 INFO sink.DefaultSinkFactory: Creating instance of sink: hdfs-sink, type: hdfs
  13/01/22 17:31:48 INFO hdfs.HDFSEventSink: Hadoop Security enabled: false
  13/01/22 17:31:48 INFO nodemanager.DefaultLogicalNodeManager: Starting new configuration:{ sourceRunners:{exec-source=EventDrivenSourceRunner: { source:org.apache.flume.source.ExecSource{name:exec-source,state:IDLE} }} sinkRunners:{hdfs-sink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@715d44 counterGroup:{ name:null counters:{} } }} channels:{ch1=org.apache.flume.channel.MemoryChannel{name: ch1}} }
  13/01/22 17:31:48 INFO nodemanager.DefaultLogicalNodeManager: Starting Channel ch1
  13/01/22 17:31:48 INFO instrumentation.MonitoredCounterGroup: Monitoried counter group for type: CHANNEL, name: ch1, registered successfully.
  13/01/22 17:31:48 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: ch1 started
  13/01/22 17:31:48 INFO nodemanager.DefaultLogicalNodeManager: Starting Sink hdfs-sink
  13/01/22 17:31:48 INFO nodemanager.DefaultLogicalNodeManager: Starting Source exec-source
  13/01/22 17:31:48 INFO source.ExecSource: Exec source starting with command:tail -F /var/log/apache2/access.log
  13/01/22 17:31:48 INFO instrumentation.MonitoredCounterGroup: Monitoried counter group for type: SINK, name: hdfs-sink, registered successfully.
  13/01/22 17:31:48 INFO instrumentation.MonitoredCounterGroup: Component type: SINK, name: hdfs-sink started
  </code></pre>
  
  <p>But it's not writing logs to HDFS. </p>
  
  <p>When I run <code>cat /var/log/apache2/access.log</code> instead of <code>tail –F /var/log/apache2/access.log</code> it runs and my files are created on HDFS.</p>
  
  <p>Please help me out.</p>