title: Kinesis agent not sending .log files through firehose
body: <p>I've setup a Kinesis firehose and the installed the Kinesis agent as described in the <a href="https://d1.awsstatic.com/Projects/P4113850/aws-projects_build-log-analytics-solution-on-aws.pdf" rel="nofollow noreferrer">AWS docs</a>.  I can get test data through to the S3 bucket, but the Kinesis agent won't send any .log files through.  I suspect a problem connecting the agent to the firehose.  </p>
  
  <p>My <code>/etc/aws-kinesis/agent.json</code> file is below.  I've also tried with the <code>"firehose.endpoint"</code> without the <code>https://</code> but I still can't get any data through.</p>
  
  <p>I've verified that the aws-kinesis-agent service is running.</p>
  
  <p>I'm not using the kinesis.endpoint/kinesisStream, but I've left the flow in the agent.json file.  Could this be a problem?</p>
  
  <p>What am I missing?</p>
  
  <pre><code>{
    "cloudwatch.emitMetrics": true,
    "kinesis.endpoint": "",
    "firehose.endpoint": "https://firehose.us-west-2.amazonaws.com",
  
    "flows": [
      {
        "filePattern": "/home/ec2-user/src/Fake-Apache-Log-Generator/*.log*",
        "kinesisStream": "yourkinesisstream",
        "partitionKeyOption": "RANDOM"
      },
      {
        "filePattern": "/home/ec2-user/src/Fake-Apache-Log-Generator/*.log*",
        "deliveryStream": "apachelogfilesdeliverystream"
      }
    ]
  }
  </code></pre>
  
  <p>EDIT:</p>
  
  <p>The log file at <code>/var/log/aws-kinesis-agent/aws-kinesis-agent.log</code> showed 0 records being parsed.  The log message led me to <a href="https://stackoverflow.com/questions/44706843/kinesis-agent-not-parsing-the-file">this post</a>, and I made the recommended fixes.  In addition I had to remove the flow for kinesis from the /etc/aws-kinesis/agent.json file to avoid an Exception that showed up in the log files.</p>
  
  <p>Botton line is that the aws-kinesis-agent can't read files from /home/ec2-user/ or its subdirectories, and you have to fix up the agent.json file.</p>