title: CloudWatch agent can not keep up with logging
body: <p>I have an EC2 machine running a java application with the following logback configuration:</p>
  
  <pre class="lang-xml prettyprint-override"><code>&lt;appender name="ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender"&gt;
    &lt;file&gt;/var/log/my-application/application.log&lt;/file&gt;
    &lt;rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"&gt;
      &lt;fileNamePattern&gt;/var/log/my-application/application-%i.log.gz&lt;/fileNamePattern&gt;
      &lt;minIndex&gt;1&lt;/minIndex&gt;
      &lt;maxIndex&gt;10&lt;/maxIndex&gt;
    &lt;/rollingPolicy&gt;
    &lt;triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy"&gt;
      &lt;maxFileSize&gt;800MB&lt;/maxFileSize&gt;
    &lt;/triggeringPolicy&gt;
    &lt;encoder&gt;
      &lt;pattern&gt;%level --- [%thread] %logger{36} : %msg%n
      &lt;/pattern&gt;
    &lt;/encoder&gt;
  &lt;/appender&gt;
  </code></pre>
  
  <p>If you are not familiar with logback, this configuration boils down to my application appending logging statements to the <code>/var/log/my-application/application.log</code> file until it reaches 800MB. Once that size is reached, the current logfile is renamed and compressed to <code>application-1.log.gz</code> and a new <code>application.log</code> file is created to start appending new logging there.</p>
  
  <p>This is my unified CloudWatch agent configuration:</p>
  
  <pre><code>{
    "agent": {
      "run_as_user": "ec2-user",
      "region": "${AWS::Region}",
      "debug": true
    },
    "logs": {
      "logs_collected": {
        "files": {
          "collect_list": [
            {
              "file_path": "/var/log/my-application/application*",
              "log_group_name": "my-application-log-group",
              "log_stream_name": "my-application-log-{instance_id}"
            }
          ]
        }
      }
    }
  }
  </code></pre>
  
  <p>This setup works, but once my application generates a lot of logging (10,000,000 logging statements in 5 minutes), only ~6,000,000 of these end up in CloudWatch. The size of each logging line is estimated to be 228 bytes.</p>
  
  <p>Does anyone know how I can make sure that the CloudWatch agent is able to send all logging to CloudWatch?</p>