<p>I have followed <a href="https://aws.amazon.com/blogs/containers/fluent-bit-for-amazon-eks-on-aws-fargate-is-here/" rel="noreferrer">this guide</a> to configure Fluent Bit and Cloudwatch on my EKS cluster, but currently all of the logs go to one log group. I tried to follow a separate tutorial that used a kubernetes plugin for Fluent Bit to tag the services before the reached the [OUTPUT] configuration. This caused issues because Fargate EKS currently does not handle Fluent Bit [INPUT] configurations as per the <a href="https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html" rel="noreferrer">bottom of this doc</a>.</p>
<p>Has anyone encountered this before? I'd like to split the logs up into separate services.</p>
<p>Here is my current YAML file .. I added the parser and filter to see if I could gain any additional information to work with over on Cloudwatch.</p>
<pre><code>kind: Namespace
apiVersion: v1
metadata:
  name: aws-observability
  labels:
    aws-observability: enabled
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: aws-logging
  namespace: aws-observability
data:
  parsers.conf: |
    [PARSER]
        Name docker
        Format json
        Time_Key time
        Time_Format %Y-%m-%dT%H:%M:%S.%L
        Time_Keep On
        
  filters.conf: |
    [FILTER]
        Name kubernetes
        Match kube.*
        Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
        # Kube_Tag_Prefix kube.var.log.containers.
        Kube_URL https://kubernetes.default.svc:443
        Merge_Log On
        Merge_Log_Key log_processed
        Use_Kubelet true
        Buffer_Size 0
        Dummy_Meta true
  
  output.conf: |
    [OUTPUT]
        Name cloudwatch_logs
        Match   *
        region us-east-1
        log_group_name fluent-bit-cloudwatch2
        log_stream_prefix from-fluent-bit-
        auto_create_group On
</code></pre>

Frederick Haug
<p>So I found out that it is actually simple to do this.</p>
<p>The default tag of input on fluent bit contains the name of the service you are logging from, so you can actually stack multiple [OUTPUT] blocks each using the wildcard operator around the name of your service <em></em>. That was all I had to do to get the streams to get sent to different log groups. Here is my YAML for reference.</p>
<pre><code>kind: Namespace
apiVersion: v1
metadata:
  name: aws-observability
  labels:
    aws-observability: enabled
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: aws-logging
  namespace: aws-observability
data:
  output.conf: |
    [OUTPUT]
        Name cloudwatch_logs
        Match   *logger*
        region us-east-1
        log_group_name logger-fluent-bit-cloudwatch
        log_stream_prefix from-fluent-bit-
        auto_create_group On
        
    [OUTPUT]
        Name cloudwatch_logs
        Match   *alb*
        region us-east-1
        log_group_name alb-fluent-bit-cloudwatch
        log_stream_prefix from-fluent-bit-
        auto_create_group On
</code></pre>
