<p>I am attempting to get fluent-bit multiline logs working for my apps running on kubernetes. These are java springboot applications. Is there a way to send the logs through the docker parser (so that they are formatted in json), and then use a custom multiline parser to concatenate the logs that are broken up by <code>\n</code>? I am attempting to use the date format as the start of the multiline parser, and am starting off with <code>This.*</code> to capture the next line that should be included in the first log, rather than broken up into different events. I am also using the default kubernetes filter for the logs. Here are the config files with the input, filter, and parsers:</p>
<pre><code>    [INPUT]
        Name                tail
        Tag                 application.*
        Path                /var/log/containers/test-container*
        DB                  /var/fluent-bit/state/flb_container.db
        Mem_Buf_Limit       50MB
        Skip_Long_Lines     On
        Refresh_Interval    10
        Rotate_Wait         30
        storage.type        filesystem
        Read_from_Head      On

    [FILTER]
        Name                  multiline
        Match                 application.*
        multiline.key_content log
        multiline.parser      docker, multi_docker_java

    [MULTILINE_PARSER]
        name                      multi_docker_java
        type                      regex
        flush_timeout             1000
        key_content               log
        rule      &quot;start_state&quot;   &quot;/\d{4}-\d{1,2}-\d{1,2}T\d{2}:\d{2}:\d{2}.\d{3}Z.*&quot;           &quot;cont&quot;
        rule      &quot;cont&quot;          &quot;/This.*&quot;                                                      &quot;cont&quot;

    [FILTER]
        Name                kubernetes
        Match               application.*
        Kube_URL            https://kubernetes.default.svc:443
        Kube_Tag_Prefix     application.var.log.containers.
        Merge_Log           On
        Merge_Log_Key       log_processed
        K8S-Logging.Parser  On
        K8S-Logging.Exclude Off
        Labels              Off
        Annotations         Off
        Use_Kubelet         On
        Kubelet_Port        10250
        Buffer_Size         0
</code></pre>
<p>The log I am testing with is below (kubernetes info is being included but not shown here):</p>
<pre><code>{
    &quot;log&quot;: &quot;2023-05-05T13:46:47.628Z  INFO 1 --- [nio-8080-exec-9] c.e.sampleApp.controller.MainController  : This is line one of the log message.\n&quot;,
    &quot;stream&quot;: &quot;stdout&quot;,
    &quot;kubernetes&quot;: {
        &quot;pod_name&quot;: &quot;&quot;,
        &quot;namespace_name&quot;: &quot;&quot;,
        &quot;pod_id&quot;: &quot;&quot;,
        &quot;host&quot;: &quot;&quot;,
        &quot;container_name&quot;: &quot;&quot;,
        &quot;docker_id&quot;: &quot;&quot;,
        &quot;container_hash&quot;: &quot;&quot;,
        &quot;container_image&quot;: &quot;&quot;
    }
}


{
    &quot;log&quot;: &quot;This is line two of the log message.\n&quot;,
    &quot;stream&quot;: &quot;stdout&quot;,
    &quot;kubernetes&quot;: {
        &quot;pod_name&quot;: &quot;&quot;,
        &quot;namespace_name&quot;: &quot;&quot;,
        &quot;pod_id&quot;: &quot;&quot;,
        &quot;host&quot;: &quot;&quot;,
        &quot;container_name&quot;: &quot;&quot;,
        &quot;docker_id&quot;: &quot;&quot;,
        &quot;container_hash&quot;: &quot;&quot;,
        &quot;container_image&quot;: &quot;&quot;
    }
}


{
    &quot;log&quot;: &quot;Here I am!\n&quot;,
    &quot;stream&quot;: &quot;stdout&quot;,
    &quot;kubernetes&quot;: {
        &quot;pod_name&quot;: &quot;&quot;,
        &quot;namespace_name&quot;: &quot;&quot;,
        &quot;pod_id&quot;: &quot;&quot;,
        &quot;host&quot;: &quot;&quot;,
        &quot;container_name&quot;: &quot;&quot;,
        &quot;docker_id&quot;: &quot;&quot;,
        &quot;container_hash&quot;: &quot;&quot;,
        &quot;container_image&quot;: &quot;&quot;
    }
}
</code></pre>

mmiara
<p>You are confusing how the parsers actually work in Fluentbit.</p>
<p>You will need to have your <code>conf</code> in 1 file but your <code>parsers</code> in another eg.</p>
<pre><code># fluentbit.conf
[SERVICE]
    Parsers_File /path/to/parsers.conf

...

[FILTER]
    name                  multiline
    match                 *
    multiline.key_content log
    multiline.parser      go, multiline-regex-test

...
</code></pre>
<p>and then</p>
<pre><code># /path/to/parsers.conf

[MULTILINE_PARSER]
    name          multiline-regex-test
    type          regex
    flush_timeout 1000
    #
    # Regex rules for multiline parsing
    # ---------------------------------
    #
    # configuration hints:
    #
    #  - first state always has the name: start_state
    #  - every field in the rule must be inside double quotes
    #
    # rules |   state name  | regex pattern                  | next state
    # ------|---------------|--------------------------------------------
    rule      &quot;start_state&quot;   &quot;/(Dec \d+ \d+\:\d+\:\d+)(.*)/&quot;  &quot;cont&quot;
    rule      &quot;cont&quot;          &quot;/^\s+at.*/&quot;                     &quot;cont&quot;
   
</code></pre>
<p>See <a href="https://docs.fluentbit.io/manual/pipeline/filters/multiline-stacktrace" rel="nofollow noreferrer">multiline-stacktrace Docs</a>.</p>
<p><strong>Note</strong>: The screenshot below shows tabs for each configuration file required</p>
<p><a href="https://i.stack.imgur.com/fGRa5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/fGRa5.png" alt="Screenshot of the Fluentbit multiline-stacktrace docs" /></a></p>
<hr />
<p>Looking at your actual parser. If you are trying to parse the following logs:</p>
<pre><code>2023-05-05T13:46:47.628Z  INFO 1 --- [nio-8080-exec-9] c.e.sampleApp.controller.MainController  : This is line one of the log message
something else
hello
2023-05-05T13:47:00.132Z  INFO 1 --- [nio-8080-exec-9] c.e.sampleApp.controller.MainController  : This is line one of the log message.
</code></pre>
<p>Once way you can achieve this is using a positive and negitive search pattern for your multiline parser eg.</p>
<pre><code>[MULTILINE_PARSER]
        name                      multi_docker_java
        type                      regex
        flush_timeout             1000
        key_content               log
        rule      &quot;start_state&quot;   &quot;/\d{4}-\d{1,2}-\d{1,2}T\d{2}:\d{2}:\d{2}.\d{3}Z.*/&quot;  &quot;cont&quot;
        rule      &quot;cont&quot;          &quot;/^(?!\d{4}-\d{1,2}-\d{1,2}T).+/&quot;  &quot;cont&quot;
</code></pre>
