<p>I tried to implement EFK stack into our current environment with Fluentd.</p>
<p>There is a configuration I have is</p>
<pre><code>    &lt;source&gt;
      ...
      path /var/log/containers/*.log
      ...
    &lt;/source&gt;
</code></pre>
<p>which supposed to grab all the standard output of all pods on the worker node. But when I ssh into that node and take inspect of the format of the output, I found the standard output log with multiple line was broken into different log entries, for example:</p>
<pre><code>{&quot;log&quot;:&quot;Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.011079366Z&quot;}
{&quot;log&quot;:&quot;    at ServerResponse.setHeader (_http_outgoing.js:530:11)\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.011130167Z&quot;}
{&quot;log&quot;:&quot;    at sendEtagResponse (/app/node_modules/next/dist/next-server/server/send-payload.js:6:12)\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.011145267Z&quot;}
{&quot;log&quot;:&quot;    at sendData (/app/node_modules/next/dist/next-server/server/api-utils.js:32:479)\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.011229869Z&quot;}
{&quot;log&quot;:&quot;    at ServerResponse.apiRes.send (/app/node_modules/next/dist/next-server/server/api-utils.js:6:250)\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.011242369Z&quot;}
{&quot;log&quot;:&quot;    at exports.modules.3626.__webpack_exports__.default (/app/.next/server/pages/api/users/[id]/organizations.js:350:34)\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.011252769Z&quot;}
{&quot;log&quot;:&quot;    at runMicrotasks (\u003canonymous\u003e)\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.011264269Z&quot;}
{&quot;log&quot;:&quot;    at processTicksAndRejections (internal/process/task_queues.js:97:5)\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.011275069Z&quot;}
{&quot;log&quot;:&quot;    at async apiResolver (/app/node_modules/next/dist/next-server/server/api-utils.js:8:1)\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.011284869Z&quot;}
{&quot;log&quot;:&quot;    at async Server.handleApiRequest (/app/node_modules/next/dist/next-server/server/next-server.js:66:462)\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.01129647Z&quot;}
{&quot;log&quot;:&quot;    at async Object.fn (/app/node_modules/next/dist/next-server/server/next-server.js:58:580) {\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.01130717Z&quot;}
{&quot;log&quot;:&quot;  code: 'ERR_HTTP_HEADERS_SENT'\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.01131707Z&quot;}
{&quot;log&quot;:&quot;}\n&quot;,&quot;stream&quot;:&quot;stderr&quot;,&quot;time&quot;:&quot;2021-10-29T18:26:26.01132747Z&quot;}
</code></pre>
<p>then all of those lines are broken into separate log pieces and got transported into Elasticsearch, is there a way that we can make those mutliple lines into one single piece?</p>
<p>Appreciated to the help of any kinds.</p>

pyy
<p>You can use the multiline plugin to achieve that.</p>
<p>This provides the <code>format_firstline </code> parameter where you can use a regex.</p>
<p>You didn't share much of your regular log output, so here is an example for a timestamp with format <code>YYYY-MM-dd HH:mm:ss,zzz</code></p>
<pre><code>firstline: /\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2},\d{3}/
</code></pre>
<p>You could also try to match on the beginning of the line e.g. <code>^(Info|Error)</code>.</p>
<p>That way fluentd will recognize multiple lines as one log entry.</p>
<p>Check <a href="https://docs.fluentd.org/parser/multiline" rel="nofollow noreferrer">docs</a> for more info about configuring the plugin.</p>
