title: Docker filebeat autodiscover not detecting nginx logs
body: <p>On my mac I am running nginx in a docker file and filebeat in a docker file.</p>
  <pre><code>docker run -p 80:80 nginx
  </code></pre>
  <p>The above command successfully runs nginx which I can visit in the browser and the output is printed to the console.</p>
  <p>This is my docker file that I am building and then running:</p>
  <pre><code>FROM docker.elastic.co/beats/filebeat:7.8.0
  COPY filebeat.yml /usr/share/filebeat/filebeat.yml
  USER root
  </code></pre>
  <p>And this is my filebeat.yml referenced in the above:</p>
  <pre><code>  filebeat.autodiscover:
      providers:
        - type: docker
          templates:
            - condition:
                contains:
                  docker.container.image: nginx
              config:
                - type: docker
                  containers.ids:
                      - &quot;${data.docker.container.id}&quot;
                - module: nginx
                  access:
                    enabled: true
                    containers:
                        stream: &quot;stdout&quot;
                  error:
                    enabled: true
                    containers:
                        stream: &quot;stderr&quot;
  output.console:
      pretty: true  
  </code></pre>
  <p>I run the filebeat docker image using:</p>
  <pre><code> sudo docker run -it  -v /var/run/docker.sock:/var/run/docker.sock filebeat
  </code></pre>
  <p>The docker.sock stuff is because of an unable to connect to docker socket error I was getting (some kind of docker user error permissions I presume).</p>
  <p>All of the above gets the filebeat running inside the docker container.</p>
  <pre><code>INFO    [autodiscover]  autodiscover/autodiscover.go:113    Starting autodiscover manager
  </code></pre>
  <p>However when I visit localhost in the browser and trigger the nginx log, the only output is occasional docker system metrics every 30 seconds:</p>
  <pre><code>{&quot;monitoring&quot;: {&quot;metrics&quot;: {&quot;beat&quot;:{&quot;cpu&quot;:{&quot;system&quot;:{&quot;ticks&quot;:40,&quot;time&quot;:{&quot;ms&quot;:43}},&quot;total&quot;:{&quot;ticks&quot;:100,&quot;time&quot;:{&quot;ms&quot;:109},&quot;value&quot;:100},&quot;user&quot;:{&quot;ticks&quot;:60,&quot;time&quot;:{&quot;ms&quot;:66}}},&quot;handles&quot;:{&quot;limit&quot;:{&quot;hard&quot;:1048576,&quot;soft&quot;:1048576},&quot;open&quot;:8},&quot;info&quot;:{&quot;ephemeral_id&quot;:&quot;d3e79d62-6949-4d79-89e8-c595332c18ed&quot;,&quot;uptime&quot;:{&quot;ms&quot;:30054}},&quot;memstats&quot;:{&quot;gc_next&quot;:10249440,&quot;memory_alloc&quot;:5520104,&quot;memory_total&quot;:17591608,&quot;rss&quot;:55390208},&quot;runtime&quot;:{&quot;goroutines&quot;:23}},&quot;filebeat&quot;:{&quot;harvester&quot;:{&quot;open_files&quot;:0,&quot;running&quot;:0}},&quot;libbeat&quot;:{&quot;config&quot;:{&quot;module&quot;:{&quot;running&quot;:0}},&quot;output&quot;:{&quot;type&quot;:&quot;console&quot;},&quot;pipeline&quot;:{&quot;clients&quot;:0,&quot;events&quot;:{&quot;active&quot;:0}}},&quot;registrar&quot;:{&quot;states&quot;:{&quot;current&quot;:0},&quot;writes&quot;:{&quot;success&quot;:1,&quot;total&quot;:1}},&quot;system&quot;:{&quot;cpu&quot;:{&quot;cores&quot;:2},&quot;load&quot;:{&quot;1&quot;:0,&quot;15&quot;:0,&quot;5&quot;:0,&quot;norm&quot;:{&quot;1&quot;:0,&quot;15&quot;:0,&quot;5&quot;:0}}}}}}
  </code></pre>
  <p>The nginx log is not being detected.</p>
  <p>This is a potentially relevant warning I am getting:</p>
  <pre><code>Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled
  </code></pre>
  <p>Edit: I think this is the crux of the issue, the harvester is not finding any logs:</p>
  <pre><code>filebeat&quot;:{&quot;harvester&quot;:{&quot;open_files&quot;:0,&quot;running&quot;:0}}
  </code></pre>
  <p>Edit2: The root cause of the problem seems to be that on the Mac the terminal can't access the log files: /var/lib/docker/containers/${data.docker.container.id}/*.log as this aren't where they are stored on the Mac... so it seems running filebeat on mac is not supported this way between dockers</p>