title: Elasticsearch Docker: flood stage disk watermark [95%] exceeded
body: <p>I have a problem with my Elasticsearch nodes running in a docker environment. I'm starting them up with docker-compose and after a few minutes they tell me:
  <em>flood stage disk watermark [95%] exceeded</em></p>
  <p>I'm running it on a cluster with rather high storage capacity and I already tried to increase the watermark settings in the elasticsearch.yml file, but I still get the error. Maybe it has to do with the size of the docker containers.</p>
  <p>Does anyone know what could be the problem? Any help is much appreciated.</p>
  <p>The docker-compose.yml for reference:</p>
  <pre><code>version: '3.4'
  services:
    es01:
      image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1
      container_name: es01
      environment:
        #- discovery.type=single-node
        - node.name=es01
        - cluster.name=es-docker-cluster
        - discovery.seed_hosts=es02,es03
        - cluster.initial_master_nodes=es01,es02,es03
        - bootstrap.memory_lock=true
        - xpack.security.enabled=false
        - &quot;ES_JAVA_OPTS=-Xms512m -Xmx512m&quot;
      ulimits:
        memlock:
          soft: -1
          hard: -1
      volumes:
        - data01:/usr/share/elasticsearch/data
      ports:
        - 9200:9200
      networks:
        - elastic
  
    es02:
      image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1
      container_name: es02
      environment:
        - node.name=es02
        - cluster.name=es-docker-cluster
        - discovery.seed_hosts=es01,es03
        - cluster.initial_master_nodes=es01,es02,es03
        - bootstrap.memory_lock=true
        - &quot;ES_JAVA_OPTS=-Xms512m -Xmx512m&quot;
      ulimits:
        memlock:
          soft: -1
          hard: -1
      volumes:
        - data02:/usr/share/elasticsearch/data
      networks:
        - elastic
  
    es03:
      image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1
      container_name: es03
      environment:
        - node.name=es03
        - cluster.name=es-docker-cluster
        - discovery.seed_hosts=es01,es02
        - cluster.initial_master_nodes=es01,es02,es03
        - bootstrap.memory_lock=true
        - &quot;ES_JAVA_OPTS=-Xms512m -Xmx512m&quot;
      ulimits:
        memlock:
          soft: -1
          hard: -1
      volumes:
        - data03:/usr/share/elasticsearch/data
      networks:
        - elastic
  
    kib01:
      image: docker.elastic.co/kibana/kibana:7.8.1
      container_name: kib01
      depends_on:
        - es01
        - es02
        - es03
      ports:
        - 5601:5601
      environment:
        ELASTICSEARCH_URL: http://es01:9200
        ELASTICSEARCH_HOSTS: http://es01:9200
      networks:
        - elastic
  
    client:
      image: appropriate/curl:latest
      depends_on:
        - es01
        - es02
        - es03
      networks:
        - elastic
      command: sh -c &quot;curl es01:9200 &amp;&amp; curl kib01:5601&quot;
  
    dash_app:
      build: .
      ports:
      - 0.0.0.0:8050:8050
      depends_on:
        - es01
        - es02
        - es03
        - kib01
      networks:
        - elastic
  
  #mapping:
  #  image: appropriate/curl:latest
  #  depends_on:
  #    - es01
  #    - es02
  #    - es03
  #  networks:
  #    - elastic
  #  command:    &quot;curl -v -XPUT 'es01:9200/urteile' -H 'Content-Type: application/json' -d '
  #       {
  #         'mappings': {
  #           'properties': {
  #             'date': {
  #               'type': 'date'
  #             }
  #           }
  #         }
  #       }
  #    '&quot;
  
    #web:
     # build: .
     # ports:
      #  - 8000:8000
      #depends_on:
      #  - es01
      #  - es02
      #  - es03
      #networks:
      #  - elastic
  
  
  volumes:
    data01:
      driver: local
    data02:
      driver: local
    data03:
      driver: local
  
  networks:
    elastic:
      driver: bridge
  </code></pre>
  <p>And docker info:</p>
  <pre><code>Server:
   Containers: 6
    Running: 3
    Paused: 0
    Stopped: 3
   Images: 185
   Server Version: 19.03.12
   Storage Driver: overlay
    Backing Filesystem: extfs
    Supports d_type: true
   Logging Driver: json-file
   Cgroup Driver: cgroupfs
   Plugins:
    Volume: local
    Network: bridge host ipvlan macvlan null overlay
    Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
   Swarm: inactive
   Runtimes: runc nvidia
   Default Runtime: runc
   Init Binary: docker-init
   containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
   runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
   init version: fec3683
   Security Options:
    seccomp
     Profile: default
   Kernel Version: 5.7.2-kd-cluster
   Operating System: Debian GNU/Linux 9 (stretch)
   OSType: linux
   Architecture: x86_64
   CPUs: 32
   Total Memory: 125.8GiB
   Name: dpl01
   ID: KBGO:2E6L:NIHR:UQAL:K5CN:XWBI:R7TK:WWZF:MZBT:BCHE:HUQW:UKKM
   Docker Root Dir: /data/docker
   Debug Mode: false
   Registry: https://index.docker.io/v1/
   Labels:
   Experimental: false
   Insecure Registries:
    127.0.0.0/8
   Live Restore Enabled: false
  </code></pre>