<p>I'm developing an application with microservice architecture using jhipster. I can run my services in dev mode even though i get this warning but when I run it on kubernetes cluster after i get this warning pod restarts itself over and over on loop. I have 4 microservies and a gateway. All the same. 
Thank you in advance.</p>

<p>This is the warning: </p>

<pre><code>2020-05-06 06:06:51.415 WARN 1 --- [scoveryClient-1] c.netflix.discovery.TimedSupervisorTask : task supervisor timed out 
java.util.concurrent.TimeoutException: null 
at java.base/java.util.concurrent.FutureTask.get(Unknown Source) 
at com.netflix.discovery.TimedSupervisorTask.run(TimedSupervisorTask.java:68) 
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) 
at java.base/java.util.concurrent.FutureTask.run(Unknown Source) 
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) 
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 
at java.base/java.lang.Thread.run(Unknown Source)
</code></pre>

<p>application.yml file: </p>

<pre><code># ===================================================================
# Spring Boot configuration.
#
# This configuration will be overridden by the Spring profile you use,
# for example application-dev.yml if you use the "dev" profile.
#
# More information on profiles: https://www.jhipster.tech/profiles/
# More information on configuration properties: https://www.jhipster.tech/common-application-properties/
# ===================================================================

# ===================================================================
# Standard Spring Boot properties.
# Full reference is available at:
# http://docs.spring.io/spring-boot/docs/current/reference/html/common-application-properties.html
# ===================================================================

eureka:
  client:
    enabled: true
    healthcheck:
      enabled: true
    fetch-registry: true
    register-with-eureka: true
    instance-info-replication-interval-seconds: 10
    registry-fetch-interval-seconds: 10
  instance:
    appname: derinconfiguration
    instanceId: derinconfiguration:${spring.application.instance-id:${random.value}}
    lease-renewal-interval-in-seconds: 5
    lease-expiration-duration-in-seconds: 10
    status-page-url-path: ${management.endpoints.web.base-path}/info
    health-check-url-path: ${management.endpoints.web.base-path}/health
    metadata-map:
      zone: primary # This is needed for the load balancer
      profile: ${spring.profiles.active}
      version: #project.version#
      git-version: ${git.commit.id.describe:}
      git-commit: ${git.commit.id.abbrev:}
      git-branch: ${git.branch:}


# See https://github.com/Netflix/Hystrix/wiki/Configuration
hystrix:
  command:
    default:
      execution:
        isolation:
          thread:
            timeoutInMilliseconds: 30000
ribbon:
  ReadTimeout: 60000
  connection-timeout: 3000
  eureka:
    enabled: true
zuul:
  ignoredServices: '*'
  host:
    time-to-live: -1
    connect-timeout-millis: 5000
    max-per-route-connections: 10000
    max-total-connections: 5000
    socket-timeout-millis: 60000
  semaphore:
    max-semaphores: 500

management:
  endpoints:
    web:
      base-path: /management
      exposure:
        include: ['configprops', 'env', 'health', 'info', 'jhimetrics', 'logfile', 'loggers', 'prometheus', 'threaddump']
  endpoint:
    health:
      show-details: when_authorized
      roles: 'ROLE_ADMIN'
    jhimetrics:
      enabled: true
  info:
    git:
      mode: full
  health:
    mail:
      enabled: false # When using the MailService, configure an SMTP server and set this to true
  metrics:
    export:
      # Prometheus is the default metrics backend
      prometheus:
        enabled: true
        step: 60
    enable:
      http: true
      jvm: true
      logback: true
      process: true
      system: true
    distribution:
      percentiles-histogram:
        all: true
      percentiles:
        all: 0, 0.5, 0.75, 0.95, 0.99, 1.0
    tags:
      application: ${spring.application.name}
    web:
      server:
        request:
          autotime:
            enabled: true

spring:
  autoconfigure:
    exclude: org.springframework.boot.actuate.autoconfigure.metrics.jdbc.DataSourcePoolMetricsAutoConfiguration
  application:
    name: derinconfiguration
  jmx:
    enabled: false
  data:
    jpa:
      repositories:
        bootstrap-mode: deferred
  jpa:
    open-in-view: false
    properties:
      hibernate.jdbc.time_zone: UTC
      hibernate.id.new_generator_mappings: true
      hibernate.connection.provider_disables_autocommit: true
      hibernate.cache.use_second_level_cache: true
      hibernate.cache.use_query_cache: false
      hibernate.generate_statistics: false
      # modify batch size as necessary
      hibernate.jdbc.batch_size: 25
      hibernate.order_inserts: true
      hibernate.order_updates: true
      hibernate.query.fail_on_pagination_over_collection_fetch: true
      hibernate.query.in_clause_parameter_padding: true
      hibernate.cache.region.factory_class: com.hazelcast.hibernate.HazelcastCacheRegionFactory
      hibernate.cache.use_minimal_puts: true
      hibernate.cache.hazelcast.instance_name: derinconfiguration
      hibernate.cache.hazelcast.use_lite_member: true
    hibernate:
      ddl-auto: none
      naming:
        physical-strategy: org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
        implicit-strategy: org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
  messages:
    basename: i18n/messages
  main:
    allow-bean-definition-overriding: true
  mvc:
    favicon:
      enabled: false
  task:
    execution:
      thread-name-prefix: derinconfiguration-task-
      pool:
        core-size: 2
        max-size: 50
        queue-capacity: 10000
    scheduling:
      thread-name-prefix: derinconfiguration-scheduling-
      pool:
        size: 2
  thymeleaf:
    mode: HTML
  output:
    ansi:
      console-available: true
security:
  oauth2:
    client:
      access-token-uri: http://localhost:9080/auth/realms/jhipster/protocol/openid-connect/token
      user-authorization-uri: http://localhost:9080/auth/realms/jhipster/protocol/openid-connect/auth
      client-id: web_app
      client-secret: web_app
      scope: openid profile email
    resource:
      filter-order: 3
      user-info-uri: http://localhost:9080/auth/realms/jhipster/protocol/openid-connect/userinfo

server:
  servlet:
    session:
      cookie:
        http-only: true

# Properties to be exposed on the /info management endpoint
info:
  # Comma separated list of profiles that will trigger the ribbon to show
  display-ribbon-on-profiles: 'dev'

# ===================================================================
# JHipster specific properties
#
# Full reference is available at: https://www.jhipster.tech/common-application-properties/
# ===================================================================

jhipster:
  clientApp:
    name: 'derinconfigurationApp'
  # By default CORS is disabled. Uncomment to enable.
  # cors:
  #     allowed-origins: "*"
  #     allowed-methods: "*"
  #     allowed-headers: "*"
  #     exposed-headers: "Authorization,Link,X-Total-Count"
  #     allow-credentials: true
  #     max-age: 1800
  mail:
    from: derinconfiguration@localhost
  swagger:
    default-include-pattern: /api/.*
    title: derinconfiguration API
    description: derinconfiguration API documentation
    version: 0.0.1
    terms-of-service-url:
    contact-name:
    contact-url:
    contact-email:
    license:
    license-url:
kafka:
  bootstrap-servers: localhost:9092
  consumer:
    key.deserializer: org.apache.kafka.common.serialization.StringDeserializer
    value.deserializer: org.apache.kafka.common.serialization.StringDeserializer
    group.id: derinconfiguration
    auto.offset.reset: earliest
  producer:
    key.serializer: org.apache.kafka.common.serialization.StringSerializer
    value.serializer: org.apache.kafka.common.serialization.StringSerializer
# ===================================================================
# Application specific properties
# Add your own application properties here, see the ApplicationProperties class
# to have type-safe configuration, like in the JHipsterProperties above
#
# More documentation is available at:
# https://www.jhipster.tech/common-application-properties/
# ===================================================================

# application:
</code></pre>

<p>Kubernetes configuration: </p>

<pre><code>apiVersion: apps/v1
kind: Deployment
metadata:
  name: derinconfiguration
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: derinconfiguration
      version: 'v1'
  template:
    metadata:
      labels:
        app: derinconfiguration
        version: 'v1'
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - podAffinityTerm:
                labelSelector:
                  matchExpressions:
                    - key: app
                      operator: In
                      values:
                        - derinconfiguration
                topologyKey: kubernetes.io/hostname
              weight: 100
      initContainers:
        - name: init-ds
          image: busybox:latest
          command:
            - '/bin/sh'
            - '-c'
            - |
              while true
              do
                rt=$(nc -z -w 1 192.168.1.156 5432)
                if [ $? -eq 0 ]; then
                  echo "DB is UP"
                  break
                fi
                echo "DB is not yet reachable;sleep for 10s before retry"
                sleep 10
              done
      containers:
        - name: derinconfiguration-app
          image: 192.168.1.150:5000/derin/derinconfiguration
          env:
            - name: SPRING_PROFILES_ACTIVE
              value: prod
            - name: SPRING_CLOUD_CONFIG_URI
              value: http://admin:${jhipster.registry.password}@jhipster-registry.default.svc.cluster.local:8761/config
            - name: JHIPSTER_REGISTRY_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: registry-secret
                  key: registry-admin-password
            - name: EUREKA_CLIENT_SERVICE_URL_DEFAULTZONE
              value: http://admin:${jhipster.registry.password}@jhipster-registry.default.svc.cluster.local:8761/eureka/
            - name: SPRING_DATASOURCE_URL
              value: jdbc:postgresql://192.168.1.156:5432/derinfw
            - name: SPRING_DATASOURCE_USERNAME
              value: admin
            - name: SPRING_DATASOURCE_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgresql-secret
                  key: postgresql-admin-password
            - name: KAFKA_CONSUMER_KEY_DESERIALIZER
              value: 'org.apache.kafka.common.serialization.StringDeserializer'
            - name: KAFKA_CONSUMER_VALUE_DESERIALIZER
              value: 'org.apache.kafka.common.serialization.StringDeserializer'
            - name: KAFKA_CONSUMER_BOOTSTRAP_SERVERS
              value: 'jhipster-kafka.default.svc.cluster.local:9092'
            - name: KAFKA_CONSUMER_GROUP_ID
              value: 'derinconfiguration'
            - name: KAFKA_CONSUMER_AUTO_OFFSET_RESET
              value: 'earliest'
            - name: KAFKA_PRODUCER_BOOTSTRAP_SERVERS
              value: 'jhipster-kafka.default.svc.cluster.local:9092'
            - name: KAFKA_PRODUCER_KEY_DESERIALIZER
              value: 'org.apache.kafka.common.serialization.StringDeserializer'
            - name: KAFKA_PRODUCER_VALUE_DESERIALIZER
              value: 'org.apache.kafka.common.serialization.StringDeserializer'
            - name: JHIPSTER_METRICS_LOGS_ENABLED
              value: 'true'
            - name: JHIPSTER_LOGGING_LOGSTASH_ENABLED
              value: 'true'
            - name: JHIPSTER_LOGGING_LOGSTASH_HOST
              value: jhipster-logstash
            - name: SPRING_ZIPKIN_ENABLED
              value: 'true'
            - name: SPRING_ZIPKIN_BASE_URL
              value: http://jhipster-zipkin
            - name: SPRING_SLEUTH_PROPAGATION_KEYS
              value: 'x-request-id,x-ot-span-context'
            - name: JAVA_OPTS
              value: ' -Xmx256m -Xms256m'
          resources:
            requests:
              memory: '512Mi'
              cpu: '500m'
            limits:
              memory: '1Gi'
              cpu: '1'
          ports:
            - name: http
              containerPort: 8084
          readinessProbe:
            httpGet:
              path: /management/health
              port: http
            initialDelaySeconds: 20
            periodSeconds: 15
            failureThreshold: 6
          livenessProbe:
            httpGet:
              path: /management/health
              port: http
            initialDelaySeconds: 120
</code></pre>

mehmet
<blockquote>
<p>have you been trying disabling readiness and liveness probes? – Nick</p>
<p>After I removed readiness and liveness probes, it working now. It seems that needs extra time to run. – mehmet</p>
</blockquote>
<p>That is quite common situation when for some reason it takes long time for probe to return result.</p>
<p>Sometimes, you have to deal with legacy applications that might require an additional startup time on their first initialization. In such cases, it can be tricky to set up liveness probe parameters without compromising the fast response to deadlocks that motivated such a probe.</p>
<p>The trick is to set up a startup probe with the same command, HTTP or TCP check, with a <code>failureThreshold * periodSeconds</code> long enough to cover the worse case startup time.</p>
<pre><code>livenessProbe:
  httpGet:
    path: /healthz
    port: liveness-port
  failureThreshold: 1
  periodSeconds: 10
</code></pre>
<p><code>Probes</code> have a number of fields that you can use to more precisely control the behavior of liveness and readiness checks:</p>
<p><code>initialDelaySeconds</code>: Number of seconds after the container has started before liveness or readiness probes are initiated. Defaults to 0 seconds. Minimum value is 0.</p>
<p><code>periodSeconds</code>: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.</p>
<p>Looking at your YAML file I'd start with changes to <code>readiness probe</code> first</p>
<pre><code>readinessProbe:
 httpGet:
   path: /management/health
 initialDelaySeconds: 20
 periodSeconds: 15
 failureThreshold: 6

livenessProbe:
 httpGet:
   path: /management/health
 initialDelaySeconds: 120
</code></pre>
<p>That means that after 20+(6*15)=110 sec the Readiness probe will fail. At the same time the <code>liveness</code> probe delay is set to 120 sec. I'd increase delay for the <code>readiness</code> probe to the same value as <code>livenes</code> one in order to prevent a situation when traffic is sent to the Pod that is not completely up and running.</p>
<p>Below is some info from <a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/" rel="nofollow noreferrer">Kubernetes official documentation</a> on probes:</p>
<p><strong>Liveness probe</strong></p>
<p>Many applications running for long periods of time eventually transition to broken states, and cannot recover except by being restarted. Kubernetes provides liveness probes to detect and remedy such situations.</p>
<p><strong>Readiness probe</strong></p>
<p>Sometimes, applications are temporarily unable to serve traffic. For example, an application might need to load large data or configuration files during startup, or depend on external services after startup. In such cases, you don’t want to kill the application, but you don’t want to send it requests either. Kubernetes provides readiness probes to detect and mitigate these situations. A pod with containers reporting that they are not ready does not receive traffic through Kubernetes Services.</p>
<blockquote>
<p>Note: Readiness probes runs on the container during its whole lifecycle.</p>
</blockquote>
<p>Readiness probes are configured similarly to liveness probes. The only difference is that you use the <code>readinessProbe</code> field instead of the <code>livenessProbe</code> field.</p>
<pre><code>readinessProbe:
  exec:
    command:
    - cat
    - /tmp/healthy
  initialDelaySeconds: 5
  periodSeconds: 5
</code></pre>
<p>Hope that helps.</p>
