title: How to configure OTEL collector with Spring app using micrometer library and java instrumentation agent to get metrics
body: <p><strong>Issue</strong></p>
  <p>Otel collector instrumentation agent does not forward prometheus metrics from Spring app correctly</p>
  <p><strong>Local reproduction steps</strong></p>
  <ol>
  <li>Simple java springboot app with gradle</li>
  </ol>
  <pre><code>dependencies {
      implementation 'org.springframework.boot:spring-boot-starter-web'
      implementation 'org.springframework.boot:spring-boot-starter-actuator'
  }
  </code></pre>
  <ol start="2">
  <li>OTEL Collector running as a docker container with the following files</li>
  </ol>
  <p><code>docker-compose.yml</code></p>
  <pre><code>  otel-collector:
        image: otel/opentelemetry-collector
        command: [&quot;--config=/etc/otel-collector-config.yaml&quot;]
        volumes:
          - ./config.yaml:/etc/otel-collector-config.yaml
        ports:
          - &quot;1888:1888&quot;   # pprof extension
          - &quot;8888:8888&quot;   # Prometheus metrics exposed by the collector
          - &quot;8889:8889&quot;   # Prometheus exporter metrics
          - &quot;13133:13133&quot; # health_check extension
          - &quot;4317:4317&quot;   # OTLP gRPC receiver
          - &quot;4318:4318&quot;   # OTLP http receiver
          - &quot;55679:55679&quot; # zpages extension
  </code></pre>
  <p><code>config.yml</code></p>
  <pre><code>receivers:
    prometheus:
      config:
        scrape_configs:
          - job_name: &quot;Book&quot;
            scrape_interval: 5s
            static_configs:
              - targets: [&quot;localhost:9080&quot;]
  processors:
    batch:
  
  exporters:
    prometheus:
      endpoint: &quot;localhost:9090&quot;
  
  service:
    pipelines:
      metrics:
        receivers: [prometheus]
        processors: [batch]
        exporters: [prometheus]
  </code></pre>
  <ol start="3">
  <li>Prometheus running as docker container</li>
  </ol>
  <p><code>docker-compose.yml</code></p>
  <pre><code>  prometheus:
      image: prom/prometheus:v2.17.1
      container_name: prometheus
      volumes:
        - ./prometheus:/etc/prometheus
        - prometheus_data:/prometheus
      command:
        - '--config.file=/etc/prometheus/prometheus.yml'
        - '--storage.tsdb.path=/prometheus'
        - '--web.console.libraries=/etc/prometheus/console_libraries'
        - '--web.console.templates=/etc/prometheus/consoles'
        - '--storage.tsdb.retention.time=200h'
        - '--web.enable-lifecycle'
      restart: unless-stopped
      expose:
        - 9090
  </code></pre>
  <p><code>prometheus.yaml</code></p>
  <pre><code>global:
    scrape_interval:     15s
    evaluation_interval: 15s
  </code></pre>
  <p><strong>Scenario</strong></p>
  <ol>
  <li>I start the OTEL Collector and Prometheus using docker</li>
  <li>I run the following command below to auto instrument my spring java app</li>
  </ol>
  <pre><code>java -javaagent:./opentelemetry-javaagent.jar \
  -Dotel.javaagent.extensions=./opentelemetry-micrometer-1.5-1.13.0-alpha.jar \
  -Dotel.instrumentation.spring-boot-actuator-autoconfigure.enabled=false \
  -Dotel.javaagent.debug=true \
  -Dotel.metrics.exporter=prometheus \
  -Dotel.traces.exporter=none \
  -Dotel.exporter.prometheus.metrics.endpoint=http://127.0.0.1:4317 \
  -Dotel.exporter.prometheus.port=9080 \
  -Dotel.resource.attributes=&quot;service.name=helloapp&quot; \
  -Dotel.instrumentation.micrometer.base-time-unit=s \
  -Dotel.instrumentation.micrometer.prometheus-mode.enabled=true \
  -jar build/libs/hello-0.0.1-SNAPSHOT.jar
  </code></pre>
  <p><strong>Issue:</strong></p>
  <ul>
  <li><p>Prometheus metrics exported by Instrumentation agent are different compared to Metrics received by collector and subsequently forwarded to Prometheus docker app</p>
  </li>
  <li><p>Expected prometheus metrics</p>
  </li>
  </ul>
  <blockquote>
  <pre><code># TYPE process_runtime_jvm_system_cpu_utilization gauge
  # HELP process_runtime_jvm_system_cpu_utilization Recent cpu utilization for the whole system
  process_runtime_jvm_system_cpu_utilization 0.0 1660037708660
  # TYPE process_runtime_jvm_memory_usage gauge
  # HELP process_runtime_jvm_memory_usage Measure of memory used process_runtime_jvm_memory_usage{pool=&quot;Metaspace&quot;,type=&quot;non_heap&quot;}
  4.289408E7 1660037708660 process_runtime_jvm_memory_usage{pool=&quot;G1 Eden Space&quot;,type=&quot;heap&quot;} 6291456.0 1660037708660
  process_runtime_jvm_memory_usage{pool=&quot;G1 Old Gen&quot;,type=&quot;heap&quot;}
  2.0279808E7 1660037708660 process_runtime_jvm_memory_usage{pool=&quot;Compressed Class
  Space&quot;,type=&quot;non_heap&quot;} 5806728.0 1660037708660
  process_runtime_jvm_memory_usage{pool=&quot;CodeHeap 'profiled
  nmethods'&quot;,type=&quot;non_heap&quot;} 1.2102912E7 1660037708660
  process_runtime_jvm_memory_usage{pool=&quot;CodeHeap
  'non-nmethods'&quot;,type=&quot;non_heap&quot;} 1271552.0 1660037708660
  process_runtime_jvm_memory_usage{pool=&quot;G1 Survivor Space&quot;,type=&quot;heap&quot;}
  5671744.0 1660037708660 process_runtime_jvm_memory_usage{pool=&quot;CodeHeap 'non-profiled
  nmethods'&quot;,type=&quot;non_heap&quot;} 3989888.0 1660037708660
  # TYPE process_runtime_jvm_threads_count gauge
  # HELP process_runtime_jvm_threads_count Number of executing threads process_runtime_jvm_threads_count 12.0 1660037708660
  # TYPE process_runtime_jvm_memory_limit gauge
  # HELP process_runtime_jvm_memory_limit Measure of max obtainable memory process_runtime_jvm_memory_limit{pool=&quot;G1 Old Gen&quot;,type=&quot;heap&quot;}
  4.294967296E9 1660037708660 process_runtime_jvm_memory_limit{pool=&quot;Compressed Class
  Space&quot;,type=&quot;non_heap&quot;} 1.073741824E9 1660037708660
  process_runtime_jvm_memory_limit{pool=&quot;CodeHeap 'profiled
  nmethods'&quot;,type=&quot;non_heap&quot;} 1.22908672E8 1660037708660
  process_runtime_jvm_memory_limit{pool=&quot;CodeHeap
  'non-nmethods'&quot;,type=&quot;non_heap&quot;} 5840896.0 1660037708660
  process_runtime_jvm_memory_limit{pool=&quot;CodeHeap 'non-profiled
  nmethods'&quot;,type=&quot;non_heap&quot;} 1.22908672E8 1660037708660
  # TYPE process_runtime_jvm_memory_init gauge
  # HELP process_runtime_jvm_memory_init Measure of initial memory requested
  process_runtime_jvm_memory_init{pool=&quot;Metaspace&quot;,type=&quot;non_heap&quot;} 0.0
  1660037708660 process_runtime_jvm_memory_init{pool=&quot;G1 Eden
  Space&quot;,type=&quot;heap&quot;} 2.5165824E7 1660037708660
  process_runtime_jvm_memory_init{pool=&quot;G1 Old Gen&quot;,type=&quot;heap&quot;}
  2.43269632E8 1660037708660 process_runtime_jvm_memory_init{pool=&quot;Compressed Class
  Space&quot;,type=&quot;non_heap&quot;} 0.0 1660037708660
  process_runtime_jvm_memory_init{pool=&quot;CodeHeap 'profiled
  nmethods'&quot;,type=&quot;non_heap&quot;} 2555904.0 1660037708660
  process_runtime_jvm_memory_init{pool=&quot;CodeHeap
  'non-nmethods'&quot;,type=&quot;non_heap&quot;} 2555904.0 1660037708660
  process_runtime_jvm_memory_init{pool=&quot;G1 Survivor Space&quot;,type=&quot;heap&quot;}
  0.0 1660037708660 process_runtime_jvm_memory_init{pool=&quot;CodeHeap 'non-profiled nmethods'&quot;,type=&quot;non_heap&quot;} 2555904.0 1660037708660
  # TYPE process_runtime_jvm_classes_loaded_total counter
  # HELP process_runtime_jvm_classes_loaded_total Number of classes loaded since JVM start process_runtime_jvm_classes_loaded_total 9284.0
  1660037708660
  # TYPE process_runtime_jvm_memory_committed gauge
  # HELP process_runtime_jvm_memory_committed Measure of memory committed
  process_runtime_jvm_memory_committed{pool=&quot;Metaspace&quot;,type=&quot;non_heap&quot;}
  4.3384832E7 1660037708660 process_runtime_jvm_memory_committed{pool=&quot;G1 Eden Space&quot;,type=&quot;heap&quot;}
  5.24288E7 1660037708660 process_runtime_jvm_memory_committed{pool=&quot;G1 Old Gen&quot;,type=&quot;heap&quot;} 3.7748736E7 1660037708660
  process_runtime_jvm_memory_committed{pool=&quot;Compressed Class
  Space&quot;,type=&quot;non_heap&quot;} 6029312.0 1660037708660
  process_runtime_jvm_memory_committed{pool=&quot;CodeHeap 'profiled
  nmethods'&quot;,type=&quot;non_heap&quot;} 1.212416E7 1660037708660
  process_runtime_jvm_memory_committed{pool=&quot;CodeHeap
  'non-nmethods'&quot;,type=&quot;non_heap&quot;} 2555904.0 1660037708660
  process_runtime_jvm_memory_committed{pool=&quot;G1 Survivor
  Space&quot;,type=&quot;heap&quot;} 6291456.0 1660037708660
  process_runtime_jvm_memory_committed{pool=&quot;CodeHeap 'non-profiled
  nmethods'&quot;,type=&quot;non_heap&quot;} 3997696.0 1660037708660
  # TYPE process_runtime_jvm_classes_current_loaded gauge
  # HELP process_runtime_jvm_classes_current_loaded Number of classes currently loaded process_runtime_jvm_classes_current_loaded 9310.0
  1660037708660
  # TYPE process_runtime_jvm_cpu_utilization gauge
  # HELP process_runtime_jvm_cpu_utilization Recent cpu utilization for the process process_runtime_jvm_cpu_utilization 0.0 1660037708660
  # TYPE process_runtime_jvm_classes_unloaded_total counter
  # HELP process_runtime_jvm_classes_unloaded_total Number of classes unloaded since JVM start process_runtime_jvm_classes_unloaded_total
  1.0 1660037708660
  # TYPE process_runtime_jvm_system_cpu_load_1m gauge
  # HELP process_runtime_jvm_system_cpu_load_1m Average CPU load of the whole system for the last minute
  process_runtime_jvm_system_cpu_load_1m 2.15087890625 1660037708660
  # TYPE system_cpu_usage gauge
  # HELP system_cpu_usage The &quot;recent cpu usage&quot; of the system the application is running in system_cpu_usage 0.4666666666666667
  1660037708660
  # TYPE jvm_threads_states_threads gauge
  # HELP jvm_threads_states_threads The current number of threads having NEW state jvm_threads_states_threads{state=&quot;runnable&quot;} 6.0
  1660037708660 jvm_threads_states_threads{state=&quot;timed-waiting&quot;} 4.0
  1660037708660 jvm_threads_states_threads{state=&quot;terminated&quot;} 0.0
  1660037708660 jvm_threads_states_threads{state=&quot;new&quot;} 0.0
  1660037708660 jvm_threads_states_threads{state=&quot;blocked&quot;} 0.0
  1660037708660 jvm_threads_states_threads{state=&quot;waiting&quot;} 2.0
  1660037708660
  # TYPE jvm_gc_max_data_size_bytes gauge
  # HELP jvm_gc_max_data_size_bytes Max size of long-lived heap memory pool jvm_gc_max_data_size_bytes 4.294967296E9 1660037708660
  # TYPE system_cpu_count gauge
  # HELP system_cpu_count The number of processors available to the Java virtual machine system_cpu_count 8.0 1660037708660
  # TYPE jvm_threads_live_threads gauge
  # HELP jvm_threads_live_threads The current number of live threads including both daemon and non-daemon threads jvm_threads_live_threads
  12.0 1660037708660
  # TYPE jvm_threads_daemon_threads gauge
  # HELP jvm_threads_daemon_threads The current number of live daemon threads jvm_threads_daemon_threads 8.0 1660037708660
  # TYPE jvm_memory_usage_after_gc_percent gauge
  # HELP jvm_memory_usage_after_gc_percent The percentage of long-lived heap pool used after the last GC event, in the range [0..1]
  jvm_memory_usage_after_gc_percent{area=&quot;heap&quot;,pool=&quot;long-lived&quot;}
  0.0047217607498168945 1660037708660
  # TYPE jvm_gc_memory_allocated_bytes_total counter
  # HELP jvm_gc_memory_allocated_bytes_total Incremented for an increase in the size of the (young) heap memory pool after one GC to before the
  next jvm_gc_memory_allocated_bytes_total 5.0331648E7 1660037708660
  # TYPE jvm_gc_overhead_percent gauge
  # HELP jvm_gc_overhead_percent An approximation of the percent of CPU time used by GC activities over the last lookback period or since
  monitoring began, whichever is shorter, in the range [0..1]
  jvm_gc_overhead_percent 0.0034007331860589907 1660037708660
  # TYPE disk_free_bytes gauge
  # HELP disk_free_bytes Usable space for path disk_free_bytes{path=&quot;/Users/anuragk/workspace/ts/demo/otel/java/hello/.&quot;} 1.89101027328E11 1660037708660
  # TYPE jvm_memory_committed_bytes gauge
  # HELP jvm_memory_committed_bytes The amount of memory in bytes that is committed for the Java virtual machine to use
  jvm_memory_committed_bytes{area=&quot;nonheap&quot;,id=&quot;Compressed Class Space&quot;}
  6029312.0 1660037708660 jvm_memory_committed_bytes{area=&quot;nonheap&quot;,id=&quot;Metaspace&quot;} 4.358144E7
  1660037708660 jvm_memory_committed_bytes{area=&quot;nonheap&quot;,id=&quot;CodeHeap
  'profiled nmethods'&quot;} 1.212416E7 1660037708660
  jvm_memory_committed_bytes{area=&quot;heap&quot;,id=&quot;G1 Survivor Space&quot;}
  6291456.0 1660037708660 jvm_memory_committed_bytes{area=&quot;heap&quot;,id=&quot;G1 Eden Space&quot;} 5.24288E7 1660037708660
  jvm_memory_committed_bytes{area=&quot;nonheap&quot;,id=&quot;CodeHeap
  'non-nmethods'&quot;} 2555904.0 1660037708660
  jvm_memory_committed_bytes{area=&quot;heap&quot;,id=&quot;G1 Old Gen&quot;} 3.7748736E7
  1660037708660 jvm_memory_committed_bytes{area=&quot;nonheap&quot;,id=&quot;CodeHeap
  'non-profiled nmethods'&quot;} 4063232.0 1660037708660
  # TYPE jvm_gc_live_data_size_bytes gauge
  # HELP jvm_gc_live_data_size_bytes Size of long-lived heap memory pool after reclamation jvm_gc_live_data_size_bytes 0.0 1660037708660
  # TYPE system_load_average_1m gauge
  # HELP system_load_average_1m The sum of the number of runnable entities queued to available processors and the number of runnable
  entities running on the available processors averaged over a period of
  time system_load_average_1m 2.15087890625 1660037708660
  # TYPE process_uptime_seconds gauge
  # HELP process_uptime_seconds The uptime of the Java virtual machine process_uptime_seconds 11.542 1660037708660
  # TYPE jvm_buffer_memory_used_bytes gauge
  # HELP jvm_buffer_memory_used_bytes An estimate of the memory that the Java virtual machine is using for this buffer pool
  jvm_buffer_memory_used_bytes{id=&quot;mapped - 'non-volatile memory'&quot;} 0.0
  1660037708660 jvm_buffer_memory_used_bytes{id=&quot;mapped&quot;} 0.0
  1660037708660 jvm_buffer_memory_used_bytes{id=&quot;direct&quot;} 16384.0
  1660037708660
  # TYPE jvm_buffer_count_buffers gauge
  # HELP jvm_buffer_count_buffers An estimate of the number of buffers in the pool jvm_buffer_count_buffers{id=&quot;mapped - 'non-volatile
  memory'&quot;} 0.0 1660037708660 jvm_buffer_count_buffers{id=&quot;mapped&quot;} 0.0
  1660037708660 jvm_buffer_count_buffers{id=&quot;direct&quot;} 2.0 1660037708660
  # TYPE process_cpu_usage gauge
  # HELP process_cpu_usage The &quot;recent cpu usage&quot; for the Java Virtual Machine process process_cpu_usage 0.3913746438746439 1660037708660
  # TYPE jvm_classes_unloaded_classes_total counter
  # HELP jvm_classes_unloaded_classes_total The total number of classes unloaded since the Java virtual machine has started execution
  jvm_classes_unloaded_classes_total 1.0 1660037708660
  # TYPE process_files_max_files gauge
  # HELP process_files_max_files The maximum file descriptor count process_files_max_files 10240.0 1660037708660
  # TYPE process_files_open_files gauge
  # HELP process_files_open_files The open file descriptor count process_files_open_files 19.0 1660037708660
  # TYPE jvm_threads_peak_threads gauge
  # HELP jvm_threads_peak_threads The peak live thread count since the Java virtual machine started or peak was reset
  jvm_threads_peak_threads 12.0 1660037708660
  # TYPE process_start_time_seconds gauge
  # HELP process_start_time_seconds Start time of the process since unix epoch. process_start_time_seconds 1.66003769719E9 1660037708660
  # TYPE jvm_memory_max_bytes gauge
  # HELP jvm_memory_max_bytes The maximum amount of memory in bytes that can be used for memory management
  jvm_memory_max_bytes{area=&quot;nonheap&quot;,id=&quot;Compressed Class Space&quot;}
  1.073741824E9 1660037708660 jvm_memory_max_bytes{area=&quot;nonheap&quot;,id=&quot;Metaspace&quot;} -1.0 1660037708660
  jvm_memory_max_bytes{area=&quot;nonheap&quot;,id=&quot;CodeHeap 'profiled nmethods'&quot;}
  1.22908672E8 1660037708660 jvm_memory_max_bytes{area=&quot;heap&quot;,id=&quot;G1 Survivor Space&quot;} -1.0 1660037708660
  jvm_memory_max_bytes{area=&quot;heap&quot;,id=&quot;G1 Eden Space&quot;} -1.0
  1660037708660 jvm_memory_max_bytes{area=&quot;nonheap&quot;,id=&quot;CodeHeap
  'non-nmethods'&quot;} 5840896.0 1660037708660
  jvm_memory_max_bytes{area=&quot;heap&quot;,id=&quot;G1 Old Gen&quot;} 4.294967296E9
  1660037708660 jvm_memory_max_bytes{area=&quot;nonheap&quot;,id=&quot;CodeHeap
  'non-profiled nmethods'&quot;} 1.22908672E8 1660037708660
  # TYPE jvm_gc_pause_seconds histogram
  # HELP jvm_gc_pause_seconds Time spent in GC pause jvm_gc_pause_seconds_count{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_sum{action=&quot;end of minor GC&quot;,cause=&quot;G1 Evacuation
  Pause&quot;} 0.003 1660037708660 jvm_gc_pause_seconds_bucket{action=&quot;end of
  minor GC&quot;,cause=&quot;G1 Evacuation Pause&quot;,le=&quot;5.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;10.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;25.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;50.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;75.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;100.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;250.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;500.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;750.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;1000.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;2500.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;5000.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;7500.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;10000.0&quot;} 1.0 1660037708660
  jvm_gc_pause_seconds_bucket{action=&quot;end of minor GC&quot;,cause=&quot;G1
  Evacuation Pause&quot;,le=&quot;+Inf&quot;} 1.0 1660037708660
  # TYPE jvm_classes_loaded_classes gauge
  # HELP jvm_classes_loaded_classes The number of classes that are currently loaded in the Java virtual machine
  jvm_classes_loaded_classes 9409.0 1660037708660
  # TYPE jvm_memory_used_bytes gauge
  # HELP jvm_memory_used_bytes The amount of used memory jvm_memory_used_bytes{area=&quot;nonheap&quot;,id=&quot;Compressed Class Space&quot;}
  5853584.0 1660037708660 jvm_memory_used_bytes{area=&quot;nonheap&quot;,id=&quot;Metaspace&quot;} 4.315976E7
  1660037708660 jvm_memory_used_bytes{area=&quot;nonheap&quot;,id=&quot;CodeHeap
  'profiled nmethods'&quot;} 1.2106752E7 1660037708660
  jvm_memory_used_bytes{area=&quot;heap&quot;,id=&quot;G1 Survivor Space&quot;} 5671744.0
  1660037708660 jvm_memory_used_bytes{area=&quot;heap&quot;,id=&quot;G1 Eden Space&quot;}
  8388608.0 1660037708660 jvm_memory_used_bytes{area=&quot;nonheap&quot;,id=&quot;CodeHeap 'non-nmethods'&quot;}
  1277696.0 1660037708660 jvm_memory_used_bytes{area=&quot;heap&quot;,id=&quot;G1 Old Gen&quot;} 2.0279808E7 1660037708660
  jvm_memory_used_bytes{area=&quot;nonheap&quot;,id=&quot;CodeHeap 'non-profiled
  nmethods'&quot;} 4001920.0 1660037708660
  # TYPE disk_total_bytes gauge
  # HELP disk_total_bytes Total space for path disk_total_bytes{path=&quot;/Users/anuragk/workspace/ts/demo/otel/java/hello/.&quot;}
  4.94384795648E11 1660037708660
  # TYPE jvm_buffer_total_capacity_bytes gauge
  # HELP jvm_buffer_total_capacity_bytes An estimate of the total capacity of the buffers in this pool
  jvm_buffer_total_capacity_bytes{id=&quot;mapped - 'non-volatile memory'&quot;}
  0.0 1660037708660 jvm_buffer_total_capacity_bytes{id=&quot;mapped&quot;} 0.0 1660037708660 jvm_buffer_total_capacity_bytes{id=&quot;direct&quot;} 16384.0
  1660037708660 ```
  </code></pre>
  </blockquote>
  <ul>
  <li>Received metrics</li>
  </ul>
  <blockquote>
  <pre><code># HELP otelcol_exporter_enqueue_failed_log_records Number of log records failed to be added to the sending queue.
  # TYPE otelcol_exporter_enqueue_failed_log_records counter otelcol_exporter_enqueue_failed_log_records{exporter=&quot;prometheus&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  0
  # HELP otelcol_exporter_enqueue_failed_metric_points Number of metric points failed to be added to the sending queue.
  # TYPE otelcol_exporter_enqueue_failed_metric_points counter otelcol_exporter_enqueue_failed_metric_points{exporter=&quot;prometheus&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  0
  # HELP otelcol_exporter_enqueue_failed_spans Number of spans failed to be added to the sending queue.
  # TYPE otelcol_exporter_enqueue_failed_spans counter otelcol_exporter_enqueue_failed_spans{exporter=&quot;prometheus&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  0
  # HELP otelcol_exporter_sent_metric_points Number of metric points successfully sent to destination.
  # TYPE otelcol_exporter_sent_metric_points counter otelcol_exporter_sent_metric_points{exporter=&quot;prometheus&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  5576
  # HELP otelcol_process_cpu_seconds Total CPU user and system time in seconds
  # TYPE otelcol_process_cpu_seconds counter otelcol_process_cpu_seconds{service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  2.17
  # HELP otelcol_process_memory_rss Total physical memory (resident set size)
  # TYPE otelcol_process_memory_rss gauge otelcol_process_memory_rss{service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  6.5077248e+07
  # HELP otelcol_process_runtime_heap_alloc_bytes Bytes of allocated heap objects (see 'go doc runtime.MemStats.HeapAlloc')
  # TYPE otelcol_process_runtime_heap_alloc_bytes gauge otelcol_process_runtime_heap_alloc_bytes{service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  9.784448e+06
  # HELP otelcol_process_runtime_total_alloc_bytes Cumulative bytes allocated for heap objects (see 'go doc runtime.MemStats.TotalAlloc')
  # TYPE otelcol_process_runtime_total_alloc_bytes counter otelcol_process_runtime_total_alloc_bytes{service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  3.9931936e+07
  # HELP otelcol_process_runtime_total_sys_memory_bytes Total bytes of memory obtained from the OS (see 'go doc runtime.MemStats.Sys')
  # TYPE otelcol_process_runtime_total_sys_memory_bytes gauge otelcol_process_runtime_total_sys_memory_bytes{service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  2.944308e+07
  # HELP otelcol_process_uptime Uptime of the process
  # TYPE otelcol_process_uptime counter otelcol_process_uptime{service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  388.883801875
  # HELP otelcol_processor_batch_batch_send_size Number of units in the batch
  # TYPE otelcol_processor_batch_batch_send_size histogram otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;10&quot;}
  26
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;25&quot;}
  28
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;50&quot;}
  32
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;75&quot;}
  32
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;100&quot;}
  32
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;250&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;500&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;750&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;1000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;2000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;3000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;4000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;5000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;6000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;7000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;8000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;9000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;10000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;20000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;30000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;50000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;100000&quot;}
  75
  otelcol_processor_batch_batch_send_size_bucket{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,le=&quot;+Inf&quot;}
  75
  otelcol_processor_batch_batch_send_size_sum{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  5576.000000000002 otelcol_processor_batch_batch_send_size_count{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  75
  # HELP otelcol_processor_batch_timeout_trigger_send Number of times the batch was sent due to a timeout trigger
  # TYPE otelcol_processor_batch_timeout_trigger_send counter otelcol_processor_batch_timeout_trigger_send{processor=&quot;batch&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;}
  75
  # HELP otelcol_receiver_accepted_metric_points Number of metric points successfully pushed into the pipeline.
  # TYPE otelcol_receiver_accepted_metric_points counter otelcol_receiver_accepted_metric_points{receiver=&quot;prometheus&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,transport=&quot;http&quot;}
  5576
  # HELP otelcol_receiver_refused_metric_points Number of metric points that could not be pushed into the pipeline.
  # TYPE otelcol_receiver_refused_metric_points counter otelcol_receiver_refused_metric_points{receiver=&quot;prometheus&quot;,service_instance_id=&quot;57f37a9a-5825-48b3-a8a2-240ee4c52656&quot;,service_version=&quot;0.56.0&quot;,transport=&quot;http&quot;}
  0 ```
  </code></pre>
  </blockquote>
  <p><a href="https://i.stack.imgur.com/nSf8x.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/nSf8x.png" alt="Architecture diagram" /></a></p>