{"action":"closed","issue":{"url":"https://api.github.com/repos/prometheus/prometheus/issues/2220","repository_url":"https://api.github.com/repos/prometheus/prometheus","labels_url":"https://api.github.com/repos/prometheus/prometheus/issues/2220/labels{/name}","comments_url":"https://api.github.com/repos/prometheus/prometheus/issues/2220/comments","events_url":"https://api.github.com/repos/prometheus/prometheus/issues/2220/events","html_url":"https://github.com/prometheus/prometheus/issues/2220","id":191754839,"number":2220,"title":"targets from consul never show up after upgrade to 1.4","user":{"login":"onorua","id":407040,"avatar_url":"https://avatars.githubusercontent.com/u/407040?v=3","gravatar_id":"","url":"https://api.github.com/users/onorua","html_url":"https://github.com/onorua","followers_url":"https://api.github.com/users/onorua/followers","following_url":"https://api.github.com/users/onorua/following{/other_user}","gists_url":"https://api.github.com/users/onorua/gists{/gist_id}","starred_url":"https://api.github.com/users/onorua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/onorua/subscriptions","organizations_url":"https://api.github.com/users/onorua/orgs","repos_url":"https://api.github.com/users/onorua/repos","events_url":"https://api.github.com/users/onorua/events{/privacy}","received_events_url":"https://api.github.com/users/onorua/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-11-25T19:06:41Z","updated_at":"2016-11-28T07:00:42Z","closed_at":"2016-11-28T07:00:42Z","body":"**What did you do?**\r\nWe have a server which was upgraded couple of hours ago, it got all the data loaded, but apparently did not do anything useful since that moment. \r\n\r\nWhen I press targets on GUI, it doesn't see anything, downgrade to 1.3.1 made the trick. I've checked and it seems no major changes to consul autodiscovery happened. \r\n\r\n**What did you expect to see?**\r\nSmooth upgrade, restart, load files and continue to scrape \r\n\r\n**Environment**\r\n\r\n* System information:\r\n```\r\nLinux 4.4.0-45-generic x86_64\r\n```\r\n\r\n* Prometheus version: \r\n```\r\nprometheus, version 1.4.0 (branch: master, revision: ecad074e46ef60536722c5b01e6c5277f2b50a3d)\r\n  build user:       root@5fb8bc2a8e57\r\n  build date:       20161125-12:45:05\r\n  go version:       go1.7.3\r\n```\r\n\r\n* Prometheus configuration file:\r\n```\r\n# my global config\r\nglobal:\r\n  scrape_interval:     30s # By default, scrape targets every 15 seconds.\r\n  evaluation_interval: 30s # By default, scrape targets every 15 seconds.\r\n  # scrape_timeout is set to the global default (10s).\r\n\r\n  # Attach these labels to any time series or alerts when communicating with\r\n  # external systems (federation, remote storage, Alertmanager).\r\n  external_labels:\r\n      monitor: 'cloud'\r\n\r\n# Load and evaluate rules in this file every 'evaluation_interval' seconds.\r\nrule_files:\r\n  - \"alert.rules\"\r\n  # - \"first.rules\"\r\n  # - \"second.rules\"\r\n\r\n# A scrape configuration containing exactly one endpoint to scrape:\r\n# Here it's Prometheus itself.\r\nscrape_configs:\r\n  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.\r\n  # Auto-discovery via consul tags by Registrator\r\n  - job_name: 'consul-services'\r\n\r\n    consul_sd_configs:\r\n    - server: 'consul:8500'\r\n      scheme: 'http'\r\n\r\n    relabel_configs:\r\n    - source_labels: [ '__meta_consul_tags' ]\r\n      action: keep\r\n      regex: .*exporter.*\r\n    - source_labels: [ '__meta_consul_address', '__meta_consul_service_port' ]\r\n      action: replace\r\n      regex: (.+)(?::\\d+);(\\d+)\r\n      replacement: $1:$2\r\n      target_label: __address__\r\n    - source_labels: [ '__meta_consul_service' ]\r\n      action: keep\r\n      regex: (.+)\r\n      replacement: $1\r\n      target_label: __name__\r\n    - source_labels: [ '__meta_consul_service_id' ]\r\n      action: replace\r\n      regex: (.*):(.*):(.*)\r\n      replacement: $2\r\n      target_label: container_name\r\n    - source_labels: [ '__meta_consul_service_id' ]\r\n      action: replace\r\n      regex: (.*):(.*):(.*)\r\n      replacement: $1\r\n      target_label: host_name\r\n    - source_labels: [ '__meta_consul_service_id' ]\r\n      action: replace\r\n      regex: .*-(\\w*)-(\\w*)-(\\d*):(.*):(.*)\r\n      replacement: $1\r\n      target_label: colo\r\n```\r\n\r\n* Logs:\r\n```\r\ntime=\"2016-11-25T13:53:49Z\" level=info msg=\"All requests for rebuilding the label indexes queued. (Actual processing may lag behind.)\" source=\"crashrecovery.go:529\"\r\ntime=\"2016-11-25T13:53:49Z\" level=warning msg=\"Crash recovery complete.\" source=\"crashrecovery.go:152\"\r\ntime=\"2016-11-25T13:53:49Z\" level=info msg=\"1657859 series loaded.\" source=\"storage.go:359\"\r\ntime=\"2016-11-25T13:53:49Z\" level=info msg=\"Starting target manager...\" source=\"targetmanager.go:63\"\r\ntime=\"2016-11-25T13:53:49Z\" level=info msg=\"Listening on :9090\" source=\"web.go:248\"\r\ntime=\"2016-11-25T13:54:25Z\" level=info msg=\"Checkpointing in-memory metrics and chunks...\" source=\"persistence.go:549\"\r\ntime=\"2016-11-25T13:55:27Z\" level=info msg=\"Done checkpointing in-memory metrics and chunks in 1m1.956668514s.\" source=\"persistence.go:573\"\r\n```\r\nand it was checkpointing all the time about once per minute. "}}
{"actor":{"display_login":"fabxc"}}