{"action":"closed","issue":{"url":"https://api.github.com/repos/prometheus/prometheus/issues/4541","repository_url":"https://api.github.com/repos/prometheus/prometheus","labels_url":"https://api.github.com/repos/prometheus/prometheus/issues/4541/labels{/name}","comments_url":"https://api.github.com/repos/prometheus/prometheus/issues/4541/comments","events_url":"https://api.github.com/repos/prometheus/prometheus/issues/4541/events","html_url":"https://github.com/prometheus/prometheus/issues/4541","id":354233623,"node_id":"MDU6SXNzdWUzNTQyMzM2MjM=","number":4541,"title":"file_sd never stop update 'custom_sd.json' file","user":{"login":"earthdiaosi","id":10106196,"node_id":"MDQ6VXNlcjEwMTA2MTk2","avatar_url":"https://avatars2.githubusercontent.com/u/10106196?v=4","gravatar_id":"","url":"https://api.github.com/users/earthdiaosi","html_url":"https://github.com/earthdiaosi","followers_url":"https://api.github.com/users/earthdiaosi/followers","following_url":"https://api.github.com/users/earthdiaosi/following{/other_user}","gists_url":"https://api.github.com/users/earthdiaosi/gists{/gist_id}","starred_url":"https://api.github.com/users/earthdiaosi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/earthdiaosi/subscriptions","organizations_url":"https://api.github.com/users/earthdiaosi/orgs","repos_url":"https://api.github.com/users/earthdiaosi/repos","events_url":"https://api.github.com/users/earthdiaosi/events{/privacy}","received_events_url":"https://api.github.com/users/earthdiaosi/received_events","type":"User","site_admin":false},"labels":[{"id":365567641,"node_id":"MDU6TGFiZWwzNjU1Njc2NDE=","url":"https://api.github.com/repos/prometheus/prometheus/labels/component/service%20discovery","name":"component/service discovery","color":"0052cc","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2018-08-27T08:49:10Z","updated_at":"2018-11-30T09:32:17Z","closed_at":"2018-11-30T09:32:17Z","author_association":"NONE","body":"<!--\r\n\r\n    Please do *NOT* ask usage questions in Github issues.\r\n\r\n    If your issue is not a feature request or bug report use:\r\n    https://groups.google.com/forum/#!forum/prometheus-users. If\r\n    you are unsure whether you hit a bug, search and ask in the\r\n    mailing list first.\r\n\r\n    You can find more information at: https://prometheus.io/community/\r\n\r\n-->\r\n## Proposal\r\n**Use case. Why is this important?**\r\n  I want to implement custom service discovery, so i read the follow blog and wirte a demo; \r\n  *  https://prometheus.io/blog/2018/07/05/implementing-custom-sd/\r\n \r\n  but i found \"custom_sd.json\" file was updated each time. so i debug the follow [code](https://github.com/prometheus/prometheus/blob/master/documentation/examples/custom-sd/adapter/adapter.go#L59), and find `key` and `i` is different each time: \r\n```\r\n// Parses incoming target groups updates. If the update contains changes to the target groups\r\n// Adapter already knows about, or new target groups, we Marshal to JSON and write to file.\r\nfunc (a *Adapter) generateTargetGroups(allTargetGroups map[string][]*targetgroup.Group) {\r\n\ttempGroups := make(map[string]*customSD)\r\n\tfor k, sdTargetGroups := range allTargetGroups {\r\n\t\tfor i, group := range sdTargetGroups {\r\n\t\t\tnewTargets := make([]string, 0)\r\n\t\t\tnewLabels := make(map[string]string)\r\n\r\n\t\t\tfor _, targets := range group.Targets {\r\n\t\t\t\tfor _, target := range targets {\r\n\t\t\t\t\tnewTargets = append(newTargets, string(target))\r\n\t\t\t\t}\r\n\t\t\t}\r\n\r\n\t\t\tfor name, value := range group.Labels {\r\n\t\t\t\tnewLabels[string(name)] = string(value)\r\n\t\t\t}\r\n\t\t\t// Make a unique key, including the current index, in case the sd_type (map key) and group.Source is not unique.\r\n\t\t\tkey := fmt.Sprintf(\"%s:%s:%d\", k, group.Source, i)\r\n\t\t\ttempGroups[key] = &customSD{\r\n\t\t\t\tTargets: newTargets,\r\n\t\t\t\tLabels:  newLabels,\r\n\t\t\t}\r\n\t\t}\r\n\t}\r\n\tif !reflect.DeepEqual(a.groups, tempGroups) {\r\n\t\ta.groups = tempGroups\r\n\t\terr := a.writeOutput()\r\n\t\tif err != nil {\r\n\t\t\tlevel.Error(log.With(a.logger, \"component\", \"sd-adapter\")).Log(\"err\", err)\r\n\t\t}\r\n\t}\r\n\r\n}\r\n```\r\n  then i search code, fount the [reason](https://github.com/prometheus/prometheus/blob/master/discovery/manager.go#L198): the `tsets` is a map, so the results are different for each iteration;\r\n```\r\nfunc (m *Manager) allGroups() map[string][]*targetgroup.Group {\r\n\tm.mtx.Lock()\r\n\tdefer m.mtx.Unlock()\r\n\r\n\ttSets := map[string][]*targetgroup.Group{}\r\n\tfor pkey, tsets := range m.targets {\r\n\t\tfor _, tg := range tsets {\r\n\t\t\t// Even if the target group 'tg' is empty we still need to send it to the 'Scrape manager'\r\n\t\t\t// to signal that it needs to stop all scrape loops for this target set.\r\n\t\t\ttSets[pkey.setName] = append(tSets[pkey.setName], tg)\r\n\t\t}\r\n\t}\r\n\treturn tSets\r\n}\r\n```\r\nsolution: sort map by key\r\n```\r\nfunc (m *Manager) allGroups() map[string][]*targetgroup.Group {\r\n\tm.mtx.Lock()\r\n\tdefer m.mtx.Unlock()\r\n\r\n\ttSets := map[string][]*targetgroup.Group{}\r\n\tfor pkey, tsets := range m.targets {\r\n\t\t//sort tsets\r\n\t\tkeys := make([]string, 0, len(tsets))\r\n\t\tfor key := range tsets {\r\n\t\t\tkeys = append(keys, key)\r\n\t\t}\r\n\t\tsort.Strings(keys)\r\n\t\tfor _, k := range keys {\r\n\t\t\t// Even if the target group 'tg' is empty we still need to send it to the 'Scrape manager'\r\n\t\t\t// to signal that it needs to stop all scrape loops for this target set.\r\n\t\t\ttSets[pkey.setName] = append(tSets[pkey.setName], tsets[k])\r\n\t\t}\r\n\t}\r\n\treturn tSets\r\n```\r\n\r\n## Bug Report\r\n**What did you do?**\r\nWrite code to implement custom service discovery;\r\n\r\n**What did you expect to see?**\r\nDo not update file \"custom_sd.json\" when no changes;\r\n\r\n**What did you see instead? Under which circumstances?**\r\n i found \"custom_sd.json\" file was updated each time.\r\n\r\n**Environment**\r\n\r\n* System information:\r\n\r\n\tWindows7 64bit\r\n\r\n* Prometheus version:\r\n\r\n\t2.3.1\r\n\r\n"}}
{"actor":{"display_login":"simonpasquier"}}