{"action":"created","issue":{"url":"https://api.github.com/repos/untergeek/zabbix-grab-bag/issues/10","repository_url":"https://api.github.com/repos/untergeek/zabbix-grab-bag","labels_url":"https://api.github.com/repos/untergeek/zabbix-grab-bag/issues/10/labels{/name}","comments_url":"https://api.github.com/repos/untergeek/zabbix-grab-bag/issues/10/comments","events_url":"https://api.github.com/repos/untergeek/zabbix-grab-bag/issues/10/events","html_url":"https://github.com/untergeek/zabbix-grab-bag/issues/10","id":144005053,"number":10,"title":"No data from trapper items","user":{"login":"mdiorio","id":10030851,"avatar_url":"https://avatars.githubusercontent.com/u/10030851?v=3","gravatar_id":"","url":"https://api.github.com/users/mdiorio","html_url":"https://github.com/mdiorio","followers_url":"https://api.github.com/users/mdiorio/followers","following_url":"https://api.github.com/users/mdiorio/following{/other_user}","gists_url":"https://api.github.com/users/mdiorio/gists{/gist_id}","starred_url":"https://api.github.com/users/mdiorio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mdiorio/subscriptions","organizations_url":"https://api.github.com/users/mdiorio/orgs","repos_url":"https://api.github.com/users/mdiorio/repos","events_url":"https://api.github.com/users/mdiorio/events{/privacy}","received_events_url":"https://api.github.com/users/mdiorio/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-03-28T15:34:31Z","updated_at":"2016-04-12T20:29:30Z","closed_at":null,"body":"Hi,\r\n\r\nIf I can get this to work, this is the best use for Zabbix and Elastic monitoring.\r\nI get Zabbix agent results back for master_node and cluster health status, but none of the Zabbix Trapper items are returning any data.\r\n\r\nIn the es_stats_zabbix.log I'm seeing the following:\r\n\r\n`2016-03-28 11:21:22,202 DEBUG            es_stats_zabbix                  batch:131  Batch mode with named batch: thirty_seconds\r\n2016-03-28 11:21:22,203 DEBUG            es_stats_zabbix                  batch:137  Batch config args: {'key3': 'health[initializing_shards]', 'key2': 'health[relocating_shards]', 'key1': 'health[unassigned_shards]', 'server': 'ghq-1pmonap01.globalspec.net', 'host': 'ghq-1delasticnode01.globalspec.net', 'key4': 'health[delayed_unassigned_shards]', 'port': '10051'}\r\n2016-03-28 11:21:22,203 DEBUG            es_stats_zabbix                  batch:143  Batch keys: {'key3': 'health[initializing_shards]', 'key2': 'health[relocating_shards]', 'key1': 'health[unassigned_shards]', 'key4': 'health[delayed_unassigned_shards]'}\r\n2016-03-28 11:21:22,204 DEBUG      es_stats_zabbix.utils              parse_key:142  API: health  Node: None  Key: initializing_shards\r\n2016-03-28 11:21:22,204 DEBUG      es_stats_zabbix.utils              parse_key:142  API: health  Node: None  Key: relocating_shards\r\n2016-03-28 11:21:22,204 DEBUG      es_stats_zabbix.utils              parse_key:142  API: health  Node: None  Key: unassigned_shards\r\n2016-03-28 11:21:22,204 DEBUG      es_stats_zabbix.utils              parse_key:142  API: health  Node: None  Key: delayed_unassigned_shards\r\n2016-03-28 11:21:22,204 DEBUG            es_stats_zabbix                  batch:150  API-separated keys: {'nodeinfo': [], 'clusterstate': [], 'health': [('health', None, 'initializing_shards'), ('health', None, 'relocating_shards'), ('health', None, 'unassigned_shards'), ('health', None, 'delayed_unassigned_shards')], 'nodestats': [], 'clusterstats': []}\r\n2016-03-28 11:21:22,205 DEBUG         urllib3.util.retry               from_int:155  Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0)\r\n2016-03-28 11:21:22,208 DEBUG     urllib3.connectionpool          _make_request:383  \"GET /_cluster/health HTTP/1.1\" 200 390\r\n2016-03-28 11:21:22,208 INFO               elasticsearch    log_request_success:63   GET http://127.0.0.1:9200/_cluster/health [status:200 request:0.003s]\r\n2016-03-28 11:21:22,209 DEBUG              elasticsearch    log_request_success:65   > None\r\n2016-03-28 11:21:22,209 DEBUG              elasticsearch    log_request_success:66   < {\"cluster_name\":\"artemis_dev\",\"status\":\"yellow\",\"timed_out\":false,\"number_of_nodes\":1,\"number_of_data_nodes\":1,\"active_primary_shards\":62,\"active_shards\":62,\"relocating_shards\":0,\"initializing_shards\":0,\"unassigned_shards\":62,\"delayed_unassigned_shards\":0,\"number_of_pending_tasks\":0,\"number_of_in_flight_fetch\":0,\"task_max_waiting_in_queue_millis\":0,\"active_shards_percent_as_number\":50.0}\r\n2016-03-28 11:21:22,210 DEBUG            es_stats_zabbix                  batch:165  Metrics: [Metric('ghq-1delasticnode01.globalspec.net', 'health[initializing_shards]', 0), Metric('ghq-1delasticnode01.globalspec.net', 'health[relocating_shards]', 0), Metric('ghq-1delasticnode01.globalspec.net', 'health[unassigned_shards]', 62), Metric('ghq-1delasticnode01.globalspec.net', 'health[delayed_unassigned_shards]', 0)]\r\n**2016-03-28 11:21:22,214 DEBUG                  zbxsender         send_to_zabbix:58   Got response from Zabbix: {u'info': u'processed: 0; failed: 4; total: 4; seconds spent: 0.000018', u'response': u'success'}\r\n2016-03-28 11:21:22,214 INFO                   zbxsender         send_to_zabbix:59   processed: 0; failed: 4; total: 4; seconds spent: 0.000018\r\n**2016-03-28 11:21:22,214 DEBUG            es_stats_zabbix                  batch:167  Result = True\r\n2016-03-28 11:21:22,215 INFO             es_stats_zabbix                  batch:170  Job completed.\r\n`\r\n\r\nIn zabbix server log set to debug, I do see:\r\n  9185:20160328:112635.725 __zbx_zbx_setproctitle() title:'trapper #4 [processing data]'\r\n  9185:20160328:112635.725 trapper got '{\r\n        \"request\":\"sender data\",\r\n        \"data\":[\r\n                {\r\n                        \"host\":\"ghq-1delasticnode01.globalspec.net\",\r\n                        \"key\":\"health[initializing_shards]\",\r\n                        \"value\":0,\r\n                        \"clock\":1459178782.36},\r\n                {\r\n                        \"host\":\"ghq-1delasticnode01.globalspec.net\",\r\n                        \"key\":\"health[relocating_shards]\",\r\n                        \"value\":0,\r\n                        \"clock\":1459178782.36},\r\n                {\r\n                        \"host\":\"ghq-1delasticnode01.globalspec.net\",\r\n                        \"key\":\"health[unassigned_shards]\",\r\n                        \"value\":62,\r\n                        \"clock\":1459178782.36},\r\n                {\r\n                        \"host\":\"ghq-1delasticnode01.globalspec.net\",\r\n                        \"key\":\"health[delayed_unassigned_shards]\",\r\n                        \"value\":0,\r\n                        \"clock\":1459178782.36}]\r\n}'\r\n\r\nWhich tells me that it's getting the data, but when I view the last data for the host, there is no data there.\r\n\r\nAny ideas?  I've only been using Zabbix for two weeks as I'm evaluating new monitoring solutions, and this is really my first major hurdle for it.  \r\n\r\nThanks.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/untergeek/zabbix-grab-bag/issues/comments/209089839","html_url":"https://github.com/untergeek/zabbix-grab-bag/issues/10#issuecomment-209089839","issue_url":"https://api.github.com/repos/untergeek/zabbix-grab-bag/issues/10","id":209089839,"user":{"login":"mdiorio","id":10030851,"avatar_url":"https://avatars.githubusercontent.com/u/10030851?v=3","gravatar_id":"","url":"https://api.github.com/users/mdiorio","html_url":"https://github.com/mdiorio","followers_url":"https://api.github.com/users/mdiorio/followers","following_url":"https://api.github.com/users/mdiorio/following{/other_user}","gists_url":"https://api.github.com/users/mdiorio/gists{/gist_id}","starred_url":"https://api.github.com/users/mdiorio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mdiorio/subscriptions","organizations_url":"https://api.github.com/users/mdiorio/orgs","repos_url":"https://api.github.com/users/mdiorio/repos","events_url":"https://api.github.com/users/mdiorio/events{/privacy}","received_events_url":"https://api.github.com/users/mdiorio/received_events","type":"User","site_admin":false},"created_at":"2016-04-12T20:29:30Z","updated_at":"2016-04-12T20:29:30Z","body":"I resolved at least part of my issue.  Doing direct calls for nodestats was returning no data when doing:\r\n\r\n`# ./zabbix_get -s ghq-1delasticnode01.globalspec.net -p 10050 -k nodestats[node-1,process.open_file_descriptors]\r\n`\r\n\r\nThe es_stats_zabbix.log file was saying no node specified, and looking at the logs, the following was showing up.  \r\n parse_key:142  API: nodestats  Node:   Key: node-1\r\n\r\nI edited the es_stats_zabbix.userparm:\r\n`UserParameter=nodestats[*],/usr/bin/es_stats_zabbix --configuration /etc/zabbix/es_stats_zabbix.ini single nodestats[$1,$2]\r\n`\r\n\r\nThis allows parsing both the node name, and key.\r\n\r\nStill not sure about the batch processing at this point though."}}