{"action":"closed","issue":{"url":"https://api.github.com/repos/influxdata/influxdb/issues/5601","repository_url":"https://api.github.com/repos/influxdata/influxdb","labels_url":"https://api.github.com/repos/influxdata/influxdb/issues/5601/labels{/name}","comments_url":"https://api.github.com/repos/influxdata/influxdb/issues/5601/comments","events_url":"https://api.github.com/repos/influxdata/influxdb/issues/5601/events","html_url":"https://github.com/influxdata/influxdb/issues/5601","id":132524357,"number":5601,"title":"[0.10.0] Nodes joins as meta node instead of data nodes in cluster of more than 4 nodes","user":{"login":"supershal","id":6110931,"avatar_url":"https://avatars.githubusercontent.com/u/6110931?v=3","gravatar_id":"","url":"https://api.github.com/users/supershal","html_url":"https://github.com/supershal","followers_url":"https://api.github.com/users/supershal/followers","following_url":"https://api.github.com/users/supershal/following{/other_user}","gists_url":"https://api.github.com/users/supershal/gists{/gist_id}","starred_url":"https://api.github.com/users/supershal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/supershal/subscriptions","organizations_url":"https://api.github.com/users/supershal/orgs","repos_url":"https://api.github.com/users/supershal/repos","events_url":"https://api.github.com/users/supershal/events{/privacy}","received_events_url":"https://api.github.com/users/supershal/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/influxdata/influxdb/labels/category/clustering","name":"category/clustering","color":"d111e2"},{"url":"https://api.github.com/repos/influxdata/influxdb/labels/category/configuration","name":"category/configuration","color":"009800"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2016-02-09T20:27:14Z","updated_at":"2016-02-15T17:30:01Z","closed_at":"2016-02-15T17:30:01Z","body":"I am trying to run influxdb cluster of more than 4 nodes. The resulting cluster adds newer node as meta node instead of data node. The cluster only have one data node (first node) and 4 meta node instead of atleast 3 meta nodes.\r\nAll nodes in the cluster is getting ID value as 1 as per meta/node.json\r\nInfluxdb version: 0.10.0-1\r\nOS: 4.2.2-coreos-r2\r\nMy config file is the default config file generated from \"influxd config\" command.\r\nBinding Address are overridden using environment variables as follows. \r\nThe cluster is created using docker containers on Kubernetes.\r\n\r\nFollowing are steps and results:\r\n`\r\n-----------------------------------------------------------------------------\r\nNode 1:\r\n-----------------------------------------------------------------------------\r\n++ hostname -i\r\n+ HOSTIP=10.244.1.4\r\n+ export INFLUXDB_META_HTTP_BIND_ADDRESS=10.244.1.4:8091\r\n+ INFLUXDB_META_HTTP_BIND_ADDRESS=10.244.1.4:8091\r\n+ export INFLUXDB_META_BIND_ADDRESS=10.244.1.4:8088\r\n+ INFLUXDB_META_BIND_ADDRESS=10.244.1.4:8088\r\n+ influxd --config /etc/influxdb.toml\r\n\r\nNode 1 - node.json\r\n-----------------------------------------\r\nroot@influxdb-raft-rc-bg2hr:/data/meta# cat node.json\r\n{\"ID\":1,\"MetaServers\":[\"10.244.1.4:8091\",\"10.244.2.188:8091\",\"10.244.2.205:8091\",\"10.244.1.190:8091\"]}\r\n\r\nNode 1 - CLI:\r\n-----------------------------------------\r\n```\r\n> show servers\r\nname: data_nodes\r\n----------------\r\nid\thttp_addr\ttcp_addr\r\n1\tlocalhost:8086\t10.244.1.4:8088\r\n\r\nname: meta_nodes\r\n----------------\r\nid\thttp_addr\ttcp_addr\r\n1\t10.244.1.4:8091\t10.244.1.4:8088\r\n```\r\n\r\n\r\n-----------------------------------------------------------------------------\r\nNode 2:\r\n-----------------------------------------------------------------------------\r\n++ INFLUXD_OPTS='-join 10.244.1.4:8091'\r\n++ hostname -i\r\n+ HOSTIP=10.244.2.188\r\n+ export INFLUXDB_META_HTTP_BIND_ADDRESS=10.244.2.188:8091\r\n+ INFLUXDB_META_HTTP_BIND_ADDRESS=10.244.2.188:8091\r\n+ export INFLUXDB_META_BIND_ADDRESS=10.244.2.188:8088\r\n+ INFLUXDB_META_BIND_ADDRESS=10.244.2.188:8088\r\n+ influxd --config /etc/influxdb.toml -join 10.244.1.4:8091\r\n\r\nNode 2 - node.json\r\n-----------------------------------------\r\nroot@influxdb-raft-rc-pdfpe:/data/meta# cat node.json\r\n{\"ID\":1,\"MetaServers\":[\"10.244.1.4:8091\",\"10.244.2.188:8091\",\"10.244.2.205:8091\",\"10.244.1.190:8091\"]}\r\n\r\nNode 2 - CLI: \r\n-----------------------------------------\r\n```\r\n> show servers\r\nname: data_nodes\r\n----------------\r\nid\thttp_addr\ttcp_addr\r\n1\tlocalhost:8086\t10.244.1.4:8088\r\n\r\n\r\nname: meta_nodes\r\n----------------\r\nid\thttp_addr\t\ttcp_addr\r\n1\t10.244.1.4:8091\t\t10.244.1.4:8088\r\n2\t10.244.2.188:8091\t10.244.2.188:8088\r\n\r\n```\r\n-----------------------------------------------------------------------------\r\nNode 3:\r\n-----------------------------------------------------------------------------\r\n++ INFLUXD_OPTS='-join 10.244.1.4:8091,10.244.2.188:8091'\r\n++ hostname -i\r\n+ HOSTIP=10.244.2.205\r\n+ export INFLUXDB_META_HTTP_BIND_ADDRESS=10.244.2.205:8091\r\n+ INFLUXDB_META_HTTP_BIND_ADDRESS=10.244.2.205:8091\r\n+ export INFLUXDB_META_BIND_ADDRESS=10.244.2.205:8088\r\n+ INFLUXDB_META_BIND_ADDRESS=10.244.2.205:8088\r\n+ influxd --config /etc/influxdb.toml -join 10.244.1.4:8091,10.244.2.188:8091\r\n\r\nNode 3 - Node.json:\r\n-----------------------------------------\r\nroot@influxdb-raft-rc-5zyef:/data/meta# cat node.json\r\n{\"ID\":1,\"MetaServers\":[\"10.244.1.4:8091\",\"10.244.2.188:8091\",\"10.244.2.205:8091\",\"10.244.1.190:8091\"]}\r\n\r\n\r\nNode 3 - CLI:\r\n-----------------------------------------\r\n```\r\n> show servers\r\nname: data_nodes\r\n----------------\r\nid\thttp_addr\ttcp_addr\r\n1\tlocalhost:8086\t10.244.1.4:8088\r\n\r\nname: meta_nodes\r\n----------------\r\nid\thttp_addr\t\ttcp_addr\r\n1\t10.244.1.4:8091\t\t10.244.1.4:8088\r\n2\t10.244.2.188:8091\t10.244.2.188:8088\r\n3\t10.244.2.205:8091\t10.244.2.205:8088\r\n```\r\n-----------------------------------------------------------------------------\r\nNode 4:\r\n-----------------------------------------------------------------------------\r\n\r\n++ INFLUXD_OPTS='-join 10.244.2.205:8091,10.244.1.4:8091,10.244.2.188:8091'\r\n++ hostname -i\r\n+ HOSTIP=10.244.1.190\r\n+ export INFLUXDB_META_HTTP_BIND_ADDRESS=10.244.1.190:8091\r\n+ INFLUXDB_META_HTTP_BIND_ADDRESS=10.244.1.190:8091\r\n+ export INFLUXDB_META_BIND_ADDRESS=10.244.1.190:8088\r\n+ INFLUXDB_META_BIND_ADDRESS=10.244.1.190:8088\r\n+ influxd --config /etc/influxdb.toml -join 10.244.2.205:8091,10.244.1.4:8091,10.244.2.188:8091\r\n\r\nNode 4: node.json\r\n-----------------------------------------\r\nroot@influxdb-peers-rc-75jd8:/data/meta# cat node.json\r\n{\"ID\":1,\"MetaServers\":[\"10.244.1.4:8091\",\"10.244.2.188:8091\",\"10.244.2.205:8091\",\"10.244.1.190:8091\"]}\r\n\r\n\r\nNode 4 - CLI:\r\n-----------------------------------------\r\n```\r\n> show servers\r\nname: data_nodes\r\n----------------\r\nid\thttp_addr\ttcp_addr\r\n1\tlocalhost:8086\t10.244.1.4:8088\r\n\r\nname: meta_nodes\r\n----------------\r\nid\thttp_addr\t\ttcp_addr\r\n1\t10.244.1.4:8091\t\t10.244.1.4:8088\r\n2\t10.244.2.188:8091\t10.244.2.188:8088\r\n3\t10.244.2.205:8091\t10.244.2.205:8088\r\n4\t10.244.1.190:8091\t10.244.1.190:8088\r\n\r\n```\r\n-----------------------------------------------------------------------------\r\nResult of cluster query shows only first host(influxdb-raft-rc-bg2hr)\r\n-----------------------------------------------------------------------------\r\n\r\n> select * from cluster where time > now() - 3h\r\nname: cluster\r\n-------------\r\ntime\t\t\t\tclusterID\t\t\thostname\t\t\tmapShardReq\tmapShardResp\tnodeID\r\n1455045073000000000\t3863944593121881570\tinfluxdb-raft-rc-bg2hr\t1\t\t1\t\t0\r\n1455045083000000000\t3863944593121881570\tinfluxdb-raft-rc-bg2hr\t2\t\t3\t\t0\r\n1455045093000000000\t3863944593121881570\tinfluxdb-raft-rc-bg2hr\t2\t\t3\t\t0\r\n1455045103000000000\t3863944593121881570\tinfluxdb-raft-rc-bg2hr\t2\t\t3\t\t0\r\n1455045113000000000\t3863944593121881570\tinfluxdb-raft-rc-bg2hr\t2\t\t3\t\t0\r\n1455045123000000000\t3863944593121881570\tinfluxdb-raft-rc-bg2hr\t3\t\t5\t\t0\r\n1455045133000000000\t3863944593121881570\tinfluxdb-raft-rc-bg2hr\t4\t\t6\t\t0\r\n1455045143000000000\t3863944593121881570\tinfluxdb-raft-rc-bg2hr\t4\t\t6\t\t0\r\n1455045153000000000\t3863944593121881570\tinfluxdb-raft-rc-bg2hr\t5\t\t8\t\t0\r\n1455045163000000000\t3863944593121881570\tinfluxdb-raft-rc-bg2hr\t5\t\t8\t\t0\r\n`"}}