{"action":"created","issue":{"url":"https://api.github.com/repos/Netflix/dyno/issues/227","repository_url":"https://api.github.com/repos/Netflix/dyno","labels_url":"https://api.github.com/repos/Netflix/dyno/issues/227/labels{/name}","comments_url":"https://api.github.com/repos/Netflix/dyno/issues/227/comments","events_url":"https://api.github.com/repos/Netflix/dyno/issues/227/events","html_url":"https://github.com/Netflix/dyno/issues/227","id":335587270,"node_id":"MDU6SXNzdWUzMzU1ODcyNzA=","number":227,"title":"Exception when I set localZone and localDatacenter","user":{"login":"marimcmurtrie","id":1384071,"node_id":"MDQ6VXNlcjEzODQwNzE=","avatar_url":"https://avatars2.githubusercontent.com/u/1384071?v=4","gravatar_id":"","url":"https://api.github.com/users/marimcmurtrie","html_url":"https://github.com/marimcmurtrie","followers_url":"https://api.github.com/users/marimcmurtrie/followers","following_url":"https://api.github.com/users/marimcmurtrie/following{/other_user}","gists_url":"https://api.github.com/users/marimcmurtrie/gists{/gist_id}","starred_url":"https://api.github.com/users/marimcmurtrie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marimcmurtrie/subscriptions","organizations_url":"https://api.github.com/users/marimcmurtrie/orgs","repos_url":"https://api.github.com/users/marimcmurtrie/repos","events_url":"https://api.github.com/users/marimcmurtrie/events{/privacy}","received_events_url":"https://api.github.com/users/marimcmurtrie/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2018-06-25T22:15:25Z","updated_at":"2018-07-09T22:44:54Z","closed_at":null,"author_association":"NONE","body":"I have Dynomite running in remote server. The cluster has 1 rack with 2 nodes.\r\n\r\nIn the Dynomite server,\r\n```\r\ncurl -X GET maritest.abc.com:22222/cluster_describe\r\n{\"dcs\": [{\"name\":\"pnp\",\"racks\": [{\"name\":\"rack1\",\"servers\": [{\"name\":\"0.0.0.0\",\"host\":\"0.0.0.0\",\"port\":8103,\r\n\"token\":4294967294\r\n},{\"name\":\"127.0.0.1\",\"host\":\"127.0.0.1\",\"port\":8101,\r\n\"token\":2147483647\r\n}]}]}]}\r\n```\r\n\r\nIn Client side, I use DynoJedisClient. I supply my own HostSupplier and TokenMapSupplier.\r\n\r\n```\r\nHost host = new Host(\"maritest.abc.com\", null, 8102, \"rack1\", \"pnp\", Host.Status.Up);\r\nHostSupplier customHostSupplier = new HostSupplier() {\r\n            @Override\r\n            public List<Host> getHosts() {\r\n                return Collections.singletonList(host);\r\n            }\r\n        };\r\n\r\nString tokenMapJson = \"\r\n[{\r\n \t\\\"token\\\": \\\"2147483647\\\",\r\n \t\\\"hostname\\\": \\\"maritest.abc.com\\\",\r\n \t\\\"zone\\\": \\\"rack1\\\",\r\n \t\\\"dc\":\\ \"pnp\\\"\r\n }, {\r\n \t\\\"token\\\": \\\"4294967294\\\",\r\n \t\\\"hostname\\\": \\\"maritest.abc.com\\\",\r\n \t\\\"zone\\\": \\\"rack1\\\",\r\n \t\\\"dc\\\": \\\"pnp\\\"\r\n }]\r\n\";\r\nTokenMapSupplier tokenSupplier = new AbstractTokenMapSupplier() {\r\n            @Override\r\n            public String getTopologyJsonPayload(Set<Host> set) {\r\n                return tokenMapJson;\r\n            }\r\n            @Override\r\n            public String getTopologyJsonPayload(String hostname) {\r\n                return tokenMapJson;\r\n            }\r\n        };\r\n```\r\n\r\nThen I instantiate DynoJedisClient\r\n\r\n```\r\n dynoClient = new DynoJedisClient.Builder()\r\n                .withApplicationName(appName)\r\n                .withHostSupplier(customHostSupplier)\r\n                .withTokenMapSupplier(tokenSupplier)\r\n                .build();\r\n```\r\n\r\nIn log, I get these warnings\r\n\r\n> WARN  [2018-06-25 15:05:58,542] com.netflix.dyno.jedis.DynoJedisClient: DynoJedisClient for app=[webproxy] is configured for local rack affinity but cannot determine the local rack! DISABLING rack affinity for this instance. To make the client aware of the local rack either use ConnectionPoolConfigurationImpl.setLocalRack() when constructing the client instance or ensure EC2_AVAILABILTY_ZONE is set as an environment variable, e.g. run with -DEC2_AVAILABILITY_ZONE=us-east-1c [main]\r\nWARN  [2018-06-25 15:05:58,550] com.netflix.dyno.connectionpool.impl.lb.AbstractTokenMapSupplier: Local Datacenter was not defined [main]\r\n\r\nSo I added those 2 variables before I instantiate DynoJedisClient.\r\n\r\n ```\r\n       System.setProperty(\"EC2_REGION\", \"pnp\");\r\n       System.setProperty(\"EC2_AVAILABILITY_ZONE\", \"rack1\");\r\n```\r\nIf I supply the above 2 variables, I get an Exception.\r\n\r\nIn Log\r\n> ERROR [2018-06-25 14:05:32,080] io.dropwizard.jersey.errors.LoggingExceptionMapper: Error handling a request: 4ec7ab819b9ca7eb [dw-51 - GET /redis/allkeys]\r\n! com.netflix.dyno.connectionpool.exception.PoolOfflineException: PoolOfflineException: [host=Host [hostname=UNKNOWN, ipAddress=UNKNOWN, port=0, rack: UNKNOWN, datacenter: UNKNOW, status: Down, hashtag=null], latency=0(0), attempts=0]host pool is offline and no Racks available for fallback\r\n! at com.netflix.dyno.connectionpool.impl.lb.HostSelectionWithFallback.getConnection(HostSelectionWithFallback.java:163)\r\n! at com.netflix.dyno.connectionpool.impl.lb.HostSelectionWithFallback.getConnectionsToRing(HostSelectionWithFallback.java:256)\r\n! at com.netflix.dyno.connectionpool.impl.ConnectionPoolImpl.executeWithRing(ConnectionPoolImpl.java:366)\r\n! at com.netflix.dyno.jedis.DynoJedisClient.d_keys(DynoJedisClient.java:2663)\r\n! at com.netflix.dyno.jedis.DynoJedisClient.keys(DynoJedisClient.java:2643)\r\n\r\nIn Console\r\n\r\n> java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 0\r\n\tat com.netflix.dyno.jedis.DynoJedisClient$Builder.startConnectionPool(DynoJedisClient.java:4059)\r\n\tat com.netflix.dyno.jedis.DynoJedisClient$Builder.createConnectionPool(DynoJedisClient.java:4031)\r\n\tat com.netflix.dyno.jedis.DynoJedisClient$Builder.buildDynoJedisClient(DynoJedisClient.java:4007)\r\n\tat com.netflix.dyno.jedis.DynoJedisClient$Builder.build(DynoJedisClient.java:3936)\r\nCaused by: java.lang.ArrayIndexOutOfBoundsException: 0\r\n\tat com.netflix.dyno.connectionpool.impl.lb.HostSelectionWithFallback.calculateReplicationFactor(HostSelectionWithFallback.java:396)\r\n\tat com.netflix.dyno.connectionpool.impl.lb.HostSelectionWithFallback.initWithHosts(HostSelectionWithFallback.java:353)\r\n\tat com.netflix.dyno.connectionpool.impl.ConnectionPoolImpl.initSelectionStrategy(ConnectionPoolImpl.java:627)\r\n\tat com.netflix.dyno.connectionpool.impl.ConnectionPoolImpl.start(ConnectionPoolImpl.java:526)\r\n\tat com.netflix.dyno.jedis.DynoJedisClient$Builder.startConnectionPool(DynoJedisClient.java:4042)\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/Netflix/dyno/issues/comments/403643358","html_url":"https://github.com/Netflix/dyno/issues/227#issuecomment-403643358","issue_url":"https://api.github.com/repos/Netflix/dyno/issues/227","id":403643358,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMzY0MzM1OA==","user":{"login":"marimcmurtrie","id":1384071,"node_id":"MDQ6VXNlcjEzODQwNzE=","avatar_url":"https://avatars2.githubusercontent.com/u/1384071?v=4","gravatar_id":"","url":"https://api.github.com/users/marimcmurtrie","html_url":"https://github.com/marimcmurtrie","followers_url":"https://api.github.com/users/marimcmurtrie/followers","following_url":"https://api.github.com/users/marimcmurtrie/following{/other_user}","gists_url":"https://api.github.com/users/marimcmurtrie/gists{/gist_id}","starred_url":"https://api.github.com/users/marimcmurtrie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marimcmurtrie/subscriptions","organizations_url":"https://api.github.com/users/marimcmurtrie/orgs","repos_url":"https://api.github.com/users/marimcmurtrie/repos","events_url":"https://api.github.com/users/marimcmurtrie/events{/privacy}","received_events_url":"https://api.github.com/users/marimcmurtrie/received_events","type":"User","site_admin":false},"created_at":"2018-07-09T22:44:54Z","updated_at":"2018-07-09T22:44:54Z","author_association":"NONE","body":"Here is my Dynomite cluster. I have 2 racks, each rack has 1 node.\r\n\r\n**redis_rack1_node_mari.yml in maritest.abc.com (216.22.222.2)**\r\n```\r\ndyn_o_mite:\r\n  datacenter: dc\r\n  rack: rack1\r\n  dyn_listen: 0.0.0.0:8101\r\n  dyn_seeds:\r\n  - 216.11.111.1:8101:rack2:dc:1383429731\r\n  listen: 0.0.0.0:8102\r\n  servers:\r\n  - 127.0.0.1:6379:1\r\n  tokens: '1383429731'\r\n  secure_server_option: datacenter\r\n  pem_key_file: conf/dynomite.pem\r\n  data_store: 0\r\n  read_consistency : DC_SAFE_QUORUM\r\n  write_consistency : DC_SAFE_QUORUM\r\n  stats_listen: 0.0.0.0:22221\r\n```\r\n**redis_rack2_node_mari.yml in maritest.zzz.com (216.11.111.1)**\r\n```\r\ndyn_o_mite:\r\n  datacenter: dc\r\n  rack: rack2\r\n  dyn_listen: 0.0.0.0:8101\r\n  dyn_seeds:\r\n  - 216.22.222.2:8101:rack1:dc:1383429731\r\n  listen: 0.0.0.0:8102\r\n  servers:\r\n  - 127.0.0.1:6377:1\r\n  tokens: '1383429731'\r\n  secure_server_option: datacenter\r\n  pem_key_file: conf/dynomite.pem\r\n  data_store: 0\r\n  read_consistency : DC_SAFE_QUORUM\r\n  write_consistency : DC_SAFE_QUORUM\r\n  stats_listen: 0.0.0.0:22222\r\n```\r\n\r\nMy Dyno client code\r\n**json for Token map** \r\n\r\n```\r\n[{\r\n\t\"token\": \"1383429731\",\r\n\t\"hostname\": \"maritest.abc.com\",\r\n\t\"zone\": \"rack1\",\r\n\t\"dc\": \"dc\"\r\n}, {\r\n\t\"token\": \"1383429731\",\r\n\t\"hostname\": \"maritest.zzz.com\",\r\n\t\"zone\": \"rack2\",\r\n\t\"dc\": \"dc\"\r\n}]\r\n```\r\n**hosts List for getHosts()** \r\n```\r\n[Host [hostname=maritest.abc.com, ipAddress=null, port=8102, rack: rack1, datacenter: dc, status: Up, hashtag=null], \r\nHost [hostname=maritest.zzz.com, ipAddress=null, port=8102, rack: rack2, datacenter: dc, status: Up, hashtag=null]]\r\n```\r\nCode snippets\r\n```\r\n      //If I comment out below 2 line, code works. If uncomment, Exception is thrown.\r\n      System.setProperty(\"EC2_REGION\", \"dc\");\r\n      System.setProperty(\"EC2_AVAILABILITY_ZONE\", \"rack1\");\r\n\r\n      HostSupplier customHostSupplier = new HostSupplier() {\r\n            @Override\r\n            public List<Host> getHosts() {\r\n                return hosts; //See above for the actual value\r\n            }\r\n        };\r\n\r\n        final String tokenMapJson = tokenMapArrayNode.toString();\r\n        TokenMapSupplier tokenSupplier = new AbstractTokenMapSupplier() {\r\n            @Override\r\n            public String getTopologyJsonPayload(Set<Host> set) {\r\n                return tokenMapJson; //See above for actual value\r\n            }\r\n            @Override\r\n            public String getTopologyJsonPayload(String hostname) {\r\n                return tokenMapJson;\r\n            }\r\n        };\r\n\r\n        dynoClient = new DynoJedisClient.Builder()\r\n                .withApplicationName(\"myApp\")\r\n                .withHostSupplier(customHostSupplier)\r\n                .withTokenMapSupplier(tokenSupplier)\r\n                .build();\r\n```"}}
{"actor":{"display_login":"marimcmurtrie"}}