{"action":"created","issue":{"url":"https://api.github.com/repos/zeebe-io/zeebe/issues/2778","repository_url":"https://api.github.com/repos/zeebe-io/zeebe","labels_url":"https://api.github.com/repos/zeebe-io/zeebe/issues/2778/labels{/name}","comments_url":"https://api.github.com/repos/zeebe-io/zeebe/issues/2778/comments","events_url":"https://api.github.com/repos/zeebe-io/zeebe/issues/2778/events","html_url":"https://github.com/zeebe-io/zeebe/issues/2778","id":463207425,"node_id":"MDU6SXNzdWU0NjMyMDc0MjU=","number":2778,"title":"Create workflow instance shows unexpected latency pattern ","user":{"login":"menski","id":516296,"node_id":"MDQ6VXNlcjUxNjI5Ng==","avatar_url":"https://avatars2.githubusercontent.com/u/516296?v=4","gravatar_id":"","url":"https://api.github.com/users/menski","html_url":"https://github.com/menski","followers_url":"https://api.github.com/users/menski/followers","following_url":"https://api.github.com/users/menski/following{/other_user}","gists_url":"https://api.github.com/users/menski/gists{/gist_id}","starred_url":"https://api.github.com/users/menski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/menski/subscriptions","organizations_url":"https://api.github.com/users/menski/orgs","repos_url":"https://api.github.com/users/menski/repos","events_url":"https://api.github.com/users/menski/events{/privacy}","received_events_url":"https://api.github.com/users/menski/received_events","type":"User","site_admin":false},"labels":[{"id":1150260843,"node_id":"MDU6TGFiZWwxMTUwMjYwODQz","url":"https://api.github.com/repos/zeebe-io/zeebe/labels/Scope:%20broker","name":"Scope: broker","color":"ffddbb","default":false},{"id":1150248446,"node_id":"MDU6TGFiZWwxMTUwMjQ4NDQ2","url":"https://api.github.com/repos/zeebe-io/zeebe/labels/Status:%20Ready","name":"Status: Ready","color":"d5db20","default":false},{"id":1150253323,"node_id":"MDU6TGFiZWwxMTUwMjUzMzIz","url":"https://api.github.com/repos/zeebe-io/zeebe/labels/Type:%20Bug","name":"Type: Bug","color":"e11d21","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/zeebe-io/zeebe/milestones/22","html_url":"https://github.com/zeebe-io/zeebe/milestone/22","labels_url":"https://api.github.com/repos/zeebe-io/zeebe/milestones/22/labels","id":4408956,"node_id":"MDk6TWlsZXN0b25lNDQwODk1Ng==","number":22,"title":"GA Release readiness","description":"","creator":{"login":"Zelldon","id":2758593,"node_id":"MDQ6VXNlcjI3NTg1OTM=","avatar_url":"https://avatars2.githubusercontent.com/u/2758593?v=4","gravatar_id":"","url":"https://api.github.com/users/Zelldon","html_url":"https://github.com/Zelldon","followers_url":"https://api.github.com/users/Zelldon/followers","following_url":"https://api.github.com/users/Zelldon/following{/other_user}","gists_url":"https://api.github.com/users/Zelldon/gists{/gist_id}","starred_url":"https://api.github.com/users/Zelldon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Zelldon/subscriptions","organizations_url":"https://api.github.com/users/Zelldon/orgs","repos_url":"https://api.github.com/users/Zelldon/repos","events_url":"https://api.github.com/users/Zelldon/events{/privacy}","received_events_url":"https://api.github.com/users/Zelldon/received_events","type":"User","site_admin":false},"open_issues":6,"closed_issues":47,"state":"open","created_at":"2019-06-14T13:03:26Z","updated_at":"2019-07-10T14:02:11Z","due_on":"2019-07-17T07:00:00Z","closed_at":null},"comments":5,"created_at":"2019-07-02T12:43:12Z","updated_at":"2019-07-10T15:14:26Z","closed_at":null,"author_association":"MEMBER","body":"Running a small benchmark with `ghz` shows an unexpected number of delayed responses around 5s.\r\n\r\n```\r\n$ ghz -insecure --proto ./gateway-protocol/src/main/proto/gateway.proto --call gateway_protocol.Gateway.CreateWorkflowInstance -d '{\"workflowKey\": 2251799813685249}' -n 2000 localhost:26500\r\n\r\nSummary:\r\n  Count:        2000\r\n  Total:        73683.84 ms\r\n  Slowest:      5075.27 ms\r\n  Fastest:      25.08 ms\r\n  Average:      1742.50 ms\r\n  Requests/sec: 27.14\r\n\r\nResponse time histogram:\r\n  25.077 [1]    |\r\n  530.096 [1339]|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\r\n  1035.115 [0]  |\r\n  1540.135 [0]  |\r\n  2045.154 [0]  |\r\n  2550.173 [0]  |\r\n  3055.192 [0]  |\r\n  3560.211 [0]  |\r\n  4065.230 [0]  |\r\n  4570.249 [0]  |\r\n  5075.268 [660]|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\r\n\r\nLatency distribution:\r\n  10% in 60.69 ms\r\n  25% in 116.77 ms\r\n  50% in 157.56 ms\r\n  75% in 5019.84 ms\r\n  90% in 5039.01 ms\r\n  95% in 5048.32 ms\r\n  99% in 5061.54 ms\r\nStatus code distribution:\r\n  [OK]  2000 responses\r\n\r\n```"},"comment":{"url":"https://api.github.com/repos/zeebe-io/zeebe/issues/comments/510104834","html_url":"https://github.com/zeebe-io/zeebe/issues/2778#issuecomment-510104834","issue_url":"https://api.github.com/repos/zeebe-io/zeebe/issues/2778","id":510104834,"node_id":"MDEyOklzc3VlQ29tbWVudDUxMDEwNDgzNA==","user":{"login":"npepinpe","id":43373,"node_id":"MDQ6VXNlcjQzMzcz","avatar_url":"https://avatars0.githubusercontent.com/u/43373?v=4","gravatar_id":"","url":"https://api.github.com/users/npepinpe","html_url":"https://github.com/npepinpe","followers_url":"https://api.github.com/users/npepinpe/followers","following_url":"https://api.github.com/users/npepinpe/following{/other_user}","gists_url":"https://api.github.com/users/npepinpe/gists{/gist_id}","starred_url":"https://api.github.com/users/npepinpe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/npepinpe/subscriptions","organizations_url":"https://api.github.com/users/npepinpe/orgs","repos_url":"https://api.github.com/users/npepinpe/repos","events_url":"https://api.github.com/users/npepinpe/events{/privacy}","received_events_url":"https://api.github.com/users/npepinpe/received_events","type":"User","site_admin":false},"created_at":"2019-07-10T15:14:26Z","updated_at":"2019-07-10T15:14:26Z","author_association":"MEMBER","body":"So I'm not sure about the test anymore...primarily because if you add more logs, it seems to be sending almost all requests before even processing a single one (so obviously last ones are slower). If you add the following before you define the `handler`:\r\n\r\n```java\r\n    // force initial connection? seems like it\r\n    netty2.registerHandler(\"connector\", (ep, data) -> CompletableFuture.completedFuture(new byte[0]));\r\n    netty1.sendAndReceive(netty2.address(), \"connector\", new byte[0]).join();\r\n```\r\n\r\n\r\nThen put a breakpoint on the next line, wait until you see \r\n\r\n```\r\n17:13:15.786 [main] DEBUG i.a.c.messaging.impl.ChannelPool - Connecting to com-camunda-npt470:45167\r\n17:13:15.890 [netty-messaging-event-epoll-client-0] DEBUG i.a.c.m.impl.NettyMessagingService - Writing client protocol version V2 for connection to com-camunda-npt470/192.168.30.69:45167\r\n17:13:15.974 [netty-messaging-event-epoll-client-0] DEBUG i.a.c.m.impl.NettyMessagingService - Activating server protocol version V2 for connection to /192.168.30.69:49420\r\n17:13:15.978 [netty-messaging-event-epoll-client-0] DEBUG i.a.c.m.impl.NettyMessagingService - Activating client protocol version V2 for connection to com-camunda-npt470/192.168.30.69:45167\r\n17:13:15.991 [netty-messaging-event-epoll-client-0] DEBUG i.a.c.messaging.impl.ChannelPool - Connected to com-camunda-npt470/192.168.30.69:45167\r\n```\r\n\r\nIn the logs...then continue, you get what seem to me to be \"normal\" latencies. But I'm not 100% sure about what's going on tbh :sweat_smile: "}}
{"actor":{"display_login":"npepinpe"}}