{"action":"created","issue":{"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/969","repository_url":"https://api.github.com/repos/datastax/spark-cassandra-connector","labels_url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/969/labels{/name}","comments_url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/969/comments","events_url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/969/events","html_url":"https://github.com/datastax/spark-cassandra-connector/pull/969","id":149501202,"number":969,"title":"SPARKC-233 use executeAsync when joining with C* table","user":{"login":"jtgrabowski","id":1613130,"avatar_url":"https://avatars.githubusercontent.com/u/1613130?v=3","gravatar_id":"","url":"https://api.github.com/users/jtgrabowski","html_url":"https://github.com/jtgrabowski","followers_url":"https://api.github.com/users/jtgrabowski/followers","following_url":"https://api.github.com/users/jtgrabowski/following{/other_user}","gists_url":"https://api.github.com/users/jtgrabowski/gists{/gist_id}","starred_url":"https://api.github.com/users/jtgrabowski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jtgrabowski/subscriptions","organizations_url":"https://api.github.com/users/jtgrabowski/orgs","repos_url":"https://api.github.com/users/jtgrabowski/repos","events_url":"https://api.github.com/users/jtgrabowski/events{/privacy}","received_events_url":"https://api.github.com/users/jtgrabowski/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2016-04-19T15:46:02Z","updated_at":"2016-04-21T14:05:40Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/pulls/969","html_url":"https://github.com/datastax/spark-cassandra-connector/pull/969","diff_url":"https://github.com/datastax/spark-cassandra-connector/pull/969.diff","patch_url":"https://github.com/datastax/spark-cassandra-connector/pull/969.patch"},"body":"This ought to speed up joining with C* table by using executeAsync\r\n(instead of execute) and prematerializing C* ResultSets during join\r\noperation. Concurrent querying is controlled by parameter\r\nspark.cassandra.input.join.concurrent.reads\r\nIt specifies maximum number of ongoing executeAsync calls and thus\r\nlimits maximum number of prematerialized ResultSets.\r\n\r\nBasically we buy some time for the cost of memory.\r\n\r\nThis also adds performance module with jmh plugin. Within module\r\nthere is a simple macrobenchmark that was used to generate following results:\r\n\r\n\r\n\r\n\r\nbefore:\r\n\r\nBenchmark |Mode | Cnt | Score | Error | Units\r\n----------|-----|-----|-------|-------|------\r\nmeasureOneElementPerPartitionsJoin | ss | 15 | 19252.805| ± 390.205| ms/op\r\nmeasureSmallPartitionsJoin | ss | 15 | 20034.163| ± 1515.168| ms/op\r\nmeasureModeratePartitionsJoin | ss | 15 | 20278.329| ± 1689.112| ms/op\r\nmeasureBigPartitionsJoin | ss | 15 | 17515.329| ± 1420.479| ms/op\r\n\r\n\r\nafter:\r\n\r\nBenchmark |Mode | Cnt | Score | Error | Units\r\n----------|-----|-----|-------|-------|------\r\nmeasureOneElementPerPartitionsJoin | ss | 15 | 8378.778 | ± 665.592 | ms/op\r\nmeasureSmallPartitionsJoin | ss | 15 | 8468.695 | ± 891.526 | ms/op\r\nmeasureModeratePartitionsJoin | ss | 15 | 8689.495 | ± 924.087 | ms/op\r\nmeasureBigPartitionsJoin | ss | 15 | 8891.763 | ± 1208.863 | ms/op\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/comments/212935134","html_url":"https://github.com/datastax/spark-cassandra-connector/pull/969#issuecomment-212935134","issue_url":"https://api.github.com/repos/datastax/spark-cassandra-connector/issues/969","id":212935134,"user":{"login":"ds-jenkins-builds","id":7905753,"avatar_url":"https://avatars.githubusercontent.com/u/7905753?v=3","gravatar_id":"","url":"https://api.github.com/users/ds-jenkins-builds","html_url":"https://github.com/ds-jenkins-builds","followers_url":"https://api.github.com/users/ds-jenkins-builds/followers","following_url":"https://api.github.com/users/ds-jenkins-builds/following{/other_user}","gists_url":"https://api.github.com/users/ds-jenkins-builds/gists{/gist_id}","starred_url":"https://api.github.com/users/ds-jenkins-builds/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ds-jenkins-builds/subscriptions","organizations_url":"https://api.github.com/users/ds-jenkins-builds/orgs","repos_url":"https://api.github.com/users/ds-jenkins-builds/repos","events_url":"https://api.github.com/users/ds-jenkins-builds/events{/privacy}","received_events_url":"https://api.github.com/users/ds-jenkins-builds/received_events","type":"User","site_admin":false},"created_at":"2016-04-21T14:05:40Z","updated_at":"2016-04-21T14:05:40Z","body":"Build against Scala 2.10 finished with success\n"}}