{"action":"created","comment":{"url":"https://api.github.com/repos/apache/spark/pulls/comments/171822744","pull_request_review_id":100744199,"id":171822744,"diff_hunk":"@@ -36,22 +42,35 @@ public MemoryBlock allocate(long size) throws OutOfMemoryError {\n \n   @Override\n   public void free(MemoryBlock memory) {\n-    assert (memory.obj == null) :\n+    if (memory == OffHeapMemoryBlock.NULL) return;\n+    assert(memory instanceof OffHeapMemoryBlock);","path":"common/unsafe/src/main/java/org/apache/spark/unsafe/memory/UnsafeMemoryAllocator.java","position":30,"original_position":30,"commit_id":"cf2d532ae9c8688ef314a51a89c76abe2fd5d857","original_commit_id":"cf2d532ae9c8688ef314a51a89c76abe2fd5d857","user":{"login":"kiszk","id":1315079,"avatar_url":"https://avatars2.githubusercontent.com/u/1315079?v=4","gravatar_id":"","url":"https://api.github.com/users/kiszk","html_url":"https://github.com/kiszk","followers_url":"https://api.github.com/users/kiszk/followers","following_url":"https://api.github.com/users/kiszk/following{/other_user}","gists_url":"https://api.github.com/users/kiszk/gists{/gist_id}","starred_url":"https://api.github.com/users/kiszk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kiszk/subscriptions","organizations_url":"https://api.github.com/users/kiszk/orgs","repos_url":"https://api.github.com/users/kiszk/repos","events_url":"https://api.github.com/users/kiszk/events{/privacy}","received_events_url":"https://api.github.com/users/kiszk/received_events","type":"User","site_admin":false},"body":"good catch","created_at":"2018-03-02T11:18:14Z","updated_at":"2018-03-02T11:18:14Z","html_url":"https://github.com/apache/spark/pull/19222#discussion_r171822744","pull_request_url":"https://api.github.com/repos/apache/spark/pulls/19222","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/apache/spark/pulls/comments/171822744"},"html":{"href":"https://github.com/apache/spark/pull/19222#discussion_r171822744"},"pull_request":{"href":"https://api.github.com/repos/apache/spark/pulls/19222"}},"in_reply_to_id":171787333},"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/19222","id":140892682,"html_url":"https://github.com/apache/spark/pull/19222","diff_url":"https://github.com/apache/spark/pull/19222.diff","patch_url":"https://github.com/apache/spark/pull/19222.patch","issue_url":"https://api.github.com/repos/apache/spark/issues/19222","number":19222,"state":"open","locked":false,"title":"[SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks to choose several types of memory block","user":{"login":"kiszk","id":1315079,"avatar_url":"https://avatars2.githubusercontent.com/u/1315079?v=4","gravatar_id":"","url":"https://api.github.com/users/kiszk","html_url":"https://github.com/kiszk","followers_url":"https://api.github.com/users/kiszk/followers","following_url":"https://api.github.com/users/kiszk/following{/other_user}","gists_url":"https://api.github.com/users/kiszk/gists{/gist_id}","starred_url":"https://api.github.com/users/kiszk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kiszk/subscriptions","organizations_url":"https://api.github.com/users/kiszk/orgs","repos_url":"https://api.github.com/users/kiszk/repos","events_url":"https://api.github.com/users/kiszk/events{/privacy}","received_events_url":"https://api.github.com/users/kiszk/received_events","type":"User","site_admin":false},"body":"## What changes were proposed in this pull request?\r\n\r\nThis PR allows us to use one of several types of `MemoryBlock`, such as byte array, int array, long array, or `java.nio.DirectByteBuffer`. To use `java.nio.DirectByteBuffer` allows to have off heap memory which is automatically deallocated by JVM. `MemoryBlock`  class has primitive accessors like `Platform.getInt()`, `Platform.putint()`, or `Platform.copyMemory()`. \r\n\r\nThis PR uses `MemoryBlock` for `OffHeapColumnVector`, `UTF8String`, and other places. This PR can improve performance of operations involving memory accesses (e.g. `UTF8String.trim`) by 1.8x.\r\n\r\nFor now, this PR does not use `MemoryBlock` for `BufferHolder` based on @cloud-fan's [suggestion](https://github.com/apache/spark/pull/11494#issuecomment-309694290).\r\n\r\nSince this PR is a successor of #11494, close #11494. Many codes were ported from #11494. Many efforts were put here. **I think this PR should credit to @yzotov.**\r\n\r\n\r\nThis PR can achieve **1.1-1.4x performance improvements** for  operations in `UTF8String` or `Murmur3_x86_32`. Other operations are almost comparable performances.\r\n\r\nWithout this PR\r\n```\r\nOpenJDK 64-Bit Server VM 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13 on Linux 4.4.0-22-generic\r\nIntel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz\r\nOpenJDK 64-Bit Server VM 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13 on Linux 4.4.0-22-generic\r\nIntel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz\r\nHash byte arrays with length 268435487:  Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\r\n------------------------------------------------------------------------------------------------\r\nMurmur3_x86_32                                 526 /  536          0.0   131399881.5       1.0X\r\n\r\nUTF8String benchmark:                    Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\r\n------------------------------------------------------------------------------------------------\r\nhashCode                                       525 /  552       1022.6           1.0       1.0X\r\nsubstring                                      414 /  423       1298.0           0.8       1.3X\r\n```\r\n\r\nWith this PR\r\n```\r\nOpenJDK 64-Bit Server VM 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13 on Linux 4.4.0-22-generic\r\nIntel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz\r\nHash byte arrays with length 268435487:  Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\r\n------------------------------------------------------------------------------------------------\r\nMurmur3_x86_32                                 474 /  488          0.0   118552232.0       1.0X\r\n\r\nUTF8String benchmark:                    Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\r\n------------------------------------------------------------------------------------------------\r\nhashCode                                       476 /  480       1127.3           0.9       1.0X\r\nsubstring                                      287 /  291       1869.9           0.5       1.7X\r\n```\r\n\r\nBenchmark program\r\n```\r\ntest(\"benchmark Murmur3_x86_32\") {\r\n  val length = 8192 * 32768 + 31\r\n  val seed = 42L\r\n  val iters = 1 << 2\r\n  val random = new Random(seed)\r\n  val arrays = Array.fill[MemoryBlock](numArrays) {\r\n    val bytes = new Array[Byte](length)\r\n    random.nextBytes(bytes)\r\n    new ByteArrayMemoryBlock(bytes, Platform.BYTE_ARRAY_OFFSET, length)\r\n  }\r\n                                                                \r\n  val benchmark = new Benchmark(\"Hash byte arrays with length \" + length,\r\n    iters * numArrays, minNumIters = 20) \r\n  benchmark.addCase(\"HiveHasher\") { _: Int =>\r\n    var sum = 0L\r\n    for (_ <- 0L until iters) {\r\n      sum += HiveHasher.hashUnsafeBytesBlock(\r\n        arrays(i), Platform.BYTE_ARRAY_OFFSET, length)\r\n    }\r\n  }\r\n  benchmark.run()   \r\n}\r\n\r\ntest(\"benchmark UTF8String\") {\r\n  val N = 512 * 1024 * 1024\r\n  val iters = 2\r\n  val benchmark = new Benchmark(\"UTF8String benchmark\", N, minNumIters = 20)\r\n  val str0 = new java.io.StringWriter() { { for (i <- 0 until N) { write(\" \") } } }.toString\r\n  val s0 = UTF8String.fromString(str0)\r\n  benchmark.addCase(\"hashCode\") { _: Int =>\r\n    var h: Int = 0\r\n    for (_ <- 0L until iters) { h += s0.hashCode }\r\n  }\r\n  benchmark.addCase(\"substring\") { _: Int =>\r\n    var s: UTF8String = null\r\n    for (_ <- 0L until iters) { s = s0.substring(N / 2 - 5, N / 2 + 5) }\r\n  }\r\n  benchmark.run()\r\n}\r\n```\r\n\r\n  \r\n  \r\nI run [this benchmark program](https://gist.github.com/kiszk/94f75b506c93a663bbbc372ffe8f05de) using [the commit](https://github.com/apache/spark/pull/19222/commits/ee5a79861c18725fb1cd9b518cdfd2489c05b81d6). I got the following results:\r\n\r\n```\r\nOpenJDK 64-Bit Server VM 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12 on Linux 4.4.0-66-generic\r\nIntel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz\r\nMemory access benchmarks:                Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\r\n------------------------------------------------------------------------------------------------\r\nByteArrayMemoryBlock get/putInt()              220 /  221        609.3           1.6       1.0X\r\nPlatform get/putInt(byte[])                    220 /  236        610.9           1.6       1.0X\r\nPlatform get/putInt(Object)                    492 /  494        272.8           3.7       0.4X\r\nOnHeapMemoryBlock get/putLong()                322 /  323        416.5           2.4       0.7X\r\nlong[]                                         221 /  221        608.0           1.6       1.0X\r\nPlatform get/putLong(long[])                   321 /  321        418.7           2.4       0.7X\r\nPlatform get/putLong(Object)                   561 /  563        239.2           4.2       0.4X\r\n```\r\n\r\nI also run [this benchmark program](https://gist.github.com/kiszk/5fdb4e03733a5d110421177e289d1fb5) for comparing performance of `Platform.copyMemory()`.\r\n```\r\nOpenJDK 64-Bit Server VM 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12 on Linux 4.4.0-66-generic\r\nIntel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz\r\nPlatform copyMemory:                     Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\r\n------------------------------------------------------------------------------------------------\r\nObject to Object                              1961 / 1967          8.6         116.9       1.0X\r\nSystem.arraycopy Object to Object             1917 / 1921          8.8         114.3       1.0X\r\nbyte array to byte array                      1961 / 1968          8.6         116.9       1.0X\r\nSystem.arraycopy byte array to byte array      1909 / 1937          8.8         113.8       1.0X\r\nint array to int array                        1921 / 1990          8.7         114.5       1.0X\r\ndouble array to double array                  1918 / 1923          8.7         114.3       1.0X\r\nObject to byte array                          1961 / 1967          8.6         116.9       1.0X\r\nObject to short array                         1965 / 1972          8.5         117.1       1.0X\r\nObject to int array                           1910 / 1915          8.8         113.9       1.0X\r\nObject to float array                         1971 / 1978          8.5         117.5       1.0X\r\nObject to double array                        1919 / 1944          8.7         114.4       1.0X\r\nbyte array to Object                          1959 / 1967          8.6         116.8       1.0X\r\nint array to Object                           1961 / 1970          8.6         116.9       1.0X\r\ndouble array to Object                        1917 / 1924          8.8         114.3       1.0X\r\n```\r\n\r\nThese results show three facts:\r\n1. According to the second/third or sixth/seventh results in the first experiment, if we use `Platform.get/putInt(Object)`, we achieve more than 2x worse performance than `Platform.get/putInt(byte[])` with concrete type (i.e. `byte[]`).\r\n2. According to the second/third or fourth/fifth/sixth results in the first experiment, the fastest way to access an array element on Java heap is `array[]`. **Cons of `array[]` is that it is not possible to support unaligned-8byte access.**\r\n3. According to the first/second/third or fourth/sixth/seventh results in the first experiment, `getInt()/putInt() or getLong()/putLong()` in subclasses of `MemoryBlock` can achieve comparable performance to `Platform.get/putInt()` or `Platform.get/putLong()` with concrete type (second or sixth result). There is no overhead regarding virtual call.\r\n4. According to results in the second experiment, for `Platform.copy()`, to pass `Object` can achieve the same performance as to pass any type of primitive array as source or destination.\r\n5. According to second/fourth results in the second experiment, `Platform.copy()` can achieve the same performance as `System.arrayCopy`. **It would be good to use `Platform.copy()` since `Platform.copy()` can take any types for src and dst.**\r\n\r\n## How was this patch tested?\r\n\r\nAdded `UnsafeMemoryAllocator`\r\n","created_at":"2017-09-13T18:05:08Z","updated_at":"2018-03-02T11:18:14Z","closed_at":null,"merged_at":null,"merge_commit_sha":"2ae8eb59fdf6185bf0fad2add650212471fb10f8","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/spark/pulls/19222/commits","review_comments_url":"https://api.github.com/repos/apache/spark/pulls/19222/comments","review_comment_url":"https://api.github.com/repos/apache/spark/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/spark/issues/19222/comments","statuses_url":"https://api.github.com/repos/apache/spark/statuses/cf2d532ae9c8688ef314a51a89c76abe2fd5d857","head":{"label":"kiszk:SPARK-10399","ref":"SPARK-10399","sha":"cf2d532ae9c8688ef314a51a89c76abe2fd5d857","user":{"login":"kiszk","id":1315079,"avatar_url":"https://avatars2.githubusercontent.com/u/1315079?v=4","gravatar_id":"","url":"https://api.github.com/users/kiszk","html_url":"https://github.com/kiszk","followers_url":"https://api.github.com/users/kiszk/followers","following_url":"https://api.github.com/users/kiszk/following{/other_user}","gists_url":"https://api.github.com/users/kiszk/gists{/gist_id}","starred_url":"https://api.github.com/users/kiszk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kiszk/subscriptions","organizations_url":"https://api.github.com/users/kiszk/orgs","repos_url":"https://api.github.com/users/kiszk/repos","events_url":"https://api.github.com/users/kiszk/events{/privacy}","received_events_url":"https://api.github.com/users/kiszk/received_events","type":"User","site_admin":false},"repo":{"id":45008789,"name":"spark","full_name":"kiszk/spark","owner":{"login":"kiszk","id":1315079,"avatar_url":"https://avatars2.githubusercontent.com/u/1315079?v=4","gravatar_id":"","url":"https://api.github.com/users/kiszk","html_url":"https://github.com/kiszk","followers_url":"https://api.github.com/users/kiszk/followers","following_url":"https://api.github.com/users/kiszk/following{/other_user}","gists_url":"https://api.github.com/users/kiszk/gists{/gist_id}","starred_url":"https://api.github.com/users/kiszk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kiszk/subscriptions","organizations_url":"https://api.github.com/users/kiszk/orgs","repos_url":"https://api.github.com/users/kiszk/repos","events_url":"https://api.github.com/users/kiszk/events{/privacy}","received_events_url":"https://api.github.com/users/kiszk/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/kiszk/spark","description":"Mirror of Apache Spark","fork":true,"url":"https://api.github.com/repos/kiszk/spark","forks_url":"https://api.github.com/repos/kiszk/spark/forks","keys_url":"https://api.github.com/repos/kiszk/spark/keys{/key_id}","collaborators_url":"https://api.github.com/repos/kiszk/spark/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/kiszk/spark/teams","hooks_url":"https://api.github.com/repos/kiszk/spark/hooks","issue_events_url":"https://api.github.com/repos/kiszk/spark/issues/events{/number}","events_url":"https://api.github.com/repos/kiszk/spark/events","assignees_url":"https://api.github.com/repos/kiszk/spark/assignees{/user}","branches_url":"https://api.github.com/repos/kiszk/spark/branches{/branch}","tags_url":"https://api.github.com/repos/kiszk/spark/tags","blobs_url":"https://api.github.com/repos/kiszk/spark/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/kiszk/spark/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/kiszk/spark/git/refs{/sha}","trees_url":"https://api.github.com/repos/kiszk/spark/git/trees{/sha}","statuses_url":"https://api.github.com/repos/kiszk/spark/statuses/{sha}","languages_url":"https://api.github.com/repos/kiszk/spark/languages","stargazers_url":"https://api.github.com/repos/kiszk/spark/stargazers","contributors_url":"https://api.github.com/repos/kiszk/spark/contributors","subscribers_url":"https://api.github.com/repos/kiszk/spark/subscribers","subscription_url":"https://api.github.com/repos/kiszk/spark/subscription","commits_url":"https://api.github.com/repos/kiszk/spark/commits{/sha}","git_commits_url":"https://api.github.com/repos/kiszk/spark/git/commits{/sha}","comments_url":"https://api.github.com/repos/kiszk/spark/comments{/number}","issue_comment_url":"https://api.github.com/repos/kiszk/spark/issues/comments{/number}","contents_url":"https://api.github.com/repos/kiszk/spark/contents/{+path}","compare_url":"https://api.github.com/repos/kiszk/spark/compare/{base}...{head}","merges_url":"https://api.github.com/repos/kiszk/spark/merges","archive_url":"https://api.github.com/repos/kiszk/spark/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/kiszk/spark/downloads","issues_url":"https://api.github.com/repos/kiszk/spark/issues{/number}","pulls_url":"https://api.github.com/repos/kiszk/spark/pulls{/number}","milestones_url":"https://api.github.com/repos/kiszk/spark/milestones{/number}","notifications_url":"https://api.github.com/repos/kiszk/spark/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/kiszk/spark/labels{/name}","releases_url":"https://api.github.com/repos/kiszk/spark/releases{/id}","deployments_url":"https://api.github.com/repos/kiszk/spark/deployments","created_at":"2015-10-27T01:25:07Z","updated_at":"2017-09-24T18:21:56Z","pushed_at":"2018-03-02T06:19:14Z","git_url":"git://github.com/kiszk/spark.git","ssh_url":"git@github.com:kiszk/spark.git","clone_url":"https://github.com/kiszk/spark.git","svn_url":"https://github.com/kiszk/spark","homepage":null,"size":261137,"stargazers_count":1,"watchers_count":1,"language":"Scala","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":1,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":0,"open_issues":1,"watchers":1,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"862fa697d829cdddf0f25e5613c91b040f9d9652","user":{"login":"apache","id":47359,"avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":17165658,"name":"spark","full_name":"apache/spark","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/spark","description":"Mirror of Apache Spark","fork":false,"url":"https://api.github.com/repos/apache/spark","forks_url":"https://api.github.com/repos/apache/spark/forks","keys_url":"https://api.github.com/repos/apache/spark/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/spark/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/spark/teams","hooks_url":"https://api.github.com/repos/apache/spark/hooks","issue_events_url":"https://api.github.com/repos/apache/spark/issues/events{/number}","events_url":"https://api.github.com/repos/apache/spark/events","assignees_url":"https://api.github.com/repos/apache/spark/assignees{/user}","branches_url":"https://api.github.com/repos/apache/spark/branches{/branch}","tags_url":"https://api.github.com/repos/apache/spark/tags","blobs_url":"https://api.github.com/repos/apache/spark/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/spark/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/spark/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/spark/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/spark/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/spark/languages","stargazers_url":"https://api.github.com/repos/apache/spark/stargazers","contributors_url":"https://api.github.com/repos/apache/spark/contributors","subscribers_url":"https://api.github.com/repos/apache/spark/subscribers","subscription_url":"https://api.github.com/repos/apache/spark/subscription","commits_url":"https://api.github.com/repos/apache/spark/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/spark/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/spark/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/spark/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/spark/contents/{+path}","compare_url":"https://api.github.com/repos/apache/spark/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/spark/merges","archive_url":"https://api.github.com/repos/apache/spark/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/spark/downloads","issues_url":"https://api.github.com/repos/apache/spark/issues{/number}","pulls_url":"https://api.github.com/repos/apache/spark/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/spark/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/spark/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/spark/labels{/name}","releases_url":"https://api.github.com/repos/apache/spark/releases{/id}","deployments_url":"https://api.github.com/repos/apache/spark/deployments","created_at":"2014-02-25T08:00:08Z","updated_at":"2018-03-02T10:11:28Z","pushed_at":"2018-03-02T06:27:52Z","git_url":"git://github.com/apache/spark.git","ssh_url":"git@github.com:apache/spark.git","clone_url":"https://github.com/apache/spark.git","svn_url":"https://github.com/apache/spark","homepage":null,"size":268453,"stargazers_count":16317,"watchers_count":16317,"language":"Scala","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":15078,"mirror_url":null,"archived":false,"open_issues_count":450,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":15078,"open_issues":450,"watchers":16317,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/spark/pulls/19222"},"html":{"href":"https://github.com/apache/spark/pull/19222"},"issue":{"href":"https://api.github.com/repos/apache/spark/issues/19222"},"comments":{"href":"https://api.github.com/repos/apache/spark/issues/19222/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/spark/pulls/19222/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/spark/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/spark/pulls/19222/commits"},"statuses":{"href":"https://api.github.com/repos/apache/spark/statuses/cf2d532ae9c8688ef314a51a89c76abe2fd5d857"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"kiszk"}}