{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15071","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15071/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15071/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15071/events","html_url":"https://github.com/apache/spark/pull/15071","id":176554831,"number":15071,"title":"[SPARK-17517][SQL]Improve generated Code for BroadcastHashJoinExec","user":{"login":"yaooqinn","id":8326978,"avatar_url":"https://avatars.githubusercontent.com/u/8326978?v=3","gravatar_id":"","url":"https://api.github.com/users/yaooqinn","html_url":"https://github.com/yaooqinn","followers_url":"https://api.github.com/users/yaooqinn/followers","following_url":"https://api.github.com/users/yaooqinn/following{/other_user}","gists_url":"https://api.github.com/users/yaooqinn/gists{/gist_id}","starred_url":"https://api.github.com/users/yaooqinn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yaooqinn/subscriptions","organizations_url":"https://api.github.com/users/yaooqinn/orgs","repos_url":"https://api.github.com/users/yaooqinn/repos","events_url":"https://api.github.com/users/yaooqinn/events{/privacy}","received_events_url":"https://api.github.com/users/yaooqinn/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":9,"created_at":"2016-09-13T05:45:59Z","updated_at":"2016-09-22T01:43:07Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15071","html_url":"https://github.com/apache/spark/pull/15071","diff_url":"https://github.com/apache/spark/pull/15071.diff","patch_url":"https://github.com/apache/spark/pull/15071.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nFor current `BroadcastHashJoinExec`, we generate join code for key is not unique like this: \r\n\r\n```\r\nwhile (matches.hasnext) {\r\n    matched = matches.next\r\n    check and read stream side row fields\r\n    check and read build side row fieldes\r\n    reset result row\r\n    write stream side row fields to result row\r\n    write build side row fields to result row\r\n    append(result row)\r\n}\r\n```\r\n\r\nFor some cases, we don't need to check/read/write the steam side repeatedly in such while circle, e.g. `Inner Join with BuildRight`, or `BuildLeft &&  all left side fields are fixed length` and so on. we may generate the code as below:\r\n\r\n```\r\ncheck and read stream side row fields\r\nreset result row\r\nwrite stream side row fields to result row\r\nwhile (matches.hasnext)\r\n{\r\n    matched = matches.next\r\n    check and read build side row fieldes\r\n    write build side row fields to result row\r\n    append(result row)\r\n}\r\n```\r\n\r\n## How was this patch tested?\r\n\r\n### 1. add ut\r\n\r\n### 2. benchmark\r\n```\r\n   Java HotSpot(TM) 64-Bit Server VM 1.8.0_65-b17 on Mac OS X 10.11.6\r\n    Intel(R) Core(TM) i5-5287U CPU @ 2.90GHz\r\n\r\n    BroadcastHashJoin w duplicated keys:     Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\r\n    ------------------------------------------------------------------------------------------------\r\n    codegen = F                                 13933 / 13974          7.5         132.9       1.0X\r\n    codegen = T avoid = F                          253 /  273        414.0           2.4      55.0X\r\n    codegen = T avoid = T                          202 /  211        518.7           1.9      68.9X\r\n    \r\n```\r\n### 3. manually check\r\n\r\n#### before\r\n```java\r\n/* 105 */     while (scan_batch != null) {\r\n/* 106 */       int numRows = scan_batch.numRows();\r\n/* 107 */       while (scan_batchIdx < numRows) {\r\n/* 108 */         int scan_rowIdx = scan_batchIdx++;\r\n/* 109 */         boolean scan_isNull = scan_colInstance0.isNullAt(scan_rowIdx);\r\n/* 110 */         int scan_value = scan_isNull ? -1 : (scan_colInstance0.getInt(scan_rowIdx));\r\n/* 111 */\r\n/* 112 */         if (!(!(scan_isNull))) continue;\r\n/* 113 */\r\n/* 114 */         boolean filter_isNull2 = false;\r\n/* 115 */\r\n/* 116 */         boolean filter_value2 = false;\r\n/* 117 */         filter_value2 = scan_value == 1;\r\n/* 118 */         if (!filter_value2) continue;\r\n/* 119 */\r\n/* 120 */         filter_numOutputRows.add(1);\r\n/* 121 */\r\n/* 122 */         // generate join key for stream side\r\n/* 123 */\r\n/* 124 */         boolean bhj_isNull = false;\r\n/* 125 */         long bhj_value = -1L;\r\n/* 126 */         if (!false) {\r\n/* 127 */           bhj_value = (long) scan_value;\r\n/* 128 */         }\r\n/* 129 */         // find matches from HashRelation\r\n/* 130 */         scala.collection.Iterator bhj_matches =\r\n/* 131 */         bhj_isNull ? null : (scala.collection.Iterator)bhj_relation.get(bhj_value);\r\n/* 132 */         if (bhj_matches == null) continue;\r\n/* 133 */         while (bhj_matches.hasNext()) {\r\n/* 134 */           UnsafeRow bhj_matched = (UnsafeRow) bhj_matches.next();\r\n/* 135 */\r\n/* 136 */           bhj_numOutputRows.add(1);\r\n/* 137 */\r\n/* 138 */           boolean scan_isNull1 = scan_colInstance1.isNullAt(scan_rowIdx);\r\n/* 139 */           UTF8String scan_value1 = scan_isNull1 ? null : (scan_colInstance1.getUTF8String(scan_rowIdx));\r\n/* 140 */           boolean scan_isNull2 = scan_colInstance2.isNullAt(scan_rowIdx);\r\n/* 141 */           UTF8String scan_value2 = scan_isNull2 ? null : (scan_colInstance2.getUTF8String(scan_rowIdx));\r\n/* 142 */           boolean scan_isNull3 = scan_colInstance3.isNullAt(scan_rowIdx);\r\n/* 143 */           int scan_value3 = scan_isNull3 ? -1 : (scan_colInstance3.getInt(scan_rowIdx));\r\n/* 144 */           boolean scan_isNull4 = scan_colInstance4.isNullAt(scan_rowIdx);\r\n/* 145 */           UTF8String scan_value4 = scan_isNull4 ? null : (scan_colInstance4.getUTF8String(scan_rowIdx));\r\n/* 146 */           boolean scan_isNull5 = scan_colInstance5.isNullAt(scan_rowIdx);\r\n/* 147 */           UTF8String scan_value5 = scan_isNull5 ? null : (scan_colInstance5.getUTF8String(scan_rowIdx));\r\n/* 148 */           boolean scan_isNull6 = scan_colInstance6.isNullAt(scan_rowIdx);\r\n/* 149 */           UTF8String scan_value6 = scan_isNull6 ? null : (scan_colInstance6.getUTF8String(scan_rowIdx));\r\n/* 150 */           boolean scan_isNull7 = scan_colInstance7.isNullAt(scan_rowIdx);\r\n/* 151 */           UTF8String scan_value7 = scan_isNull7 ? null : (scan_colInstance7.getUTF8String(scan_rowIdx));\r\n/* 152 */           boolean scan_isNull8 = scan_colInstance8.isNullAt(scan_rowIdx);\r\n/* 153 */           int scan_value8 = scan_isNull8 ? -1 : (scan_colInstance8.getInt(scan_rowIdx));\r\n/* 154 */           boolean bhj_isNull2 = bhj_matched.isNullAt(0);\r\n/* 155 */           int bhj_value2 = bhj_isNull2 ? -1 : (bhj_matched.getInt(0));\r\n/* 156 */           boolean bhj_isNull3 = bhj_matched.isNullAt(1);\r\n/* 157 */           UTF8String bhj_value3 = bhj_isNull3 ? null : (bhj_matched.getUTF8String(1));\r\n/* 158 */           boolean bhj_isNull4 = bhj_matched.isNullAt(2);\r\n/* 159 */           UTF8String bhj_value4 = bhj_isNull4 ? null : (bhj_matched.getUTF8String(2));\r\n/* 160 */           boolean bhj_isNull5 = bhj_matched.isNullAt(3);\r\n/* 161 */           int bhj_value5 = bhj_isNull5 ? -1 : (bhj_matched.getInt(3));\r\n/* 162 */           boolean bhj_isNull6 = bhj_matched.isNullAt(4);\r\n/* 163 */           UTF8String bhj_value6 = bhj_isNull6 ? null : (bhj_matched.getUTF8String(4));\r\n/* 164 */           boolean bhj_isNull7 = bhj_matched.isNullAt(5);\r\n/* 165 */           UTF8String bhj_value7 = bhj_isNull7 ? null : (bhj_matched.getUTF8String(5));\r\n/* 166 */           boolean bhj_isNull8 = bhj_matched.isNullAt(6);\r\n/* 167 */           UTF8String bhj_value8 = bhj_isNull8 ? null : (bhj_matched.getUTF8String(6));\r\n/* 168 */           boolean bhj_isNull9 = bhj_matched.isNullAt(7);\r\n/* 169 */           UTF8String bhj_value9 = bhj_isNull9 ? null : (bhj_matched.getUTF8String(7));\r\n/* 170 */           boolean bhj_isNull10 = bhj_matched.isNullAt(8);\r\n/* 171 */           int bhj_value10 = bhj_isNull10 ? -1 : (bhj_matched.getInt(8));\r\n/* 172 */           bhj_holder.reset();\r\n/* 173 */\r\n/* 174 */           bhj_rowWriter.zeroOutNullBytes();\r\n/* 175 */\r\n/* 176 */           bhj_rowWriter.write(0, scan_value);\r\n/* 177 */\r\n/* 178 */           if (scan_isNull1) {\r\n/* 179 */             bhj_rowWriter.setNullAt(1);\r\n/* 180 */           } else {\r\n/* 181 */             bhj_rowWriter.write(1, scan_value1);\r\n/* 182 */           }\r\n/* 183 */\r\n/* 184 */           if (scan_isNull2) {\r\n/* 185 */             bhj_rowWriter.setNullAt(2);\r\n/* 186 */           } else {\r\n/* 187 */             bhj_rowWriter.write(2, scan_value2);\r\n/* 188 */           }\r\n/* 189 */\r\n/* 190 */           if (scan_isNull3) {\r\n/* 191 */             bhj_rowWriter.setNullAt(3);\r\n/* 192 */           } else {\r\n/* 193 */             bhj_rowWriter.write(3, scan_value3);\r\n/* 194 */           }\r\n/* 195 */\r\n/* 196 */           if (scan_isNull4) {\r\n/* 197 */             bhj_rowWriter.setNullAt(4);\r\n/* 198 */           } else {\r\n/* 199 */             bhj_rowWriter.write(4, scan_value4);\r\n/* 200 */           }\r\n/* 201 */\r\n/* 202 */           if (scan_isNull5) {\r\n/* 203 */             bhj_rowWriter.setNullAt(5);\r\n/* 204 */           } else {\r\n/* 205 */             bhj_rowWriter.write(5, scan_value5);\r\n/* 206 */           }\r\n/* 207 */\r\n/* 208 */           if (scan_isNull6) {\r\n/* 209 */             bhj_rowWriter.setNullAt(6);\r\n/* 210 */           } else {\r\n/* 211 */             bhj_rowWriter.write(6, scan_value6);\r\n/* 212 */           }\r\n/* 213 */\r\n/* 214 */           if (scan_isNull7) {\r\n/* 215 */             bhj_rowWriter.setNullAt(7);\r\n/* 216 */           } else {\r\n/* 217 */             bhj_rowWriter.write(7, scan_value7);\r\n/* 218 */           }\r\n/* 219 */\r\n/* 220 */           if (scan_isNull8) {\r\n/* 221 */             bhj_rowWriter.setNullAt(8);\r\n/* 222 */           } else {\r\n/* 223 */             bhj_rowWriter.write(8, scan_value8);\r\n/* 224 */           }\r\n/* 225 */\r\n/* 226 */           if (bhj_isNull2) {\r\n/* 227 */             bhj_rowWriter.setNullAt(9);\r\n/* 228 */           } else {\r\n/* 229 */             bhj_rowWriter.write(9, bhj_value2);\r\n/* 230 */           }\r\n/* 231 */\r\n/* 232 */           if (bhj_isNull3) {\r\n/* 233 */             bhj_rowWriter.setNullAt(10);\r\n/* 234 */           } else {\r\n/* 235 */             bhj_rowWriter.write(10, bhj_value3);\r\n/* 236 */           }\r\n/* 237 */\r\n/* 238 */           if (bhj_isNull4) {\r\n/* 239 */             bhj_rowWriter.setNullAt(11);\r\n/* 240 */           } else {\r\n/* 241 */             bhj_rowWriter.write(11, bhj_value4);\r\n/* 242 */           }\r\n/* 243 */\r\n/* 244 */           if (bhj_isNull5) {\r\n/* 245 */             bhj_rowWriter.setNullAt(12);\r\n/* 246 */           } else {\r\n/* 247 */             bhj_rowWriter.write(12, bhj_value5);\r\n/* 248 */           }\r\n/* 249 */\r\n/* 250 */           if (bhj_isNull6) {\r\n/* 251 */             bhj_rowWriter.setNullAt(13);\r\n/* 252 */           } else {\r\n/* 253 */             bhj_rowWriter.write(13, bhj_value6);\r\n/* 254 */           }\r\n/* 255 */\r\n/* 256 */           if (bhj_isNull7) {\r\n/* 257 */             bhj_rowWriter.setNullAt(14);\r\n/* 258 */           } else {\r\n/* 259 */             bhj_rowWriter.write(14, bhj_value7);\r\n/* 260 */           }\r\n/* 261 */\r\n/* 262 */           if (bhj_isNull8) {\r\n/* 263 */             bhj_rowWriter.setNullAt(15);\r\n/* 264 */           } else {\r\n/* 265 */             bhj_rowWriter.write(15, bhj_value8);\r\n/* 266 */           }\r\n/* 267 */\r\n/* 268 */           if (bhj_isNull9) {\r\n/* 269 */             bhj_rowWriter.setNullAt(16);\r\n/* 270 */           } else {\r\n/* 271 */             bhj_rowWriter.write(16, bhj_value9);\r\n/* 272 */           }\r\n/* 273 */\r\n/* 274 */           if (bhj_isNull10) {\r\n/* 275 */             bhj_rowWriter.setNullAt(17);\r\n/* 276 */           } else {\r\n/* 277 */             bhj_rowWriter.write(17, bhj_value10);\r\n/* 278 */           }\r\n/* 279 */           bhj_result.setTotalSize(bhj_holder.totalSize());\r\n/* 280 */\r\n/* 281 */           append(bhj_result.copy());\r\n/* 282 */\r\n/* 283 */         }\r\n/* 284 */         if (shouldStop()) return;\r\n/* 285 */       }\r\n/* 286 */       scan_batch = null;\r\n/* 287 */       scan_nextBatch();\r\n/* 288 */     }\r\n```\r\n\r\n#### after \r\n\r\n```java\r\n/* 105 */     while (scan_batch != null) {\r\n/* 106 */       int numRows = scan_batch.numRows();\r\n/* 107 */       while (scan_batchIdx < numRows) {\r\n/* 108 */         int scan_rowIdx = scan_batchIdx++;\r\n/* 109 */         boolean scan_isNull = scan_colInstance0.isNullAt(scan_rowIdx);\r\n/* 110 */         int scan_value = scan_isNull ? -1 : (scan_colInstance0.getInt(scan_rowIdx));\r\n/* 111 */\r\n/* 112 */         if (!(!(scan_isNull))) continue;\r\n/* 113 */\r\n/* 114 */         boolean filter_isNull2 = false;\r\n/* 115 */\r\n/* 116 */         boolean filter_value2 = false;\r\n/* 117 */         filter_value2 = scan_value == 1;\r\n/* 118 */         if (!filter_value2) continue;\r\n/* 119 */\r\n/* 120 */         filter_numOutputRows.add(1);\r\n/* 121 */\r\n/* 122 */         // generate join key for stream side\r\n/* 123 */\r\n/* 124 */         boolean bhj_isNull = false;\r\n/* 125 */         long bhj_value = -1L;\r\n/* 126 */         if (!false) {\r\n/* 127 */           bhj_value = (long) scan_value;\r\n/* 128 */         }\r\n/* 129 */         // find matches from HashRelation\r\n/* 130 */         scala.collection.Iterator bhj_matches =\r\n/* 131 */         bhj_isNull ? null : (scala.collection.Iterator)bhj_relation.get(bhj_value);\r\n/* 132 */         if (bhj_matches == null) continue;\r\n/* 133 */\r\n/* 134 */         boolean scan_isNull1 = scan_colInstance1.isNullAt(scan_rowIdx);\r\n/* 135 */         UTF8String scan_value1 = scan_isNull1 ? null : (scan_colInstance1.getUTF8String(scan_rowIdx));\r\n/* 136 */         boolean scan_isNull2 = scan_colInstance2.isNullAt(scan_rowIdx);\r\n/* 137 */         UTF8String scan_value2 = scan_isNull2 ? null : (scan_colInstance2.getUTF8String(scan_rowIdx));\r\n/* 138 */         boolean scan_isNull3 = scan_colInstance3.isNullAt(scan_rowIdx);\r\n/* 139 */         int scan_value3 = scan_isNull3 ? -1 : (scan_colInstance3.getInt(scan_rowIdx));\r\n/* 140 */         boolean scan_isNull4 = scan_colInstance4.isNullAt(scan_rowIdx);\r\n/* 141 */         UTF8String scan_value4 = scan_isNull4 ? null : (scan_colInstance4.getUTF8String(scan_rowIdx));\r\n/* 142 */         boolean scan_isNull5 = scan_colInstance5.isNullAt(scan_rowIdx);\r\n/* 143 */         UTF8String scan_value5 = scan_isNull5 ? null : (scan_colInstance5.getUTF8String(scan_rowIdx));\r\n/* 144 */         boolean scan_isNull6 = scan_colInstance6.isNullAt(scan_rowIdx);\r\n/* 145 */         UTF8String scan_value6 = scan_isNull6 ? null : (scan_colInstance6.getUTF8String(scan_rowIdx));\r\n/* 146 */         boolean scan_isNull7 = scan_colInstance7.isNullAt(scan_rowIdx);\r\n/* 147 */         UTF8String scan_value7 = scan_isNull7 ? null : (scan_colInstance7.getUTF8String(scan_rowIdx));\r\n/* 148 */         boolean scan_isNull8 = scan_colInstance8.isNullAt(scan_rowIdx);\r\n/* 149 */         int scan_value8 = scan_isNull8 ? -1 : (scan_colInstance8.getInt(scan_rowIdx));\r\n/* 150 */         bhj_holder.reset();\r\n/* 151 */         bhj_rowWriter.zeroOutNullBytes();\r\n/* 152 */\r\n/* 153 */         bhj_rowWriter.write(0, scan_value);\r\n/* 154 */\r\n/* 155 */         if (scan_isNull1) {\r\n/* 156 */           bhj_rowWriter.setNullAt(1);\r\n/* 157 */         } else {\r\n/* 158 */           bhj_rowWriter.write(1, scan_value1);\r\n/* 159 */         }\r\n/* 160 */\r\n/* 161 */         if (scan_isNull2) {\r\n/* 162 */           bhj_rowWriter.setNullAt(2);\r\n/* 163 */         } else {\r\n/* 164 */           bhj_rowWriter.write(2, scan_value2);\r\n/* 165 */         }\r\n/* 166 */\r\n/* 167 */         if (scan_isNull3) {\r\n/* 168 */           bhj_rowWriter.setNullAt(3);\r\n/* 169 */         } else {\r\n/* 170 */           bhj_rowWriter.write(3, scan_value3);\r\n/* 171 */         }\r\n/* 172 */\r\n/* 173 */         if (scan_isNull4) {\r\n/* 174 */           bhj_rowWriter.setNullAt(4);\r\n/* 175 */         } else {\r\n/* 176 */           bhj_rowWriter.write(4, scan_value4);\r\n/* 177 */         }\r\n/* 178 */\r\n/* 179 */         if (scan_isNull5) {\r\n/* 180 */           bhj_rowWriter.setNullAt(5);\r\n/* 181 */         } else {\r\n/* 182 */           bhj_rowWriter.write(5, scan_value5);\r\n/* 183 */         }\r\n/* 184 */\r\n/* 185 */         if (scan_isNull6) {\r\n/* 186 */           bhj_rowWriter.setNullAt(6);\r\n/* 187 */         } else {\r\n/* 188 */           bhj_rowWriter.write(6, scan_value6);\r\n/* 189 */         }\r\n/* 190 */\r\n/* 191 */         if (scan_isNull7) {\r\n/* 192 */           bhj_rowWriter.setNullAt(7);\r\n/* 193 */         } else {\r\n/* 194 */           bhj_rowWriter.write(7, scan_value7);\r\n/* 195 */         }\r\n/* 196 */\r\n/* 197 */         if (scan_isNull8) {\r\n/* 198 */           bhj_rowWriter.setNullAt(8);\r\n/* 199 */         } else {\r\n/* 200 */           bhj_rowWriter.write(8, scan_value8);\r\n/* 201 */         }\r\n/* 202 */\r\n/* 203 */         while (bhj_matches.hasNext()) {\r\n/* 204 */           UnsafeRow bhj_matched = (UnsafeRow) bhj_matches.next();\r\n/* 205 */\r\n/* 206 */           bhj_numOutputRows.add(1);\r\n/* 207 */\r\n/* 208 */           boolean bhj_isNull2 = bhj_matched.isNullAt(0);\r\n/* 209 */           int bhj_value2 = bhj_isNull2 ? -1 : (bhj_matched.getInt(0));\r\n/* 210 */           boolean bhj_isNull3 = bhj_matched.isNullAt(1);\r\n/* 211 */           UTF8String bhj_value3 = bhj_isNull3 ? null : (bhj_matched.getUTF8String(1));\r\n/* 212 */           boolean bhj_isNull4 = bhj_matched.isNullAt(2);\r\n/* 213 */           UTF8String bhj_value4 = bhj_isNull4 ? null : (bhj_matched.getUTF8String(2));\r\n/* 214 */           boolean bhj_isNull5 = bhj_matched.isNullAt(3);\r\n/* 215 */           int bhj_value5 = bhj_isNull5 ? -1 : (bhj_matched.getInt(3));\r\n/* 216 */           boolean bhj_isNull6 = bhj_matched.isNullAt(4);\r\n/* 217 */           UTF8String bhj_value6 = bhj_isNull6 ? null : (bhj_matched.getUTF8String(4));\r\n/* 218 */           boolean bhj_isNull7 = bhj_matched.isNullAt(5);\r\n/* 219 */           UTF8String bhj_value7 = bhj_isNull7 ? null : (bhj_matched.getUTF8String(5));\r\n/* 220 */           boolean bhj_isNull8 = bhj_matched.isNullAt(6);\r\n/* 221 */           UTF8String bhj_value8 = bhj_isNull8 ? null : (bhj_matched.getUTF8String(6));\r\n/* 222 */           boolean bhj_isNull9 = bhj_matched.isNullAt(7);\r\n/* 223 */           UTF8String bhj_value9 = bhj_isNull9 ? null : (bhj_matched.getUTF8String(7));\r\n/* 224 */           boolean bhj_isNull10 = bhj_matched.isNullAt(8);\r\n/* 225 */           int bhj_value10 = bhj_isNull10 ? -1 : (bhj_matched.getInt(8));\r\n/* 226 */           if (bhj_isNull2) {\r\n/* 227 */             bhj_rowWriter.setNullAt(9);\r\n/* 228 */           } else {\r\n/* 229 */             bhj_rowWriter.write(9, bhj_value2);\r\n/* 230 */           }\r\n/* 231 */\r\n/* 232 */           if (bhj_isNull3) {\r\n/* 233 */             bhj_rowWriter.setNullAt(10);\r\n/* 234 */           } else {\r\n/* 235 */             bhj_rowWriter.write(10, bhj_value3);\r\n/* 236 */           }\r\n/* 237 */\r\n/* 238 */           if (bhj_isNull4) {\r\n/* 239 */             bhj_rowWriter.setNullAt(11);\r\n/* 240 */           } else {\r\n/* 241 */             bhj_rowWriter.write(11, bhj_value4);\r\n/* 242 */           }\r\n/* 243 */\r\n/* 244 */           if (bhj_isNull5) {\r\n/* 245 */             bhj_rowWriter.setNullAt(12);\r\n/* 246 */           } else {\r\n/* 247 */             bhj_rowWriter.write(12, bhj_value5);\r\n/* 248 */           }\r\n/* 249 */\r\n/* 250 */           if (bhj_isNull6) {\r\n/* 251 */             bhj_rowWriter.setNullAt(13);\r\n/* 252 */           } else {\r\n/* 253 */             bhj_rowWriter.write(13, bhj_value6);\r\n/* 254 */           }\r\n/* 255 */\r\n/* 256 */           if (bhj_isNull7) {\r\n/* 257 */             bhj_rowWriter.setNullAt(14);\r\n/* 258 */           } else {\r\n/* 259 */             bhj_rowWriter.write(14, bhj_value7);\r\n/* 260 */           }\r\n/* 261 */\r\n/* 262 */           if (bhj_isNull8) {\r\n/* 263 */             bhj_rowWriter.setNullAt(15);\r\n/* 264 */           } else {\r\n/* 265 */             bhj_rowWriter.write(15, bhj_value8);\r\n/* 266 */           }\r\n/* 267 */\r\n/* 268 */           if (bhj_isNull9) {\r\n/* 269 */             bhj_rowWriter.setNullAt(16);\r\n/* 270 */           } else {\r\n/* 271 */             bhj_rowWriter.write(16, bhj_value9);\r\n/* 272 */           }\r\n/* 273 */\r\n/* 274 */           if (bhj_isNull10) {\r\n/* 275 */             bhj_rowWriter.setNullAt(17);\r\n/* 276 */           } else {\r\n/* 277 */             bhj_rowWriter.write(17, bhj_value10);\r\n/* 278 */           }\r\n/* 279 */           bhj_result.setTotalSize(bhj_holder.totalSize());\r\n/* 280 */\r\n/* 281 */           append(bhj_result.copy());\r\n/* 282 */\r\n/* 283 */         }\r\n/* 284 */         if (shouldStop()) return;\r\n/* 285 */       }\r\n/* 286 */       scan_batch = null;\r\n/* 287 */       scan_nextBatch();\r\n/* 288 */     }\r\n\r\n```\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/248790826","html_url":"https://github.com/apache/spark/pull/15071#issuecomment-248790826","issue_url":"https://api.github.com/repos/apache/spark/issues/15071","id":248790826,"user":{"login":"yaooqinn","id":8326978,"avatar_url":"https://avatars.githubusercontent.com/u/8326978?v=3","gravatar_id":"","url":"https://api.github.com/users/yaooqinn","html_url":"https://github.com/yaooqinn","followers_url":"https://api.github.com/users/yaooqinn/followers","following_url":"https://api.github.com/users/yaooqinn/following{/other_user}","gists_url":"https://api.github.com/users/yaooqinn/gists{/gist_id}","starred_url":"https://api.github.com/users/yaooqinn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yaooqinn/subscriptions","organizations_url":"https://api.github.com/users/yaooqinn/orgs","repos_url":"https://api.github.com/users/yaooqinn/repos","events_url":"https://api.github.com/users/yaooqinn/events{/privacy}","received_events_url":"https://api.github.com/users/yaooqinn/received_events","type":"User","site_admin":false},"created_at":"2016-09-22T01:43:07Z","updated_at":"2016-09-22T01:43:07Z","body":"@hvanhovell\r\nI think unfixed length fields may lead to memory overlapping when ```BuildLeft```, since we are reusing the ```BufferHolder``` to avoid writing the stream side repeatedly. In this case, the holder can not ```grow``` properly to avoid the left side overlap the right side.\r\n\r\nWhen ```BuildRight```, there is no such a problem."}}
{"actor":{"display_login":"yaooqinn"}}