{"push_id":1169872327,"size":1,"distinct_size":1,"ref":"refs/heads/master","head":"7580f3041a1a3757a0b14b9d8afeb720f261fff6","before":"d77c4e6e2eef24f4276c38b3add8c29bb885f4db","commits":[{"sha":"7580f3041a1a3757a0b14b9d8afeb720f261fff6","author":{"name":"hyukjinkwon","email":"4c114bf3676ebcdb1616a11ca478d9181f4025da@gmail.com"},"message":"[SPARK-16104] [SQL] Do not creaate CSV writer object for every flush when writing\n\n## What changes were proposed in this pull request?\n\nThis PR let `CsvWriter` object is not created for each time but able to be reused. This way was taken after from JSON data source.\n\nOriginal `CsvWriter` was being created for each row but it was enhanced in https://github.com/apache/spark/pull/13229. However, it still creates `CsvWriter` object for each `flush()` in `LineCsvWriter`. It seems it does not have to close the object and re-create this for every flush.\n\nIt follows the original logic as it is but `CsvWriter` is reused by reseting `CharArrayWriter`.\n\n## How was this patch tested?\n\nExisting tests should cover this.\n\nAuthor: hyukjinkwon <gurwls223@gmail.com>\n\nCloses #13809 from HyukjinKwon/write-perf.","distinct":true,"url":"https://api.github.com/repos/dbtsai/spark/commits/7580f3041a1a3757a0b14b9d8afeb720f261fff6"}]}
{"actor":{"display_login":"dbtsai-bot"}}