{"push_id":3108228936,"size":2,"distinct_size":2,"ref":"refs/heads/master","head":"7143e9d7220bd98ceb82c5c5f045108a8a664ec1","before":"35f9163adf5c067229afbe57ed60d5dd5f2422c8","commits":[{"sha":"180f969c97a66b4c265e5fad8272665a00572f1a","author":{"name":"Imran Rashid","email":"b54fbb0e17aa598fdc04bb3d79ba76b5f0737666@cloudera.com"},"message":"[SPARK-26094][CORE][STREAMING] createNonEcFile creates parent dirs.\n\n## What changes were proposed in this pull request?\n\nWe explicitly avoid files with hdfs erasure coding for the streaming WAL\nand for event logs, as hdfs EC does not support all relevant apis.\nHowever, the new builder api used has different semantics -- it does not\ncreate parent dirs, and it does not resolve relative paths.  This\nupdates createNonEcFile to have similar semantics to the old api.\n\n## How was this patch tested?\n\nRan tests with the WAL pointed at a non-existent dir, which failed before this change.  Manually tested the new function with a relative path as well.\nUnit tests via jenkins.\n\nCloses #23092 from squito/SPARK-26094.\n\nAuthored-by: Imran Rashid <irashid@cloudera.com>\nSigned-off-by: Marcelo Vanzin <vanzin@cloudera.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/180f969c97a66b4c265e5fad8272665a00572f1a"},{"sha":"7143e9d7220bd98ceb82c5c5f045108a8a664ec1","author":{"name":"Marco Gaido","email":"368e8ff04b169d99d4d85a32ca87c1aa4ab0fa7e@gmail.com"},"message":"[SPARK-25829][SQL][FOLLOWUP] Refactor MapConcat in order to check properly the limit size\n\n## What changes were proposed in this pull request?\n\nThe PR starts from the [comment](https://github.com/apache/spark/pull/23124#discussion_r236112390) in the main one and it aims at:\n - simplifying the code for `MapConcat`;\n - be more precise in checking the limit size.\n\n## How was this patch tested?\n\nexisting tests\n\nCloses #23217 from mgaido91/SPARK-25829_followup.\n\nAuthored-by: Marco Gaido <marcogaido91@gmail.com>\nSigned-off-by: Wenchen Fan <wenchen@databricks.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/7143e9d7220bd98ceb82c5c5f045108a8a664ec1"}]}
{"actor":{"display_login":"ueshin"}}