{"push_id":1122276551,"size":4,"distinct_size":4,"ref":"refs/heads/branch-2.0","head":"c7e013f1888f084d6a02d6dd905a77c4f20d1db4","before":"89e29870bb73dac9dfebd3c3663320e4fdc6c03a","commits":[{"sha":"ea92f7e7e5fbc28747b528c8c60982f8ce7f739f","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-15454][SQL] Filter out files starting with _\n\n## What changes were proposed in this pull request?\nMany other systems (e.g. Impala) uses _xxx as staging, and Spark should not be reading those files.\n\n## How was this patch tested?\nAdded a unit test case.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #13227 from rxin/SPARK-15454.\n\n(cherry picked from commit dcac8e6f49918a809fb3f2b8bf666582c479a6eb)\nSigned-off-by: Reynold Xin <rxin@databricks.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/ea92f7e7e5fbc28747b528c8c60982f8ce7f739f"},{"sha":"f31c24a6afd964bdf6dd334454e50bbd64106290","author":{"email":"22f3e725b314afeca4d45e12683c560674029d21@gmail.com","name":"tedyu"},"message":"[SPARK-15273] YarnSparkHadoopUtil#getOutOfMemoryErrorArgument should respect OnOutOfMemoryError parameter given by user\n\n## What changes were proposed in this pull request?\n\nAs Nirav reported in this thread:\nhttp://search-hadoop.com/m/q3RTtdF3yNLMd7u\n\nYarnSparkHadoopUtil#getOutOfMemoryErrorArgument previously specified 'kill %p' unconditionally.\nWe should respect the parameter given by user.\n\n## How was this patch tested?\n\nExisting tests\n\nAuthor: tedyu <yuzhihong@gmail.com>\n\nCloses #13057 from tedyu/master.\n\n(cherry picked from commit 06c9f520714e07259c6f8ce6f9ea5a230a278cb5)\nSigned-off-by: Sean Owen <sowen@cloudera.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/f31c24a6afd964bdf6dd334454e50bbd64106290"},{"sha":"68416786206d7f6c48f6517ff8da1e76927f0713","author":{"email":"21ef94aec54fae1a36abbd011347596ae66d9959@foxmail.com","name":"Zheng RuiFeng"},"message":"[SPARK-15031][EXAMPLE] Use SparkSession in examples\n\n## What changes were proposed in this pull request?\nUse `SparkSession` according to [SPARK-15031](https://issues.apache.org/jira/browse/SPARK-15031)\n\n`MLLLIB` is not recommended to use now, so examples in `MLLIB` are ignored in this PR.\n`StreamingContext` can not be directly obtained from `SparkSession`, so example in `Streaming` are ignored too.\n\ncc andrewor14\n\n## How was this patch tested?\nmanual tests with spark-submit\n\nAuthor: Zheng RuiFeng <ruifengz@foxmail.com>\n\nCloses #13164 from zhengruifeng/use_sparksession_ii.\n\n(cherry picked from commit 127bf1bb07967e2e4f99ad7abaa7f6fab3b3f407)\nSigned-off-by: Andrew Or <andrew@databricks.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/68416786206d7f6c48f6517ff8da1e76927f0713"},{"sha":"c7e013f1888f084d6a02d6dd905a77c4f20d1db4","author":{"email":"060dd14c05a8ffa8f201883ef57d0676b7c9c04a@gmail.com","name":"Bryan Cutler"},"message":"[SPARK-15456][PYSPARK] Fixed PySpark shell context initialization when HiveConf not present\n\n## What changes were proposed in this pull request?\n\nWhen PySpark shell cannot find HiveConf, it will fallback to create a SparkSession from a SparkContext.  This fixes a bug caused by using a variable to SparkContext before it was initialized.\n\n## How was this patch tested?\n\nManually starting PySpark shell and using the SparkContext\n\nAuthor: Bryan Cutler <cutlerb@gmail.com>\n\nCloses #13237 from BryanCutler/pyspark-shell-session-context-SPARK-15456.\n\n(cherry picked from commit 021c19702c720b4466b016498917d47f99000e13)\nSigned-off-by: Andrew Or <andrew@databricks.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/c7e013f1888f084d6a02d6dd905a77c4f20d1db4"}]}
{"actor":{"display_login":"ueshin"}}