{"push_id":1129302952,"size":1,"distinct_size":1,"ref":"refs/heads/master","head":"589cce93c821ac28e9090a478f6e7465398b7c30","before":"9c297df3d4d5fa4bbfdffdaad15f362586db384b","commits":[{"sha":"589cce93c821ac28e9090a478f6e7465398b7c30","author":{"email":"91fc8c8ce21861271961485268a4e92855a7f420@linkedin.com","name":"Gio Borje"},"message":"Log warnings for numIterations * miniBatchFraction < 1.0\n\n## What changes were proposed in this pull request?\n\nAdd a warning log for the case that `numIterations * miniBatchFraction <1.0` during gradient descent. If the product of those two numbers is less than `1.0`, then not all training examples will be used during optimization. To put this concretely, suppose that `numExamples = 100`, `miniBatchFraction = 0.2` and `numIterations = 3`. Then, 3 iterations will occur each sampling approximately 6 examples each. In the best case, each of the 6 examples are unique; hence 18/100 examples are used.\n\nThis may be counter-intuitive to most users and led to the issue during the development of another Spark  ML model: https://github.com/zhengruifeng/spark-libFM/issues/11. If a user actually does not require the training data set, it would be easier and more intuitive to use `RDD.sample`.\n\n## How was this patch tested?\n\n`build/mvn -DskipTests clean package` build succeeds\n\nAuthor: Gio Borje <gborje@linkedin.com>\n\nCloses #13265 from Hydrotoast/master.","distinct":true,"url":"https://api.github.com/repos/dbtsai/spark/commits/589cce93c821ac28e9090a478f6e7465398b7c30"}]}
{"actor":{"display_login":"dbtsai-bot"}}