{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/14034","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/14034/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/14034/comments","events_url":"https://api.github.com/repos/apache/spark/issues/14034/events","html_url":"https://github.com/apache/spark/pull/14034","id":163549174,"number":14034,"title":"[SPARK-16355] [SPARK-16354] [SQL] Fix Bugs When LIMIT/TABLESAMPLE is Non-foldable, Zero or Negative","user":{"login":"gatorsmile","id":11567269,"avatar_url":"https://avatars.githubusercontent.com/u/11567269?v=3","gravatar_id":"","url":"https://api.github.com/users/gatorsmile","html_url":"https://github.com/gatorsmile","followers_url":"https://api.github.com/users/gatorsmile/followers","following_url":"https://api.github.com/users/gatorsmile/following{/other_user}","gists_url":"https://api.github.com/users/gatorsmile/gists{/gist_id}","starred_url":"https://api.github.com/users/gatorsmile/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gatorsmile/subscriptions","organizations_url":"https://api.github.com/users/gatorsmile/orgs","repos_url":"https://api.github.com/users/gatorsmile/repos","events_url":"https://api.github.com/users/gatorsmile/events{/privacy}","received_events_url":"https://api.github.com/users/gatorsmile/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2016-07-03T06:05:19Z","updated_at":"2016-07-08T22:56:14Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/14034","html_url":"https://github.com/apache/spark/pull/14034","diff_url":"https://github.com/apache/spark/pull/14034.diff","patch_url":"https://github.com/apache/spark/pull/14034.patch"},"body":"#### What changes were proposed in this pull request?\r\n**Issue 1:** When a query containing LIMIT/TABLESAMPLE 0, the statistics could be zero. Results are correct but it could cause a huge performance regression. For example,\r\n```Scala\r\nSeq((\"one\", 1), (\"two\", 2), (\"three\", 3), (\"four\", 4)).toDF(\"k\", \"v\")\r\n  .createOrReplaceTempView(\"test\")\r\nval df1 = spark.table(\"test\")\r\nval df2 = spark.table(\"test\").limit(0)\r\nval df = df1.join(df2, Seq(\"k\"), \"left\")\r\n```\r\nThe statistics of both `df` and `df2` are zero. The statistics values should never be zero; otherwise `sizeInBytes` of `BinaryNode` will also be zero (product of children). This PR is to increase it to `1` when the num of rows is equal to 0.\r\n\r\n**Issue 2:** When a query containing negative LIMIT/TABLESAMPLE, we should issue exceptions. Negative values could break the implementation assumption of multiple parts. For example, statistics calculation.  Below is the example query.\r\n```SQL\r\nSELECT * FROM testData TABLESAMPLE (-1 rows)\r\nSELECT * FROM testData LIMIT -1\r\n``` \r\nThis PR is to issue an appropriate exception in this case.\r\n\r\n**Issue 3:** Spark SQL follows the restriction of LIMIT clause in Hive. The argument to the LIMIT clause must evaluate to a constant value. It can be a numeric literal, or another kind of numeric expression involving operators, casts, and function return values. You cannot refer to a column or use a subquery. Currently, we do not detect whether the expression in LIMIT clause is foldable or not. If non-foldable, we might issue a strange error message. For example, \r\n```SQL\r\nSELECT * FROM testData LIMIT rand() > 0.2\r\n```\r\nThen, a misleading error message is issued, like\r\n```\r\nassertion failed: No plan for GlobalLimit (_nondeterministic#203 > 0.2)\r\n+- Project [key#11, value#12, rand(-1441968339187861415) AS _nondeterministic#203]\r\n   +- LocalLimit (_nondeterministic#202 > 0.2)\r\n      +- Project [key#11, value#12, rand(-1308350387169017676) AS _nondeterministic#202]\r\n         +- LogicalRDD [key#11, value#12]\r\n\r\njava.lang.AssertionError: assertion failed: No plan for GlobalLimit (_nondeterministic#203 > 0.2)\r\n+- Project [key#11, value#12, rand(-1441968339187861415) AS _nondeterministic#203]\r\n   +- LocalLimit (_nondeterministic#202 > 0.2)\r\n      +- Project [key#11, value#12, rand(-1308350387169017676) AS _nondeterministic#202]\r\n         +- LogicalRDD [key#11, value#12]\r\n```\r\nThis PR detects it and then issues a meaningful error message. \r\n\r\n#### How was this patch tested?\r\nAdded test cases."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/231491846","html_url":"https://github.com/apache/spark/pull/14034#issuecomment-231491846","issue_url":"https://api.github.com/repos/apache/spark/issues/14034","id":231491846,"user":{"login":"gatorsmile","id":11567269,"avatar_url":"https://avatars.githubusercontent.com/u/11567269?v=3","gravatar_id":"","url":"https://api.github.com/users/gatorsmile","html_url":"https://github.com/gatorsmile","followers_url":"https://api.github.com/users/gatorsmile/followers","following_url":"https://api.github.com/users/gatorsmile/following{/other_user}","gists_url":"https://api.github.com/users/gatorsmile/gists{/gist_id}","starred_url":"https://api.github.com/users/gatorsmile/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gatorsmile/subscriptions","organizations_url":"https://api.github.com/users/gatorsmile/orgs","repos_url":"https://api.github.com/users/gatorsmile/repos","events_url":"https://api.github.com/users/gatorsmile/events{/privacy}","received_events_url":"https://api.github.com/users/gatorsmile/received_events","type":"User","site_admin":false},"created_at":"2016-07-08T22:56:14Z","updated_at":"2016-07-08T22:56:14Z","body":"retest this please"}}
{"actor":{"display_login":"gatorsmile"}}