{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/11301","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/11301/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/11301/comments","events_url":"https://api.github.com/repos/apache/spark/issues/11301/events","html_url":"https://github.com/apache/spark/pull/11301","id":135372383,"number":11301,"title":"[SPARK-13432][SQL] add the source file name and line into a generated Java code and stderr","user":{"login":"kiszk","id":1315079,"avatar_url":"https://avatars.githubusercontent.com/u/1315079?v=3","gravatar_id":"","url":"https://api.github.com/users/kiszk","html_url":"https://github.com/kiszk","followers_url":"https://api.github.com/users/kiszk/followers","following_url":"https://api.github.com/users/kiszk/following{/other_user}","gists_url":"https://api.github.com/users/kiszk/gists{/gist_id}","starred_url":"https://api.github.com/users/kiszk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kiszk/subscriptions","organizations_url":"https://api.github.com/users/kiszk/orgs","repos_url":"https://api.github.com/users/kiszk/repos","events_url":"https://api.github.com/users/kiszk/events{/privacy}","received_events_url":"https://api.github.com/users/kiszk/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2016-02-22T10:05:47Z","updated_at":"2016-02-25T17:49:09Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/11301","html_url":"https://github.com/apache/spark/pull/11301","diff_url":"https://github.com/apache/spark/pull/11301.diff","patch_url":"https://github.com/apache/spark/pull/11301.patch"},"body":"## What changes were proposed in this pull request?\r\nThis PR adds the source file name and line into a comment of a Java code generated by Catalyst. It would be helpful to quickly identify the original source file from a position where an error occurs during a problem determination of a customer. It supports only DataFrame and SQL.\r\n\r\nThis PR consists of three parts:\r\n- add line information to Origin\r\n- pass Origin from a master to executors\r\n- add Origin into a comment\r\n\r\nThis PR adds the information to a comment for existing operations. Other PRs will address the followings:\r\n\r\n- Add a comment for Dataset\r\n- Insert a comment for other places\r\n\r\n\r\nHere is an example. The original Java program.\r\n\r\n    object Test {\r\n      ...\r\n      df.filter(\"v <= 3\")\r\n        .filter(\"v % 2 == 0\")\r\n        .show()\r\n      ...\r\n    }\r\n\r\n\r\nGenerated Java code\r\n\r\n    ...\r\n    /* 031 */   protected void processNext() throws java.io.IOException {\r\n    /* 032 */     while (input.hasNext()) {\r\n    /* 033 */       InternalRow inputadapter_row = (InternalRow) input.next();\r\n    /* 034 */       /* input[0, string] @ filter at Test.scala:23 */\r\n    /* 035 */       boolean inputadapter_isNull = inputadapter_row.isNullAt(0);\r\n    /* 036 */       UTF8String inputadapter_value = inputadapter_isNull ? null : (inputadapter_row.getUTF8String(0));\r\n    /* 037 */       /* input[1, int] @ filter at Test.scala:23 */\r\n    /* 038 */       boolean inputadapter_isNull1 = inputadapter_row.isNullAt(1);\r\n    /* 039 */       int inputadapter_value1 = inputadapter_isNull1 ? -1 : (inputadapter_row.getInt(1));\r\n    /* 040 */       /* ((input[1, int] <= 3) && ((input[1, int] % 2) = 0)) @ filter at Test.scala:23 */\r\n    /* 041 */       /* (input[1, int] <= 3) @ filter at Dataset1.scala:22 */\r\n    ...\r\n\r\n\r\n\r\n## How was the this patch tested?\r\nUnit test  (add a test to keep Origin during SerDe)\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/188901193","html_url":"https://github.com/apache/spark/pull/11301#issuecomment-188901193","issue_url":"https://api.github.com/repos/apache/spark/issues/11301","id":188901193,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-02-25T17:49:09Z","updated_at":"2016-02-25T17:49:09Z","body":"**[Test build #51977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51977/consoleFull)** for PR 11301 at commit [`7671742`](https://github.com/apache/spark/commit/7671742949428b5846a9756bafe42967b5acdebb).\n * This patch **fails Spark unit tests**.\n * This patch merges cleanly.\n * This patch adds no public classes."}}