{"action":"created","comment":{"url":"https://api.github.com/repos/apache/zeppelin/pulls/comments/175297711","pull_request_review_id":104805661,"id":175297711,"diff_hunk":"@@ -365,6 +365,51 @@ myScalaDataFrame = DataFrame(z.get(\"myScalaDataFrame\"), sqlContext)\n   </div>\n </div>\n \n+### Object Interpolation\n+Some interpreters can interpolate object values from `z` into the command string by using the ","path":"docs/interpreter/spark.md","position":5,"original_position":5,"commit_id":"b7ddf6ba202c2624a289aff67cfad669510ca80d","original_commit_id":"b7ddf6ba202c2624a289aff67cfad669510ca80d","user":{"login":"felixcheung","id":8969467,"avatar_url":"https://avatars3.githubusercontent.com/u/8969467?v=4","gravatar_id":"","url":"https://api.github.com/users/felixcheung","html_url":"https://github.com/felixcheung","followers_url":"https://api.github.com/users/felixcheung/followers","following_url":"https://api.github.com/users/felixcheung/following{/other_user}","gists_url":"https://api.github.com/users/felixcheung/gists{/gist_id}","starred_url":"https://api.github.com/users/felixcheung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/felixcheung/subscriptions","organizations_url":"https://api.github.com/users/felixcheung/orgs","repos_url":"https://api.github.com/users/felixcheung/repos","events_url":"https://api.github.com/users/felixcheung/events{/privacy}","received_events_url":"https://api.github.com/users/felixcheung/received_events","type":"User","site_admin":false},"body":"question: perhaps this is talked about but the meaning of \"command string\" isn't very clear to me as an user.\r\nAre we referring to the paragraph text?","created_at":"2018-03-18T17:58:14Z","updated_at":"2018-03-18T18:00:13Z","html_url":"https://github.com/apache/zeppelin/pull/2834#discussion_r175297711","pull_request_url":"https://api.github.com/repos/apache/zeppelin/pulls/2834","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/apache/zeppelin/pulls/comments/175297711"},"html":{"href":"https://github.com/apache/zeppelin/pull/2834#discussion_r175297711"},"pull_request":{"href":"https://api.github.com/repos/apache/zeppelin/pulls/2834"}}},"pull_request":{"url":"https://api.github.com/repos/apache/zeppelin/pulls/2834","id":172769506,"html_url":"https://github.com/apache/zeppelin/pull/2834","diff_url":"https://github.com/apache/zeppelin/pull/2834.diff","patch_url":"https://github.com/apache/zeppelin/pull/2834.patch","issue_url":"https://api.github.com/repos/apache/zeppelin/issues/2834","number":2834,"state":"open","locked":false,"title":"[ZEPPELIN-1967] Passing Z variables to Shell and SQL Interpreters","user":{"login":"sanjaydasgupta","id":477015,"avatar_url":"https://avatars0.githubusercontent.com/u/477015?v=4","gravatar_id":"","url":"https://api.github.com/users/sanjaydasgupta","html_url":"https://github.com/sanjaydasgupta","followers_url":"https://api.github.com/users/sanjaydasgupta/followers","following_url":"https://api.github.com/users/sanjaydasgupta/following{/other_user}","gists_url":"https://api.github.com/users/sanjaydasgupta/gists{/gist_id}","starred_url":"https://api.github.com/users/sanjaydasgupta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sanjaydasgupta/subscriptions","organizations_url":"https://api.github.com/users/sanjaydasgupta/orgs","repos_url":"https://api.github.com/users/sanjaydasgupta/repos","events_url":"https://api.github.com/users/sanjaydasgupta/events{/privacy}","received_events_url":"https://api.github.com/users/sanjaydasgupta/received_events","type":"User","site_admin":false},"body":"### What is this PR for?\r\n\r\nThe code in this PR enables embedding/interpolating Z variables into command strings passed to Spark's SQL and Shell interpreters. It implements the functionality described in issue [ZEPPELIN-1967](https://issues.apache.org/jira/browse/ZEPPELIN-1967)\r\n\r\nThis PR resumes a fresh effort while taking into consideration all of the discussion in the context of the earlier [PR-2502](https://github.com/apache/zeppelin/pull/2502). The earlier PR-2502 was closed due to a corruption in my repo that could not be corrected.\r\n\r\nThe code in this PR resolves all of the discussion and suggestions in the body of the earlier [PR-2502](https://github.com/apache/zeppelin/pull/2502). The following description is a summary of the current implementation:\r\n\r\nPatterns of the form `{var-name}` within commands will be interpolated only if a predefined object of the specified name exists in `z`. Additionally, all such patterns within the command line should also be translatable for any interpolation to occur. Partial translation of a command line (where only some of the patterns are translated, and others are not) is never performed.\r\n\r\nPatterns of the form `{{any-text}}` are translated into `{any-text}`. This feature is an escaping mechanism that allows `{` and `}` characters to be passed into a command without invoking the interpolation mechanism.\r\n\r\nThe translations described above are performed only when all occurrences of `{`, `}`, `{{`, and `}}` in any command string conform to one of the two forms described above. A command that contains `{` and/or `}` characters used in any other way (than `{var-name}` and `{{any-text}}` as described above) will be used as-is without attempting any translations whatsoever -- even if the command also contains legal, translate-able and/or escape-able, constructs of the above two forms.\r\n\r\nNo error is flagged in any case. This behavior is identical to the implementation of a similar feature in\r\nJupyter's shell invocation using the ! magic command.\r\n\r\nAt present only the SQL and Shell interpreters support object interpolation.\r\n\r\n### What type of PR is it?\r\n[Improvement]\r\n\r\n### Todos\r\n* [ ] - Task\r\n\r\n### What is the Jira issue?\r\n\r\nhttps://issues.apache.org/jira/browse/ZEPPELIN-1967\r\n\r\n### How should this be tested?\r\n\r\nA new unit-test class ZeppCtxtVariableTest.java (see below) has been added. The attached screenshots below also show tests of the functionality.\r\n\r\n### Screenshots (if appropriate)\r\n\r\n![figure-1](https://user-images.githubusercontent.com/477015/36956999-5f8cca92-2057-11e8-8b76-f4ccd2a21d50.png)\r\n![figure-2](https://user-images.githubusercontent.com/477015/36957001-650f271c-2057-11e8-8e94-4805fd24e796.png)\r\n![figure-3](https://user-images.githubusercontent.com/477015/36957005-6a747dec-2057-11e8-9c72-4ebef17b52db.png)\r\n![figure-4](https://user-images.githubusercontent.com/477015/36957006-6df95ad2-2057-11e8-8585-3eb679e3a146.png)\r\n![figure-5](https://user-images.githubusercontent.com/477015/36957011-7284be02-2057-11e8-9204-3774121397e6.png)\r\n\r\n\r\n### Questions:\r\n* Does the licenses files need update? No\r\n* Is there breaking changes for older versions? No\r\n* Does this needs documentation? Yes, and detailed documentation has been added to the part describing ZeppelinContext variables (see file spark.md below).\r\n","created_at":"2018-03-05T04:03:16Z","updated_at":"2018-03-18T18:00:13Z","closed_at":null,"merged_at":null,"merge_commit_sha":"e69529a68d98597685ef9d01de6842dc478ae611","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/zeppelin/pulls/2834/commits","review_comments_url":"https://api.github.com/repos/apache/zeppelin/pulls/2834/comments","review_comment_url":"https://api.github.com/repos/apache/zeppelin/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/zeppelin/issues/2834/comments","statuses_url":"https://api.github.com/repos/apache/zeppelin/statuses/b7ddf6ba202c2624a289aff67cfad669510ca80d","head":{"label":"sanjaydasgupta:ZEPPELIN-1967","ref":"ZEPPELIN-1967","sha":"b7ddf6ba202c2624a289aff67cfad669510ca80d","user":{"login":"sanjaydasgupta","id":477015,"avatar_url":"https://avatars0.githubusercontent.com/u/477015?v=4","gravatar_id":"","url":"https://api.github.com/users/sanjaydasgupta","html_url":"https://github.com/sanjaydasgupta","followers_url":"https://api.github.com/users/sanjaydasgupta/followers","following_url":"https://api.github.com/users/sanjaydasgupta/following{/other_user}","gists_url":"https://api.github.com/users/sanjaydasgupta/gists{/gist_id}","starred_url":"https://api.github.com/users/sanjaydasgupta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sanjaydasgupta/subscriptions","organizations_url":"https://api.github.com/users/sanjaydasgupta/orgs","repos_url":"https://api.github.com/users/sanjaydasgupta/repos","events_url":"https://api.github.com/users/sanjaydasgupta/events{/privacy}","received_events_url":"https://api.github.com/users/sanjaydasgupta/received_events","type":"User","site_admin":false},"repo":{"id":123680175,"name":"zeppelin","full_name":"sanjaydasgupta/zeppelin","owner":{"login":"sanjaydasgupta","id":477015,"avatar_url":"https://avatars0.githubusercontent.com/u/477015?v=4","gravatar_id":"","url":"https://api.github.com/users/sanjaydasgupta","html_url":"https://github.com/sanjaydasgupta","followers_url":"https://api.github.com/users/sanjaydasgupta/followers","following_url":"https://api.github.com/users/sanjaydasgupta/following{/other_user}","gists_url":"https://api.github.com/users/sanjaydasgupta/gists{/gist_id}","starred_url":"https://api.github.com/users/sanjaydasgupta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sanjaydasgupta/subscriptions","organizations_url":"https://api.github.com/users/sanjaydasgupta/orgs","repos_url":"https://api.github.com/users/sanjaydasgupta/repos","events_url":"https://api.github.com/users/sanjaydasgupta/events{/privacy}","received_events_url":"https://api.github.com/users/sanjaydasgupta/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/sanjaydasgupta/zeppelin","description":"Mirror of Apache Zeppelin","fork":true,"url":"https://api.github.com/repos/sanjaydasgupta/zeppelin","forks_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/forks","keys_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/keys{/key_id}","collaborators_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/teams","hooks_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/hooks","issue_events_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/issues/events{/number}","events_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/events","assignees_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/assignees{/user}","branches_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/branches{/branch}","tags_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/tags","blobs_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/git/refs{/sha}","trees_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/git/trees{/sha}","statuses_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/statuses/{sha}","languages_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/languages","stargazers_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/stargazers","contributors_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/contributors","subscribers_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/subscribers","subscription_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/subscription","commits_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/commits{/sha}","git_commits_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/git/commits{/sha}","comments_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/comments{/number}","issue_comment_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/issues/comments{/number}","contents_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/contents/{+path}","compare_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/compare/{base}...{head}","merges_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/merges","archive_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/downloads","issues_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/issues{/number}","pulls_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/pulls{/number}","milestones_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/milestones{/number}","notifications_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/labels{/name}","releases_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/releases{/id}","deployments_url":"https://api.github.com/repos/sanjaydasgupta/zeppelin/deployments","created_at":"2018-03-03T10:03:01Z","updated_at":"2018-03-03T10:03:07Z","pushed_at":"2018-03-18T09:54:32Z","git_url":"git://github.com/sanjaydasgupta/zeppelin.git","ssh_url":"git@github.com:sanjaydasgupta/zeppelin.git","clone_url":"https://github.com/sanjaydasgupta/zeppelin.git","svn_url":"https://github.com/sanjaydasgupta/zeppelin","homepage":"","size":65071,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"233c09873456ac0893ce30970b7caa64204d99ca","user":{"login":"apache","id":47359,"avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":32848140,"name":"zeppelin","full_name":"apache/zeppelin","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars0.githubusercontent.com/u/47359?v=4","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/zeppelin","description":"Mirror of Apache Zeppelin","fork":false,"url":"https://api.github.com/repos/apache/zeppelin","forks_url":"https://api.github.com/repos/apache/zeppelin/forks","keys_url":"https://api.github.com/repos/apache/zeppelin/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/zeppelin/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/zeppelin/teams","hooks_url":"https://api.github.com/repos/apache/zeppelin/hooks","issue_events_url":"https://api.github.com/repos/apache/zeppelin/issues/events{/number}","events_url":"https://api.github.com/repos/apache/zeppelin/events","assignees_url":"https://api.github.com/repos/apache/zeppelin/assignees{/user}","branches_url":"https://api.github.com/repos/apache/zeppelin/branches{/branch}","tags_url":"https://api.github.com/repos/apache/zeppelin/tags","blobs_url":"https://api.github.com/repos/apache/zeppelin/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/zeppelin/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/zeppelin/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/zeppelin/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/zeppelin/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/zeppelin/languages","stargazers_url":"https://api.github.com/repos/apache/zeppelin/stargazers","contributors_url":"https://api.github.com/repos/apache/zeppelin/contributors","subscribers_url":"https://api.github.com/repos/apache/zeppelin/subscribers","subscription_url":"https://api.github.com/repos/apache/zeppelin/subscription","commits_url":"https://api.github.com/repos/apache/zeppelin/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/zeppelin/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/zeppelin/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/zeppelin/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/zeppelin/contents/{+path}","compare_url":"https://api.github.com/repos/apache/zeppelin/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/zeppelin/merges","archive_url":"https://api.github.com/repos/apache/zeppelin/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/zeppelin/downloads","issues_url":"https://api.github.com/repos/apache/zeppelin/issues{/number}","pulls_url":"https://api.github.com/repos/apache/zeppelin/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/zeppelin/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/zeppelin/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/zeppelin/labels{/name}","releases_url":"https://api.github.com/repos/apache/zeppelin/releases{/id}","deployments_url":"https://api.github.com/repos/apache/zeppelin/deployments","created_at":"2015-03-25T07:00:06Z","updated_at":"2018-03-18T03:10:34Z","pushed_at":"2018-03-18T09:54:37Z","git_url":"git://github.com/apache/zeppelin.git","ssh_url":"git@github.com:apache/zeppelin.git","clone_url":"https://github.com/apache/zeppelin.git","svn_url":"https://github.com/apache/zeppelin","homepage":"","size":65112,"stargazers_count":3448,"watchers_count":3448,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":1775,"mirror_url":null,"archived":false,"open_issues_count":180,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":1775,"open_issues":180,"watchers":3448,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/zeppelin/pulls/2834"},"html":{"href":"https://github.com/apache/zeppelin/pull/2834"},"issue":{"href":"https://api.github.com/repos/apache/zeppelin/issues/2834"},"comments":{"href":"https://api.github.com/repos/apache/zeppelin/issues/2834/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/zeppelin/pulls/2834/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/zeppelin/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/zeppelin/pulls/2834/commits"},"statuses":{"href":"https://api.github.com/repos/apache/zeppelin/statuses/b7ddf6ba202c2624a289aff67cfad669510ca80d"}},"author_association":"NONE"}}
{"actor":{"display_login":"felixcheung"}}