{"action":"created","comment":{"url":"https://api.github.com/repos/prestodb/presto/pulls/comments/239297572","pull_request_review_id":182035329,"id":239297572,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIzOTI5NzU3Mg==","diff_hunk":"@@ -0,0 +1,171 @@\n+=======================\n+Elasticsearch Connector\n+=======================\n+\n+.. contents::\n+    :local:\n+    :backlinks: none\n+    :depth: 1\n+\n+Overview\n+--------\n+\n+Elasticsearch maintains documents and indexes in a highly scalable way. It provides\n+full-text search and analytics capabilities.\n+\n+The Elasticsearch Connector allows access to Elasticsearch data from Presto. This document describes how to setup the Elasticsearch Connector to run SQL on Elasticsearch.\n+\n+.. note::\n+\n+    It is highly recommended to use Elasticsearch 6.0.0 or later.\n+\n+Configuration\n+-------------\n+\n+To configure the Elasticsearch connector, create a catalog properties file\n+``etc/catalog/elasticsearch.properties`` with the following contents,\n+replacing the properties as appropriate:\n+\n+.. code-block:: none\n+\n+    connector.name=elasticsearch\n+    elasticsearch.table-names=schema.table1,schema.table2\n+    elasticsearch.default-schema=default\n+    elasticsearch.table-description-dir=etc/elasticsearch/\n+    elasticsearch.scroll-size=1000\n+    elasticsearch.scroll-time=60000\n+    elasticsearch.request-timeout=2s\n+\n+Configuration Properties\n+------------------------\n+\n+The following configuration properties are available:\n+\n+======================================= ==============================================================================\n+Property Name                           Description\n+======================================= ==============================================================================\n+``elasticsearch.table-names``           List of tables provided by the catalog\n+``elasticsearch.default-schema``        Default schema name for tables\n+``elasticsearch.table-description-dir`` Directory containing JSON table description files\n+``elasticsearch.scroll-size``           Maximum number of hits to be returned with each batch of Elasticsearch scroll\n+``elasticsearch.scroll-timeout``        Number of milliseconds Elasticsearch would keep live for a search scroll result\n+``elasticsearch.max-hits``              Maximum number of hits a single Elasticsearch request could fetch\n+``elasticsearch.request-timeout``       Timeout for Elasticsearch requests\n+======================================= ==============================================================================\n+\n+``elasticsearch.table-names``\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+Comma-separated list of all tables provided by this catalog. A table name\n+can be unqualified (simple name) and will be put into the default schema\n+(see below) or qualified with a schema name (``<schema-name>.<table-name>``).\n+\n+For each table defined here, a table description file must exist (see below).\n+\n+This property is required; there is no default and at least one table must be defined.\n+\n+``elasticsearch.default-schema``\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+Defines the schema that will contain all the tables defined without\n+a qualifying schema name.\n+\n+This property is optional; the default is ``default``.\n+\n+``elasticsearch.table-description-dir``\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+Specifies a path under the Presto deployment directory that contains\n+one or more JSON files with table descriptions (must end with ``.json``).\n+\n+This property is optional; the default is ``etc/elasticsearch``.\n+\n+``elasticsearch.scroll-size``\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+This property defines the maximum number of hits to be returned with each\n+batch of Elasticsearch scroll.\n+\n+This property is optional; the default is ``1000``.\n+\n+``elasticsearch.scroll-time``\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+\n+This property defines number of milliseconds Elasticsearch client would\n+keep live for a search scroll result.\n+\n+This property is optional; the default is ``20s``.\n+\n+``elasticsearch.max-hits``\n+^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+This property defines the maximum number of hits an Elasticsearch request can fetch.\n+\n+This property is optional; the default is ``1000000``.\n+\n+``elasticsearch.request-timeout``\n+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n+\n+This property defines timeout for all Elasticsearch request.\n+\n+This property is optional; the default is ``10s``.\n+\n+Table Definition Files\n+----------------------\n+\n+For Presto, Elasticsearch data must be mapped to tables and columns to allow queries against the data.\n+\n+A table definition file consists of a JSON definition for a table. The\n+name of the file can be arbitrary, but it must end with ``.json``.\n+\n+.. code-block:: none\n+\n+    {\n+        \"tableName\": ...,\n+        \"schemaName\": ...,\n+        \"hostAddress\": ...,\n+        \"port\": ...,\n+        \"clusterName\": ...,\n+        \"index\": ...,\n+        \"indexExactMatch\": ...,\n+        \"type\": ...\n+        \"columns\": [\n+            {\n+                \"name\": ...,\n+                \"type\": ...,\n+                \"jsonPath\": ...,\n+                \"jsonType\": ...,\n+                \"ordinalPosition\": ...\n+            }\n+        ]\n+    }\n+\n+=================== ========= ============== =============================\n+Field               Required  Type           Description\n+=================== ========= ============== =============================\n+``tableName``       required  string         Presto table name defined by this file.\n+``schemaName``      optional  string         Schema that contains the table. If omitted, the default schema name is used.\n+``host``            required  string         Elasticsearch search node host name.\n+``port``            required  integer        Elasticsearch search node port number.\n+``clusterName``     required  string         Elasticsearch cluster name.\n+``index``           required  string         Elasticsearch index that is backing this table.\n+``indexExactMatch`` optional  boolean        true requires index name exact match, false indicates sending request to all indices with the same prefix.\n+``type``            required  string         Elasticsearch type, which represents a group of similar documents.\n+``columns``         optional  list           List of column metadata information.\n+=================== ========= ============== =============================\n+\n+Elasticsearch Column Metadata\n+-----------------------------\n+\n+Optionally, Elasticsearch column metadata can be configured in json file, with the following fields:\n+\n+===================== ========= ============== =============================\n+Field                 Required  Type           Description\n+===================== ========= ============== =============================\n+``name``              optional  string         Column name of Elasticsearch index field.","path":"presto-docs/src/main/sphinx/connector/elasticsearch.rst","position":166,"original_position":166,"commit_id":"850d76aae1fec686a0ffd23799b34475a807bccc","original_commit_id":"850d76aae1fec686a0ffd23799b34475a807bccc","user":{"login":"nezihyigitbasi","id":1223839,"node_id":"MDQ6VXNlcjEyMjM4Mzk=","avatar_url":"https://avatars3.githubusercontent.com/u/1223839?v=4","gravatar_id":"","url":"https://api.github.com/users/nezihyigitbasi","html_url":"https://github.com/nezihyigitbasi","followers_url":"https://api.github.com/users/nezihyigitbasi/followers","following_url":"https://api.github.com/users/nezihyigitbasi/following{/other_user}","gists_url":"https://api.github.com/users/nezihyigitbasi/gists{/gist_id}","starred_url":"https://api.github.com/users/nezihyigitbasi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nezihyigitbasi/subscriptions","organizations_url":"https://api.github.com/users/nezihyigitbasi/orgs","repos_url":"https://api.github.com/users/nezihyigitbasi/repos","events_url":"https://api.github.com/users/nezihyigitbasi/events{/privacy}","received_events_url":"https://api.github.com/users/nezihyigitbasi/received_events","type":"User","site_admin":false},"body":"`Column name of the index field.`","created_at":"2018-12-06T01:09:57Z","updated_at":"2018-12-06T01:11:05Z","html_url":"https://github.com/prestodb/presto/pull/11942#discussion_r239297572","pull_request_url":"https://api.github.com/repos/prestodb/presto/pulls/11942","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/prestodb/presto/pulls/comments/239297572"},"html":{"href":"https://github.com/prestodb/presto/pull/11942#discussion_r239297572"},"pull_request":{"href":"https://api.github.com/repos/prestodb/presto/pulls/11942"}}},"pull_request":{"url":"https://api.github.com/repos/prestodb/presto/pulls/11942","id":231428375,"node_id":"MDExOlB1bGxSZXF1ZXN0MjMxNDI4Mzc1","html_url":"https://github.com/prestodb/presto/pull/11942","diff_url":"https://github.com/prestodb/presto/pull/11942.diff","patch_url":"https://github.com/prestodb/presto/pull/11942.patch","issue_url":"https://api.github.com/repos/prestodb/presto/issues/11942","number":11942,"state":"open","locked":false,"title":"Presto Elasticsearch Connector","user":{"login":"zhenxiao","id":1274155,"node_id":"MDQ6VXNlcjEyNzQxNTU=","avatar_url":"https://avatars2.githubusercontent.com/u/1274155?v=4","gravatar_id":"","url":"https://api.github.com/users/zhenxiao","html_url":"https://github.com/zhenxiao","followers_url":"https://api.github.com/users/zhenxiao/followers","following_url":"https://api.github.com/users/zhenxiao/following{/other_user}","gists_url":"https://api.github.com/users/zhenxiao/gists{/gist_id}","starred_url":"https://api.github.com/users/zhenxiao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhenxiao/subscriptions","organizations_url":"https://api.github.com/users/zhenxiao/orgs","repos_url":"https://api.github.com/users/zhenxiao/repos","events_url":"https://api.github.com/users/zhenxiao/events{/privacy}","received_events_url":"https://api.github.com/users/zhenxiao/received_events","type":"User","site_admin":false},"body":"continue Presto Elasticsearch Connector:\r\nhttps://github.com/prestodb/presto/pull/9459\r\n\r\n@nezihyigitbasi @kokosing ","created_at":"2018-11-16T06:26:54Z","updated_at":"2018-12-06T01:11:04Z","closed_at":null,"merged_at":null,"merge_commit_sha":"bfde6c27041b585db482ea932683fb979630582b","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":196992743,"node_id":"MDU6TGFiZWwxOTY5OTI3NDM=","url":"https://api.github.com/repos/prestodb/presto/labels/CLA%20Signed","name":"CLA Signed","color":"ededed","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/prestodb/presto/pulls/11942/commits","review_comments_url":"https://api.github.com/repos/prestodb/presto/pulls/11942/comments","review_comment_url":"https://api.github.com/repos/prestodb/presto/pulls/comments{/number}","comments_url":"https://api.github.com/repos/prestodb/presto/issues/11942/comments","statuses_url":"https://api.github.com/repos/prestodb/presto/statuses/850d76aae1fec686a0ffd23799b34475a807bccc","head":{"label":"zhenxiao:elk","ref":"elk","sha":"850d76aae1fec686a0ffd23799b34475a807bccc","user":{"login":"zhenxiao","id":1274155,"node_id":"MDQ6VXNlcjEyNzQxNTU=","avatar_url":"https://avatars2.githubusercontent.com/u/1274155?v=4","gravatar_id":"","url":"https://api.github.com/users/zhenxiao","html_url":"https://github.com/zhenxiao","followers_url":"https://api.github.com/users/zhenxiao/followers","following_url":"https://api.github.com/users/zhenxiao/following{/other_user}","gists_url":"https://api.github.com/users/zhenxiao/gists{/gist_id}","starred_url":"https://api.github.com/users/zhenxiao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhenxiao/subscriptions","organizations_url":"https://api.github.com/users/zhenxiao/orgs","repos_url":"https://api.github.com/users/zhenxiao/repos","events_url":"https://api.github.com/users/zhenxiao/events{/privacy}","received_events_url":"https://api.github.com/users/zhenxiao/received_events","type":"User","site_admin":false},"repo":{"id":17963913,"node_id":"MDEwOlJlcG9zaXRvcnkxNzk2MzkxMw==","name":"presto","full_name":"zhenxiao/presto","private":false,"owner":{"login":"zhenxiao","id":1274155,"node_id":"MDQ6VXNlcjEyNzQxNTU=","avatar_url":"https://avatars2.githubusercontent.com/u/1274155?v=4","gravatar_id":"","url":"https://api.github.com/users/zhenxiao","html_url":"https://github.com/zhenxiao","followers_url":"https://api.github.com/users/zhenxiao/followers","following_url":"https://api.github.com/users/zhenxiao/following{/other_user}","gists_url":"https://api.github.com/users/zhenxiao/gists{/gist_id}","starred_url":"https://api.github.com/users/zhenxiao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhenxiao/subscriptions","organizations_url":"https://api.github.com/users/zhenxiao/orgs","repos_url":"https://api.github.com/users/zhenxiao/repos","events_url":"https://api.github.com/users/zhenxiao/events{/privacy}","received_events_url":"https://api.github.com/users/zhenxiao/received_events","type":"User","site_admin":false},"html_url":"https://github.com/zhenxiao/presto","description":"Distributed SQL query engine for big data","fork":true,"url":"https://api.github.com/repos/zhenxiao/presto","forks_url":"https://api.github.com/repos/zhenxiao/presto/forks","keys_url":"https://api.github.com/repos/zhenxiao/presto/keys{/key_id}","collaborators_url":"https://api.github.com/repos/zhenxiao/presto/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/zhenxiao/presto/teams","hooks_url":"https://api.github.com/repos/zhenxiao/presto/hooks","issue_events_url":"https://api.github.com/repos/zhenxiao/presto/issues/events{/number}","events_url":"https://api.github.com/repos/zhenxiao/presto/events","assignees_url":"https://api.github.com/repos/zhenxiao/presto/assignees{/user}","branches_url":"https://api.github.com/repos/zhenxiao/presto/branches{/branch}","tags_url":"https://api.github.com/repos/zhenxiao/presto/tags","blobs_url":"https://api.github.com/repos/zhenxiao/presto/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/zhenxiao/presto/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/zhenxiao/presto/git/refs{/sha}","trees_url":"https://api.github.com/repos/zhenxiao/presto/git/trees{/sha}","statuses_url":"https://api.github.com/repos/zhenxiao/presto/statuses/{sha}","languages_url":"https://api.github.com/repos/zhenxiao/presto/languages","stargazers_url":"https://api.github.com/repos/zhenxiao/presto/stargazers","contributors_url":"https://api.github.com/repos/zhenxiao/presto/contributors","subscribers_url":"https://api.github.com/repos/zhenxiao/presto/subscribers","subscription_url":"https://api.github.com/repos/zhenxiao/presto/subscription","commits_url":"https://api.github.com/repos/zhenxiao/presto/commits{/sha}","git_commits_url":"https://api.github.com/repos/zhenxiao/presto/git/commits{/sha}","comments_url":"https://api.github.com/repos/zhenxiao/presto/comments{/number}","issue_comment_url":"https://api.github.com/repos/zhenxiao/presto/issues/comments{/number}","contents_url":"https://api.github.com/repos/zhenxiao/presto/contents/{+path}","compare_url":"https://api.github.com/repos/zhenxiao/presto/compare/{base}...{head}","merges_url":"https://api.github.com/repos/zhenxiao/presto/merges","archive_url":"https://api.github.com/repos/zhenxiao/presto/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/zhenxiao/presto/downloads","issues_url":"https://api.github.com/repos/zhenxiao/presto/issues{/number}","pulls_url":"https://api.github.com/repos/zhenxiao/presto/pulls{/number}","milestones_url":"https://api.github.com/repos/zhenxiao/presto/milestones{/number}","notifications_url":"https://api.github.com/repos/zhenxiao/presto/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/zhenxiao/presto/labels{/name}","releases_url":"https://api.github.com/repos/zhenxiao/presto/releases{/id}","deployments_url":"https://api.github.com/repos/zhenxiao/presto/deployments","created_at":"2014-03-21T01:03:42Z","updated_at":"2018-03-22T11:08:39Z","pushed_at":"2018-12-04T05:57:07Z","git_url":"git://github.com/zhenxiao/presto.git","ssh_url":"git@github.com:zhenxiao/presto.git","clone_url":"https://github.com/zhenxiao/presto.git","svn_url":"https://github.com/zhenxiao/presto","homepage":"http://prestodb.io/","size":103874,"stargazers_count":2,"watchers_count":2,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":2,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":2,"open_issues":0,"watchers":2,"default_branch":"master"}},"base":{"label":"prestodb:master","ref":"master","sha":"d7ae9113dd3fcbb2831ecc05e39c58d230c67ca0","user":{"login":"prestodb","id":6882181,"node_id":"MDEyOk9yZ2FuaXphdGlvbjY4ODIxODE=","avatar_url":"https://avatars0.githubusercontent.com/u/6882181?v=4","gravatar_id":"","url":"https://api.github.com/users/prestodb","html_url":"https://github.com/prestodb","followers_url":"https://api.github.com/users/prestodb/followers","following_url":"https://api.github.com/users/prestodb/following{/other_user}","gists_url":"https://api.github.com/users/prestodb/gists{/gist_id}","starred_url":"https://api.github.com/users/prestodb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prestodb/subscriptions","organizations_url":"https://api.github.com/users/prestodb/orgs","repos_url":"https://api.github.com/users/prestodb/repos","events_url":"https://api.github.com/users/prestodb/events{/privacy}","received_events_url":"https://api.github.com/users/prestodb/received_events","type":"Organization","site_admin":false},"repo":{"id":5349565,"node_id":"MDEwOlJlcG9zaXRvcnk1MzQ5NTY1","name":"presto","full_name":"prestodb/presto","private":false,"owner":{"login":"prestodb","id":6882181,"node_id":"MDEyOk9yZ2FuaXphdGlvbjY4ODIxODE=","avatar_url":"https://avatars0.githubusercontent.com/u/6882181?v=4","gravatar_id":"","url":"https://api.github.com/users/prestodb","html_url":"https://github.com/prestodb","followers_url":"https://api.github.com/users/prestodb/followers","following_url":"https://api.github.com/users/prestodb/following{/other_user}","gists_url":"https://api.github.com/users/prestodb/gists{/gist_id}","starred_url":"https://api.github.com/users/prestodb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/prestodb/subscriptions","organizations_url":"https://api.github.com/users/prestodb/orgs","repos_url":"https://api.github.com/users/prestodb/repos","events_url":"https://api.github.com/users/prestodb/events{/privacy}","received_events_url":"https://api.github.com/users/prestodb/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/prestodb/presto","description":"Distributed SQL query engine for big data","fork":false,"url":"https://api.github.com/repos/prestodb/presto","forks_url":"https://api.github.com/repos/prestodb/presto/forks","keys_url":"https://api.github.com/repos/prestodb/presto/keys{/key_id}","collaborators_url":"https://api.github.com/repos/prestodb/presto/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/prestodb/presto/teams","hooks_url":"https://api.github.com/repos/prestodb/presto/hooks","issue_events_url":"https://api.github.com/repos/prestodb/presto/issues/events{/number}","events_url":"https://api.github.com/repos/prestodb/presto/events","assignees_url":"https://api.github.com/repos/prestodb/presto/assignees{/user}","branches_url":"https://api.github.com/repos/prestodb/presto/branches{/branch}","tags_url":"https://api.github.com/repos/prestodb/presto/tags","blobs_url":"https://api.github.com/repos/prestodb/presto/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/prestodb/presto/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/prestodb/presto/git/refs{/sha}","trees_url":"https://api.github.com/repos/prestodb/presto/git/trees{/sha}","statuses_url":"https://api.github.com/repos/prestodb/presto/statuses/{sha}","languages_url":"https://api.github.com/repos/prestodb/presto/languages","stargazers_url":"https://api.github.com/repos/prestodb/presto/stargazers","contributors_url":"https://api.github.com/repos/prestodb/presto/contributors","subscribers_url":"https://api.github.com/repos/prestodb/presto/subscribers","subscription_url":"https://api.github.com/repos/prestodb/presto/subscription","commits_url":"https://api.github.com/repos/prestodb/presto/commits{/sha}","git_commits_url":"https://api.github.com/repos/prestodb/presto/git/commits{/sha}","comments_url":"https://api.github.com/repos/prestodb/presto/comments{/number}","issue_comment_url":"https://api.github.com/repos/prestodb/presto/issues/comments{/number}","contents_url":"https://api.github.com/repos/prestodb/presto/contents/{+path}","compare_url":"https://api.github.com/repos/prestodb/presto/compare/{base}...{head}","merges_url":"https://api.github.com/repos/prestodb/presto/merges","archive_url":"https://api.github.com/repos/prestodb/presto/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/prestodb/presto/downloads","issues_url":"https://api.github.com/repos/prestodb/presto/issues{/number}","pulls_url":"https://api.github.com/repos/prestodb/presto/pulls{/number}","milestones_url":"https://api.github.com/repos/prestodb/presto/milestones{/number}","notifications_url":"https://api.github.com/repos/prestodb/presto/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/prestodb/presto/labels{/name}","releases_url":"https://api.github.com/repos/prestodb/presto/releases{/id}","deployments_url":"https://api.github.com/repos/prestodb/presto/deployments","created_at":"2012-08-09T01:03:37Z","updated_at":"2018-12-05T21:34:17Z","pushed_at":"2018-12-06T00:35:33Z","git_url":"git://github.com/prestodb/presto.git","ssh_url":"git@github.com:prestodb/presto.git","clone_url":"https://github.com/prestodb/presto.git","svn_url":"https://github.com/prestodb/presto","homepage":"https://prestodb.io","size":104949,"stargazers_count":8388,"watchers_count":8388,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":2849,"mirror_url":null,"archived":false,"open_issues_count":1470,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":2849,"open_issues":1470,"watchers":8388,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/prestodb/presto/pulls/11942"},"html":{"href":"https://github.com/prestodb/presto/pull/11942"},"issue":{"href":"https://api.github.com/repos/prestodb/presto/issues/11942"},"comments":{"href":"https://api.github.com/repos/prestodb/presto/issues/11942/comments"},"review_comments":{"href":"https://api.github.com/repos/prestodb/presto/pulls/11942/comments"},"review_comment":{"href":"https://api.github.com/repos/prestodb/presto/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/prestodb/presto/pulls/11942/commits"},"statuses":{"href":"https://api.github.com/repos/prestodb/presto/statuses/850d76aae1fec686a0ffd23799b34475a807bccc"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"nezihyigitbasi"}}