{"action":"created","issue":{"url":"https://api.github.com/repos/unipop-graph/unipop/issues/135","repository_url":"https://api.github.com/repos/unipop-graph/unipop","labels_url":"https://api.github.com/repos/unipop-graph/unipop/issues/135/labels{/name}","comments_url":"https://api.github.com/repos/unipop-graph/unipop/issues/135/comments","events_url":"https://api.github.com/repos/unipop-graph/unipop/issues/135/events","html_url":"https://github.com/unipop-graph/unipop/issues/135","id":354671719,"node_id":"MDU6SXNzdWUzNTQ2NzE3MTk=","number":135,"title":"Unipop Elasticsearch plugin working only on 10000 docuements","user":{"login":"gaurav6041","id":40815733,"node_id":"MDQ6VXNlcjQwODE1NzMz","avatar_url":"https://avatars0.githubusercontent.com/u/40815733?v=4","gravatar_id":"","url":"https://api.github.com/users/gaurav6041","html_url":"https://github.com/gaurav6041","followers_url":"https://api.github.com/users/gaurav6041/followers","following_url":"https://api.github.com/users/gaurav6041/following{/other_user}","gists_url":"https://api.github.com/users/gaurav6041/gists{/gist_id}","starred_url":"https://api.github.com/users/gaurav6041/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gaurav6041/subscriptions","organizations_url":"https://api.github.com/users/gaurav6041/orgs","repos_url":"https://api.github.com/users/gaurav6041/repos","events_url":"https://api.github.com/users/gaurav6041/events{/privacy}","received_events_url":"https://api.github.com/users/gaurav6041/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":27,"created_at":"2018-08-28T10:39:17Z","updated_at":"2018-10-18T04:02:20Z","closed_at":null,"author_association":"NONE","body":"I have an index where one field has has a value of  department in which the employee is present in.\r\nBelow is present Unipop elasticsearch configurations\r\n\r\n```\r\n{\r\n  \"class\": \"org.unipop.elastic.ElasticSourceProvider\",\r\n  \"addresses\": \"http://192.168.10.121:6968\",\r\n  \"vertices\": [\r\n    {\r\n      \"index\": \"employee_records\",\r\n      \"id\": \"@department\",\r\n      \"label\": \"hastags\",\r\n      \"properties\": {\r\n        \"value\": \"@department\"\r\n      }\r\n    },\r\n    {\r\n      \"index\": \"employee_records\",\r\n      \"id\": \"@unique\",\r\n      \"label\": \"unique\",\r\n      \"properties\": {\r\n        \"value\": \"@unique\"\r\n      }\r\n    }\r\n  ],\r\n  \"edges\": [\r\n    {\r\n      \"index\": \"employee_records\",\r\n      \"id\": \"@_id\",\r\n      \"label\": \"connects\",\r\n      \"properties\": {},\r\n      \"outVertex\": {\r\n        \"ref\": true,\r\n        \"id\": \"@unique\",\r\n        \"label\": \"unique\"\r\n      },\r\n      \"inVertex\": {\r\n        \"ref\": true,\r\n        \"id\": \"@department\",\r\n        \"label\": \"hastags\"\r\n      }\r\n    }\r\n  ]\r\n}\r\n\r\n```\r\nAnd below are results\r\n{v[Tech]=3159, v[Analysis]=3726, v[Admin]=2111, v[Support]=1003}\r\n\r\nAnd Here is my gremlin query => **g.V().outE('connects').inV().groupCount()**\r\nThere are total of 65000 employee records. But the above result totals to 9999\r\nIn my above configurations i have mapped **department of employees** with **employee id**.\r\nSo count of all the departments should total out to 9999 whereas it should be 65000.\r\n\r\nThis means that unipop is picking only 10000 documents from elasticsearch.\r\nThis might me due to **limit of elasticsearch of fetching 10000 documents** in one request and **scroll api is not been used**.\r\n\r\n\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/unipop-graph/unipop/issues/comments/430868116","html_url":"https://github.com/unipop-graph/unipop/issues/135#issuecomment-430868116","issue_url":"https://api.github.com/repos/unipop-graph/unipop/issues/135","id":430868116,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMDg2ODExNg==","user":{"login":"eyalmar100","id":5397091,"node_id":"MDQ6VXNlcjUzOTcwOTE=","avatar_url":"https://avatars0.githubusercontent.com/u/5397091?v=4","gravatar_id":"","url":"https://api.github.com/users/eyalmar100","html_url":"https://github.com/eyalmar100","followers_url":"https://api.github.com/users/eyalmar100/followers","following_url":"https://api.github.com/users/eyalmar100/following{/other_user}","gists_url":"https://api.github.com/users/eyalmar100/gists{/gist_id}","starred_url":"https://api.github.com/users/eyalmar100/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eyalmar100/subscriptions","organizations_url":"https://api.github.com/users/eyalmar100/orgs","repos_url":"https://api.github.com/users/eyalmar100/repos","events_url":"https://api.github.com/users/eyalmar100/events{/privacy}","received_events_url":"https://api.github.com/users/eyalmar100/received_events","type":"User","site_admin":false},"created_at":"2018-10-18T04:02:20Z","updated_at":"2018-10-18T04:02:20Z","author_association":"COLLABORATOR","body":"Hi @gaurav6041 \r\nSorry for the delay ..had some problems,\r\nI will deploy it next week ( hopefully ) \r\nThanks"}}
{"actor":{"display_login":"eyalmar100"}}