{"action":"created","issue":{"url":"https://api.github.com/repos/apache/carbondata/issues/2310","repository_url":"https://api.github.com/repos/apache/carbondata","labels_url":"https://api.github.com/repos/apache/carbondata/issues/2310/labels{/name}","comments_url":"https://api.github.com/repos/apache/carbondata/issues/2310/comments","events_url":"https://api.github.com/repos/apache/carbondata/issues/2310/events","html_url":"https://github.com/apache/carbondata/pull/2310","id":323246354,"number":2310,"title":"[CARBONDATA-2484][LUCENE]Refactor distributable code and lauch job to clear the datamap from executor(clears segmentMap and remove datamap from cache)","user":{"login":"akashrn5","id":21074821,"avatar_url":"https://avatars0.githubusercontent.com/u/21074821?v=4","gravatar_id":"","url":"https://api.github.com/users/akashrn5","html_url":"https://github.com/akashrn5","followers_url":"https://api.github.com/users/akashrn5/followers","following_url":"https://api.github.com/users/akashrn5/following{/other_user}","gists_url":"https://api.github.com/users/akashrn5/gists{/gist_id}","starred_url":"https://api.github.com/users/akashrn5/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akashrn5/subscriptions","organizations_url":"https://api.github.com/users/akashrn5/orgs","repos_url":"https://api.github.com/users/akashrn5/repos","events_url":"https://api.github.com/users/akashrn5/events{/privacy}","received_events_url":"https://api.github.com/users/akashrn5/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2018-05-15T14:35:22Z","updated_at":"2018-05-17T11:30:11Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/carbondata/pulls/2310","html_url":"https://github.com/apache/carbondata/pull/2310","diff_url":"https://github.com/apache/carbondata/pull/2310.diff","patch_url":"https://github.com/apache/carbondata/pull/2310.patch"},"body":"Problem:\r\nDuring query, blockletDataMapFactory maintains a segmentMap which has mapping of \r\nsegmentId -> list of index file, and this will be used while getting the extended blocklet\r\nby checking whether the blocklet present in the index or not.\r\nIn case of Lucene, the datamap job will be launched and during pruning the segmentMap will be added\r\nin executor and this map will be cleared in driver when drop table is called, but it will not be cleared in executor.\r\nso when the query is fired after table or datamap is dropped, the lucene query fails.\r\n\r\nSolution:\r\nSo when drop table or drop datamap is called a job is launched which clears the datamaps from segmentMap and cache and\r\nthen clears in driver.\r\n\r\nThis PR also refactors the datamap job classes and other common classes.\r\n\r\nBe sure to do all of the following checklist to help us incorporate \r\nyour contribution quickly and easily:\r\n\r\n - [x] Any interfaces changed?\r\n YES\r\n - [x] Any backward compatibility impacted?\r\n NA\r\n - [x] Document update required?\r\nNA\r\n - [x] Testing done\r\ntested in cluster\r\n        Please provide details on \r\n        - Whether new unit test cases have been added or why no new tests are required?\r\n        - How it is tested? Please attach test report.\r\n        - Is it a performance related change? Please attach the performance test report.\r\n        - Any additional information to help reviewers in testing this change.\r\n       \r\n - [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. \r\nNA\r\n"},"comment":{"url":"https://api.github.com/repos/apache/carbondata/issues/comments/389835025","html_url":"https://github.com/apache/carbondata/pull/2310#issuecomment-389835025","issue_url":"https://api.github.com/repos/apache/carbondata/issues/2310","id":389835025,"user":{"login":"CarbonDataQA","id":24402133,"avatar_url":"https://avatars0.githubusercontent.com/u/24402133?v=4","gravatar_id":"","url":"https://api.github.com/users/CarbonDataQA","html_url":"https://github.com/CarbonDataQA","followers_url":"https://api.github.com/users/CarbonDataQA/followers","following_url":"https://api.github.com/users/CarbonDataQA/following{/other_user}","gists_url":"https://api.github.com/users/CarbonDataQA/gists{/gist_id}","starred_url":"https://api.github.com/users/CarbonDataQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/CarbonDataQA/subscriptions","organizations_url":"https://api.github.com/users/CarbonDataQA/orgs","repos_url":"https://api.github.com/users/CarbonDataQA/repos","events_url":"https://api.github.com/users/CarbonDataQA/events{/privacy}","received_events_url":"https://api.github.com/users/CarbonDataQA/received_events","type":"User","site_admin":false},"created_at":"2018-05-17T11:30:11Z","updated_at":"2018-05-17T11:30:11Z","author_association":"NONE","body":"Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5937/\n"}}
{"actor":{"display_login":"CarbonDataQA"}}