{"action":"created","comment":{"url":"https://api.github.com/repos/eclipse/che/pulls/comments/69380206","id":69380206,"diff_hunk":"@@ -0,0 +1,63 @@\n+/*******************************************************************************\n+ * Copyright (c) 2012-2016 Codenvy, S.A.\n+ * All rights reserved. This program and the accompanying materials\n+ * are made available under the terms of the Eclipse Public License v1.0\n+ * which accompanies this distribution, and is available at\n+ * http://www.eclipse.org/legal/epl-v10.html\n+ *\n+ * Contributors:\n+ *   Codenvy, S.A. - initial API and implementation\n+ *******************************************************************************/\n+package org.eclipse.che.api.vfs.impl.file.event;\n+\n+import com.google.common.annotations.Beta;\n+\n+import org.slf4j.Logger;\n+\n+import javax.annotation.PostConstruct;\n+import javax.annotation.PreDestroy;\n+import java.util.concurrent.ExecutorService;\n+import java.util.concurrent.atomic.AtomicBoolean;\n+\n+import static java.util.concurrent.Executors.newSingleThreadExecutor;\n+import static org.slf4j.LoggerFactory.getLogger;\n+\n+/**\n+ * @author Dmitry Kuleshov\n+ *\n+ * @since 4.5\n+ */\n+@Beta\n+abstract class VfsEventService {\n+    private static final Logger LOG = getLogger(VfsEventService.class);\n+\n+    private final ExecutorService executor;\n+    private final AtomicBoolean   running;\n+\n+    public VfsEventService() {\n+        this.executor = newSingleThreadExecutor();","path":"wsagent/che-core-api-project/src/main/java/org/eclipse/che/api/vfs/impl/file/event/VfsEventService.java","position":38,"original_position":38,"commit_id":"e23d5b71ad7e4801f0dbaa930c11409b84df48ea","original_commit_id":"e23d5b71ad7e4801f0dbaa930c11409b84df48ea","user":{"login":"garagatyi","id":1851863,"avatar_url":"https://avatars.githubusercontent.com/u/1851863?v=3","gravatar_id":"","url":"https://api.github.com/users/garagatyi","html_url":"https://github.com/garagatyi","followers_url":"https://api.github.com/users/garagatyi/followers","following_url":"https://api.github.com/users/garagatyi/following{/other_user}","gists_url":"https://api.github.com/users/garagatyi/gists{/gist_id}","starred_url":"https://api.github.com/users/garagatyi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/garagatyi/subscriptions","organizations_url":"https://api.github.com/users/garagatyi/orgs","repos_url":"https://api.github.com/users/garagatyi/repos","events_url":"https://api.github.com/users/garagatyi/events{/privacy}","received_events_url":"https://api.github.com/users/garagatyi/received_events","type":"User","site_admin":false},"body":"Please use thread builder factory to set name of the thread. That will help in case we will need to analyze thread dump. Here is an example of the way we usually use it https://github.com/eclipse/che/blob/25e6cc2360133ead8847feccf1f36a5c3f03a2fb/wsmaster/che-core-api-workspace/src/main/java/org/eclipse/che/api/workspace/server/WorkspaceManager.java#L110-L110","created_at":"2016-07-02T16:53:29Z","updated_at":"2016-07-02T16:53:29Z","html_url":"https://github.com/eclipse/che/pull/1634#discussion_r69380206","pull_request_url":"https://api.github.com/repos/eclipse/che/pulls/1634","_links":{"self":{"href":"https://api.github.com/repos/eclipse/che/pulls/comments/69380206"},"html":{"href":"https://github.com/eclipse/che/pull/1634#discussion_r69380206"},"pull_request":{"href":"https://api.github.com/repos/eclipse/che/pulls/1634"}}},"pull_request":{"url":"https://api.github.com/repos/eclipse/che/pulls/1634","id":76064540,"html_url":"https://github.com/eclipse/che/pull/1634","diff_url":"https://github.com/eclipse/che/pull/1634.diff","patch_url":"https://github.com/eclipse/che/pull/1634.patch","issue_url":"https://api.github.com/repos/eclipse/che/issues/1634","number":1634,"state":"open","locked":false,"title":"[WIP] CHE-11: VFS event handling system","user":{"login":"dkuleshov","id":1632360,"avatar_url":"https://avatars.githubusercontent.com/u/1632360?v=3","gravatar_id":"","url":"https://api.github.com/users/dkuleshov","html_url":"https://github.com/dkuleshov","followers_url":"https://api.github.com/users/dkuleshov/followers","following_url":"https://api.github.com/users/dkuleshov/following{/other_user}","gists_url":"https://api.github.com/users/dkuleshov/gists{/gist_id}","starred_url":"https://api.github.com/users/dkuleshov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dkuleshov/subscriptions","organizations_url":"https://api.github.com/users/dkuleshov/orgs","repos_url":"https://api.github.com/users/dkuleshov/repos","events_url":"https://api.github.com/users/dkuleshov/events{/privacy}","received_events_url":"https://api.github.com/users/dkuleshov/received_events","type":"User","site_admin":false},"body":"Hi,\r\n\r\nI'm working on a new VFS event handling system, I prepared a pull request with initial implementation and I would like to discuss the approach.\r\n\r\n### Reasoning\r\nWe need a service to be able to detect virtual file system (and physical file system correspondingly) events and properly react. In this there is a need because now there is a number of problems. For example right now some file system related events are not handled at all or handled partially. Another issue: there is no automated event management on VFS level, that means that a developer has to care by himself to raise an event to handle it later. For example when we implement an editor to be able to properly react on file changes first we have to manually detect (somehow) those changes and then manually raise corresponding event while that could be done on VFS level automatically and a developer would only care about how to react on an event. Finally, there is one more thing that is worth mentioning. The matter is that without proper means of event handling on file system level we have no possibility to detect (and as a consequence to react on) events that have their origin outside of IDE. For example, when a user edits a file (that is already opened in IDE editor) using terminal or MC editor we are unable detect that event thus we can't refresh edited file's content in IDE editor.\r\n\r\n### Idea\r\nGenerally speaking that should not be a problem to catch file or directory modification event but there are many examples where it is just not enough for our purposes. When we create a project it is not enough to track folder creation and when we check out a branch it is not enough to track `.git/HEAD` file modifications. Often to see the picture we need to track the whole sequence of events or vice versa the sequence leads to the final event that is the only one that is meaningful for us. For example, if we create a java project we track root folder, `src` folder and almost certainly project object model file (let's assume that it is a maven project). One more example, when we edit a file within a few seconds (a sequence of file modifications) there is no real need to track intermediate file states, so we can skip most of events except the last one with the final file state. So the challenge I see here is to implement a clever system that will not only handle events but also will provide means to easily and effectively analyze both single and sequential events.\r\n\r\nIn the first place, I propose to distinguish events according to their nature and logical level. The low level includes events related to modification of items in context of file system (e.g. creation of a directory, modification of a file). While the high level includes events related to complex logical events that represents a higher level of abstraction (e.g. modification of POM file, compiling project sources).\r\n\r\nNext thing is event segments. If we imagine a timeline with events then the event segment will be a segment of that line that contains all events that are close to each other in terms of time. If time interval between two events is less than some defined value, then we consider those events related to each other and we attribute them to a common event segment. And the opposite, if timestamps of two consequent events differs greatly we consider those events as belonging to two different event segments and are not related. Thus we will split an event timeline into segments containing sequences of related events.\r\n\r\nIn order to simplify analysis of those segments, I assume we can introduce a tree-like structures that will correspond to trees of items of a file system and hold data related to events that we caught. If an item is modified on a file system it is also noted in an event tree in a corresponding place with all related data set.\r\n\r\nThe last important thing I would like to introduce is so called event detectors. Briefly, they are responsible for event tree analysis and also produce corresponding high level events. How it should work: we apply those detectors to each event tree that we get in real time to see if corresponding event happened, when appropriate conditions are met we raise high level event.\r\n\r\n### Implementation\r\nProbably it is yet to be discussed but I have implemented a very basic draft of a system and I would like to provide a brief description of the most valuable components.\r\n\r\n`LoEventListener` class is the beginning. The only thing it does is listens for `LoEvent` - low level file system events - and populates a `LoEventQueueHolder`. \r\n\r\n`LoEventService` is responsible for event timeline segmentation and populating of  `EventTreeQueueHolder`.\r\n\r\nGenerally speaking, a couple of `QueueHolder` classes are used to share data access between entities using dependency injection while queues themselves are used to conform with variation of producer/consumer pattern.\r\n\r\nFinally, `HiEventService` uses `HiEventDetectorManager` and `HiEventBroadcasterManager` to correspondingly detect `HiEvent` with `HiEventDetector` implementations and broadcast them.\r\n\r\n### Conclusion\r\nIn conclusion I will emphasize the benefits that I see:\r\n\r\n- A good idea of the overall picture. When we create event trees we have quite a good amount of data to make more accurate decision, in comparison to processing event sequence when we separately and independently analyze events.\r\n- Source of modification does not matter. We have a huge amount of problems because of inability to effectively track events caused by outer sources (e.g. separate SSH). While new approach will handle such issues automatically because it does not care about source of modification, though it may take it into account if necessary.\r\n- Ease of extending. If a developer wants to track a new type of events there is no need to do extra coding, he just need to provide corresponding HiEventDetector. This is an incontestable benefit of dependency injection framework Guice and well-decoupled logics approach.\r\n- Simplification of recognition of complex events. This is achieved thanks to separation of logical (high level) and physical (low level) events.\r\n- Still effective detection of simple events. We can easily detect any simple event inside complex one due to the fact that the complex event is represented by a handy data structure.\r\n\r\n\r\nAnd the impediments that I foresee:\r\n\r\n- Performance. While the implementation performance has already been tested on staging server, I still believe that there is a risk of performance issues. The thing is that along with good opportunities for an accurate event timeline analysis we get overhead in memory consumption (which is within the normal right now) and a bit more usage of processor time. In comparison to plain real time event handling of course. \r\n- Depends on low level implementations. I mean that we don't get events directly form file system. Before we know that something happens the event passes several intermediate layers which are not under full our control. It is a potential place for problems and may cause misunderstandings or system malfunctions.\r\n- Real time file system event processing. Any real time processing always makes things more complex because no one clearly know what will happen and when. We assume that events related to each other happen almost simultaneously but in fact that may not always be true.\r\n- More complex event we want to track, more complex event detector we must provide. Some events may require quite a big amount of effort to track, especially if they affect several event segments or require extra data.","created_at":"2016-07-02T13:00:17Z","updated_at":"2016-07-02T16:53:29Z","closed_at":null,"merged_at":null,"merge_commit_sha":"aefbbc0f07619c880d99cf6ad17d8e9bd839aa37","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/eclipse/che/pulls/1634/commits","review_comments_url":"https://api.github.com/repos/eclipse/che/pulls/1634/comments","review_comment_url":"https://api.github.com/repos/eclipse/che/pulls/comments{/number}","comments_url":"https://api.github.com/repos/eclipse/che/issues/1634/comments","statuses_url":"https://api.github.com/repos/eclipse/che/statuses/e23d5b71ad7e4801f0dbaa930c11409b84df48ea","head":{"label":"dkuleshov:CHE-11","ref":"CHE-11","sha":"e23d5b71ad7e4801f0dbaa930c11409b84df48ea","user":{"login":"dkuleshov","id":1632360,"avatar_url":"https://avatars.githubusercontent.com/u/1632360?v=3","gravatar_id":"","url":"https://api.github.com/users/dkuleshov","html_url":"https://github.com/dkuleshov","followers_url":"https://api.github.com/users/dkuleshov/followers","following_url":"https://api.github.com/users/dkuleshov/following{/other_user}","gists_url":"https://api.github.com/users/dkuleshov/gists{/gist_id}","starred_url":"https://api.github.com/users/dkuleshov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dkuleshov/subscriptions","organizations_url":"https://api.github.com/users/dkuleshov/orgs","repos_url":"https://api.github.com/users/dkuleshov/repos","events_url":"https://api.github.com/users/dkuleshov/events{/privacy}","received_events_url":"https://api.github.com/users/dkuleshov/received_events","type":"User","site_admin":false},"repo":{"id":59562244,"name":"che","full_name":"dkuleshov/che","owner":{"login":"dkuleshov","id":1632360,"avatar_url":"https://avatars.githubusercontent.com/u/1632360?v=3","gravatar_id":"","url":"https://api.github.com/users/dkuleshov","html_url":"https://github.com/dkuleshov","followers_url":"https://api.github.com/users/dkuleshov/followers","following_url":"https://api.github.com/users/dkuleshov/following{/other_user}","gists_url":"https://api.github.com/users/dkuleshov/gists{/gist_id}","starred_url":"https://api.github.com/users/dkuleshov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dkuleshov/subscriptions","organizations_url":"https://api.github.com/users/dkuleshov/orgs","repos_url":"https://api.github.com/users/dkuleshov/repos","events_url":"https://api.github.com/users/dkuleshov/events{/privacy}","received_events_url":"https://api.github.com/users/dkuleshov/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/dkuleshov/che","description":"Eclipse Che: Next-generation Eclipse IDE.  Open source workspace server and cloud IDE.","fork":true,"url":"https://api.github.com/repos/dkuleshov/che","forks_url":"https://api.github.com/repos/dkuleshov/che/forks","keys_url":"https://api.github.com/repos/dkuleshov/che/keys{/key_id}","collaborators_url":"https://api.github.com/repos/dkuleshov/che/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/dkuleshov/che/teams","hooks_url":"https://api.github.com/repos/dkuleshov/che/hooks","issue_events_url":"https://api.github.com/repos/dkuleshov/che/issues/events{/number}","events_url":"https://api.github.com/repos/dkuleshov/che/events","assignees_url":"https://api.github.com/repos/dkuleshov/che/assignees{/user}","branches_url":"https://api.github.com/repos/dkuleshov/che/branches{/branch}","tags_url":"https://api.github.com/repos/dkuleshov/che/tags","blobs_url":"https://api.github.com/repos/dkuleshov/che/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/dkuleshov/che/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/dkuleshov/che/git/refs{/sha}","trees_url":"https://api.github.com/repos/dkuleshov/che/git/trees{/sha}","statuses_url":"https://api.github.com/repos/dkuleshov/che/statuses/{sha}","languages_url":"https://api.github.com/repos/dkuleshov/che/languages","stargazers_url":"https://api.github.com/repos/dkuleshov/che/stargazers","contributors_url":"https://api.github.com/repos/dkuleshov/che/contributors","subscribers_url":"https://api.github.com/repos/dkuleshov/che/subscribers","subscription_url":"https://api.github.com/repos/dkuleshov/che/subscription","commits_url":"https://api.github.com/repos/dkuleshov/che/commits{/sha}","git_commits_url":"https://api.github.com/repos/dkuleshov/che/git/commits{/sha}","comments_url":"https://api.github.com/repos/dkuleshov/che/comments{/number}","issue_comment_url":"https://api.github.com/repos/dkuleshov/che/issues/comments{/number}","contents_url":"https://api.github.com/repos/dkuleshov/che/contents/{+path}","compare_url":"https://api.github.com/repos/dkuleshov/che/compare/{base}...{head}","merges_url":"https://api.github.com/repos/dkuleshov/che/merges","archive_url":"https://api.github.com/repos/dkuleshov/che/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/dkuleshov/che/downloads","issues_url":"https://api.github.com/repos/dkuleshov/che/issues{/number}","pulls_url":"https://api.github.com/repos/dkuleshov/che/pulls{/number}","milestones_url":"https://api.github.com/repos/dkuleshov/che/milestones{/number}","notifications_url":"https://api.github.com/repos/dkuleshov/che/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/dkuleshov/che/labels{/name}","releases_url":"https://api.github.com/repos/dkuleshov/che/releases{/id}","deployments_url":"https://api.github.com/repos/dkuleshov/che/deployments","created_at":"2016-05-24T10:05:28Z","updated_at":"2016-05-24T10:05:45Z","pushed_at":"2016-07-01T14:44:32Z","git_url":"git://github.com/dkuleshov/che.git","ssh_url":"git@github.com:dkuleshov/che.git","clone_url":"https://github.com/dkuleshov/che.git","svn_url":"https://github.com/dkuleshov/che","homepage":"http://eclipse.org/che","size":27809,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"eclipse:master","ref":"master","sha":"e8759326ae7a4e34bb1c79ff034347b1294e6937","user":{"login":"eclipse","id":56974,"avatar_url":"https://avatars.githubusercontent.com/u/56974?v=3","gravatar_id":"","url":"https://api.github.com/users/eclipse","html_url":"https://github.com/eclipse","followers_url":"https://api.github.com/users/eclipse/followers","following_url":"https://api.github.com/users/eclipse/following{/other_user}","gists_url":"https://api.github.com/users/eclipse/gists{/gist_id}","starred_url":"https://api.github.com/users/eclipse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eclipse/subscriptions","organizations_url":"https://api.github.com/users/eclipse/orgs","repos_url":"https://api.github.com/users/eclipse/repos","events_url":"https://api.github.com/users/eclipse/events{/privacy}","received_events_url":"https://api.github.com/users/eclipse/received_events","type":"Organization","site_admin":false},"repo":{"id":32935745,"name":"che","full_name":"eclipse/che","owner":{"login":"eclipse","id":56974,"avatar_url":"https://avatars.githubusercontent.com/u/56974?v=3","gravatar_id":"","url":"https://api.github.com/users/eclipse","html_url":"https://github.com/eclipse","followers_url":"https://api.github.com/users/eclipse/followers","following_url":"https://api.github.com/users/eclipse/following{/other_user}","gists_url":"https://api.github.com/users/eclipse/gists{/gist_id}","starred_url":"https://api.github.com/users/eclipse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eclipse/subscriptions","organizations_url":"https://api.github.com/users/eclipse/orgs","repos_url":"https://api.github.com/users/eclipse/repos","events_url":"https://api.github.com/users/eclipse/events{/privacy}","received_events_url":"https://api.github.com/users/eclipse/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/eclipse/che","description":"Eclipse Che: Next-generation Eclipse IDE.  Open source workspace server and cloud IDE.","fork":false,"url":"https://api.github.com/repos/eclipse/che","forks_url":"https://api.github.com/repos/eclipse/che/forks","keys_url":"https://api.github.com/repos/eclipse/che/keys{/key_id}","collaborators_url":"https://api.github.com/repos/eclipse/che/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/eclipse/che/teams","hooks_url":"https://api.github.com/repos/eclipse/che/hooks","issue_events_url":"https://api.github.com/repos/eclipse/che/issues/events{/number}","events_url":"https://api.github.com/repos/eclipse/che/events","assignees_url":"https://api.github.com/repos/eclipse/che/assignees{/user}","branches_url":"https://api.github.com/repos/eclipse/che/branches{/branch}","tags_url":"https://api.github.com/repos/eclipse/che/tags","blobs_url":"https://api.github.com/repos/eclipse/che/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/eclipse/che/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/eclipse/che/git/refs{/sha}","trees_url":"https://api.github.com/repos/eclipse/che/git/trees{/sha}","statuses_url":"https://api.github.com/repos/eclipse/che/statuses/{sha}","languages_url":"https://api.github.com/repos/eclipse/che/languages","stargazers_url":"https://api.github.com/repos/eclipse/che/stargazers","contributors_url":"https://api.github.com/repos/eclipse/che/contributors","subscribers_url":"https://api.github.com/repos/eclipse/che/subscribers","subscription_url":"https://api.github.com/repos/eclipse/che/subscription","commits_url":"https://api.github.com/repos/eclipse/che/commits{/sha}","git_commits_url":"https://api.github.com/repos/eclipse/che/git/commits{/sha}","comments_url":"https://api.github.com/repos/eclipse/che/comments{/number}","issue_comment_url":"https://api.github.com/repos/eclipse/che/issues/comments{/number}","contents_url":"https://api.github.com/repos/eclipse/che/contents/{+path}","compare_url":"https://api.github.com/repos/eclipse/che/compare/{base}...{head}","merges_url":"https://api.github.com/repos/eclipse/che/merges","archive_url":"https://api.github.com/repos/eclipse/che/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/eclipse/che/downloads","issues_url":"https://api.github.com/repos/eclipse/che/issues{/number}","pulls_url":"https://api.github.com/repos/eclipse/che/pulls{/number}","milestones_url":"https://api.github.com/repos/eclipse/che/milestones{/number}","notifications_url":"https://api.github.com/repos/eclipse/che/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/eclipse/che/labels{/name}","releases_url":"https://api.github.com/repos/eclipse/che/releases{/id}","deployments_url":"https://api.github.com/repos/eclipse/che/deployments","created_at":"2015-03-26T15:31:01Z","updated_at":"2016-07-02T02:46:52Z","pushed_at":"2016-07-02T15:15:42Z","git_url":"git://github.com/eclipse/che.git","ssh_url":"git@github.com:eclipse/che.git","clone_url":"https://github.com/eclipse/che.git","svn_url":"https://github.com/eclipse/che","homepage":"http://eclipse.org/che","size":40732,"stargazers_count":2953,"watchers_count":2953,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":456,"mirror_url":null,"open_issues_count":148,"forks":456,"open_issues":148,"watchers":2953,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/eclipse/che/pulls/1634"},"html":{"href":"https://github.com/eclipse/che/pull/1634"},"issue":{"href":"https://api.github.com/repos/eclipse/che/issues/1634"},"comments":{"href":"https://api.github.com/repos/eclipse/che/issues/1634/comments"},"review_comments":{"href":"https://api.github.com/repos/eclipse/che/pulls/1634/comments"},"review_comment":{"href":"https://api.github.com/repos/eclipse/che/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/eclipse/che/pulls/1634/commits"},"statuses":{"href":"https://api.github.com/repos/eclipse/che/statuses/e23d5b71ad7e4801f0dbaa930c11409b84df48ea"}}}}
{"actor":{"display_login":"garagatyi"}}