{"action":"created","comment":{"url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/comments/73912988","id":73912988,"diff_hunk":"@@ -396,6 +396,53 @@\n             #\"shutdown-on-error triggered because of exception\"\n             :error))))))\n \n+(deftest ^:integration debouncing-test\n+  (testing \"Default debouncing behavior\"\n+    (let [root-dir (fs/temp-dir \"debounce\")\n+          actual-events (atom [])\n+          actual-calls (atom 0)\n+          callback (fn [e]\n+                     (swap! actual-events concat e)\n+                     (swap! actual-calls inc))\n+          app (tk/boot-services-with-config watch-service-and-deps {})\n+          svc (tk-app/get-service app :FilesystemWatchService)\n+          event-cadence (quot watch-core/window-min 2)]\n+      ;; We trigger half the number of events that could independently occur\n+      ;; without exceeding the window-max, we trigger them at half the wait\n+      ;; of window-min to ensure they will all be caught even if the host\n+      ;; running the tests is slow. This requires that window-max be at least\n+      ;; twice the duration as window-min.\n+      (testing \"multiple events trigger one callback when within `window-min` and `window-max`\"\n+        (let [test-dir (fs/file root-dir \"window-min\")\n+              event-nums (quot (quot watch-core/window-max watch-core/window-min) 2)\n+              files (map #(fs/file test-dir (str % \".txt\")) (range 0 event-nums))\n+              expected-events (set (map (fn [f] {:path f :type :create}) files))]\n+          (is (> event-nums 1))\n+          (is (fs/mkdirs test-dir))\n+          (watch! svc test-dir callback)\n+          (doseq [f files] (fs/touch f) (Thread/sleep event-cadence))\n+          (is (= expected-events (wait-for-events actual-events expected-events)))\n+          (is (= @actual-calls 1))))\n+      ;; We trigger events at the same cadence as above to ensure that we will\n+      ;; be causing events within window-min and continuing the polling loop.\n+      ;; We trigger them for what should be twice the duration of window-max\n+      ;; however to see the loop properly short circuited. Depending on the\n+      ;; speed of the test runner host, we could see between 2 and 4 callbacks\n+      ;; without seeing failures elsewhere in the suite.\n+      (testing \"multiple callbacks are triggered if polling exceeds `window-max`\"\n+        (let [test-dir (fs/file root-dir \"window-max\")\n+              event-nums (* (quot watch-core/window-max watch-core/window-min) 4)\n+              files (map #(fs/file test-dir (str % \".txt\")) (range 0 event-nums))\n+              expected-events (set (map (fn [f] {:path f :type :create}) files))]\n+          (is (> event-nums 2))","path":"test/puppetlabs/trapperkeeper/services/watcher/filesystem_watch_service_test.clj","position":42,"original_position":42,"commit_id":"1b897dc86276e42c48a12a3850fc18221d7b85e9","original_commit_id":"1b897dc86276e42c48a12a3850fc18221d7b85e9","user":{"login":"justinstoller","id":241423,"avatar_url":"https://avatars.githubusercontent.com/u/241423?v=3","gravatar_id":"","url":"https://api.github.com/users/justinstoller","html_url":"https://github.com/justinstoller","followers_url":"https://api.github.com/users/justinstoller/followers","following_url":"https://api.github.com/users/justinstoller/following{/other_user}","gists_url":"https://api.github.com/users/justinstoller/gists{/gist_id}","starred_url":"https://api.github.com/users/justinstoller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinstoller/subscriptions","organizations_url":"https://api.github.com/users/justinstoller/orgs","repos_url":"https://api.github.com/users/justinstoller/repos","events_url":"https://api.github.com/users/justinstoller/events{/privacy}","received_events_url":"https://api.github.com/users/justinstoller/received_events","type":"User","site_admin":false},"body":"It's not true of the code itself, but in order to write tests around it I had to make some assumptions. This could be put a different way: testing that we are debouncing events, but are allowing multiple callbacks when those events come in outside of our cutoff, requires at least more than two events to test.","created_at":"2016-08-08T17:04:05Z","updated_at":"2016-08-08T17:04:05Z","html_url":"https://github.com/puppetlabs/trapperkeeper-filesystem-watcher/pull/8#discussion_r73912988","pull_request_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/8","_links":{"self":{"href":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/comments/73912988"},"html":{"href":"https://github.com/puppetlabs/trapperkeeper-filesystem-watcher/pull/8#discussion_r73912988"},"pull_request":{"href":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/8"}}},"pull_request":{"url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/8","id":80190628,"html_url":"https://github.com/puppetlabs/trapperkeeper-filesystem-watcher/pull/8","diff_url":"https://github.com/puppetlabs/trapperkeeper-filesystem-watcher/pull/8.diff","patch_url":"https://github.com/puppetlabs/trapperkeeper-filesystem-watcher/pull/8.patch","issue_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/issues/8","number":8,"state":"open","locked":false,"title":"(TK-389) Debounce Events","user":{"login":"justinstoller","id":241423,"avatar_url":"https://avatars.githubusercontent.com/u/241423?v=3","gravatar_id":"","url":"https://api.github.com/users/justinstoller","html_url":"https://github.com/justinstoller","followers_url":"https://api.github.com/users/justinstoller/followers","following_url":"https://api.github.com/users/justinstoller/following{/other_user}","gists_url":"https://api.github.com/users/justinstoller/gists{/gist_id}","starred_url":"https://api.github.com/users/justinstoller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinstoller/subscriptions","organizations_url":"https://api.github.com/users/justinstoller/orgs","repos_url":"https://api.github.com/users/justinstoller/repos","events_url":"https://api.github.com/users/justinstoller/events{/privacy}","received_events_url":"https://api.github.com/users/justinstoller/received_events","type":"User","site_admin":false},"body":"This patch set follows `Implementation Option #2` in the corresponding ticket.\r\n\r\nAfter an event occurs it begins a polling loop to see if there are queued events that occurred concurrently to the one being processed. Once the queue has been drained it will wait until `window-min` (currently set to 100 milliseconds) to see if any additional events occur. It will break and report the events that it has accumulated thus far if `window-max` (currently set to 2 seconds) is exceeded during its polling loop.\r\n\r\nThis PR contains two additional refactors. It consolidates our internal representation of Events so that we have one collection to represent them that all internal functions can use (and has appropriate helpers to translate them so that consumers do not need to change). It also moves the TK shutdown service function to wrap the entire event processing rather than just the consumer specified callback. This allows any unforeseen issues (eg schema errors) to be logged and raised when they would have normally failed silently in the processing future.","created_at":"2016-08-05T05:23:57Z","updated_at":"2016-08-08T17:04:05Z","closed_at":null,"merged_at":null,"merge_commit_sha":"306dea54369ebf29d4cbbc143595b09792a027f2","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/8/commits","review_comments_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/8/comments","review_comment_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/comments{/number}","comments_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/issues/8/comments","statuses_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/statuses/1b897dc86276e42c48a12a3850fc18221d7b85e9","head":{"label":"justinstoller:tk389-debounce+","ref":"tk389-debounce+","sha":"1b897dc86276e42c48a12a3850fc18221d7b85e9","user":{"login":"justinstoller","id":241423,"avatar_url":"https://avatars.githubusercontent.com/u/241423?v=3","gravatar_id":"","url":"https://api.github.com/users/justinstoller","html_url":"https://github.com/justinstoller","followers_url":"https://api.github.com/users/justinstoller/followers","following_url":"https://api.github.com/users/justinstoller/following{/other_user}","gists_url":"https://api.github.com/users/justinstoller/gists{/gist_id}","starred_url":"https://api.github.com/users/justinstoller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinstoller/subscriptions","organizations_url":"https://api.github.com/users/justinstoller/orgs","repos_url":"https://api.github.com/users/justinstoller/repos","events_url":"https://api.github.com/users/justinstoller/events{/privacy}","received_events_url":"https://api.github.com/users/justinstoller/received_events","type":"User","site_admin":false},"repo":{"id":63891650,"name":"trapperkeeper-filesystem-watcher","full_name":"justinstoller/trapperkeeper-filesystem-watcher","owner":{"login":"justinstoller","id":241423,"avatar_url":"https://avatars.githubusercontent.com/u/241423?v=3","gravatar_id":"","url":"https://api.github.com/users/justinstoller","html_url":"https://github.com/justinstoller","followers_url":"https://api.github.com/users/justinstoller/followers","following_url":"https://api.github.com/users/justinstoller/following{/other_user}","gists_url":"https://api.github.com/users/justinstoller/gists{/gist_id}","starred_url":"https://api.github.com/users/justinstoller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinstoller/subscriptions","organizations_url":"https://api.github.com/users/justinstoller/orgs","repos_url":"https://api.github.com/users/justinstoller/repos","events_url":"https://api.github.com/users/justinstoller/events{/privacy}","received_events_url":"https://api.github.com/users/justinstoller/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/justinstoller/trapperkeeper-filesystem-watcher","description":"Trapperkeeper filesystem watcher service. https://tickets.puppetlabs.com/browse/TK","fork":true,"url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher","forks_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/forks","keys_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/keys{/key_id}","collaborators_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/teams","hooks_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/hooks","issue_events_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/issues/events{/number}","events_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/events","assignees_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/assignees{/user}","branches_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/branches{/branch}","tags_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/tags","blobs_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/git/refs{/sha}","trees_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/git/trees{/sha}","statuses_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/statuses/{sha}","languages_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/languages","stargazers_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/stargazers","contributors_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/contributors","subscribers_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/subscribers","subscription_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/subscription","commits_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/commits{/sha}","git_commits_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/git/commits{/sha}","comments_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/comments{/number}","issue_comment_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/issues/comments{/number}","contents_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/contents/{+path}","compare_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/compare/{base}...{head}","merges_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/merges","archive_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/downloads","issues_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/issues{/number}","pulls_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/pulls{/number}","milestones_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/milestones{/number}","notifications_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/labels{/name}","releases_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/releases{/id}","deployments_url":"https://api.github.com/repos/justinstoller/trapperkeeper-filesystem-watcher/deployments","created_at":"2016-07-21T18:06:31Z","updated_at":"2016-07-21T18:06:32Z","pushed_at":"2016-08-06T16:57:57Z","git_url":"git://github.com/justinstoller/trapperkeeper-filesystem-watcher.git","ssh_url":"git@github.com:justinstoller/trapperkeeper-filesystem-watcher.git","clone_url":"https://github.com/justinstoller/trapperkeeper-filesystem-watcher.git","svn_url":"https://github.com/justinstoller/trapperkeeper-filesystem-watcher","homepage":null,"size":81,"stargazers_count":0,"watchers_count":0,"language":"Clojure","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"puppetlabs:master","ref":"master","sha":"2a7e18f2af557c90b0f35ef712f5935f724712d7","user":{"login":"puppetlabs","id":234268,"avatar_url":"https://avatars.githubusercontent.com/u/234268?v=3","gravatar_id":"","url":"https://api.github.com/users/puppetlabs","html_url":"https://github.com/puppetlabs","followers_url":"https://api.github.com/users/puppetlabs/followers","following_url":"https://api.github.com/users/puppetlabs/following{/other_user}","gists_url":"https://api.github.com/users/puppetlabs/gists{/gist_id}","starred_url":"https://api.github.com/users/puppetlabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/puppetlabs/subscriptions","organizations_url":"https://api.github.com/users/puppetlabs/orgs","repos_url":"https://api.github.com/users/puppetlabs/repos","events_url":"https://api.github.com/users/puppetlabs/events{/privacy}","received_events_url":"https://api.github.com/users/puppetlabs/received_events","type":"Organization","site_admin":false},"repo":{"id":63367859,"name":"trapperkeeper-filesystem-watcher","full_name":"puppetlabs/trapperkeeper-filesystem-watcher","owner":{"login":"puppetlabs","id":234268,"avatar_url":"https://avatars.githubusercontent.com/u/234268?v=3","gravatar_id":"","url":"https://api.github.com/users/puppetlabs","html_url":"https://github.com/puppetlabs","followers_url":"https://api.github.com/users/puppetlabs/followers","following_url":"https://api.github.com/users/puppetlabs/following{/other_user}","gists_url":"https://api.github.com/users/puppetlabs/gists{/gist_id}","starred_url":"https://api.github.com/users/puppetlabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/puppetlabs/subscriptions","organizations_url":"https://api.github.com/users/puppetlabs/orgs","repos_url":"https://api.github.com/users/puppetlabs/repos","events_url":"https://api.github.com/users/puppetlabs/events{/privacy}","received_events_url":"https://api.github.com/users/puppetlabs/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/puppetlabs/trapperkeeper-filesystem-watcher","description":"Trapperkeeper filesystem watcher service. https://tickets.puppetlabs.com/browse/TK","fork":false,"url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher","forks_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/forks","keys_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/keys{/key_id}","collaborators_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/teams","hooks_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/hooks","issue_events_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/issues/events{/number}","events_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/events","assignees_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/assignees{/user}","branches_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/branches{/branch}","tags_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/tags","blobs_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/git/refs{/sha}","trees_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/git/trees{/sha}","statuses_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/statuses/{sha}","languages_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/languages","stargazers_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/stargazers","contributors_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/contributors","subscribers_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/subscribers","subscription_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/subscription","commits_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/commits{/sha}","git_commits_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/git/commits{/sha}","comments_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/comments{/number}","issue_comment_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/issues/comments{/number}","contents_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/contents/{+path}","compare_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/compare/{base}...{head}","merges_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/merges","archive_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/downloads","issues_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/issues{/number}","pulls_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls{/number}","milestones_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/milestones{/number}","notifications_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/labels{/name}","releases_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/releases{/id}","deployments_url":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/deployments","created_at":"2016-07-14T20:34:27Z","updated_at":"2016-07-15T10:36:28Z","pushed_at":"2016-08-06T16:57:57Z","git_url":"git://github.com/puppetlabs/trapperkeeper-filesystem-watcher.git","ssh_url":"git@github.com:puppetlabs/trapperkeeper-filesystem-watcher.git","clone_url":"https://github.com/puppetlabs/trapperkeeper-filesystem-watcher.git","svn_url":"https://github.com/puppetlabs/trapperkeeper-filesystem-watcher","homepage":null,"size":65,"stargazers_count":0,"watchers_count":0,"language":"Clojure","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":3,"mirror_url":null,"open_issues_count":1,"forks":3,"open_issues":1,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/8"},"html":{"href":"https://github.com/puppetlabs/trapperkeeper-filesystem-watcher/pull/8"},"issue":{"href":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/issues/8"},"comments":{"href":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/issues/8/comments"},"review_comments":{"href":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/8/comments"},"review_comment":{"href":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/pulls/8/commits"},"statuses":{"href":"https://api.github.com/repos/puppetlabs/trapperkeeper-filesystem-watcher/statuses/1b897dc86276e42c48a12a3850fc18221d7b85e9"}}}}
{"actor":{"display_login":"justinstoller"}}