{"action":"created","comment":{"url":"https://api.github.com/repos/citusdata/citus/pulls/comments/190597945","pull_request_review_id":122895395,"id":190597945,"diff_hunk":"@@ -0,0 +1,234 @@\n+SET citus.next_shard_id TO 100300;\n+ALTER SEQUENCE pg_catalog.pg_dist_placement_placementid_seq RESTART 40;\n+SELECT citus.mitmproxy('flow.kill()');\n+ mitmproxy \n+-----------\n+ \n+(1 row)\n+\n+SELECT citus.mitmproxy('recorder.reset()');\n+ mitmproxy \n+-----------\n+ \n+(1 row)\n+\n+-- none of these commands generate network traffic\n+SET citus.shard_replication_factor TO 2;\n+CREATE TABLE append (id int);\n+SELECT create_distributed_table('append', 'id', 'append');\n+ create_distributed_table \n+--------------------------\n+ \n+(1 row)\n+\n+DROP TABLE append;\n+-- prove no network traffic happend\n+SELECT count(1) FROM citus.mitmproxy('recorder.dump()');\n+ count \n+-------","path":"src/test/regress/expected/failure_drop_table.out","position":28,"original_position":28,"commit_id":"05e22e28b9456f0c7629cb9932522f95ca861ade","original_commit_id":"05e22e28b9456f0c7629cb9932522f95ca861ade","user":{"login":"onderkalaci","id":7829470,"avatar_url":"https://avatars2.githubusercontent.com/u/7829470?v=4","gravatar_id":"","url":"https://api.github.com/users/onderkalaci","html_url":"https://github.com/onderkalaci","followers_url":"https://api.github.com/users/onderkalaci/followers","following_url":"https://api.github.com/users/onderkalaci/following{/other_user}","gists_url":"https://api.github.com/users/onderkalaci/gists{/gist_id}","starred_url":"https://api.github.com/users/onderkalaci/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/onderkalaci/subscriptions","organizations_url":"https://api.github.com/users/onderkalaci/orgs","repos_url":"https://api.github.com/users/onderkalaci/repos","events_url":"https://api.github.com/users/onderkalaci/events{/privacy}","received_events_url":"https://api.github.com/users/onderkalaci/received_events","type":"User","site_admin":false},"body":"why do we see `1` if we're to prove no network traffic happened?","created_at":"2018-05-24T14:04:00Z","updated_at":"2018-05-24T14:34:17Z","html_url":"https://github.com/citusdata/citus/pull/2119#discussion_r190597945","pull_request_url":"https://api.github.com/repos/citusdata/citus/pulls/2119","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/citusdata/citus/pulls/comments/190597945"},"html":{"href":"https://github.com/citusdata/citus/pull/2119#discussion_r190597945"},"pull_request":{"href":"https://api.github.com/repos/citusdata/citus/pulls/2119"}}},"pull_request":{"url":"https://api.github.com/repos/citusdata/citus/pulls/2119","id":182913725,"html_url":"https://github.com/citusdata/citus/pull/2119","diff_url":"https://github.com/citusdata/citus/pull/2119.diff","patch_url":"https://github.com/citusdata/citus/pull/2119.patch","issue_url":"https://api.github.com/repos/citusdata/citus/issues/2119","number":2119,"state":"open","locked":false,"title":"[WIP] Mitmproxy-based automated failure testing","user":{"login":"lithp","id":466333,"avatar_url":"https://avatars3.githubusercontent.com/u/466333?v=4","gravatar_id":"","url":"https://api.github.com/users/lithp","html_url":"https://github.com/lithp","followers_url":"https://api.github.com/users/lithp/followers","following_url":"https://api.github.com/users/lithp/following{/other_user}","gists_url":"https://api.github.com/users/lithp/gists{/gist_id}","starred_url":"https://api.github.com/users/lithp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lithp/subscriptions","organizations_url":"https://api.github.com/users/lithp/orgs","repos_url":"https://api.github.com/users/lithp/repos","events_url":"https://api.github.com/users/lithp/events{/privacy}","received_events_url":"https://api.github.com/users/lithp/received_events","type":"User","site_admin":false},"body":"A modernized #2044. This one works at the network level instead of the process level. A drawback is: we don't have a good way of exposing timing problems. An example might be, maybe if a packet comes in before the process is ready for it the process crashes. That class of bugs seems rare to me, so I think this approach is safe to move forward with.\r\n\r\nIt's also much easier! This doesn't require sudo, just an extra daemon which packets flow through. It also has the advantage of not caring at all how the process runs, so the same tests should work modulo any refactoring which we choose to do.\r\n\r\nSome work left to be done:\r\n\r\n- [x] There's a race condition which causes mitmproxy to sometimes return `'NoneType' object has no attribute 'tell'`. What is going wrong?\r\n- [ ] Should we really be using a fifo? Maybe a socket makes more sense? A hostname:port is much more portable (though we would still need some way of blocking to make sure changes have been made, and would have to rely on a tool such as `netcat` in order to keep using `COPY`)\r\n- [ ] Refactor fluent.py into something more maintainable, the double-queue solution probably isn't optimal.\r\n- [ ] The `Handler` and `Mixin` hierarchy is cute but there's definitely a better way. Separating out the Building and the Handling seems like it'd improve things.\r\n- [ ] Convert all of our manual tests into automated tests under this framework!\r\n- [ ] Either `pg_regress_multi.pl` or `fluent.py` should redirect the proxy output to a log file\r\n- [ ] How hard would it be to extend this to cancellation testing?\r\n- [ ] Instead of bumping the Citus version, add `citus.mitmproxy()` to the test functions?\r\n- [ ] Currently it's only possible to fail one of the workers, should we add a second mitmproxy so we can play with that worker as well?\r\n- [ ] `DROP TABLE` crashes if you drop the connection after `DROP TABLE IF EXISTS` is sent. Open a ticket!\r\n- [ ] What do we do about test output which includes PIDs?\r\n- [ ] Profile this and figure out how to improve performance? 6 tests take 10 seconds to run.\r\n- [ ] Some of these tests could be made faster. Instead of recreating tables with different settings of shard_replication_factor, we can start by using the higher replication factor, then manually set some of the shards inactive.\r\n- [ ] Make failure_drop_table consistent, prepared transactions have random ids, introduce something like `enable_unique_job_ids` -> `enable_unique_transaction_ids`\r\n- [ ] Make the `failure_task_tracker_executor` test reproducible (don't hard-code filepaths)","created_at":"2018-04-19T22:18:16Z","updated_at":"2018-05-24T14:34:17Z","closed_at":null,"merged_at":null,"merge_commit_sha":"e649153b41f58ef0ea93f3d929c81b783df0c54f","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":324367882,"url":"https://api.github.com/repos/citusdata/citus/labels/needs%20review","name":"needs review","color":"eeeeee","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/citusdata/citus/pulls/2119/commits","review_comments_url":"https://api.github.com/repos/citusdata/citus/pulls/2119/comments","review_comment_url":"https://api.github.com/repos/citusdata/citus/pulls/comments{/number}","comments_url":"https://api.github.com/repos/citusdata/citus/issues/2119/comments","statuses_url":"https://api.github.com/repos/citusdata/citus/statuses/05e22e28b9456f0c7629cb9932522f95ca861ade","head":{"label":"citusdata:mitmproxy-failure-testing","ref":"mitmproxy-failure-testing","sha":"05e22e28b9456f0c7629cb9932522f95ca861ade","user":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars2.githubusercontent.com/u/2545424?v=4","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"repo":{"id":50874442,"name":"citus","full_name":"citusdata/citus","owner":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars2.githubusercontent.com/u/2545424?v=4","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/citusdata/citus","description":"Scalable PostgreSQL for multi-tenant and real-time workloads","fork":false,"url":"https://api.github.com/repos/citusdata/citus","forks_url":"https://api.github.com/repos/citusdata/citus/forks","keys_url":"https://api.github.com/repos/citusdata/citus/keys{/key_id}","collaborators_url":"https://api.github.com/repos/citusdata/citus/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/citusdata/citus/teams","hooks_url":"https://api.github.com/repos/citusdata/citus/hooks","issue_events_url":"https://api.github.com/repos/citusdata/citus/issues/events{/number}","events_url":"https://api.github.com/repos/citusdata/citus/events","assignees_url":"https://api.github.com/repos/citusdata/citus/assignees{/user}","branches_url":"https://api.github.com/repos/citusdata/citus/branches{/branch}","tags_url":"https://api.github.com/repos/citusdata/citus/tags","blobs_url":"https://api.github.com/repos/citusdata/citus/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/citusdata/citus/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/citusdata/citus/git/refs{/sha}","trees_url":"https://api.github.com/repos/citusdata/citus/git/trees{/sha}","statuses_url":"https://api.github.com/repos/citusdata/citus/statuses/{sha}","languages_url":"https://api.github.com/repos/citusdata/citus/languages","stargazers_url":"https://api.github.com/repos/citusdata/citus/stargazers","contributors_url":"https://api.github.com/repos/citusdata/citus/contributors","subscribers_url":"https://api.github.com/repos/citusdata/citus/subscribers","subscription_url":"https://api.github.com/repos/citusdata/citus/subscription","commits_url":"https://api.github.com/repos/citusdata/citus/commits{/sha}","git_commits_url":"https://api.github.com/repos/citusdata/citus/git/commits{/sha}","comments_url":"https://api.github.com/repos/citusdata/citus/comments{/number}","issue_comment_url":"https://api.github.com/repos/citusdata/citus/issues/comments{/number}","contents_url":"https://api.github.com/repos/citusdata/citus/contents/{+path}","compare_url":"https://api.github.com/repos/citusdata/citus/compare/{base}...{head}","merges_url":"https://api.github.com/repos/citusdata/citus/merges","archive_url":"https://api.github.com/repos/citusdata/citus/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/citusdata/citus/downloads","issues_url":"https://api.github.com/repos/citusdata/citus/issues{/number}","pulls_url":"https://api.github.com/repos/citusdata/citus/pulls{/number}","milestones_url":"https://api.github.com/repos/citusdata/citus/milestones{/number}","notifications_url":"https://api.github.com/repos/citusdata/citus/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/citusdata/citus/labels{/name}","releases_url":"https://api.github.com/repos/citusdata/citus/releases{/id}","deployments_url":"https://api.github.com/repos/citusdata/citus/deployments","created_at":"2016-02-01T21:58:39Z","updated_at":"2018-05-24T10:31:31Z","pushed_at":"2018-05-24T11:04:43Z","git_url":"git://github.com/citusdata/citus.git","ssh_url":"git@github.com:citusdata/citus.git","clone_url":"https://github.com/citusdata/citus.git","svn_url":"https://github.com/citusdata/citus","homepage":"https://www.citusdata.com","size":10458,"stargazers_count":2709,"watchers_count":2709,"language":"C","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":194,"mirror_url":null,"archived":false,"open_issues_count":522,"license":{"key":"agpl-3.0","name":"GNU Affero General Public License v3.0","spdx_id":"AGPL-3.0","url":"https://api.github.com/licenses/agpl-3.0"},"forks":194,"open_issues":522,"watchers":2709,"default_branch":"master"}},"base":{"label":"citusdata:master","ref":"master","sha":"f069e7efb69daa7a96b9d82ac50de885c7623253","user":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars2.githubusercontent.com/u/2545424?v=4","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"repo":{"id":50874442,"name":"citus","full_name":"citusdata/citus","owner":{"login":"citusdata","id":2545424,"avatar_url":"https://avatars2.githubusercontent.com/u/2545424?v=4","gravatar_id":"","url":"https://api.github.com/users/citusdata","html_url":"https://github.com/citusdata","followers_url":"https://api.github.com/users/citusdata/followers","following_url":"https://api.github.com/users/citusdata/following{/other_user}","gists_url":"https://api.github.com/users/citusdata/gists{/gist_id}","starred_url":"https://api.github.com/users/citusdata/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/citusdata/subscriptions","organizations_url":"https://api.github.com/users/citusdata/orgs","repos_url":"https://api.github.com/users/citusdata/repos","events_url":"https://api.github.com/users/citusdata/events{/privacy}","received_events_url":"https://api.github.com/users/citusdata/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/citusdata/citus","description":"Scalable PostgreSQL for multi-tenant and real-time workloads","fork":false,"url":"https://api.github.com/repos/citusdata/citus","forks_url":"https://api.github.com/repos/citusdata/citus/forks","keys_url":"https://api.github.com/repos/citusdata/citus/keys{/key_id}","collaborators_url":"https://api.github.com/repos/citusdata/citus/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/citusdata/citus/teams","hooks_url":"https://api.github.com/repos/citusdata/citus/hooks","issue_events_url":"https://api.github.com/repos/citusdata/citus/issues/events{/number}","events_url":"https://api.github.com/repos/citusdata/citus/events","assignees_url":"https://api.github.com/repos/citusdata/citus/assignees{/user}","branches_url":"https://api.github.com/repos/citusdata/citus/branches{/branch}","tags_url":"https://api.github.com/repos/citusdata/citus/tags","blobs_url":"https://api.github.com/repos/citusdata/citus/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/citusdata/citus/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/citusdata/citus/git/refs{/sha}","trees_url":"https://api.github.com/repos/citusdata/citus/git/trees{/sha}","statuses_url":"https://api.github.com/repos/citusdata/citus/statuses/{sha}","languages_url":"https://api.github.com/repos/citusdata/citus/languages","stargazers_url":"https://api.github.com/repos/citusdata/citus/stargazers","contributors_url":"https://api.github.com/repos/citusdata/citus/contributors","subscribers_url":"https://api.github.com/repos/citusdata/citus/subscribers","subscription_url":"https://api.github.com/repos/citusdata/citus/subscription","commits_url":"https://api.github.com/repos/citusdata/citus/commits{/sha}","git_commits_url":"https://api.github.com/repos/citusdata/citus/git/commits{/sha}","comments_url":"https://api.github.com/repos/citusdata/citus/comments{/number}","issue_comment_url":"https://api.github.com/repos/citusdata/citus/issues/comments{/number}","contents_url":"https://api.github.com/repos/citusdata/citus/contents/{+path}","compare_url":"https://api.github.com/repos/citusdata/citus/compare/{base}...{head}","merges_url":"https://api.github.com/repos/citusdata/citus/merges","archive_url":"https://api.github.com/repos/citusdata/citus/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/citusdata/citus/downloads","issues_url":"https://api.github.com/repos/citusdata/citus/issues{/number}","pulls_url":"https://api.github.com/repos/citusdata/citus/pulls{/number}","milestones_url":"https://api.github.com/repos/citusdata/citus/milestones{/number}","notifications_url":"https://api.github.com/repos/citusdata/citus/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/citusdata/citus/labels{/name}","releases_url":"https://api.github.com/repos/citusdata/citus/releases{/id}","deployments_url":"https://api.github.com/repos/citusdata/citus/deployments","created_at":"2016-02-01T21:58:39Z","updated_at":"2018-05-24T10:31:31Z","pushed_at":"2018-05-24T11:04:43Z","git_url":"git://github.com/citusdata/citus.git","ssh_url":"git@github.com:citusdata/citus.git","clone_url":"https://github.com/citusdata/citus.git","svn_url":"https://github.com/citusdata/citus","homepage":"https://www.citusdata.com","size":10458,"stargazers_count":2709,"watchers_count":2709,"language":"C","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":194,"mirror_url":null,"archived":false,"open_issues_count":522,"license":{"key":"agpl-3.0","name":"GNU Affero General Public License v3.0","spdx_id":"AGPL-3.0","url":"https://api.github.com/licenses/agpl-3.0"},"forks":194,"open_issues":522,"watchers":2709,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/citusdata/citus/pulls/2119"},"html":{"href":"https://github.com/citusdata/citus/pull/2119"},"issue":{"href":"https://api.github.com/repos/citusdata/citus/issues/2119"},"comments":{"href":"https://api.github.com/repos/citusdata/citus/issues/2119/comments"},"review_comments":{"href":"https://api.github.com/repos/citusdata/citus/pulls/2119/comments"},"review_comment":{"href":"https://api.github.com/repos/citusdata/citus/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/citusdata/citus/pulls/2119/commits"},"statuses":{"href":"https://api.github.com/repos/citusdata/citus/statuses/05e22e28b9456f0c7629cb9932522f95ca861ade"}},"author_association":"MEMBER"}}
{"actor":{"display_login":"onderkalaci"}}