{"action":"created","comment":{"url":"https://api.github.com/repos/ansible/ansible/pulls/comments/228733400","pull_request_review_id":169077613,"id":228733400,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIyODczMzQwMA==","diff_hunk":"@@ -380,9 +380,6 @@ def receive(self, command=None, prompts=None, answer=None, newline=True, prompt_\n         command_prompt_matched = False\n         matched_prompt_window = window_count = 0\n \n-        command_timeout = self.get_option('persistent_command_timeout')","path":"lib/ansible/plugins/connection/network_cli.py","position":4,"original_position":4,"commit_id":"79989cbda52e097020205eddb588dc2f15acd012","original_commit_id":"79989cbda52e097020205eddb588dc2f15acd012","user":{"login":"ganeshrn","id":5292461,"node_id":"MDQ6VXNlcjUyOTI0NjE=","avatar_url":"https://avatars0.githubusercontent.com/u/5292461?v=4","gravatar_id":"","url":"https://api.github.com/users/ganeshrn","html_url":"https://github.com/ganeshrn","followers_url":"https://api.github.com/users/ganeshrn/followers","following_url":"https://api.github.com/users/ganeshrn/following{/other_user}","gists_url":"https://api.github.com/users/ganeshrn/gists{/gist_id}","starred_url":"https://api.github.com/users/ganeshrn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ganeshrn/subscriptions","organizations_url":"https://api.github.com/users/ganeshrn/orgs","repos_url":"https://api.github.com/users/ganeshrn/repos","events_url":"https://api.github.com/users/ganeshrn/events{/privacy}","received_events_url":"https://api.github.com/users/ganeshrn/received_events","type":"User","site_admin":false},"body":"> it does not happen under normal conditions or during regular network activity and it has a very long delay\r\n`persistent_command_timeout` default timeout value triggered from `ansible-connection` is 10 secs, not sure how you concluded it is a 'very long delay'.","created_at":"2018-10-28T03:53:34Z","updated_at":"2018-10-28T03:53:34Z","html_url":"https://github.com/ansible/ansible/pull/47702#discussion_r228733400","pull_request_url":"https://api.github.com/repos/ansible/ansible/pulls/47702","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/ansible/ansible/pulls/comments/228733400"},"html":{"href":"https://github.com/ansible/ansible/pull/47702#discussion_r228733400"},"pull_request":{"href":"https://api.github.com/repos/ansible/ansible/pulls/47702"}},"in_reply_to_id":228720534},"pull_request":{"url":"https://api.github.com/repos/ansible/ansible/pulls/47702","id":226279356,"node_id":"MDExOlB1bGxSZXF1ZXN0MjI2Mjc5MzU2","html_url":"https://github.com/ansible/ansible/pull/47702","diff_url":"https://github.com/ansible/ansible/pull/47702.diff","patch_url":"https://github.com/ansible/ansible/pull/47702.patch","issue_url":"https://api.github.com/repos/ansible/ansible/issues/47702","number":47702,"state":"open","locked":false,"title":"fix network_cli timeout","user":{"login":"f-bor","id":42858062,"node_id":"MDQ6VXNlcjQyODU4MDYy","avatar_url":"https://avatars1.githubusercontent.com/u/42858062?v=4","gravatar_id":"","url":"https://api.github.com/users/f-bor","html_url":"https://github.com/f-bor","followers_url":"https://api.github.com/users/f-bor/followers","following_url":"https://api.github.com/users/f-bor/following{/other_user}","gists_url":"https://api.github.com/users/f-bor/gists{/gist_id}","starred_url":"https://api.github.com/users/f-bor/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/f-bor/subscriptions","organizations_url":"https://api.github.com/users/f-bor/orgs","repos_url":"https://api.github.com/users/f-bor/repos","events_url":"https://api.github.com/users/f-bor/events{/privacy}","received_events_url":"https://api.github.com/users/f-bor/received_events","type":"User","site_admin":false},"body":"##### SUMMARY\r\nPR #47345 and #47004 are causing a random timeout to occur. By removing either, the timeout disappears. I suppose this is a problem between signals and paramiko's critical sections/conditions used in recv_ready, or maybe time.sleep. However, there is no need anymore to use signals to manage the buffer_read timeout since #47345 introduce such management.\r\n\r\n##### ISSUE TYPE\r\n- Bugfix Pull Request\r\n\r\n##### COMPONENT NAME\r\nnetwork_cli\r\n\r\n##### ANSIBLE VERSION\r\n<!--- Paste verbatim output from \"ansible --version\" between quotes -->\r\n```paste below\r\nansible 2.8.0.dev0 (timeout_fix 79989cbda5) last updated 2018/10/27 09:42:05 (GMT +200)\r\n  config file = /etc/ansible/ansible.cfg\r\n  configured module search path = [u'/home/fbor/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']\r\n  ansible python module location = /home/fbor/ansible/lib/ansible\r\n  executable location = /home/fbor/ansible/bin/ansible\r\n  python version = 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516]\r\n```\r\n\r\n##### ADDITIONAL INFORMATION\r\nTo reproduce the issue, just setup an integration environnement for vyos and run:\r\n```\r\nansible-test network-integration -vvvv --inventory inventory.networking vyos_.*\r\n```\r\n\r\nEDIT: After digging a little bit more, the timeout is caused by a deadlock. It seems that when signaled, python's execution is interrupted and does not resume in certain cases. Then, Paramiko's lock is staying in an acquired state and never get released. Here is a trivial way to reproduce the receive loop and the deadlock:\r\n\r\n```\r\nimport signal\r\nimport time\r\nimport threading\r\nimport math\r\n\r\nclass DeadLock:\r\n\r\n    def __init__(self):\r\n        self.lock = threading.Lock()\r\n\r\n    def sighandler(self, signum, frame):\r\n        print('signaled')\r\n        raise Exception\r\n\r\n    def compute(self):\r\n        print('locking')\r\n        self.lock.acquire()\r\n        print('locked')\r\n        try:\r\n            return math.acos(1)\r\n        finally:\r\n            print('unlocking')\r\n            self.lock.release()\r\n            print('unlocked')\r\n            return -1\r\n\r\n\r\n    def run(self):\r\n        while True:\r\n            signal.signal(signal.SIGALRM, self.sighandler)\r\n            signal.setitimer(signal.ITIMER_REAL, 0.1)\r\n            try:\r\n                while True:\r\n                    self.compute()\r\n                    sleep(0.001)\r\n            except:\r\n                pass\r\n\r\ndl = DeadLock()\r\ndl.run()\r\n```\r\n\r\nIt always gives this after a while:\r\n> locking\r\n> locked\r\n> unlocking\r\n> signaled\r\n> locking\r\n","created_at":"2018-10-27T08:34:27Z","updated_at":"2018-10-28T03:53:34Z","closed_at":null,"merged_at":null,"merge_commit_sha":"9f49e4690f9d6adce868240e1a3395f2d4151e2c","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":1043397454,"node_id":"MDU6TGFiZWwxMDQzMzk3NDU0","url":"https://api.github.com/repos/ansible/ansible/labels/affects_2.8","name":"affects_2.8","color":"ededed","default":false},{"id":785271699,"node_id":"MDU6TGFiZWw3ODUyNzE2OTk=","url":"https://api.github.com/repos/ansible/ansible/labels/bug","name":"bug","color":"fbca04","default":true},{"id":513652538,"node_id":"MDU6TGFiZWw1MTM2NTI1Mzg=","url":"https://api.github.com/repos/ansible/ansible/labels/core_review","name":"core_review","color":"ededed","default":false},{"id":339653091,"node_id":"MDU6TGFiZWwzMzk2NTMwOTE=","url":"https://api.github.com/repos/ansible/ansible/labels/networking","name":"networking","color":"c7def8","default":false},{"id":636352796,"node_id":"MDU6TGFiZWw2MzYzNTI3OTY=","url":"https://api.github.com/repos/ansible/ansible/labels/support:core","name":"support:core","color":"ededed","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/ansible/ansible/pulls/47702/commits","review_comments_url":"https://api.github.com/repos/ansible/ansible/pulls/47702/comments","review_comment_url":"https://api.github.com/repos/ansible/ansible/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ansible/ansible/issues/47702/comments","statuses_url":"https://api.github.com/repos/ansible/ansible/statuses/79989cbda52e097020205eddb588dc2f15acd012","head":{"label":"f-bor:timeout_fix","ref":"timeout_fix","sha":"79989cbda52e097020205eddb588dc2f15acd012","user":{"login":"f-bor","id":42858062,"node_id":"MDQ6VXNlcjQyODU4MDYy","avatar_url":"https://avatars1.githubusercontent.com/u/42858062?v=4","gravatar_id":"","url":"https://api.github.com/users/f-bor","html_url":"https://github.com/f-bor","followers_url":"https://api.github.com/users/f-bor/followers","following_url":"https://api.github.com/users/f-bor/following{/other_user}","gists_url":"https://api.github.com/users/f-bor/gists{/gist_id}","starred_url":"https://api.github.com/users/f-bor/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/f-bor/subscriptions","organizations_url":"https://api.github.com/users/f-bor/orgs","repos_url":"https://api.github.com/users/f-bor/repos","events_url":"https://api.github.com/users/f-bor/events{/privacy}","received_events_url":"https://api.github.com/users/f-bor/received_events","type":"User","site_admin":false},"repo":{"id":153735104,"node_id":"MDEwOlJlcG9zaXRvcnkxNTM3MzUxMDQ=","name":"ansible","full_name":"f-bor/ansible","private":false,"owner":{"login":"f-bor","id":42858062,"node_id":"MDQ6VXNlcjQyODU4MDYy","avatar_url":"https://avatars1.githubusercontent.com/u/42858062?v=4","gravatar_id":"","url":"https://api.github.com/users/f-bor","html_url":"https://github.com/f-bor","followers_url":"https://api.github.com/users/f-bor/followers","following_url":"https://api.github.com/users/f-bor/following{/other_user}","gists_url":"https://api.github.com/users/f-bor/gists{/gist_id}","starred_url":"https://api.github.com/users/f-bor/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/f-bor/subscriptions","organizations_url":"https://api.github.com/users/f-bor/orgs","repos_url":"https://api.github.com/users/f-bor/repos","events_url":"https://api.github.com/users/f-bor/events{/privacy}","received_events_url":"https://api.github.com/users/f-bor/received_events","type":"User","site_admin":false},"html_url":"https://github.com/f-bor/ansible","description":"Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications — automate in a language that approaches plain English, using SSH, with no agents to install on remote systems. https://docs.ansible.com/ansible/","fork":true,"url":"https://api.github.com/repos/f-bor/ansible","forks_url":"https://api.github.com/repos/f-bor/ansible/forks","keys_url":"https://api.github.com/repos/f-bor/ansible/keys{/key_id}","collaborators_url":"https://api.github.com/repos/f-bor/ansible/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/f-bor/ansible/teams","hooks_url":"https://api.github.com/repos/f-bor/ansible/hooks","issue_events_url":"https://api.github.com/repos/f-bor/ansible/issues/events{/number}","events_url":"https://api.github.com/repos/f-bor/ansible/events","assignees_url":"https://api.github.com/repos/f-bor/ansible/assignees{/user}","branches_url":"https://api.github.com/repos/f-bor/ansible/branches{/branch}","tags_url":"https://api.github.com/repos/f-bor/ansible/tags","blobs_url":"https://api.github.com/repos/f-bor/ansible/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/f-bor/ansible/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/f-bor/ansible/git/refs{/sha}","trees_url":"https://api.github.com/repos/f-bor/ansible/git/trees{/sha}","statuses_url":"https://api.github.com/repos/f-bor/ansible/statuses/{sha}","languages_url":"https://api.github.com/repos/f-bor/ansible/languages","stargazers_url":"https://api.github.com/repos/f-bor/ansible/stargazers","contributors_url":"https://api.github.com/repos/f-bor/ansible/contributors","subscribers_url":"https://api.github.com/repos/f-bor/ansible/subscribers","subscription_url":"https://api.github.com/repos/f-bor/ansible/subscription","commits_url":"https://api.github.com/repos/f-bor/ansible/commits{/sha}","git_commits_url":"https://api.github.com/repos/f-bor/ansible/git/commits{/sha}","comments_url":"https://api.github.com/repos/f-bor/ansible/comments{/number}","issue_comment_url":"https://api.github.com/repos/f-bor/ansible/issues/comments{/number}","contents_url":"https://api.github.com/repos/f-bor/ansible/contents/{+path}","compare_url":"https://api.github.com/repos/f-bor/ansible/compare/{base}...{head}","merges_url":"https://api.github.com/repos/f-bor/ansible/merges","archive_url":"https://api.github.com/repos/f-bor/ansible/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/f-bor/ansible/downloads","issues_url":"https://api.github.com/repos/f-bor/ansible/issues{/number}","pulls_url":"https://api.github.com/repos/f-bor/ansible/pulls{/number}","milestones_url":"https://api.github.com/repos/f-bor/ansible/milestones{/number}","notifications_url":"https://api.github.com/repos/f-bor/ansible/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/f-bor/ansible/labels{/name}","releases_url":"https://api.github.com/repos/f-bor/ansible/releases{/id}","deployments_url":"https://api.github.com/repos/f-bor/ansible/deployments","created_at":"2018-10-19T06:16:00Z","updated_at":"2018-10-25T13:23:36Z","pushed_at":"2018-10-27T09:24:18Z","git_url":"git://github.com/f-bor/ansible.git","ssh_url":"git@github.com:f-bor/ansible.git","clone_url":"https://github.com/f-bor/ansible.git","svn_url":"https://github.com/f-bor/ansible","homepage":"https://www.ansible.com/","size":136339,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"gpl-3.0","name":"GNU General Public License v3.0","spdx_id":"GPL-3.0","url":"https://api.github.com/licenses/gpl-3.0","node_id":"MDc6TGljZW5zZTk="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"devel"}},"base":{"label":"ansible:devel","ref":"devel","sha":"64171f5524372c316564b7625d528ae42906ec78","user":{"login":"ansible","id":1507452,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE1MDc0NTI=","avatar_url":"https://avatars2.githubusercontent.com/u/1507452?v=4","gravatar_id":"","url":"https://api.github.com/users/ansible","html_url":"https://github.com/ansible","followers_url":"https://api.github.com/users/ansible/followers","following_url":"https://api.github.com/users/ansible/following{/other_user}","gists_url":"https://api.github.com/users/ansible/gists{/gist_id}","starred_url":"https://api.github.com/users/ansible/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ansible/subscriptions","organizations_url":"https://api.github.com/users/ansible/orgs","repos_url":"https://api.github.com/users/ansible/repos","events_url":"https://api.github.com/users/ansible/events{/privacy}","received_events_url":"https://api.github.com/users/ansible/received_events","type":"Organization","site_admin":false},"repo":{"id":3638964,"node_id":"MDEwOlJlcG9zaXRvcnkzNjM4OTY0","name":"ansible","full_name":"ansible/ansible","private":false,"owner":{"login":"ansible","id":1507452,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE1MDc0NTI=","avatar_url":"https://avatars2.githubusercontent.com/u/1507452?v=4","gravatar_id":"","url":"https://api.github.com/users/ansible","html_url":"https://github.com/ansible","followers_url":"https://api.github.com/users/ansible/followers","following_url":"https://api.github.com/users/ansible/following{/other_user}","gists_url":"https://api.github.com/users/ansible/gists{/gist_id}","starred_url":"https://api.github.com/users/ansible/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ansible/subscriptions","organizations_url":"https://api.github.com/users/ansible/orgs","repos_url":"https://api.github.com/users/ansible/repos","events_url":"https://api.github.com/users/ansible/events{/privacy}","received_events_url":"https://api.github.com/users/ansible/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/ansible/ansible","description":"Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications — automate in a language that approaches plain English, using SSH, with no agents to install on remote systems. https://docs.ansible.com/ansible/","fork":false,"url":"https://api.github.com/repos/ansible/ansible","forks_url":"https://api.github.com/repos/ansible/ansible/forks","keys_url":"https://api.github.com/repos/ansible/ansible/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ansible/ansible/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ansible/ansible/teams","hooks_url":"https://api.github.com/repos/ansible/ansible/hooks","issue_events_url":"https://api.github.com/repos/ansible/ansible/issues/events{/number}","events_url":"https://api.github.com/repos/ansible/ansible/events","assignees_url":"https://api.github.com/repos/ansible/ansible/assignees{/user}","branches_url":"https://api.github.com/repos/ansible/ansible/branches{/branch}","tags_url":"https://api.github.com/repos/ansible/ansible/tags","blobs_url":"https://api.github.com/repos/ansible/ansible/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ansible/ansible/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ansible/ansible/git/refs{/sha}","trees_url":"https://api.github.com/repos/ansible/ansible/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ansible/ansible/statuses/{sha}","languages_url":"https://api.github.com/repos/ansible/ansible/languages","stargazers_url":"https://api.github.com/repos/ansible/ansible/stargazers","contributors_url":"https://api.github.com/repos/ansible/ansible/contributors","subscribers_url":"https://api.github.com/repos/ansible/ansible/subscribers","subscription_url":"https://api.github.com/repos/ansible/ansible/subscription","commits_url":"https://api.github.com/repos/ansible/ansible/commits{/sha}","git_commits_url":"https://api.github.com/repos/ansible/ansible/git/commits{/sha}","comments_url":"https://api.github.com/repos/ansible/ansible/comments{/number}","issue_comment_url":"https://api.github.com/repos/ansible/ansible/issues/comments{/number}","contents_url":"https://api.github.com/repos/ansible/ansible/contents/{+path}","compare_url":"https://api.github.com/repos/ansible/ansible/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ansible/ansible/merges","archive_url":"https://api.github.com/repos/ansible/ansible/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ansible/ansible/downloads","issues_url":"https://api.github.com/repos/ansible/ansible/issues{/number}","pulls_url":"https://api.github.com/repos/ansible/ansible/pulls{/number}","milestones_url":"https://api.github.com/repos/ansible/ansible/milestones{/number}","notifications_url":"https://api.github.com/repos/ansible/ansible/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ansible/ansible/labels{/name}","releases_url":"https://api.github.com/repos/ansible/ansible/releases{/id}","deployments_url":"https://api.github.com/repos/ansible/ansible/deployments","created_at":"2012-03-06T14:58:02Z","updated_at":"2018-10-28T03:29:50Z","pushed_at":"2018-10-28T03:33:36Z","git_url":"git://github.com/ansible/ansible.git","ssh_url":"git@github.com:ansible/ansible.git","clone_url":"https://github.com/ansible/ansible.git","svn_url":"https://github.com/ansible/ansible","homepage":"https://www.ansible.com/","size":144499,"stargazers_count":33416,"watchers_count":33416,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":13280,"mirror_url":null,"archived":false,"open_issues_count":5631,"license":{"key":"gpl-3.0","name":"GNU General Public License v3.0","spdx_id":"GPL-3.0","url":"https://api.github.com/licenses/gpl-3.0","node_id":"MDc6TGljZW5zZTk="},"forks":13280,"open_issues":5631,"watchers":33416,"default_branch":"devel"}},"_links":{"self":{"href":"https://api.github.com/repos/ansible/ansible/pulls/47702"},"html":{"href":"https://github.com/ansible/ansible/pull/47702"},"issue":{"href":"https://api.github.com/repos/ansible/ansible/issues/47702"},"comments":{"href":"https://api.github.com/repos/ansible/ansible/issues/47702/comments"},"review_comments":{"href":"https://api.github.com/repos/ansible/ansible/pulls/47702/comments"},"review_comment":{"href":"https://api.github.com/repos/ansible/ansible/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ansible/ansible/pulls/47702/commits"},"statuses":{"href":"https://api.github.com/repos/ansible/ansible/statuses/79989cbda52e097020205eddb588dc2f15acd012"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"ganeshrn"}}