{"action":"created","issue":{"url":"https://api.github.com/repos/aws/aws-codedeploy-agent/issues/133","repository_url":"https://api.github.com/repos/aws/aws-codedeploy-agent","labels_url":"https://api.github.com/repos/aws/aws-codedeploy-agent/issues/133/labels{/name}","comments_url":"https://api.github.com/repos/aws/aws-codedeploy-agent/issues/133/comments","events_url":"https://api.github.com/repos/aws/aws-codedeploy-agent/issues/133/events","html_url":"https://github.com/aws/aws-codedeploy-agent/issues/133","id":258943522,"number":133,"title":"Agent starts to fail and unable to correct self.","user":{"login":"sc-chad","id":6853614,"avatar_url":"https://avatars3.githubusercontent.com/u/6853614?v=4","gravatar_id":"","url":"https://api.github.com/users/sc-chad","html_url":"https://github.com/sc-chad","followers_url":"https://api.github.com/users/sc-chad/followers","following_url":"https://api.github.com/users/sc-chad/following{/other_user}","gists_url":"https://api.github.com/users/sc-chad/gists{/gist_id}","starred_url":"https://api.github.com/users/sc-chad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sc-chad/subscriptions","organizations_url":"https://api.github.com/users/sc-chad/orgs","repos_url":"https://api.github.com/users/sc-chad/repos","events_url":"https://api.github.com/users/sc-chad/events{/privacy}","received_events_url":"https://api.github.com/users/sc-chad/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":21,"created_at":"2017-09-19T19:53:00Z","updated_at":"2018-04-24T19:25:25Z","closed_at":null,"author_association":"NONE","body":"**OS**: Amazon Linux\r\n**Version**: OFFICIAL_1.0-1.1231_rpm\r\n**Issue**:  Agent will be unresponsive and unable to receive deployments. Reboot / restarting the agent does not fix.\r\n**Logs**:\r\n```\r\n2017-09-19 14:47:51 INFO  [codedeploy-agent(4128)]: master 4128: Received CHLD - cleaning dead child process\r\n2017-09-19 14:47:51 INFO  [codedeploy-agent(4128)]: master 4128: been told to replace child 4383\r\n2017-09-19 14:47:51 INFO  [codedeploy-agent(4128)]: master 4128: not enough child processes running - missing at least 1 - respawning\r\n2017-09-19 14:47:56 INFO  [codedeploy-agent(4128)]: master 4128: Spawned child 1/1\r\n2017-09-19 14:47:56 ERROR [codedeploy-agent(4396)]: booting child: error during start or run: NoMethodError - undefined method `underscore' for Seahorse::Util:Module - /opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:45:in `command'\r\n/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:70:in `<class:CommandExecutor>'\r\n/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:14:in `<module:CodeDeployPlugin>'\r\n/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:12:in `<module:Plugins>'\r\n/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:11:in `<module:InstanceAgent>'\r\n/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/command_executor.rb:10:in `<top (required)>'\r\n/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/register_plugin.rb:2:in `require_relative'\r\n/opt/codedeploy-agent/lib/instance_agent/plugins/codedeploy/register_plugin.rb:2:in `<top (required)>'\r\n/usr/local/rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:in `require'\r\n/usr/local/rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:in `require'\r\n/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:16:in `block in load_plugins'\r\n/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:12:in `each'\r\n/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:12:in `load_plugins'\r\n/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:28:in `prepare_run'\r\n/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:64:in `block in prepare_run_with_error_handling'\r\n/opt/codedeploy-agent/lib/instance_agent/runner/child.rb:55:in `with_error_handling'\r\n/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:63:in `prepare_run_with_error_handling'\r\n/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/child.rb:20:in `start'\r\n/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:206:in `block in spawn_child'\r\n/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:204:in `fork'\r\n/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:204:in `spawn_child'\r\n/opt/codedeploy-agent/vendor/gems/process_manager-0.0.13/lib/process_manager/master.rb:283:in `block (2 levels) in replace_terminated_children'\r\n/opt/codedeploy-agent/vendor/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'\r\n```\r\n\r\nSolution is to remove all locally installed gems via:\r\n```\r\ngem uninstall -aIx\r\n```\r\nUninstall the codedeploy agent, remove everything from /opt/codedeploy and reinstall.\r\n\r\nWe are using the puppet module to install which has been working great.\r\n\r\nThe one thing that I think that might be happening is our rails apps are installing a local version of the aws-sdk that is older or newer than what the agent supports.  But that's just a hunch.\r\n\r\nThis issue has been randomly popping up during deployments and has been causing a great amount of grief lately.  To the point we are looking to switch to blue / green just to get around it.  Unfortunately that deployment strategy has it's own issues to deal with.\r\n"},"comment":{"url":"https://api.github.com/repos/aws/aws-codedeploy-agent/issues/comments/384050695","html_url":"https://github.com/aws/aws-codedeploy-agent/issues/133#issuecomment-384050695","issue_url":"https://api.github.com/repos/aws/aws-codedeploy-agent/issues/133","id":384050695,"user":{"login":"rohkat-aws","id":38056420,"avatar_url":"https://avatars1.githubusercontent.com/u/38056420?v=4","gravatar_id":"","url":"https://api.github.com/users/rohkat-aws","html_url":"https://github.com/rohkat-aws","followers_url":"https://api.github.com/users/rohkat-aws/followers","following_url":"https://api.github.com/users/rohkat-aws/following{/other_user}","gists_url":"https://api.github.com/users/rohkat-aws/gists{/gist_id}","starred_url":"https://api.github.com/users/rohkat-aws/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rohkat-aws/subscriptions","organizations_url":"https://api.github.com/users/rohkat-aws/orgs","repos_url":"https://api.github.com/users/rohkat-aws/repos","events_url":"https://api.github.com/users/rohkat-aws/events{/privacy}","received_events_url":"https://api.github.com/users/rohkat-aws/received_events","type":"User","site_admin":false},"created_at":"2018-04-24T19:25:25Z","updated_at":"2018-04-24T19:25:25Z","author_association":"NONE","body":"@sc-chad  can you please provide the above mentioned details . So that we can help solve the issue . Or should i just close this."}}
{"actor":{"display_login":"rohkat-aws"}}