{"action":"created","issue":{"url":"https://api.github.com/repos/theforeman/smart-proxy/issues/358","labels_url":"https://api.github.com/repos/theforeman/smart-proxy/issues/358/labels{/name}","comments_url":"https://api.github.com/repos/theforeman/smart-proxy/issues/358/comments","events_url":"https://api.github.com/repos/theforeman/smart-proxy/issues/358/events","html_url":"https://github.com/theforeman/smart-proxy/pull/358","id":126215463,"number":358,"title":"Fixes #12715 - simple log buffer and API","user":{"login":"lzap","id":49752,"avatar_url":"https://avatars.githubusercontent.com/u/49752?v=3","gravatar_id":"","url":"https://api.github.com/users/lzap","html_url":"https://github.com/lzap","followers_url":"https://api.github.com/users/lzap/followers","following_url":"https://api.github.com/users/lzap/following{/other_user}","gists_url":"https://api.github.com/users/lzap/gists{/gist_id}","starred_url":"https://api.github.com/users/lzap/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lzap/subscriptions","organizations_url":"https://api.github.com/users/lzap/orgs","repos_url":"https://api.github.com/users/lzap/repos","events_url":"https://api.github.com/users/lzap/events{/privacy}","received_events_url":"https://api.github.com/users/lzap/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/theforeman/smart-proxy/labels/Needs%20re-review","name":"Needs re-review","color":"0052cc"},{"url":"https://api.github.com/repos/theforeman/smart-proxy/labels/Needs%20testing","name":"Needs testing","color":"fbca04"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":33,"created_at":"2016-01-12T16:20:24Z","updated_at":"2016-01-18T15:20:19Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/theforeman/smart-proxy/pulls/358","html_url":"https://github.com/theforeman/smart-proxy/pull/358","diff_url":"https://github.com/theforeman/smart-proxy/pull/358.diff","patch_url":"https://github.com/theforeman/smart-proxy/pull/358.patch"},"body":"As per our discussion on the IRC, here is my attempt to implement Dmitri's\nevent system in a much more simplified way. Since we want something very simple\nand efficient, this is essentially a ring buffer of structs (events), which is\nsynchronized.\n\nThere is only one buffer for all modules, there is a special \"tail\" buffer that\nholds all the old values from the main buffer that has WARNING or ERROR level.\nThis way we won't loose important errors. The buffer can be large as this\nimplementation is fast and efficient.\n\nThe API is trivial, backtraces are not passed in the parameter, but you get the\nidea. Our main goal is to get logs and important messages from proxy to\nforeman. This accomplishes it easily and this buys us time to integrate with\nother systems that does this thing well. Collectd comes to my mind, it actually\ncontains exactly same mechanism we need (it's called notifications and I\nalready have an implementation for smart-proxy).\n\nI created this POC because I don't want us to reinvent the wheel building our\nown messaging system for value readouts and notifications. Instead, let's\ncreate a simple API and save time for other important things.\n\n```\n$ curl -sk http://localhost:8448/events | json_reformat \n{\n    \"events\": [\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"'discovery' settings were initialized with default values: :node_port: 8443, :node_scheme: https\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'discovery'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'discovery_image'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'salt'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'abrt'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"'colly' settings were initialized with default values: :socket: /var/run/collectd-unixsock, :timeout: 5\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 100,\n            \"message\": \"Couldn't enable plugin colly: No such file or directory - \\\"/var/run/collectd-unixsock\\\"\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 400,\n            \"message\": \"No such file or directory - \\\"/var/run/collectd-unixsock\\\":/home/lzap/work/collectd-uxsock/lib/uxsock/collectd_unix_socket.rb:12:in `initialize'\\n/home/lzap/work/collectd-uxsock/lib/uxsock/collectd_unix_socket.rb:12:in `open'\\n/home/lzap/work/collectd-uxsock/lib/uxsock/collectd_unix_socket.rb:12:in `initialize'\\n/home/lzap/work/collectd-uxsock/lib/uxsock/collectd_unix_socket.rb:22:in `new'\\n/home/lzap/work/collectd-uxsock/lib/uxsock/collectd_unix_socket.rb:22:in `open'\\n/home/lzap/work/smart_proxy_colly/lib/smart_proxy_colly/writer.rb:31:in `notify'\\n/home/lzap/work/smart_proxy_colly/lib/smart_proxy_colly/colly.rb:14:in `block in <class:Plugin>'\\n/home/lzap/work/smart-proxy/lib/proxy/pluggable.rb:33:in `instance_eval'\\n/home/lzap/work/smart-proxy/lib/proxy/pluggable.rb:33:in `after_activation'\\n/home/lzap/work/smart-proxy/lib/proxy/plugin.rb:138:in `configure_plugin'\\n/home/lzap/work/smart-proxy/lib/proxy/plugin.rb:29:in `block in configure_loaded_plugins'\\n/home/lzap/work/smart-proxy/lib/proxy/plugin.rb:29:in `each'\\n/home/lzap/work/smart-proxy/lib/proxy/plugin.rb:29:in `configure_loaded_plugins'\\n/home/lzap/work/smart-proxy/lib/launcher.rb:111:in `launch'\\nbin/smart-proxy:6:in `<main>'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'pulp'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"'pulpnode' module is disabled.\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"'foreman_proxy' settings were initialized with default values: :enabled: true\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'foreman_proxy'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"'facts' module is disabled.\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"'dns' settings were initialized with default values: :dns_ttl: 86400\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'dns'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'templates'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'tftp'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"'dhcp' settings were initialized with default values: :server: 127.0.0.1\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615190,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'dhcp'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615191,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'puppetca'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615191,\n            \"level\": 300,\n            \"message\": \"'puppet' settings were initialized with default values: :puppet_ssl_ca: /var/lib/puppet/ssl/certs/ca.pem, :salt_puppetrun_cmd: puppet.run, :use_cache: true\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615191,\n            \"level\": 100,\n            \"message\": \"Couldn't enable plugin puppet: Cannot set modulepath settings in puppet.conf\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615191,\n            \"level\": 400,\n            \"message\": \"Cannot set modulepath settings in puppet.conf:/home/lzap/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/puppet-4.3.0/lib/puppet/settings.rb:1107:in `block in screen_non_puppet_conf_settings'\\n/home/lzap/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/puppet-4.3.0/lib/puppet/settings.rb:1105:in `each'\\n/home/lzap/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/puppet-4.3.0/lib/puppet/settings.rb:1105:in `screen_non_puppet_conf_settings'\\n/home/lzap/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/puppet-4.3.0/lib/puppet/settings.rb:538:in `parse_config'\\n/home/lzap/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/puppet-4.3.0/lib/puppet/settings.rb:595:in `parse_config_files'\\n/home/lzap/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/puppet-4.3.0/lib/puppet/settings.rb:262:in `initialize_global_settings'\\n/home/lzap/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/puppet-4.3.0/lib/puppet.rb:133:in `do_initialize_settings_for_run_mode'\\n/home/lzap/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/puppet-4.3.0/lib/puppet.rb:127:in `initialize_settings'\\n/home/lzap/work/smart-proxy/modules/puppet_proxy/initializer.rb:20:in `reset_puppet'\\n/home/lzap/work/smart-proxy/modules/puppet_proxy/puppet_plugin.rb:17:in `block in <class:Plugin>'\\n/home/lzap/work/smart-proxy/lib/proxy/pluggable.rb:33:in `instance_eval'\\n/home/lzap/work/smart-proxy/lib/proxy/pluggable.rb:33:in `after_activation'\\n/home/lzap/work/smart-proxy/lib/proxy/plugin.rb:138:in `configure_plugin'\\n/home/lzap/work/smart-proxy/lib/proxy/plugin.rb:29:in `block in configure_loaded_plugins'\\n/home/lzap/work/smart-proxy/lib/proxy/plugin.rb:29:in `each'\\n/home/lzap/work/smart-proxy/lib/proxy/plugin.rb:29:in `configure_loaded_plugins'\\n/home/lzap/work/smart-proxy/lib/launcher.rb:111:in `launch'\\nbin/smart-proxy:6:in `<main>'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615191,\n            \"level\": 300,\n            \"message\": \"Finished initialization of module 'bmc'\",\n            \"backtrace\": [\n\n            ]\n        },\n        {\n            \"timestamp\": 1452615191,\n            \"level\": 300,\n            \"message\": \"'realm' module is disabled.\",\n            \"backtrace\": [\n\n            ]\n        }\n    ]\n}\n```"},"comment":{"url":"https://api.github.com/repos/theforeman/smart-proxy/issues/comments/172557583","html_url":"https://github.com/theforeman/smart-proxy/pull/358#issuecomment-172557583","issue_url":"https://api.github.com/repos/theforeman/smart-proxy/issues/358","id":172557583,"user":{"login":"lzap","id":49752,"avatar_url":"https://avatars.githubusercontent.com/u/49752?v=3","gravatar_id":"","url":"https://api.github.com/users/lzap","html_url":"https://github.com/lzap","followers_url":"https://api.github.com/users/lzap/followers","following_url":"https://api.github.com/users/lzap/following{/other_user}","gists_url":"https://api.github.com/users/lzap/gists{/gist_id}","starred_url":"https://api.github.com/users/lzap/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lzap/subscriptions","organizations_url":"https://api.github.com/users/lzap/orgs","repos_url":"https://api.github.com/users/lzap/repos","events_url":"https://api.github.com/users/lzap/events{/privacy}","received_events_url":"https://api.github.com/users/lzap/received_events","type":"User","site_admin":false},"created_at":"2016-01-18T15:20:19Z","updated_at":"2016-01-18T15:20:19Z","body":"When logging is refactored per-module, then we are able to keep this \"logs\" API and still provide failed modules by showing latest error messages there."}}