{"action":"created","issue":{"url":"https://api.github.com/repos/codingteam/devops/issues/18","repository_url":"https://api.github.com/repos/codingteam/devops","labels_url":"https://api.github.com/repos/codingteam/devops/issues/18/labels{/name}","comments_url":"https://api.github.com/repos/codingteam/devops/issues/18/comments","events_url":"https://api.github.com/repos/codingteam/devops/issues/18/events","html_url":"https://github.com/codingteam/devops/issues/18","id":195769398,"number":18,"title":"Loglist fails and its logs are being flushed","user":{"login":"ForNeVeR","id":92793,"avatar_url":"https://avatars.githubusercontent.com/u/92793?v=3","gravatar_id":"","url":"https://api.github.com/users/ForNeVeR","html_url":"https://github.com/ForNeVeR","followers_url":"https://api.github.com/users/ForNeVeR/followers","following_url":"https://api.github.com/users/ForNeVeR/following{/other_user}","gists_url":"https://api.github.com/users/ForNeVeR/gists{/gist_id}","starred_url":"https://api.github.com/users/ForNeVeR/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ForNeVeR/subscriptions","organizations_url":"https://api.github.com/users/ForNeVeR/orgs","repos_url":"https://api.github.com/users/ForNeVeR/repos","events_url":"https://api.github.com/users/ForNeVeR/events{/privacy}","received_events_url":"https://api.github.com/users/ForNeVeR/received_events","type":"User","site_admin":false},"labels":[{"id":237791809,"url":"https://api.github.com/repos/codingteam/devops/labels/kind:bug","name":"kind:bug","color":"fc2929","default":false},{"id":465194554,"url":"https://api.github.com/repos/codingteam/devops/labels/topic:vm","name":"topic:vm","color":"fef2c0","default":false}],"state":"open","locked":false,"assignee":{"login":"ForNeVeR","id":92793,"avatar_url":"https://avatars.githubusercontent.com/u/92793?v=3","gravatar_id":"","url":"https://api.github.com/users/ForNeVeR","html_url":"https://github.com/ForNeVeR","followers_url":"https://api.github.com/users/ForNeVeR/followers","following_url":"https://api.github.com/users/ForNeVeR/following{/other_user}","gists_url":"https://api.github.com/users/ForNeVeR/gists{/gist_id}","starred_url":"https://api.github.com/users/ForNeVeR/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ForNeVeR/subscriptions","organizations_url":"https://api.github.com/users/ForNeVeR/orgs","repos_url":"https://api.github.com/users/ForNeVeR/repos","events_url":"https://api.github.com/users/ForNeVeR/events{/privacy}","received_events_url":"https://api.github.com/users/ForNeVeR/received_events","type":"User","site_admin":false},"assignees":[{"login":"ForNeVeR","id":92793,"avatar_url":"https://avatars.githubusercontent.com/u/92793?v=3","gravatar_id":"","url":"https://api.github.com/users/ForNeVeR","html_url":"https://github.com/ForNeVeR","followers_url":"https://api.github.com/users/ForNeVeR/followers","following_url":"https://api.github.com/users/ForNeVeR/following{/other_user}","gists_url":"https://api.github.com/users/ForNeVeR/gists{/gist_id}","starred_url":"https://api.github.com/users/ForNeVeR/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ForNeVeR/subscriptions","organizations_url":"https://api.github.com/users/ForNeVeR/orgs","repos_url":"https://api.github.com/users/ForNeVeR/repos","events_url":"https://api.github.com/users/ForNeVeR/events{/privacy}","received_events_url":"https://api.github.com/users/ForNeVeR/received_events","type":"User","site_admin":false}],"milestone":null,"comments":4,"created_at":"2016-12-15T10:30:38Z","updated_at":"2016-12-29T04:29:13Z","closed_at":null,"body":"Today loglist wasn't started, and its logs are completely empty:\r\n\r\n```console\r\n$ sudo journalctl -u loglist\r\n-- No entries --\r\n```\r\n\r\nAt the same time, `cor-site` logs are okay from `2016-12-12`:\r\n\r\n```console\r\n$ sudo journalctl -u cor-site\r\n-- Logs begin at Mon 2016-12-12 16:35:01 EST, end at Thu 2016-12-15 05:25:22 EST. --\r\nDec 12 17:17:48 xmpp1 dotnet[8164]: info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]\r\nDec 12 17:17:48 xmpp1 dotnet[8164]:       Request starting HTTP/1.1 GET http://codingteam.org.ru/robots.txt\r\n...\r\n```\r\n\r\nAfter I restart loglist, the logs are okay, too:\r\n\r\n```console\r\n$ sudo rm /opt/loglist/RUNNING_PID\r\n$ sudo systemctl restart loglist\r\n$ sudo journalctl -u loglist\r\n-- Logs begin at Mon 2016-12-12 16:35:01 EST, end at Thu 2016-12-15 05:25:38 EST. --\r\nDec 15 05:20:35 xmpp1 systemd[1]: Stopped loglist.net site.\r\nDec 15 05:20:35 xmpp1 systemd[1]: Started loglist.net site.\r\nDec 15 05:20:36 xmpp1 loglist-jvm[16122]: Play server process ID is 16122\r\nDec 15 05:20:39 xmpp1 loglist-jvm[16122]: [info] play - database [default] connected at jdbc:postgresql:loglist?user=loglist&password=xxxx\r\nDec 15 05:20:40 xmpp1 loglist-jvm[16122]: [info] play - Application started (Prod)\r\nDec 15 05:20:40 xmpp1 loglist-jvm[16122]: [info] play - Listening for HTTP on /127.0.0.1:9000\r\n```\r\n\r\nSo my conslusion is that we have two problems here:\r\n\r\n1. LogList does stop sometimes, and we don't know the reason.\r\n2. LogList logs are being cleaned or deleted by some unknown entity (or they are configured wrong)."},"comment":{"url":"https://api.github.com/repos/codingteam/devops/issues/comments/269580516","html_url":"https://github.com/codingteam/devops/issues/18#issuecomment-269580516","issue_url":"https://api.github.com/repos/codingteam/devops/issues/18","id":269580516,"user":{"login":"ForNeVeR","id":92793,"avatar_url":"https://avatars.githubusercontent.com/u/92793?v=3","gravatar_id":"","url":"https://api.github.com/users/ForNeVeR","html_url":"https://github.com/ForNeVeR","followers_url":"https://api.github.com/users/ForNeVeR/followers","following_url":"https://api.github.com/users/ForNeVeR/following{/other_user}","gists_url":"https://api.github.com/users/ForNeVeR/gists{/gist_id}","starred_url":"https://api.github.com/users/ForNeVeR/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ForNeVeR/subscriptions","organizations_url":"https://api.github.com/users/ForNeVeR/orgs","repos_url":"https://api.github.com/users/ForNeVeR/repos","events_url":"https://api.github.com/users/ForNeVeR/events{/privacy}","received_events_url":"https://api.github.com/users/ForNeVeR/received_events","type":"User","site_admin":false},"created_at":"2016-12-29T04:29:13Z","updated_at":"2016-12-29T04:29:13Z","body":"The situation is more or less clear. We're starting our LogList JVM with `-Xmx384m -Xss512k`. That means it'll eventually eat up to 384 MiB of RAM. Probably just tune this amount to 200.\r\n\r\nI have no idea why doesn't it use swap file; we have it enabled, but its usage is zero."}}
{"actor":{"display_login":"ForNeVeR"}}