{"action":"created","issue":{"url":"https://api.github.com/repos/scylladb/scylla/issues/1501","repository_url":"https://api.github.com/repos/scylladb/scylla","labels_url":"https://api.github.com/repos/scylladb/scylla/issues/1501/labels{/name}","comments_url":"https://api.github.com/repos/scylladb/scylla/issues/1501/comments","events_url":"https://api.github.com/repos/scylladb/scylla/issues/1501/events","html_url":"https://github.com/scylladb/scylla/issues/1501","id":167499883,"number":1501,"title":"HouseKeeping doesn't work in CentOS 7.2 (got systemd error of $SCYLLA_HOME)","user":{"login":"kongove","id":309708,"avatar_url":"https://avatars.githubusercontent.com/u/309708?v=3","gravatar_id":"","url":"https://api.github.com/users/kongove","html_url":"https://github.com/kongove","followers_url":"https://api.github.com/users/kongove/followers","following_url":"https://api.github.com/users/kongove/following{/other_user}","gists_url":"https://api.github.com/users/kongove/gists{/gist_id}","starred_url":"https://api.github.com/users/kongove/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kongove/subscriptions","organizations_url":"https://api.github.com/users/kongove/orgs","repos_url":"https://api.github.com/users/kongove/repos","events_url":"https://api.github.com/users/kongove/events{/privacy}","received_events_url":"https://api.github.com/users/kongove/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/scylladb/scylla/labels/fixed","name":"fixed","color":"fbca04"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/scylladb/scylla/milestones/14","html_url":"https://github.com/scylladb/scylla/milestones/1.3","labels_url":"https://api.github.com/repos/scylladb/scylla/milestones/14/labels","id":1794296,"number":14,"title":"1.3","description":"","creator":{"login":"tzach","id":170200,"avatar_url":"https://avatars.githubusercontent.com/u/170200?v=3","gravatar_id":"","url":"https://api.github.com/users/tzach","html_url":"https://github.com/tzach","followers_url":"https://api.github.com/users/tzach/followers","following_url":"https://api.github.com/users/tzach/following{/other_user}","gists_url":"https://api.github.com/users/tzach/gists{/gist_id}","starred_url":"https://api.github.com/users/tzach/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tzach/subscriptions","organizations_url":"https://api.github.com/users/tzach/orgs","repos_url":"https://api.github.com/users/tzach/repos","events_url":"https://api.github.com/users/tzach/events{/privacy}","received_events_url":"https://api.github.com/users/tzach/received_events","type":"User","site_admin":false},"open_issues":6,"closed_issues":56,"state":"open","created_at":"2016-05-29T08:33:00Z","updated_at":"2016-08-18T07:18:30Z","due_on":null,"closed_at":null},"comments":17,"created_at":"2016-07-26T01:08:31Z","updated_at":"2016-08-18T08:39:38Z","closed_at":null,"body":"*Installation details*\r\nScylla version (or git commit hash): 1.3.rc1-20160721.c84e030\r\nSystemd version: systemd-219-19.el7\r\nCluster size: single node\r\nOS (RHEL/CentOS/Ubuntu/AWS AMI): CentOS 7.2\r\nAMI ID: RHEL-7.2_HVM_GA-20151112-x86_64-1-Hourly2-GP2 (ami-2051294a)\r\n\r\nWe got this message, but I saw we set SCYLLA_HOME in EnvironmentFile (/etc/sysconfig/scylla-server)\r\n```\r\nsystemd[1]: [/usr/lib/systemd/system/scylla-server.service:15] Not an absolute path, ignoring: $SCYLLA_HOME\r\n```\r\n\r\n```\r\n$ systemctl|grep timer\r\nUNIT                                             LOAD   ACTIVE SUB       DESCRIPTION\r\ntimers.target                                    loaded active active    Timers\r\nscylla-housekeeping.timer                        loaded active elapsed   Run Scylla Housekeeping daily\r\nsystemd-tmpfiles-clean.timer                     loaded active waiting   Daily Cleanup of Temporary Directories\r\n\r\n```\r\n\r\n```\r\n$ sudo service scylla-server status -l\r\nRedirecting to /bin/systemctl status  -l scylla-server.service\r\n● scylla-server.service - Scylla Server\r\n   Loaded: loaded (/usr/lib/systemd/system/scylla-server.service; disabled; vendor preset: disabled)\r\n   Active: active (running) since Mon 2016-07-25 17:57:24 EDT; 2h 23min ago\r\n Main PID: 2770 (scylla)\r\n   Status: \"serving\"\r\n   CGroup: /system.slice/scylla-server.service\r\n           └─2770 /usr/bin/scylla --log-to-syslog 1 --log-to-stdout 0 --default-log-level info --collectd-address=127.0.0.1:25826 --collectd=1 --collectd-poll-period 3000 --network-stack posix --max-io-requests=15\r\n\r\nJul 25 17:57:25 ip-10-138-151-110.ec2.internal scylla[2770]:  [shard 0] gossip - Checking if need_features {RANGE_TOMBSTONES} in features {LARGE_PARTITIONS, RANGE_TOMBSTONES}\r\nJul 25 17:57:25 ip-10-138-151-110.ec2.internal scylla[2770]:  [shard 0] gossip - Checking if need_features {LARGE_PARTITIONS} in features {LARGE_PARTITIONS, RANGE_TOMBSTONES}\r\nJul 25 17:57:25 ip-10-138-151-110.ec2.internal scylla[2770]:  [shard 1] gossip - Checking if need_features {RANGE_TOMBSTONES} in features {LARGE_PARTITIONS, RANGE_TOMBSTONES}\r\nJul 25 17:57:25 ip-10-138-151-110.ec2.internal scylla[2770]:  [shard 1] gossip - Checking if need_features {LARGE_PARTITIONS} in features {LARGE_PARTITIONS, RANGE_TOMBSTONES}\r\nJul 25 17:57:25 ip-10-138-151-110.ec2.internal scylla[2770]:  [shard 0] gossip - Waiting for gossip to settle before accepting client requests...\r\nJul 25 17:57:33 ip-10-138-151-110.ec2.internal scylla[2770]:  [shard 0] gossip - No gossip backlog; proceeding\r\nJul 25 17:57:33 ip-10-138-151-110.ec2.internal scylla[2770]:  [shard 0] storage_service - Starting listening for CQL clients on localhost:9042...\r\nJul 25 17:57:47 ip-10-138-151-110.ec2.internal systemd[1]: [/usr/lib/systemd/system/scylla-server.service:15] Not an absolute path, ignoring: $SCYLLA_HOME\r\nJul 25 18:02:53 ip-10-138-151-110.ec2.internal systemd[1]: [/usr/lib/systemd/system/scylla-server.service:15] Not an absolute path, ignoring: $SCYLLA_HOME\r\nJul 25 18:02:59 ip-10-138-151-110.ec2.internal systemd[1]: [/usr/lib/systemd/system/scylla-server.service:15] Not an absolute path, ignoring: $SCYLLA_HOME\r\n```\r\n\r\n@amnonh Where can I found the timer log in CentOS?"},"comment":{"url":"https://api.github.com/repos/scylladb/scylla/issues/comments/240660099","html_url":"https://github.com/scylladb/scylla/issues/1501#issuecomment-240660099","issue_url":"https://api.github.com/repos/scylladb/scylla/issues/1501","id":240660099,"user":{"login":"kongove","id":309708,"avatar_url":"https://avatars.githubusercontent.com/u/309708?v=3","gravatar_id":"","url":"https://api.github.com/users/kongove","html_url":"https://github.com/kongove","followers_url":"https://api.github.com/users/kongove/followers","following_url":"https://api.github.com/users/kongove/following{/other_user}","gists_url":"https://api.github.com/users/kongove/gists{/gist_id}","starred_url":"https://api.github.com/users/kongove/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kongove/subscriptions","organizations_url":"https://api.github.com/users/kongove/orgs","repos_url":"https://api.github.com/users/kongove/repos","events_url":"https://api.github.com/users/kongove/events{/privacy}","received_events_url":"https://api.github.com/users/kongove/received_events","type":"User","site_admin":false},"created_at":"2016-08-18T08:39:38Z","updated_at":"2016-08-18T08:39:38Z","body":"Update for housekeeping in CentOS 7.2\r\n\r\n@amnonh In my original test, housekeeping service will never be run if it's not enabled.\r\nI just found 'OnBootSec=0' solved this issue, after add this option to housekeeping.timer file,\r\nreload daemon by 'sudo systemctl daemon-reload' and restart scylla-server, then housekeeping works.\r\n\r\nTimer was started, but it doesn't work (repeatedly run service)\r\nI tried to short the period to 60s, it still doesn't work.\r\n\r\n```\r\n[centos@ip-10-178-144-197 ~]$ systemctl status scylla-housekeeping.service\r\n● scylla-housekeeping.service - Scylla Housekeeping\r\n   Loaded: loaded (/usr/lib/systemd/system/scylla-housekeeping.service; disabled; vendor preset: disabled)\r\n   Active: inactive (dead)\r\n[centos@ip-10-178-144-197 ~]$ systemctl status scylla-housekeeping.timer\r\n● scylla-housekeeping.timer - Run Scylla Housekeeping daily\r\n   Loaded: loaded (/usr/lib/systemd/system/scylla-housekeeping.timer; disabled; vendor preset: disabled)\r\n   Active: active (elapsed) since Thu 2016-08-18 08:21:15 UTC; 48s ago\r\n\r\nAug 18 08:21:15 ip-10-178-144-197.ec2.internal systemd[1]: Started Run Scylla Housekeeping daily.\r\nAug 18 08:21:15 ip-10-178-144-197.ec2.internal systemd[1]: Starting Run Scylla Housekeeping daily.\r\n[centos@ip-10-178-144-197 ~]$ sudo systemctl list-timers\r\nNEXT                         LEFT       LAST PASSED UNIT                         ACTIVATES\r\nn/a                          n/a        n/a  n/a    scylla-housekeeping.timer    scylla-housekeeping.service\r\nThu 2016-08-18 08:36:05 UTC  13min left n/a  n/a    systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service\r\n\r\n2 timers listed.\r\n```\r\n\r\nBut if I added a 'OnBootSec' parameter, timer started to work, the service also be repeated run.\r\n\r\n```\r\n[centos@ip-10-178-144-197 ~]$  sudo vim /usr/lib/systemd/system/scylla-housekeeping.timer\r\n[centos@ip-10-178-144-197 ~]$ sudo systemctl daemon-reload\r\n[centos@ip-10-178-144-197 ~]$ sudo service scylla-server restart\r\nRedirecting to /bin/systemctl restart  scylla-server.service\r\n[centos@ip-10-178-144-197 ~]$ systemctl status scylla-housekeeping.service\r\n● scylla-housekeeping.service - Scylla Housekeeping\r\n   Loaded: loaded (/usr/lib/systemd/system/scylla-housekeeping.service; disabled; vendor preset: disabled)\r\n   Active: inactive (dead) since Thu 2016-08-18 08:23:32 UTC; 4s ago\r\n  Process: 2040 ExecStart=/usr/lib/scylla/scylla-housekeeping -q -c /etc/scylla.d/housekeeping.cfg version (code=exited, status=0/SUCCESS)\r\n Main PID: 2040 (code=exited, status=0/SUCCESS)\r\n\r\nAug 18 08:23:31 ip-10-178-144-197.ec2.internal systemd[1]: Started Scylla Housekeeping.\r\nAug 18 08:23:31 ip-10-178-144-197.ec2.internal systemd[1]: Starting Scylla Housekeeping...\r\nAug 18 08:23:32 ip-10-178-144-197.ec2.internal scylla-housekeeping[2040]: A new version was found, current version=1.3.rc3-20160815.b2e6a52 latest...on=1.3\r\nHint: Some lines were ellipsized, use -l to show in full.\r\n[centos@ip-10-178-144-197 ~]$ \r\n```\r\n\r\nConclusion: this issue was fixed by:\r\n[PATCH] systemd: have the first housekeeping check right after start"}}
{"actor":{"display_login":"kongove"}}