{"action":"created","issue":{"url":"https://api.github.com/repos/Metaswitch/clearwater-cassandra/issues/79","repository_url":"https://api.github.com/repos/Metaswitch/clearwater-cassandra","labels_url":"https://api.github.com/repos/Metaswitch/clearwater-cassandra/issues/79/labels{/name}","comments_url":"https://api.github.com/repos/Metaswitch/clearwater-cassandra/issues/79/comments","events_url":"https://api.github.com/repos/Metaswitch/clearwater-cassandra/issues/79/events","html_url":"https://github.com/Metaswitch/clearwater-cassandra/pull/79","id":137882706,"number":79,"title":"[Reviewer: Alex] Move to config-package infrastructure","user":{"login":"rkd-msw","id":4363492,"avatar_url":"https://avatars.githubusercontent.com/u/4363492?v=3","gravatar_id":"","url":"https://api.github.com/users/rkd-msw","html_url":"https://github.com/rkd-msw","followers_url":"https://api.github.com/users/rkd-msw/followers","following_url":"https://api.github.com/users/rkd-msw/following{/other_user}","gists_url":"https://api.github.com/users/rkd-msw/gists{/gist_id}","starred_url":"https://api.github.com/users/rkd-msw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rkd-msw/subscriptions","organizations_url":"https://api.github.com/users/rkd-msw/orgs","repos_url":"https://api.github.com/users/rkd-msw/repos","events_url":"https://api.github.com/users/rkd-msw/events{/privacy}","received_events_url":"https://api.github.com/users/rkd-msw/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"rkd-msw","id":4363492,"avatar_url":"https://avatars.githubusercontent.com/u/4363492?v=3","gravatar_id":"","url":"https://api.github.com/users/rkd-msw","html_url":"https://github.com/rkd-msw","followers_url":"https://api.github.com/users/rkd-msw/followers","following_url":"https://api.github.com/users/rkd-msw/following{/other_user}","gists_url":"https://api.github.com/users/rkd-msw/gists{/gist_id}","starred_url":"https://api.github.com/users/rkd-msw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rkd-msw/subscriptions","organizations_url":"https://api.github.com/users/rkd-msw/orgs","repos_url":"https://api.github.com/users/rkd-msw/repos","events_url":"https://api.github.com/users/rkd-msw/events{/privacy}","received_events_url":"https://api.github.com/users/rkd-msw/received_events","type":"User","site_admin":false},"milestone":null,"comments":5,"created_at":"2016-03-02T14:09:18Z","updated_at":"2016-03-21T17:30:00Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/Metaswitch/clearwater-cassandra/pulls/79","html_url":"https://github.com/Metaswitch/clearwater-cassandra/pull/79","diff_url":"https://github.com/Metaswitch/clearwater-cassandra/pull/79.diff","patch_url":"https://github.com/Metaswitch/clearwater-cassandra/pull/79.patch"},"body":"Hi Alex,\r\n\r\nThis moves clearwater-cassandra over to the structured config-package infrastructure (https://debathena.mit.edu/config-packages/) instead of just running random sed scripts. That approach caused issues on upgrade if we change a config file that has also changed upstream (#16).\r\n\r\nI strongly recommend you review commit-by-commit - I've tried to make that sensible (e.g. by pulling in files from Cassandra and modifying them in separate commits). In particular, I think you'll barely need to review https://github.com/Metaswitch/clearwater-cassandra/commit/b808fa7ee2bc205f32de5eeece2cc13c7abc4af8.\r\n\r\nSo that you don't get confused, I haven't been able to exactly follow my emailed proposal - in particular, I was wrong about how transform rules work. They don't transform files on the production system at install time - they transform files on the build system at package creation time - so the use-case there is \"check in a standard version of a file, use the transform rule to tweak it to your needs\". Instead, I've just checked in the file, modified it and used a displace rule to put it into place - i think just editing the file is generally going to be easier than writing/deciphering the sed commands to modify it.\r\n\r\nUltimately all this does is:\r\n\r\n* replaces /etc/init.d/cassandra with a version that runs in the signaling namespace\r\n* replaces /etc/default/cassandra and /usr/share/cassandra/cassandra.in.sh with a version that always uses Java 7 (we didn't need to do the latter through this mechanism, as it's not a conffile, but on reflection I thought it was best to be consistent in how to change those files)\r\n* replaces /etc/cassandra/cassandra-env.sh with an IPv4-enabled or IPv6-enabled version, based on the type of $local_ip\r\n* hides /etc/cassandra/cassandra.yaml and /etc/cassandra/cassandra-rackdc.properties so that clearwater-cluster-manager can generate them\r\n\r\nThere are two things I thought of doing while making this change, but which ultimately didn't seem worth it:\r\n- Instead of checking in files from the Cassandra repository and modifying them, I could have included Cassandra as a submodule, defined a Makefile rule that would copy the files into the right directory, and defined transform rules to make the corresponding edits. In principle this would mean that we could change our submodule to point at a different Cassandra version and everything would auto-update, but it felt like overkill.\r\n- I considered moving a lot of the cassandra.yaml-mangling logic out from the plugin into a script which the plugin could run with some arguments - especially given that there's currently a bit that shells out to check an /etc/clearwater/config value. I couldn't quite persuade myself it was worth the risk of the change, but I'd like a second opinion.\r\n\r\nTwo things I'm not doing in this PR, because they're nicely separable:\r\n* add a clearwater-cluster-manager hook we can kick to regenerate the cassandra.yaml file when the template changes\r\n* move AIO and Docker nodes over to use etcd\r\n\r\nLet me know if you're really keen to review either of those follow-on PRs, otherwise I might spread the load."},"comment":{"url":"https://api.github.com/repos/Metaswitch/clearwater-cassandra/issues/comments/199390089","html_url":"https://github.com/Metaswitch/clearwater-cassandra/pull/79#issuecomment-199390089","issue_url":"https://api.github.com/repos/Metaswitch/clearwater-cassandra/issues/79","id":199390089,"user":{"login":"rkd-msw","id":4363492,"avatar_url":"https://avatars.githubusercontent.com/u/4363492?v=3","gravatar_id":"","url":"https://api.github.com/users/rkd-msw","html_url":"https://github.com/rkd-msw","followers_url":"https://api.github.com/users/rkd-msw/followers","following_url":"https://api.github.com/users/rkd-msw/following{/other_user}","gists_url":"https://api.github.com/users/rkd-msw/gists{/gist_id}","starred_url":"https://api.github.com/users/rkd-msw/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rkd-msw/subscriptions","organizations_url":"https://api.github.com/users/rkd-msw/orgs","repos_url":"https://api.github.com/users/rkd-msw/repos","events_url":"https://api.github.com/users/rkd-msw/events{/privacy}","received_events_url":"https://api.github.com/users/rkd-msw/received_events","type":"User","site_admin":false},"created_at":"2016-03-21T17:30:00Z","updated_at":"2016-03-21T17:30:00Z","body":"I tweaked /etc/default/cassandra, bumped the version and rebuilt Cassandra. With the master code, I get a conflict reported:\r\n\r\n```\r\nSetting up cassandra (2.0.15) ...\r\nInstalling new version of config file /etc/init.d/cassandra ...\r\n\r\nConfiguration file '/etc/default/cassandra'\r\n ==> Modified (by you or by a script) since installation.\r\n ==> Package distributor has shipped an updated version.\r\n ==> Keeping old config file as default.\r\n```\r\n\r\n(I was using clearwater-upgrade, which includes `--force-confdef -o Dpkg::Options::=--force-confnew`, so doesn't ask the user.)\r\n\r\nWith my new code, no such conflict, and we can see evidence of the diversion:\r\n\r\n```\r\nSetting up cassandra (2.0.15) ...\r\nInstalling new version of config file /etc/init.d/cassandra.clearwater-orig ...\r\nInstalling new version of config file /etc/default/cassandra.clearwater-orig ...\r\nvm.max_map_count = 1048575\r\nProcessing triggers for ureadahead (0.100.0-16) ...\r\nProcessing triggers for python-support (1.0.15) ...\r\n```"}}