{"action":"created","issue":{"url":"https://api.github.com/repos/hawkular/cassalog/issues/3","repository_url":"https://api.github.com/repos/hawkular/cassalog","labels_url":"https://api.github.com/repos/hawkular/cassalog/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/hawkular/cassalog/issues/3/comments","events_url":"https://api.github.com/repos/hawkular/cassalog/issues/3/events","html_url":"https://github.com/hawkular/cassalog/issues/3","id":179848578,"number":3,"title":"Cassalog can get into inconsistent state and prevent upgrades from completing","user":{"login":"jsanda","id":168121,"avatar_url":"https://avatars.githubusercontent.com/u/168121?v=3","gravatar_id":"","url":"https://api.github.com/users/jsanda","html_url":"https://github.com/jsanda","followers_url":"https://api.github.com/users/jsanda/followers","following_url":"https://api.github.com/users/jsanda/following{/other_user}","gists_url":"https://api.github.com/users/jsanda/gists{/gist_id}","starred_url":"https://api.github.com/users/jsanda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jsanda/subscriptions","organizations_url":"https://api.github.com/users/jsanda/orgs","repos_url":"https://api.github.com/users/jsanda/repos","events_url":"https://api.github.com/users/jsanda/events{/privacy}","received_events_url":"https://api.github.com/users/jsanda/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/hawkular/cassalog/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/hawkular/cassalog/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-09-28T18:19:29Z","updated_at":"2016-10-05T07:50:02Z","closed_at":null,"body":"When Cassalog applies a schema change, it executes whatever CQL statements correspond to the change, and then it updates the change log table, `cassalog`. If Cassalog fails to update the change log table, maybe due to a request timeout, Cassalog will abort. Depending on the schema change involved, Cassalog can wind up in an inconsistent state. Let's say we have:\r\n\r\n```cassalog.groovy\r\nschemaChange {\r\n  version '1.0'\r\n  author 'jsanda'\r\n  cql \"\"\"\r\nCREATE TABLE my_table (\r\n    id text PRIMARY KEY,\r\n    value text\r\n) WITH compaction = { 'class': 'LeveledCompactionStrategy' }\r\n\"\"\"\r\n}\r\n```\r\n\r\nCassalog fails with a timeout exception while trying to update the change log table, or better yet, the JVM in which Cassalog is running gets shutdown abruptly. The next time Cassalog is run it will attempt to apply the above schema change since it was not recorded in the change log. This in turn will result in an error from Cassandra since the table already exists.\r\n\r\nIn this particular example we can sort of work around the problem by using `CREATE TABLE my_table IF NOT EXISTS...`, but that won't work in general. Initially I thought maybe the solution would be to use atomic batches, but they cannot be used with DDL statements. We need to figure something else out to prevent Cassalog from getting into an inconsistent state."},"comment":{"url":"https://api.github.com/repos/hawkular/cassalog/issues/comments/251605996","html_url":"https://github.com/hawkular/cassalog/issues/3#issuecomment-251605996","issue_url":"https://api.github.com/repos/hawkular/cassalog/issues/3","id":251605996,"user":{"login":"lucasponce","id":1662329,"avatar_url":"https://avatars.githubusercontent.com/u/1662329?v=3","gravatar_id":"","url":"https://api.github.com/users/lucasponce","html_url":"https://github.com/lucasponce","followers_url":"https://api.github.com/users/lucasponce/followers","following_url":"https://api.github.com/users/lucasponce/following{/other_user}","gists_url":"https://api.github.com/users/lucasponce/gists{/gist_id}","starred_url":"https://api.github.com/users/lucasponce/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lucasponce/subscriptions","organizations_url":"https://api.github.com/users/lucasponce/orgs","repos_url":"https://api.github.com/users/lucasponce/repos","events_url":"https://api.github.com/users/lucasponce/events{/privacy}","received_events_url":"https://api.github.com/users/lucasponce/received_events","type":"User","site_admin":false},"created_at":"2016-10-05T07:50:02Z","updated_at":"2016-10-05T07:50:02Z","body":"In the same scenario, in alerting we have introduced a checker that basically flags if the schema is ok or not querying the tables on the system tables too.\r\nPerhaps trying to do this under the hood of the user it can bring a lot of complexity.\r\nBut a checker() function on cassalog might help to identify when a query was correct.\r\nMy previous comment was more simple.\r\nTaking as assumption that when an update() call finished correctly, I was thinking on set a previous STARTED/FINISHED flags in cassalog table.\r\nSo, in that case, a preliminar check with STARTED flag should stop the process and warns user that Cassandra was in a non-stable status.\r\nTrying to automate this from the update() call means that we need to detect the type of operation and create the specific checker function. That's interesting but perhaps is better to delegate that into the user.\r\nWDYT ?"}}
{"actor":{"display_login":"lucasponce"}}