{"action":"created","issue":{"url":"https://api.github.com/repos/scylladb/scylla/issues/548","repository_url":"https://api.github.com/repos/scylladb/scylla","labels_url":"https://api.github.com/repos/scylladb/scylla/issues/548/labels{/name}","comments_url":"https://api.github.com/repos/scylladb/scylla/issues/548/comments","events_url":"https://api.github.com/repos/scylladb/scylla/issues/548/events","html_url":"https://github.com/scylladb/scylla/issues/548","id":116212735,"number":548,"title":"Column family has wrong ID","user":{"login":"glommer","id":331197,"avatar_url":"https://avatars.githubusercontent.com/u/331197?v=3","gravatar_id":"","url":"https://api.github.com/users/glommer","html_url":"https://github.com/glommer","followers_url":"https://api.github.com/users/glommer/followers","following_url":"https://api.github.com/users/glommer/following{/other_user}","gists_url":"https://api.github.com/users/glommer/gists{/gist_id}","starred_url":"https://api.github.com/users/glommer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/glommer/subscriptions","organizations_url":"https://api.github.com/users/glommer/orgs","repos_url":"https://api.github.com/users/glommer/repos","events_url":"https://api.github.com/users/glommer/events{/privacy}","received_events_url":"https://api.github.com/users/glommer/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/scylladb/scylla/labels/bug","name":"bug","color":"eb6420"},{"url":"https://api.github.com/repos/scylladb/scylla/labels/Correctness","name":"Correctness","color":"b60205"},{"url":"https://api.github.com/repos/scylladb/scylla/labels/high","name":"high","color":"d4c5f9"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/scylladb/scylla/milestones/10","html_url":"https://github.com/scylladb/scylla/milestones/Yellowfin","labels_url":"https://api.github.com/repos/scylladb/scylla/milestones/10/labels","id":1496576,"number":10,"title":"Yellowfin","description":"","creator":{"login":"tzach","id":170200,"avatar_url":"https://avatars.githubusercontent.com/u/170200?v=3","gravatar_id":"","url":"https://api.github.com/users/tzach","html_url":"https://github.com/tzach","followers_url":"https://api.github.com/users/tzach/followers","following_url":"https://api.github.com/users/tzach/following{/other_user}","gists_url":"https://api.github.com/users/tzach/gists{/gist_id}","starred_url":"https://api.github.com/users/tzach/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tzach/subscriptions","organizations_url":"https://api.github.com/users/tzach/orgs","repos_url":"https://api.github.com/users/tzach/repos","events_url":"https://api.github.com/users/tzach/events{/privacy}","received_events_url":"https://api.github.com/users/tzach/received_events","type":"User","site_admin":false},"open_issues":116,"closed_issues":34,"state":"open","created_at":"2016-01-11T08:12:15Z","updated_at":"2016-06-26T11:32:14Z","due_on":null,"closed_at":null},"comments":4,"created_at":"2015-11-10T22:04:22Z","updated_at":"2016-06-28T22:02:34Z","closed_at":null,"body":"In a cluster with 3 nodes, loaded with standard cassandra-stress write test, I am seeing this in one of the nodes:\r\n```\r\nDEBUG [shard 25] sstable - Writing TOC file\r\n/data/data/keyspace1/standard1-9005484087af11e593f7000000000001/keyspace1-standard1-ka-56-TOC.txt.tmp\r\nDEBUG [shard 25] database - Flushing done\r\nERROR [shard 25] database - failed to write sstable: std::system_error\r\n(error system:2, No such file or directory)\r\n```\r\n\r\nThe reason we fail this write, is that this column family does not exist:\r\n\r\n```\r\n[fedora@host ~]$ sudo ls /data/data/keyspace1/\r\nstandard1-9001c5d087af11e5b67a000000000000\r\n```\r\nSo the CF is there, but it has a different UUID attached to it. That\r\ndirectory is empty,  and the commitlog directory is 9G big in size.\r\n\r\nI don't know how to reproduce but something triggers the problem. I will try more tomorrow, but we should be inspecting code paths that may lead to this\r\n\r\n@penberg says on the matter: \r\n\r\n>One potential explanation I can think of is that CF was created on\r\n>different nodes simultaneously and our schema merging validation logic\r\n>in database::update_column_family() failed to kick in."},"comment":{"url":"https://api.github.com/repos/scylladb/scylla/issues/comments/229197962","html_url":"https://github.com/scylladb/scylla/issues/548#issuecomment-229197962","issue_url":"https://api.github.com/repos/scylladb/scylla/issues/548","id":229197962,"user":{"login":"tgrabiec","id":283695,"avatar_url":"https://avatars.githubusercontent.com/u/283695?v=3","gravatar_id":"","url":"https://api.github.com/users/tgrabiec","html_url":"https://github.com/tgrabiec","followers_url":"https://api.github.com/users/tgrabiec/followers","following_url":"https://api.github.com/users/tgrabiec/following{/other_user}","gists_url":"https://api.github.com/users/tgrabiec/gists{/gist_id}","starred_url":"https://api.github.com/users/tgrabiec/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tgrabiec/subscriptions","organizations_url":"https://api.github.com/users/tgrabiec/orgs","repos_url":"https://api.github.com/users/tgrabiec/repos","events_url":"https://api.github.com/users/tgrabiec/events{/privacy}","received_events_url":"https://api.github.com/users/tgrabiec/received_events","type":"User","site_admin":false},"created_at":"2016-06-28T22:02:34Z","updated_at":"2016-06-28T22:02:34Z","body":"Most likely dup of #420 "}}
{"actor":{"display_login":"tgrabiec"}}