{"action":"created","issue":{"url":"https://api.github.com/repos/ratt-ru/CubiCal/issues/221","repository_url":"https://api.github.com/repos/ratt-ru/CubiCal","labels_url":"https://api.github.com/repos/ratt-ru/CubiCal/issues/221/labels{/name}","comments_url":"https://api.github.com/repos/ratt-ru/CubiCal/issues/221/comments","events_url":"https://api.github.com/repos/ratt-ru/CubiCal/issues/221/events","html_url":"https://github.com/ratt-ru/CubiCal/issues/221","id":356515389,"node_id":"MDU6SXNzdWUzNTY1MTUzODk=","number":221,"title":"BITFLAGs not marked as properly initialized when running in single-CPU mode","user":{"login":"o-smirnov","id":6470079,"node_id":"MDQ6VXNlcjY0NzAwNzk=","avatar_url":"https://avatars2.githubusercontent.com/u/6470079?v=4","gravatar_id":"","url":"https://api.github.com/users/o-smirnov","html_url":"https://github.com/o-smirnov","followers_url":"https://api.github.com/users/o-smirnov/followers","following_url":"https://api.github.com/users/o-smirnov/following{/other_user}","gists_url":"https://api.github.com/users/o-smirnov/gists{/gist_id}","starred_url":"https://api.github.com/users/o-smirnov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/o-smirnov/subscriptions","organizations_url":"https://api.github.com/users/o-smirnov/orgs","repos_url":"https://api.github.com/users/o-smirnov/repos","events_url":"https://api.github.com/users/o-smirnov/events{/privacy}","received_events_url":"https://api.github.com/users/o-smirnov/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-09-03T13:53:48Z","updated_at":"2018-09-03T21:56:10Z","closed_at":null,"author_association":"COLLABORATOR","body":"Raised by @IanHeywood here: https://github.com/ratt-ru/CubiCal/issues/211#issuecomment-417711651\r\n\r\nI can see part of the problem. When auto-initializing BITFLAG from FLAG, I mark the BITFLAG column with an \"in progress\" keyword. When a run is successful, this keyword is cleared. The overly bright idea was, if a run is interrupted when the column is in a half-filled state, cubical knows it has to go reinitialize it the next time it is run.\r\n\r\nDue to sheer stupid oversight on my part, the keyword is *not* cleared when cubical is run in single-CPU mode. So each time it proceeds to reinitialize it all over again. If one of the runs happens to flag 99% of the data (due to user error, or other bugs in madmax or solver flagging, for example), these flags then end up in the \"legacy\" flagset (rather than \"cubical\"), at which point we're all screwed.\r\n\r\nOK, so fixing the bug and clearing the right keyword is simple enough, but I wonder if this whole design isn't misguided on my part. There's still the possibility of the user Ctrl+C'ing the first auto-initializing run, and leaving BITFLAG in a partly-filled state. What must cubical do on the next run? After the Ctrl+C, the MS is in the following state:\r\n\r\n* BITFLAG is only partially filled (part of it is filled, with the \"legacy\" bit containing a copy of FLAG/FLAG_ROW before the run, and the rest of the column in an undefined state)\r\n\r\n* The start of the FLAG column may be overflagged (since it contains a combination of \"legacy\" and \"cubical\" flags), while the rest of the column is still in its original state.\r\n\r\nAt this point, throwing away BITFLAG and re-populating it from FLAG (which the auto-init code does) is clearly the wrong thing to do. But what's the right thing? \r\n\r\n* Bomb out with a fat error message and let the user recover manually? (But how?)\r\n\r\n* Bomb out with a fat error message unless a special \"proceed anyway\" command-line option is given?\r\n\r\n* Try to keep track of how many BITFLAG rows have been initialized, and do something smart? I'd prefer to do this, but trying to do something smart is what got me into this pickle in the first place...\r\n\r\n* Prohibit auto-init of BITFLAG and changing of FLAG in the same run? (I.e. only allow it to change FLAGs when it knows a proper BITFLAG column is in place)? That may be the safest, but it does mean running cubical twice (at least the first time 'round) before any solver/madmax flags can kick in.\r\n\r\nThoughts, anyone?\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/ratt-ru/CubiCal/issues/comments/418197685","html_url":"https://github.com/ratt-ru/CubiCal/issues/221#issuecomment-418197685","issue_url":"https://api.github.com/repos/ratt-ru/CubiCal/issues/221","id":418197685,"node_id":"MDEyOklzc3VlQ29tbWVudDQxODE5NzY4NQ==","user":{"login":"o-smirnov","id":6470079,"node_id":"MDQ6VXNlcjY0NzAwNzk=","avatar_url":"https://avatars2.githubusercontent.com/u/6470079?v=4","gravatar_id":"","url":"https://api.github.com/users/o-smirnov","html_url":"https://github.com/o-smirnov","followers_url":"https://api.github.com/users/o-smirnov/followers","following_url":"https://api.github.com/users/o-smirnov/following{/other_user}","gists_url":"https://api.github.com/users/o-smirnov/gists{/gist_id}","starred_url":"https://api.github.com/users/o-smirnov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/o-smirnov/subscriptions","organizations_url":"https://api.github.com/users/o-smirnov/orgs","repos_url":"https://api.github.com/users/o-smirnov/repos","events_url":"https://api.github.com/users/o-smirnov/events{/privacy}","received_events_url":"https://api.github.com/users/o-smirnov/received_events","type":"User","site_admin":false},"created_at":"2018-09-03T21:56:10Z","updated_at":"2018-09-03T21:56:10Z","author_association":"COLLABORATOR","body":"BTW @IanHeywood, if you run in single-CPU mode, can you do us a favour, and routinely run it as\r\n\r\n```\r\npython -mcProfile -o <YOUROUTPUTNAME>.prof `which gocubical` <parameters...>\r\n```\r\n\r\nIt shouldn't affect anything, apart from making it dump a profile file at the end. These are very useful to look at, to see where the CPU time is going in your particular setup, and if anything can be speeded up.\r\n\r\n(Less useful in multi-CPU mode, I'm afraid...)\r\n\r\n\r\n\r\n"}}
{"actor":{"display_login":"o-smirnov"}}