{"action":"closed","issue":{"url":"https://api.github.com/repos/src-d/gitbase/issues/563","repository_url":"https://api.github.com/repos/src-d/gitbase","labels_url":"https://api.github.com/repos/src-d/gitbase/issues/563/labels{/name}","comments_url":"https://api.github.com/repos/src-d/gitbase/issues/563/comments","events_url":"https://api.github.com/repos/src-d/gitbase/issues/563/events","html_url":"https://github.com/src-d/gitbase/issues/563","id":368914213,"node_id":"MDU6SXNzdWUzNjg5MTQyMTM=","number":563,"title":"Progress counter on indexes is wrong (per-partition, not global)","user":{"login":"campoy","id":2237452,"node_id":"MDQ6VXNlcjIyMzc0NTI=","avatar_url":"https://avatars3.githubusercontent.com/u/2237452?v=4","gravatar_id":"","url":"https://api.github.com/users/campoy","html_url":"https://github.com/campoy","followers_url":"https://api.github.com/users/campoy/followers","following_url":"https://api.github.com/users/campoy/following{/other_user}","gists_url":"https://api.github.com/users/campoy/gists{/gist_id}","starred_url":"https://api.github.com/users/campoy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/campoy/subscriptions","organizations_url":"https://api.github.com/users/campoy/orgs","repos_url":"https://api.github.com/users/campoy/repos","events_url":"https://api.github.com/users/campoy/events{/privacy}","received_events_url":"https://api.github.com/users/campoy/received_events","type":"User","site_admin":false},"labels":[{"id":445238580,"node_id":"MDU6TGFiZWw0NDUyMzg1ODA=","url":"https://api.github.com/repos/src-d/gitbase/labels/bug","name":"bug","color":"ee0701","default":true}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/src-d/gitbase/milestones/9","html_url":"https://github.com/src-d/gitbase/milestone/9","labels_url":"https://api.github.com/repos/src-d/gitbase/milestones/9/labels","id":3743489,"node_id":"MDk6TWlsZXN0b25lMzc0MzQ4OQ==","number":9,"title":"v0.17.0","description":null,"creator":{"login":"ajnavarro","id":1196465,"node_id":"MDQ6VXNlcjExOTY0NjU=","avatar_url":"https://avatars3.githubusercontent.com/u/1196465?v=4","gravatar_id":"","url":"https://api.github.com/users/ajnavarro","html_url":"https://github.com/ajnavarro","followers_url":"https://api.github.com/users/ajnavarro/followers","following_url":"https://api.github.com/users/ajnavarro/following{/other_user}","gists_url":"https://api.github.com/users/ajnavarro/gists{/gist_id}","starred_url":"https://api.github.com/users/ajnavarro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ajnavarro/subscriptions","organizations_url":"https://api.github.com/users/ajnavarro/orgs","repos_url":"https://api.github.com/users/ajnavarro/repos","events_url":"https://api.github.com/users/ajnavarro/events{/privacy}","received_events_url":"https://api.github.com/users/ajnavarro/received_events","type":"User","site_admin":false},"open_issues":0,"closed_issues":1,"state":"open","created_at":"2018-10-16T14:55:02Z","updated_at":"2018-10-17T14:29:09Z","due_on":null,"closed_at":null},"comments":7,"created_at":"2018-10-11T01:19:56Z","updated_at":"2018-10-17T14:29:09Z","closed_at":"2018-10-17T14:29:09Z","author_association":"MEMBER","body":"I'm building an index on a really large dataset (the 96 repos with the most commits under google's github org).\r\n\r\nRan this:\r\n\r\n```sql\r\nCREATE INDEX file_by_lang ON files USING pilosa (LANGUAGE(file_path, blob_content));\r\n```\r\n\r\nAnd it started creating it right away, at 22:53:32.\r\n\r\n```\r\ntime=\"2018-10-10T22:53:32Z\" level=debug msg=\"executing query\" query=\"\\nCREATE INDEX file_by_lang ON files USING pilosa (LANGUAGE(file_path, blob_content));\\n\"\r\ntime=\"2018-10-10T22:53:32Z\" level=info msg=\"starting to save the index\" async=true driver=pilosa id=file_by_lang\r\ntime=\"2018-10-10T22:53:33Z\" level=debug msg=\"still creating index\" driver=pilosa duration=1.467891121s id=file_by_lang rows=10000\r\n```\r\n\r\nAt 01:00:51 (that's over one hour later) I see this in the logs:\r\n\r\n```\r\nnohup.out:time=\"2018-10-11T01:00:51Z\" level=debug msg=\"still creating index\" driver=pilosa duration=1.394008128s id=file_by_lang rows=33070000\r\nnohup.out:time=\"2018-10-11T01:00:59Z\" level=debug msg=\"still creating index\" driver=pilosa duration=5.915422321s id=file_by_lang rows=10000\r\n```\r\n\r\nWAT? After one hour of computing the index simply restarted and the rows went down to zero?\r\n\r\nThere's no other logs in between, so no other query was made that could have dropped the previous index or anything like that.\r\n\r\nI've also considered whether this is some kind of overflow, but I don't see how.  33070000 is close to 33554432 (2^25) but since the increase of rows is 10000 I would have expected to see 33080000.\r\n\r\nAny ideas on what might be going on in here?\r\n\r\nYou can find the logs attached: [file_by_lang.log](https://github.com/src-d/gitbase/files/2466988/file_by_lang.log)\r\n\r\nAlso, if you prefer to see *all of the logs* in my session, they're right here: [nohup.log](https://github.com/src-d/gitbase/files/2466993/nohup.log)\r\n\r\n\r\n\r\n"}}
{"actor":{"display_login":"ajnavarro"}}