{"action":"created","issue":{"url":"https://api.github.com/repos/iterative/dvc/issues/1241","repository_url":"https://api.github.com/repos/iterative/dvc","labels_url":"https://api.github.com/repos/iterative/dvc/issues/1241/labels{/name}","comments_url":"https://api.github.com/repos/iterative/dvc/issues/1241/comments","events_url":"https://api.github.com/repos/iterative/dvc/issues/1241/events","html_url":"https://github.com/iterative/dvc/issues/1241","id":372048426,"node_id":"MDU6SXNzdWUzNzIwNDg0MjY=","number":1241,"title":"assert inode >=0","user":{"login":"satya-venneti","id":33077272,"node_id":"MDQ6VXNlcjMzMDc3Mjcy","avatar_url":"https://avatars0.githubusercontent.com/u/33077272?v=4","gravatar_id":"","url":"https://api.github.com/users/satya-venneti","html_url":"https://github.com/satya-venneti","followers_url":"https://api.github.com/users/satya-venneti/followers","following_url":"https://api.github.com/users/satya-venneti/following{/other_user}","gists_url":"https://api.github.com/users/satya-venneti/gists{/gist_id}","starred_url":"https://api.github.com/users/satya-venneti/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/satya-venneti/subscriptions","organizations_url":"https://api.github.com/users/satya-venneti/orgs","repos_url":"https://api.github.com/users/satya-venneti/repos","events_url":"https://api.github.com/users/satya-venneti/events{/privacy}","received_events_url":"https://api.github.com/users/satya-venneti/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"efiop","id":5367102,"node_id":"MDQ6VXNlcjUzNjcxMDI=","avatar_url":"https://avatars1.githubusercontent.com/u/5367102?v=4","gravatar_id":"","url":"https://api.github.com/users/efiop","html_url":"https://github.com/efiop","followers_url":"https://api.github.com/users/efiop/followers","following_url":"https://api.github.com/users/efiop/following{/other_user}","gists_url":"https://api.github.com/users/efiop/gists{/gist_id}","starred_url":"https://api.github.com/users/efiop/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/efiop/subscriptions","organizations_url":"https://api.github.com/users/efiop/orgs","repos_url":"https://api.github.com/users/efiop/repos","events_url":"https://api.github.com/users/efiop/events{/privacy}","received_events_url":"https://api.github.com/users/efiop/received_events","type":"User","site_admin":false},"assignees":[{"login":"efiop","id":5367102,"node_id":"MDQ6VXNlcjUzNjcxMDI=","avatar_url":"https://avatars1.githubusercontent.com/u/5367102?v=4","gravatar_id":"","url":"https://api.github.com/users/efiop","html_url":"https://github.com/efiop","followers_url":"https://api.github.com/users/efiop/followers","following_url":"https://api.github.com/users/efiop/following{/other_user}","gists_url":"https://api.github.com/users/efiop/gists{/gist_id}","starred_url":"https://api.github.com/users/efiop/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/efiop/subscriptions","organizations_url":"https://api.github.com/users/efiop/orgs","repos_url":"https://api.github.com/users/efiop/repos","events_url":"https://api.github.com/users/efiop/events{/privacy}","received_events_url":"https://api.github.com/users/efiop/received_events","type":"User","site_admin":false}],"milestone":null,"comments":3,"created_at":"2018-10-19T17:30:37Z","updated_at":"2018-10-19T20:16:19Z","closed_at":"2018-10-19T20:15:00Z","author_association":"NONE","body":"I have ~23,000 files that I want to add to DVC. remote backing is S3 bucket. These are underneath several layers of directories. Eg S3 folder is called S1. I am trying to dvc add S1/X1/Y1/23,000 files.\r\n\r\nFirst attempt failed with \r\n\"assert inode >= 0\".\r\n\r\nI wanted to rule out the possibility of the nested subdirectories causing the issue. So on S3, I created subfolders X1 and Y1.\r\n\r\nNow it fails again.\r\n\r\nHere is the output:\r\ndvc add -v *\r\nDebug: PRAGMA user_version;\r\nDebug: fetched: [(2,)]\r\nDebug: CREATE TABLE IF NOT EXISTS state (inode INTEGER PRIMARY KEY, mtime TEXT NOT NULL, md5 TEXT NOT NULL, timestamp TEXT NOT NULL)\r\nDebug: CREATE TABLE IF NOT EXISTS state_info (count INTEGER)\r\nDebug: CREATE TABLE IF NOT EXISTS link_state (path TEXT PRIMARY KEY, inode INTEGER NOT NULL, mtime TEXT NOT NULL)\r\nDebug: INSERT OR IGNORE INTO state_info (count) SELECT 0 WHERE NOT EXISTS (SELECT * FROM state_info)\r\nDebug: PRAGMA user_version = 2;\r\nSaving '10000.wav' to cache '../../../../.dvc/cache'.\r\nDebug: SELECT count from state_info WHERE rowid=1\r\nDebug: fetched: [(0,)]\r\nDebug: UPDATE state_info SET count = 0 WHERE rowid = 1\r\nError: Traceback (most recent call last):\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/main.py\", line 22, in main\r\n    ret = cmd.run_cmd()\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/command/base.py\", line 40, in run_cmd\r\n    return self.run()\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/command/add.py\", line 10, in run\r\n    self.project.add(target, recursive=self.args.recursive)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/project.py\", line 173, in add\r\n    stage.save()\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/stage.py\", line 393, in save\r\n    out.save()\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/output/local.py\", line 113, in save\r\n    self.info = self.project.cache.local.save(self.path_info)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/remote/local.py\", line 346, in save\r\n    return self._save_file(path_info)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/remote/local.py\", line 297, in _save_file\r\n    md5 = self.state.update(path)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/state.py\", line 300, in update\r\n    return self._do_update(path)[0]\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/state.py\", line 257, in _do_update\r\n    inode = self.inode(path)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/state.py\", line 250, in inode\r\n    return System.inode(path)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/system.py\", line 188, in inode\r\n    assert inode >= 0\r\nAssertionError\r\n\r\nError: Unexpected error:\r\n\r\nHaving any troubles? Hit us up at dvc.org/support, we are always happy to help!\r\n\r\nWith pdb added to /home/ubuntu/.local/lib/python2.7/site-packages/dvc/system.py:\r\n\r\ndvc add -v *\r\nDebug: PRAGMA user_version;\r\nDebug: fetched: [(2,)]\r\nDebug: CREATE TABLE IF NOT EXISTS state (inode INTEGER PRIMARY KEY, mtime TEXT NOT NULL, md5 TEXT NOT NULL, timestamp TEXT NOT NULL)\r\nDebug: CREATE TABLE IF NOT EXISTS state_info (count INTEGER)\r\nDebug: CREATE TABLE IF NOT EXISTS link_state (path TEXT PRIMARY KEY, inode INTEGER NOT NULL, mtime TEXT NOT NULL)\r\nDebug: INSERT OR IGNORE INTO state_info (count) SELECT 0 WHERE NOT EXISTS (SELECT * FROM state_info)\r\nDebug: PRAGMA user_version = 2;\r\nSaving '10000.wav' to cache '../../../../.dvc/cache'.\r\n> /home/ubuntu/.local/lib/python2.7/site-packages/dvc/system.py(190)inode()\r\n-> assert inode >= 0\r\n(Pdb) inode\r\n-7187427192193941402\r\n(Pdb) c\r\nDebug: SELECT count from state_info WHERE rowid=1\r\nDebug: fetched: [(0,)]\r\nDebug: UPDATE state_info SET count = 0 WHERE rowid = 1\r\nError: Traceback (most recent call last):\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/main.py\", line 22, in main\r\n    ret = cmd.run_cmd()\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/command/base.py\", line 40, in run_cmd\r\n    return self.run()\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/command/add.py\", line 10, in run\r\n    self.project.add(target, recursive=self.args.recursive)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/project.py\", line 173, in add\r\n    stage.save()\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/stage.py\", line 393, in save\r\n    out.save()\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/output/local.py\", line 113, in save\r\n    self.info = self.project.cache.local.save(self.path_info)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/remote/local.py\", line 346, in save\r\n    return self._save_file(path_info)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/remote/local.py\", line 297, in _save_file\r\n    md5 = self.state.update(path)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/state.py\", line 300, in update\r\n    return self._do_update(path)[0]\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/state.py\", line 257, in _do_update\r\n    inode = self.inode(path)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/state.py\", line 250, in inode\r\n    return System.inode(path)\r\n  File \"/home/ubuntu/.local/lib/python2.7/site-packages/dvc/system.py\", line 190, in inode\r\n    assert inode >= 0\r\nAssertionError\r\n\r\nError: Unexpected error:\r\n\r\nHaving any troubles? Hit us up at dvc.org/support, we are always happy to help!\r\n\r\n================\r\ndvc --version\r\n0.19.12\r\n\r\npip install\r\n\r\nLinux  4.4.0-1060-aws #69-Ubuntu SMP S x86_64 GNU/Linux"},"comment":{"url":"https://api.github.com/repos/iterative/dvc/issues/comments/431486319","html_url":"https://github.com/iterative/dvc/issues/1241#issuecomment-431486319","issue_url":"https://api.github.com/repos/iterative/dvc/issues/1241","id":431486319,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMTQ4NjMxOQ==","user":{"login":"efiop","id":5367102,"node_id":"MDQ6VXNlcjUzNjcxMDI=","avatar_url":"https://avatars1.githubusercontent.com/u/5367102?v=4","gravatar_id":"","url":"https://api.github.com/users/efiop","html_url":"https://github.com/efiop","followers_url":"https://api.github.com/users/efiop/followers","following_url":"https://api.github.com/users/efiop/following{/other_user}","gists_url":"https://api.github.com/users/efiop/gists{/gist_id}","starred_url":"https://api.github.com/users/efiop/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/efiop/subscriptions","organizations_url":"https://api.github.com/users/efiop/orgs","repos_url":"https://api.github.com/users/efiop/repos","events_url":"https://api.github.com/users/efiop/events{/privacy}","received_events_url":"https://api.github.com/users/efiop/received_events","type":"User","site_admin":false},"created_at":"2018-10-19T20:16:19Z","updated_at":"2018-10-19T20:16:19Z","author_association":"MEMBER","body":"@satya-venneti Patch is merged and I'm preparing a new release right now(should be ready in an hour or so)."}}
{"actor":{"display_login":"efiop"}}