{"action":"closed","issue":{"url":"https://api.github.com/repos/facebookresearch/faiss/issues/555","repository_url":"https://api.github.com/repos/facebookresearch/faiss","labels_url":"https://api.github.com/repos/facebookresearch/faiss/issues/555/labels{/name}","comments_url":"https://api.github.com/repos/facebookresearch/faiss/issues/555/comments","events_url":"https://api.github.com/repos/facebookresearch/faiss/issues/555/events","html_url":"https://github.com/facebookresearch/faiss/issues/555","id":346931402,"node_id":"MDU6SXNzdWUzNDY5MzE0MDI=","number":555,"title":"failed: read error","user":{"login":"reshu-b7","id":20737958,"node_id":"MDQ6VXNlcjIwNzM3OTU4","avatar_url":"https://avatars3.githubusercontent.com/u/20737958?v=4","gravatar_id":"","url":"https://api.github.com/users/reshu-b7","html_url":"https://github.com/reshu-b7","followers_url":"https://api.github.com/users/reshu-b7/followers","following_url":"https://api.github.com/users/reshu-b7/following{/other_user}","gists_url":"https://api.github.com/users/reshu-b7/gists{/gist_id}","starred_url":"https://api.github.com/users/reshu-b7/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/reshu-b7/subscriptions","organizations_url":"https://api.github.com/users/reshu-b7/orgs","repos_url":"https://api.github.com/users/reshu-b7/repos","events_url":"https://api.github.com/users/reshu-b7/events{/privacy}","received_events_url":"https://api.github.com/users/reshu-b7/received_events","type":"User","site_admin":false},"labels":[{"id":534246314,"node_id":"MDU6TGFiZWw1MzQyNDYzMTQ=","url":"https://api.github.com/repos/facebookresearch/faiss/labels/question","name":"question","color":"cc317c","default":true}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-08-02T09:28:31Z","updated_at":"2018-08-28T09:11:28Z","closed_at":"2018-08-28T09:11:28Z","author_association":"NONE","body":"I am trying to run faiss search on a big data set having 79241051 vectors of 64 dimension using the [bench_gpu_1bn.py](https://github.com/facebookresearch/faiss/blob/master/benchs/bench_gpu_1bn.py) scrpit . I am running it on 4 P40 Tesla GPUs and have been receiving this error but couldn't understand what could have been causing this. I am able to run the same for IVF4096,Flat and combination of IVF,PQ\r\n\r\n`python bench_gpu.py big_data IVF16384,Flat -nnn 100 -ngpu 4 -altadd -noptables -tempmem $[2*1024*1024*1024] `\r\n\r\n```\r\n$[4*1024*1024*1024] -qbs 100000 -float16\r\nPreparing dataset big_data\r\npreparing base and learning set\r\npreparing query set\r\npreparing ground truth\r\nsizes: B (79241051, 64) Q (100000, 64) T (79241051, 64) gt (100000, 100)\r\ncachefiles:\r\nNone\r\n/data/index/new/cent_big_data_IVF16384.npy\r\n/data/index/new/big_data_IVF16384,Flat.index\r\npreparing resources for 4 GPUs\r\nload /data/index/new/big_data_IVF16384,Flat.index\r\nTraceback (most recent call last):\r\n  File \"bench_gpu.py\", line 789, in <module>\r\n    index = get_populated_index(preproc)\r\n  File \"bench_gpu.py\", line 669, in get_populated_index\r\n    indexall = faiss.read_index(index_cachefile)\r\nRuntimeError: Error in faiss::InvertedLists* faiss::read_InvertedLists(faiss::IOReader*, int) at index_io.cpp:613: Error: 'ret == (n * ails->code_size)' failed: read error\r\n\r\n```"}}
{"actor":{"display_login":"mdouze"}}