{"action":"created","issue":{"url":"https://api.github.com/repos/facebookresearch/faiss/issues/580","repository_url":"https://api.github.com/repos/facebookresearch/faiss","labels_url":"https://api.github.com/repos/facebookresearch/faiss/issues/580/labels{/name}","comments_url":"https://api.github.com/repos/facebookresearch/faiss/issues/580/comments","events_url":"https://api.github.com/repos/facebookresearch/faiss/issues/580/events","html_url":"https://github.com/facebookresearch/faiss/issues/580","id":357655443,"node_id":"MDU6SXNzdWUzNTc2NTU0NDM=","number":580,"title":"Does GpuIndexIVFFlat use extra memory","user":{"login":"woniu17","id":6073027,"node_id":"MDQ6VXNlcjYwNzMwMjc=","avatar_url":"https://avatars1.githubusercontent.com/u/6073027?v=4","gravatar_id":"","url":"https://api.github.com/users/woniu17","html_url":"https://github.com/woniu17","followers_url":"https://api.github.com/users/woniu17/followers","following_url":"https://api.github.com/users/woniu17/following{/other_user}","gists_url":"https://api.github.com/users/woniu17/gists{/gist_id}","starred_url":"https://api.github.com/users/woniu17/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/woniu17/subscriptions","organizations_url":"https://api.github.com/users/woniu17/orgs","repos_url":"https://api.github.com/users/woniu17/repos","events_url":"https://api.github.com/users/woniu17/events{/privacy}","received_events_url":"https://api.github.com/users/woniu17/received_events","type":"User","site_admin":false},"labels":[{"id":534246314,"node_id":"MDU6TGFiZWw1MzQyNDYzMTQ=","url":"https://api.github.com/repos/facebookresearch/faiss/labels/question","name":"question","color":"cc317c","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-09-06T13:29:05Z","updated_at":"2018-09-13T12:32:07Z","closed_at":null,"author_association":"NONE","body":"I try to run GpuIndexIVFFlat with 1 million \\* 512 \\* sizeof(float) vectors using code `tutorial/cpp/4-GPU.cpp `. ~~It seems to use more than 2G memory not including the default 18% temp memory. It actually uses 3G+ memory.~~ So I doubt that GpuIndexIVFFlat uses extra memory for some reason. Is there someone  that can explain it? Thanks!\r\n\r\nBy the way, it will fail to build `tutorial/cpp/4-GPU.cpp` and `tutorial/cpp/5-Multiple-GPUs.cpp` using CMake  because of the missing of libgpufaiss, it may need to change CMakeLists.txt to fix it.\r\n\r\n| trainning set | adding set | random vectors using GPU memory  | real vectors using GPU memory |\r\n| ------ | ------ | ------ | ------ |\r\n| 1 million | 1 million |  1490 MB | 1556 MB |\r\n| 3.5 million | 3.5 million |  4668 MB | 4886 MB |\r\n| 5 million | 5 million |  6446 MB | 8188 MB |\r\n| 5 million | 1 million |  7212 MB | 7948 MB |\r\n\r\n**random vectors** :  vectors created by drand48() function\r\n**real vectors** : vectors created from images\r\n**trainning set** : number of vectors using for train()\r\n**adding set** : number of vectors using for add()\r\n- vector dimention is 512 and useFloat16IVFStorage, so every vector is 1 KB.\r\n- the IVF has 8192 centroids \r\n\r\nI doubt the reason of using different amount of GPU memory is because of difference distribution  of vectors on centroids."},"comment":{"url":"https://api.github.com/repos/facebookresearch/faiss/issues/comments/420989884","html_url":"https://github.com/facebookresearch/faiss/issues/580#issuecomment-420989884","issue_url":"https://api.github.com/repos/facebookresearch/faiss/issues/580","id":420989884,"node_id":"MDEyOklzc3VlQ29tbWVudDQyMDk4OTg4NA==","user":{"login":"hlzz","id":4837640,"node_id":"MDQ6VXNlcjQ4Mzc2NDA=","avatar_url":"https://avatars1.githubusercontent.com/u/4837640?v=4","gravatar_id":"","url":"https://api.github.com/users/hlzz","html_url":"https://github.com/hlzz","followers_url":"https://api.github.com/users/hlzz/followers","following_url":"https://api.github.com/users/hlzz/following{/other_user}","gists_url":"https://api.github.com/users/hlzz/gists{/gist_id}","starred_url":"https://api.github.com/users/hlzz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hlzz/subscriptions","organizations_url":"https://api.github.com/users/hlzz/orgs","repos_url":"https://api.github.com/users/hlzz/repos","events_url":"https://api.github.com/users/hlzz/events{/privacy}","received_events_url":"https://api.github.com/users/hlzz/received_events","type":"User","site_admin":false},"created_at":"2018-09-13T12:32:07Z","updated_at":"2018-09-13T12:32:07Z","author_association":"CONTRIBUTOR","body":"@woniu17 do you have the cmake error message? Thanks."}}
{"actor":{"display_login":"hlzz"}}