{"action":"closed","issue":{"url":"https://api.github.com/repos/searchivarius/nmslib/issues/64","repository_url":"https://api.github.com/repos/searchivarius/nmslib","labels_url":"https://api.github.com/repos/searchivarius/nmslib/issues/64/labels{/name}","comments_url":"https://api.github.com/repos/searchivarius/nmslib/issues/64/comments","events_url":"https://api.github.com/repos/searchivarius/nmslib/issues/64/events","html_url":"https://github.com/searchivarius/nmslib/issues/64","id":114686071,"number":64,"title":"Manual improvements & changes for the upcoming version","user":{"login":"searchivarius","id":825650,"avatar_url":"https://avatars.githubusercontent.com/u/825650?v=3","gravatar_id":"","url":"https://api.github.com/users/searchivarius","html_url":"https://github.com/searchivarius","followers_url":"https://api.github.com/users/searchivarius/followers","following_url":"https://api.github.com/users/searchivarius/following{/other_user}","gists_url":"https://api.github.com/users/searchivarius/gists{/gist_id}","starred_url":"https://api.github.com/users/searchivarius/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/searchivarius/subscriptions","organizations_url":"https://api.github.com/users/searchivarius/orgs","repos_url":"https://api.github.com/users/searchivarius/repos","events_url":"https://api.github.com/users/searchivarius/events{/privacy}","received_events_url":"https://api.github.com/users/searchivarius/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"searchivarius","id":825650,"avatar_url":"https://avatars.githubusercontent.com/u/825650?v=3","gravatar_id":"","url":"https://api.github.com/users/searchivarius","html_url":"https://github.com/searchivarius","followers_url":"https://api.github.com/users/searchivarius/followers","following_url":"https://api.github.com/users/searchivarius/following{/other_user}","gists_url":"https://api.github.com/users/searchivarius/gists{/gist_id}","starred_url":"https://api.github.com/users/searchivarius/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/searchivarius/subscriptions","organizations_url":"https://api.github.com/users/searchivarius/orgs","repos_url":"https://api.github.com/users/searchivarius/repos","events_url":"https://api.github.com/users/searchivarius/events{/privacy}","received_events_url":"https://api.github.com/users/searchivarius/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/searchivarius/nmslib/milestones/6","html_url":"https://github.com/searchivarius/nmslib/milestones/Server%20version","labels_url":"https://api.github.com/repos/searchivarius/nmslib/milestones/6/labels","id":1446453,"number":6,"title":"Server version","description":"Release a version that can work as a service.","creator":{"login":"searchivarius","id":825650,"avatar_url":"https://avatars.githubusercontent.com/u/825650?v=3","gravatar_id":"","url":"https://api.github.com/users/searchivarius","html_url":"https://github.com/searchivarius","followers_url":"https://api.github.com/users/searchivarius/followers","following_url":"https://api.github.com/users/searchivarius/following{/other_user}","gists_url":"https://api.github.com/users/searchivarius/gists{/gist_id}","starred_url":"https://api.github.com/users/searchivarius/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/searchivarius/subscriptions","organizations_url":"https://api.github.com/users/searchivarius/orgs","repos_url":"https://api.github.com/users/searchivarius/repos","events_url":"https://api.github.com/users/searchivarius/events{/privacy}","received_events_url":"https://api.github.com/users/searchivarius/received_events","type":"User","site_admin":false},"open_issues":5,"closed_issues":6,"state":"open","created_at":"2015-12-06T17:29:00Z","updated_at":"2016-05-05T20:24:33Z","due_on":"2016-05-08T04:00:00Z","closed_at":null},"comments":7,"created_at":"2015-11-02T21:19:48Z","updated_at":"2016-05-05T20:24:33Z","closed_at":"2016-05-05T20:24:33Z","body":"1. Removal of HiddenClone() in Space\r\n2. Recommendation to use DISABLE_COPY_AND_ASSIGN, \r\n3. **but** Now each space should have an explicit constructor\r\n4. Object.id() function clarifications\r\n5. No functions SetQueryTimeParamsInternal & GetQueryTimeParamNames in the Index function.\r\n6. However, you have the function SetQueryTimeParams(...), CreateIndex(params), LoadIndex(string location), SaveIndex(string location);\r\n7 GetParamOptional should now **ALWAYS** have a default value.\r\n9. Index function ` DuplicateData() ` for methods that make data duplicates. \r\n10. In the test workflow specify that SetQueryTimeParams() is called before every execution. No need to call it inside the method any more!!!\r\n11. Removal of an optional dimensionality parameter\r\n12. Removal of some old ineffective methods, including: \"permutation\", \"permutation_incsort\",(permutation_incsort is superseeded by proj_incsort), permutation_vptree.\r\n13. Strongly recommend to replace all pointers members with uniq_ptr.\r\n14. Reminder about pmgr.checkParams()\r\n15. this->ResetQueryTimeParams **must** be called by CreateIndex, or by the upper-level code. For some reason **this->** is needed.\r\n16. SetQueryTimeParams shouldn't have required parameters without defaults.\r\n21. By default, if no query-time parameters are specified each method will get a set of empty query-time parameters. Usually, this means that query-time parameters are reset to their default values. However, this behavior can be overriden (though this is not recommended) by method designers. Explain how it is possible to leave the parameter value unchanged.\r\n17. Additional parameter IdType in Search()\r\n18. dbScanFrac new query time parameter in PermBinVPTree\r\n20. Command line parameters: space specifications remain the same. Method specification just accepts the name. If index is created from scratch index-time parameters are passed via createIndex. If the index is loaded from a location, one uses loadIndex instead. saveIndex specifies the location to save index. Each set of query-time parameters is specified separately using queryTimeParams.\r\n20.5 SaveIndex won't be called by the framework, if the index doesn't exist **one gotcha** \r\n300 Explain why a user may encounter the following data mutation error:\r\n```\r\n016-04-22 00:49:54 main.cc:366 (RunExper) [INFO] Loading an index for test set id 0 using location: sw-graph-sift-ver7-NN=5,efConstruction=400_0\r\n2016-04-22 00:49:54 small_world_rand_ver7.cc:918 (LoadIndex) [ERROR] Check failed: data_[nodeID]->id() == objID\r\n2016-04-22 00:49:54 main.cc:434 (RunExper) [ERROR] Exception: Check failed: A previously saved index is apparently used with a different data set, a different data set split, and/or a different gold standard file! (unexpected object ID 204 for data element with ID 203 expected object ID: 203)\r\n2016-04-22 00:49:54 main.cc:442 (RunExper) [FATAL] Failure due to an exception!\r\n```\r\n22. **To explicitly** specify a set of empty-query time parameters use --queryTimeParams \"\"\r\n23. A new synonym  perm_bin_incsort\r\n24. Describe precision of approximation in the manual\r\n25. appendToResFile became a switch flag. Instead of printProgress, there is a switch noProgressBar\r\n26. Cover options for the server & client.\r\n27. Make sure you updated all the examples & texts where ToString() is mentioned.\r\n28. small_world_rand got a synonym sw-graph *AND a new query-time par.:\" efSearch (and an index-time parameter efConstruction)\r\n29. Explain which methods support LoadIndex and SaveIndex\r\n30. Intra-query ||-ism for sequential search: parameters multiThread & threadQty. Doesn't make sense to use for small data sets and/or very light weight distances.\r\n31. VP-tree also has a random-restart parameter.\r\n32. seq_search can also be named brute_force\r\n33. Explain why we need to read data like this:\r\n```\r\nunique_ptr<DataFileInputState> inpState(space->ReadDataset(dataSet1, vExternIds1, sampleDataPrefix + dataSetFileName, maxNumRec));\r\nspace->UpdateParamsFromFile(*inpState);\r\n```\r\n34. Building with libraries installed in a non-standard location (**actually not everything is so simple, cmake file needs to be modified to be able to turnoff GSL & lshkit:\r\ncmake . -DCMAKE_LIBRARY_PATH=/usr2/srchvrs/addPackages/lib/ -DCMAKE_INCLUDE_PATH=/usr2/srchvrs/addPackages/include  -DBoost_NO_SYSTEM_PATHS=true\r\n\r\n35. Add docs for the word embedding space.\r\n\r\n36.1 Building thrift. **Must use the exact same version!** See the guide here: https://thrift.apache.org/docs/BuildingFromSource On Ubuntu, first do\r\n```\r\nsudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libboost-system-dev libboost-filesystem-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev libboost-thread-dev make\r\n```\r\n36.2 Regarding building the thrift:\r\n```\r\nYou can also disable a lot of unnecessary crap by doing:\r\n./configure --prefix=/home/srchvrs/addPackages --exec-prefix=/home/srchvrs/addPackages --without-java --without-erlang --without-nodejs --without-lua --without-python --without-php --without-ruby --without-haskell --without-go --without-d --with-cpp\r\n```\r\n\r\nFixing a problem with boost:\r\n```\r\n export BOOST_ROOT=/home/srchvrs/addPackagesSrc/boost_1_59_0\r\n```\r\n\r\n37. Building a new version on Windows, see an explanation there:\r\n\r\nhttps://github.com/ot/path_decomposed_tries\r\n\r\n38. Adding a reference to an earlier paper:\r\n\r\n[PDF] Metrized Small World Properties Based Data Structure.\r\nV Krylov, A Logvinov, A Ponomarenko, D Ponomarev - SEDE, 2008 - researchgate.net\r\n100. Code style for newer Eclipse versions:\r\n\r\n39. Setting white-space properties in the new Eclipse.\r\n\r\nC / C++\r\n    Click Window » Preferences\r\n    Expand C/C++ » Code Style\r\n    Click Formatter\r\n    Click the New button to create a new profile, then OK to continue\r\n    Click the Indentation tab\r\n    Under General Settings, set Tab policy to: Spaces only\r\n    Click OK ad nauseum to apply the changes.\r\n\r\n40. The switch --maxCacheGSQty will be replaced with --maxCacheGSRelativeQty This options defines what fraction of the result set we store. By default, the value will be, e.g., 10. This would mean storing 10x as many entries as the number of result entries.\r\n\r\n42. Describe a new (better) and more **configurable** plotting script `genplot_configurable.py`:\r\n41.1 Limits for x & y\r\n41.2 Optional title\r\n41.3 Configurable name of the methodId field\r\n41.4 Configurable axis names \r\n41.5 Configurable line types (can be anything PGFPLOTS package supports colors, line types, etc)\r\n\r\n42. multi-threaded generation of gold-standard cache (should match the number of threads during experimentation)\r\n\r\n43. Better gold-standard cache generation. There is now a coefficient K to memorize K * |size of the result| closest entries.\r\n\r\n200. Explanation for the message:\r\n```\r\n2015-10-11 12:39:45 experiments.h:327 (Execute) [INFO] >>>> Computing effectiveness metrics for small_world_rand\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -2.09754 id = 140154 -> Apr: -2.11169 id = 140154 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.97406 id = 113850 -> Apr: -2.00505 id = 113850 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.88388 id = 102001 -> Apr: -1.89802 id = 102001 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.66678 id = 58445 -> Apr: -1.67824 id = 58445 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.65475 id = 76888 -> Apr: -1.66889 id = 76888 1 - ratio: -0.000202635 diff: -0.000221372\r\n 2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.58057 id = 47669 -> Apr: -1.59471 id = 47669 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.52015 id = 65783 -> Apr: -1.49983 id = 14954 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.46885 id = 14954 -> Apr: -1.39462 id = 25564 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.45484 id = 90204 -> Apr: -1.37854 id = 120613 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.38047 id = 25564 -> Apr: -1.31906 id = 22051 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.36709 id = 120613 -> Apr: -1.20523 id = 101722 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.31842 id = 71704 -> Apr: -1.16612 id = 136738 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.3103 id = 22051 -> Apr: -1.10392 id = 52950 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.19109 id = 101722 -> Apr: -1.09269 id = 16190 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:71 (iterate) [INFO] Ex: -1.15736 id = 136738 -> Apr: -1.03483 id = 13878 1 - ratio: -0.000202635 diff: -0.000221372\r\n2015-10-11 12:39:45 eval_metrics.h:75 (iterate) [FATAL] bug: the approximate query should not return objects that are closer to the query than object returned by (exact) sequential searching! Approx: -1.09269 id = 16190 Exact: -1.09247 id = 52950\r\n```\r\n\r\n"}}