{"action":"created","comment":{"url":"https://api.github.com/repos/xapian/xapian/pulls/comments/66728442","id":66728442,"diff_hunk":"@@ -588,6 +588,114 @@ class XAPIAN_VISIBILITY_DEFAULT BM25Weight : public Weight {\n     double get_sumextra(Xapian::termcount doclen,\n \t\t\tXapian::termcount uniqterms) const;\n     double get_maxextra() const;\n+\n+  protected:\n+\t// BM25+ is disabled.\n+\tbool bm25_plus = 0;\n+};\n+\n+/** Xapian::BM25PlusWeight subclass implementing the BM25+ weighting function.\n+ *\n+ * This class implements the BM25+ Probabilistic Weighting scheme which is\n+ * described by Lv-Zhai in their paper.  BM25+ is useful when very-long\n+ * documents are present in document collection.\n+ *\n+ * BM25+ uses an extra parameter delta(δ). Formally, BM25PlusWeight(delta)\n+ * should be equivalent to BM25Weight(k1, k2, k3, b, min_normlen, delta).\n+ */\n+class XAPIAN_VISIBILITY_DEFAULT BM25PlusWeight : public BM25Weight {\n+\t/// Factor to multiply the document length by.\n+    mutable Xapian::doclength len_factor;\n+\n+    /// Factor combining all the document independent factors.\n+    mutable double termweight;\n+\n+    /// The BM25 parameters.\n+    double param_k1, param_k2, param_k3, param_b;\n+\n+    /// The minimum normalised document length value.\n+    Xapian::doclength param_min_normlen;\n+\n+\t/// Additional parameter delta in the BM25+ formula.\n+    double param_delta;\n+\n+    BM25PlusWeight * clone() const;\n+\n+    void init(double factor);\n+\n+  public:\n+    /** Construct a BM25PlusWeight.\n+     *\n+     *  @param delta  A parameter for pseudo tf value to control the scale\n+     *\t\t   of the tf lower bound. Delta(δ) can be tuned for e.g from 0.0\n+     *\t\t   to 1.5 but BM25+ can still work effectively across collections\n+     *\t\t   with a fixed δ = 1.0. (default 1.0)\n+     */\n+    BM25PlusWeight(double k1, double k2, double k3, double b, double min_normlen, double delta)\n+    : param_k1(k1), param_k2(k2), param_k3(k3), param_b(b),","path":"xapian-core/include/xapian/weight.h","position":48,"original_position":48,"commit_id":"af1c4a5f8de6be5d9864c5b1df6ad67f65b5dd39","original_commit_id":"af1c4a5f8de6be5d9864c5b1df6ad67f65b5dd39","user":{"login":"jaylett","id":80187,"avatar_url":"https://avatars.githubusercontent.com/u/80187?v=3","gravatar_id":"","url":"https://api.github.com/users/jaylett","html_url":"https://github.com/jaylett","followers_url":"https://api.github.com/users/jaylett/followers","following_url":"https://api.github.com/users/jaylett/following{/other_user}","gists_url":"https://api.github.com/users/jaylett/gists{/gist_id}","starred_url":"https://api.github.com/users/jaylett/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaylett/subscriptions","organizations_url":"https://api.github.com/users/jaylett/orgs","repos_url":"https://api.github.com/users/jaylett/repos","events_url":"https://api.github.com/users/jaylett/events{/privacy}","received_events_url":"https://api.github.com/users/jaylett/received_events","type":"User","site_admin":false},"body":"We normally indent the entire initialiser list, including the colon (\"`:`\").","created_at":"2016-06-12T22:03:34Z","updated_at":"2016-06-12T22:03:34Z","html_url":"https://github.com/xapian/xapian/pull/104#discussion_r66728442","pull_request_url":"https://api.github.com/repos/xapian/xapian/pulls/104","_links":{"self":{"href":"https://api.github.com/repos/xapian/xapian/pulls/comments/66728442"},"html":{"href":"https://github.com/xapian/xapian/pull/104#discussion_r66728442"},"pull_request":{"href":"https://api.github.com/repos/xapian/xapian/pulls/104"}}},"pull_request":{"url":"https://api.github.com/repos/xapian/xapian/pulls/104","id":72571960,"html_url":"https://github.com/xapian/xapian/pull/104","diff_url":"https://github.com/xapian/xapian/pull/104.diff","patch_url":"https://github.com/xapian/xapian/pull/104.patch","issue_url":"https://api.github.com/repos/xapian/xapian/issues/104","number":104,"state":"open","locked":false,"title":"Add support for BM25+ weighting function","user":{"login":"ivmarkp","id":17006136,"avatar_url":"https://avatars.githubusercontent.com/u/17006136?v=3","gravatar_id":"","url":"https://api.github.com/users/ivmarkp","html_url":"https://github.com/ivmarkp","followers_url":"https://api.github.com/users/ivmarkp/followers","following_url":"https://api.github.com/users/ivmarkp/following{/other_user}","gists_url":"https://api.github.com/users/ivmarkp/gists{/gist_id}","starred_url":"https://api.github.com/users/ivmarkp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ivmarkp/subscriptions","organizations_url":"https://api.github.com/users/ivmarkp/orgs","repos_url":"https://api.github.com/users/ivmarkp/repos","events_url":"https://api.github.com/users/ivmarkp/events{/privacy}","received_events_url":"https://api.github.com/users/ivmarkp/received_events","type":"User","site_admin":false},"body":"- The original `bm25weight.cc` file that contains implementation of BM25 weighting function is modified to add support for BM25+ weighting function without disturbing existing functionalities of BM25 function. \r\n- Also, modified existing test cases of BM25 to extend test coverage for BM25+ function.\r\n- Added API support in `weight.h` for the same.\r\n\r\nIssue marked as \"FIXME\" about dealing with negative weights in `bm25weight.cc` that Olly mentioned on IRC is fixed by default when BM25+ is used.  BM25+ can never assign a negative termweight as in BM25+ termweight is calculated as `N+1 / df(t)` which translates to `(total doc in collection + 1) / termfreq` in Xapian terminology. Since, term frequency can't possibly be greater than N so `log((total doc in collection + 1) / termfreq)` can never be negative. I'd say it can be regarded as one of the benefits of BM25+ function.\r\n\r\nPlease note that I haven't run any automated tests yet using `make check`. Just wanted to get the code reviewed first. Will make sure to run all the tests after getting feedback.","created_at":"2016-06-03T21:22:13Z","updated_at":"2016-06-12T22:03:34Z","closed_at":null,"merged_at":null,"merge_commit_sha":"db9ab12b6aa76ab5d8eaf868e962f048bc8a7192","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/xapian/xapian/pulls/104/commits","review_comments_url":"https://api.github.com/repos/xapian/xapian/pulls/104/comments","review_comment_url":"https://api.github.com/repos/xapian/xapian/pulls/comments{/number}","comments_url":"https://api.github.com/repos/xapian/xapian/issues/104/comments","statuses_url":"https://api.github.com/repos/xapian/xapian/statuses/af1c4a5f8de6be5d9864c5b1df6ad67f65b5dd39","head":{"label":"ivmarkp:bm25+","ref":"bm25+","sha":"af1c4a5f8de6be5d9864c5b1df6ad67f65b5dd39","user":{"login":"ivmarkp","id":17006136,"avatar_url":"https://avatars.githubusercontent.com/u/17006136?v=3","gravatar_id":"","url":"https://api.github.com/users/ivmarkp","html_url":"https://github.com/ivmarkp","followers_url":"https://api.github.com/users/ivmarkp/followers","following_url":"https://api.github.com/users/ivmarkp/following{/other_user}","gists_url":"https://api.github.com/users/ivmarkp/gists{/gist_id}","starred_url":"https://api.github.com/users/ivmarkp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ivmarkp/subscriptions","organizations_url":"https://api.github.com/users/ivmarkp/orgs","repos_url":"https://api.github.com/users/ivmarkp/repos","events_url":"https://api.github.com/users/ivmarkp/events{/privacy}","received_events_url":"https://api.github.com/users/ivmarkp/received_events","type":"User","site_admin":false},"repo":{"id":53715784,"name":"xapian","full_name":"ivmarkp/xapian","owner":{"login":"ivmarkp","id":17006136,"avatar_url":"https://avatars.githubusercontent.com/u/17006136?v=3","gravatar_id":"","url":"https://api.github.com/users/ivmarkp","html_url":"https://github.com/ivmarkp","followers_url":"https://api.github.com/users/ivmarkp/followers","following_url":"https://api.github.com/users/ivmarkp/following{/other_user}","gists_url":"https://api.github.com/users/ivmarkp/gists{/gist_id}","starred_url":"https://api.github.com/users/ivmarkp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ivmarkp/subscriptions","organizations_url":"https://api.github.com/users/ivmarkp/orgs","repos_url":"https://api.github.com/users/ivmarkp/repos","events_url":"https://api.github.com/users/ivmarkp/events{/privacy}","received_events_url":"https://api.github.com/users/ivmarkp/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/ivmarkp/xapian","description":"Mirror of the Xapian repository.  You're welcome to open pull requests on github (they'll just get merged indirectly).","fork":true,"url":"https://api.github.com/repos/ivmarkp/xapian","forks_url":"https://api.github.com/repos/ivmarkp/xapian/forks","keys_url":"https://api.github.com/repos/ivmarkp/xapian/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ivmarkp/xapian/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ivmarkp/xapian/teams","hooks_url":"https://api.github.com/repos/ivmarkp/xapian/hooks","issue_events_url":"https://api.github.com/repos/ivmarkp/xapian/issues/events{/number}","events_url":"https://api.github.com/repos/ivmarkp/xapian/events","assignees_url":"https://api.github.com/repos/ivmarkp/xapian/assignees{/user}","branches_url":"https://api.github.com/repos/ivmarkp/xapian/branches{/branch}","tags_url":"https://api.github.com/repos/ivmarkp/xapian/tags","blobs_url":"https://api.github.com/repos/ivmarkp/xapian/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ivmarkp/xapian/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ivmarkp/xapian/git/refs{/sha}","trees_url":"https://api.github.com/repos/ivmarkp/xapian/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ivmarkp/xapian/statuses/{sha}","languages_url":"https://api.github.com/repos/ivmarkp/xapian/languages","stargazers_url":"https://api.github.com/repos/ivmarkp/xapian/stargazers","contributors_url":"https://api.github.com/repos/ivmarkp/xapian/contributors","subscribers_url":"https://api.github.com/repos/ivmarkp/xapian/subscribers","subscription_url":"https://api.github.com/repos/ivmarkp/xapian/subscription","commits_url":"https://api.github.com/repos/ivmarkp/xapian/commits{/sha}","git_commits_url":"https://api.github.com/repos/ivmarkp/xapian/git/commits{/sha}","comments_url":"https://api.github.com/repos/ivmarkp/xapian/comments{/number}","issue_comment_url":"https://api.github.com/repos/ivmarkp/xapian/issues/comments{/number}","contents_url":"https://api.github.com/repos/ivmarkp/xapian/contents/{+path}","compare_url":"https://api.github.com/repos/ivmarkp/xapian/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ivmarkp/xapian/merges","archive_url":"https://api.github.com/repos/ivmarkp/xapian/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ivmarkp/xapian/downloads","issues_url":"https://api.github.com/repos/ivmarkp/xapian/issues{/number}","pulls_url":"https://api.github.com/repos/ivmarkp/xapian/pulls{/number}","milestones_url":"https://api.github.com/repos/ivmarkp/xapian/milestones{/number}","notifications_url":"https://api.github.com/repos/ivmarkp/xapian/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ivmarkp/xapian/labels{/name}","releases_url":"https://api.github.com/repos/ivmarkp/xapian/releases{/id}","deployments_url":"https://api.github.com/repos/ivmarkp/xapian/deployments","created_at":"2016-03-12T05:46:48Z","updated_at":"2016-03-12T05:46:55Z","pushed_at":"2016-06-10T20:56:15Z","git_url":"git://github.com/ivmarkp/xapian.git","ssh_url":"git@github.com:ivmarkp/xapian.git","clone_url":"https://github.com/ivmarkp/xapian.git","svn_url":"https://github.com/ivmarkp/xapian","homepage":"https://xapian.org/","size":77225,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_downloads":false,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"xapian:master","ref":"master","sha":"9d2c1f7e401e0535b2ba43f3bbefeba9bfc197b9","user":{"login":"xapian","id":312861,"avatar_url":"https://avatars.githubusercontent.com/u/312861?v=3","gravatar_id":"","url":"https://api.github.com/users/xapian","html_url":"https://github.com/xapian","followers_url":"https://api.github.com/users/xapian/followers","following_url":"https://api.github.com/users/xapian/following{/other_user}","gists_url":"https://api.github.com/users/xapian/gists{/gist_id}","starred_url":"https://api.github.com/users/xapian/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xapian/subscriptions","organizations_url":"https://api.github.com/users/xapian/orgs","repos_url":"https://api.github.com/users/xapian/repos","events_url":"https://api.github.com/users/xapian/events{/privacy}","received_events_url":"https://api.github.com/users/xapian/received_events","type":"Organization","site_admin":false},"repo":{"id":735981,"name":"xapian","full_name":"xapian/xapian","owner":{"login":"xapian","id":312861,"avatar_url":"https://avatars.githubusercontent.com/u/312861?v=3","gravatar_id":"","url":"https://api.github.com/users/xapian","html_url":"https://github.com/xapian","followers_url":"https://api.github.com/users/xapian/followers","following_url":"https://api.github.com/users/xapian/following{/other_user}","gists_url":"https://api.github.com/users/xapian/gists{/gist_id}","starred_url":"https://api.github.com/users/xapian/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xapian/subscriptions","organizations_url":"https://api.github.com/users/xapian/orgs","repos_url":"https://api.github.com/users/xapian/repos","events_url":"https://api.github.com/users/xapian/events{/privacy}","received_events_url":"https://api.github.com/users/xapian/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/xapian/xapian","description":"Mirror of the Xapian repository.  You're welcome to open pull requests on github (they'll just get merged indirectly).","fork":false,"url":"https://api.github.com/repos/xapian/xapian","forks_url":"https://api.github.com/repos/xapian/xapian/forks","keys_url":"https://api.github.com/repos/xapian/xapian/keys{/key_id}","collaborators_url":"https://api.github.com/repos/xapian/xapian/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/xapian/xapian/teams","hooks_url":"https://api.github.com/repos/xapian/xapian/hooks","issue_events_url":"https://api.github.com/repos/xapian/xapian/issues/events{/number}","events_url":"https://api.github.com/repos/xapian/xapian/events","assignees_url":"https://api.github.com/repos/xapian/xapian/assignees{/user}","branches_url":"https://api.github.com/repos/xapian/xapian/branches{/branch}","tags_url":"https://api.github.com/repos/xapian/xapian/tags","blobs_url":"https://api.github.com/repos/xapian/xapian/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/xapian/xapian/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/xapian/xapian/git/refs{/sha}","trees_url":"https://api.github.com/repos/xapian/xapian/git/trees{/sha}","statuses_url":"https://api.github.com/repos/xapian/xapian/statuses/{sha}","languages_url":"https://api.github.com/repos/xapian/xapian/languages","stargazers_url":"https://api.github.com/repos/xapian/xapian/stargazers","contributors_url":"https://api.github.com/repos/xapian/xapian/contributors","subscribers_url":"https://api.github.com/repos/xapian/xapian/subscribers","subscription_url":"https://api.github.com/repos/xapian/xapian/subscription","commits_url":"https://api.github.com/repos/xapian/xapian/commits{/sha}","git_commits_url":"https://api.github.com/repos/xapian/xapian/git/commits{/sha}","comments_url":"https://api.github.com/repos/xapian/xapian/comments{/number}","issue_comment_url":"https://api.github.com/repos/xapian/xapian/issues/comments{/number}","contents_url":"https://api.github.com/repos/xapian/xapian/contents/{+path}","compare_url":"https://api.github.com/repos/xapian/xapian/compare/{base}...{head}","merges_url":"https://api.github.com/repos/xapian/xapian/merges","archive_url":"https://api.github.com/repos/xapian/xapian/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/xapian/xapian/downloads","issues_url":"https://api.github.com/repos/xapian/xapian/issues{/number}","pulls_url":"https://api.github.com/repos/xapian/xapian/pulls{/number}","milestones_url":"https://api.github.com/repos/xapian/xapian/milestones{/number}","notifications_url":"https://api.github.com/repos/xapian/xapian/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/xapian/xapian/labels{/name}","releases_url":"https://api.github.com/repos/xapian/xapian/releases{/id}","deployments_url":"https://api.github.com/repos/xapian/xapian/deployments","created_at":"2010-06-23T15:51:03Z","updated_at":"2016-06-11T12:28:34Z","pushed_at":"2016-06-11T07:54:41Z","git_url":"git://github.com/xapian/xapian.git","ssh_url":"git@github.com:xapian/xapian.git","clone_url":"https://github.com/xapian/xapian.git","svn_url":"https://github.com/xapian/xapian","homepage":"https://xapian.org/","size":88733,"stargazers_count":260,"watchers_count":260,"language":"C++","has_issues":false,"has_downloads":false,"has_wiki":false,"has_pages":false,"forks_count":137,"mirror_url":null,"open_issues_count":19,"forks":137,"open_issues":19,"watchers":260,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/xapian/xapian/pulls/104"},"html":{"href":"https://github.com/xapian/xapian/pull/104"},"issue":{"href":"https://api.github.com/repos/xapian/xapian/issues/104"},"comments":{"href":"https://api.github.com/repos/xapian/xapian/issues/104/comments"},"review_comments":{"href":"https://api.github.com/repos/xapian/xapian/pulls/104/comments"},"review_comment":{"href":"https://api.github.com/repos/xapian/xapian/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/xapian/xapian/pulls/104/commits"},"statuses":{"href":"https://api.github.com/repos/xapian/xapian/statuses/af1c4a5f8de6be5d9864c5b1df6ad67f65b5dd39"}}}}
{"actor":{"display_login":"jaylett"}}