{"action":"closed","issue":{"url":"https://api.github.com/repos/dfcf93/MSMARCOV2/issues/21","repository_url":"https://api.github.com/repos/dfcf93/MSMARCOV2","labels_url":"https://api.github.com/repos/dfcf93/MSMARCOV2/issues/21/labels{/name}","comments_url":"https://api.github.com/repos/dfcf93/MSMARCOV2/issues/21/comments","events_url":"https://api.github.com/repos/dfcf93/MSMARCOV2/issues/21/events","html_url":"https://github.com/dfcf93/MSMARCOV2/issues/21","id":388639665,"node_id":"MDU6SXNzdWUzODg2Mzk2NjU=","number":21,"title":"BM25 relevance values for top 1000 eval/dev?","user":{"login":"amirj","id":1645137,"node_id":"MDQ6VXNlcjE2NDUxMzc=","avatar_url":"https://avatars2.githubusercontent.com/u/1645137?v=4","gravatar_id":"","url":"https://api.github.com/users/amirj","html_url":"https://github.com/amirj","followers_url":"https://api.github.com/users/amirj/followers","following_url":"https://api.github.com/users/amirj/following{/other_user}","gists_url":"https://api.github.com/users/amirj/gists{/gist_id}","starred_url":"https://api.github.com/users/amirj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amirj/subscriptions","organizations_url":"https://api.github.com/users/amirj/orgs","repos_url":"https://api.github.com/users/amirj/repos","events_url":"https://api.github.com/users/amirj/events{/privacy}","received_events_url":"https://api.github.com/users/amirj/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-12-07T12:42:25Z","updated_at":"2018-12-12T21:32:25Z","closed_at":"2018-12-12T21:32:25Z","author_association":"NONE","body":"In the documents:\r\n\r\n> We collected all unique passages(without any normalization) to make a pool of ~8.8 million unique passages. Then, for each query from the existing MSMARCO splits(train,dev, and eval) we ran a standard BM25 to produce 1000 relevant passages. **These were ordered by random so each query now has 1000 corresponding passages.** \r\n\r\nWhy you ordered the top1000 retrieved docs **by random** and didn't store the BM25 relevance values?\r\n"}}
{"actor":{"display_login":"dfcf93"}}