{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/6973","repository_url":"https://api.github.com/repos/scikit-learn/scikit-learn","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/6973/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/6973/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/6973/events","html_url":"https://github.com/scikit-learn/scikit-learn/pull/6973","id":164735691,"number":6973,"title":"Added BM25Transformer and BM25Vectorizer to feature_extraction.text. …","user":{"login":"BasilBeirouti","id":10292259,"avatar_url":"https://avatars.githubusercontent.com/u/10292259?v=3","gravatar_id":"","url":"https://api.github.com/users/BasilBeirouti","html_url":"https://github.com/BasilBeirouti","followers_url":"https://api.github.com/users/BasilBeirouti/followers","following_url":"https://api.github.com/users/BasilBeirouti/following{/other_user}","gists_url":"https://api.github.com/users/BasilBeirouti/gists{/gist_id}","starred_url":"https://api.github.com/users/BasilBeirouti/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BasilBeirouti/subscriptions","organizations_url":"https://api.github.com/users/BasilBeirouti/orgs","repos_url":"https://api.github.com/users/BasilBeirouti/repos","events_url":"https://api.github.com/users/BasilBeirouti/events{/privacy}","received_events_url":"https://api.github.com/users/BasilBeirouti/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2016-07-10T21:41:37Z","updated_at":"2016-07-13T01:26:23Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/pulls/6973","html_url":"https://github.com/scikit-learn/scikit-learn/pull/6973","diff_url":"https://github.com/scikit-learn/scikit-learn/pull/6973.diff","patch_url":"https://github.com/scikit-learn/scikit-learn/pull/6973.patch"},"body":"<!--\r\nThanks for contributing a pull request! Please ensure you have taken a look at\r\nthe contribution guidelines: https://github.com/scikit-learn/scikit-learn/blob/master/CONTRIBUTING.md#Contributing-Pull-Requests\r\n-->\r\n#### Reference Issue\r\n<!-- Example: Fixes #1234 -->\r\n\r\n\r\n#### What does this implement/fix? Explain your changes.\r\n\r\n\r\n#### Any other comments?\r\n\r\n\r\n<!--\r\nPlease be aware that we are a loose team of volunteers so patience is\r\nnecessary; assistance handling other issues is very welcome. We value\r\nall user contributions, no matter how minor they are. If we are slow to\r\nreview, either the pull request needs some benchmarking, tinkering,\r\nconvincing, etc. or more likely the reviewers are simply busy. In either\r\ncase, we ask for your understanding during the review process.\r\nFor more information, see our FAQ on this topic:\r\nhttp://scikit-learn.org/dev/faq.html#why-is-my-pull-request-not-getting-any-attention.\r\n\r\nThanks for contributing!\r\n-->\r\n\r\n…Also added a test to check validity of BM25 calculation."},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/232229682","html_url":"https://github.com/scikit-learn/scikit-learn/pull/6973#issuecomment-232229682","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/6973","id":232229682,"user":{"login":"jnothman","id":78827,"avatar_url":"https://avatars.githubusercontent.com/u/78827?v=3","gravatar_id":"","url":"https://api.github.com/users/jnothman","html_url":"https://github.com/jnothman","followers_url":"https://api.github.com/users/jnothman/followers","following_url":"https://api.github.com/users/jnothman/following{/other_user}","gists_url":"https://api.github.com/users/jnothman/gists{/gist_id}","starred_url":"https://api.github.com/users/jnothman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jnothman/subscriptions","organizations_url":"https://api.github.com/users/jnothman/orgs","repos_url":"https://api.github.com/users/jnothman/repos","events_url":"https://api.github.com/users/jnothman/events{/privacy}","received_events_url":"https://api.github.com/users/jnothman/received_events","type":"User","site_admin":false},"created_at":"2016-07-13T01:26:23Z","updated_at":"2016-07-13T01:26:23Z","body":"the method sign doesn't exist for sparse matrices, nor for numpy arrays\nAFAIK. Do you just mean (X >= 0)?\n\nThe other problem relates to reshape requiring the overall number of\nelements being the same, and apparently (I've not checked why this may be\nin your code) X.sum(axis=1) does not (always) have n_samples elements.\n\nOn 13 July 2016 at 04:38, Basil Beirouti <notifications@github.com> wrote:\n\n> Hi Joel,\n>\n> I can see now that these are the same tests that are already included in\n> the scikit-learn source code, but I'm still not sure how to run them. Is\n> there a guide or document online that explains this?\n>\n> Most of the errors that are being thrown don't make any sense to me.\n>\n> For example, why is the method sign not found here?\n> ======================================================================\n> ERROR: sklearn.feature_extraction.tests.test_text.test_bm25_no_smoothing\n> ----------------------------------------------------------------------\n> Traceback (most recent call last):\n> File\n>\n> \"/home/travis/build/scikit-learn/scikit-learn/testvenv/local/lib/python2.7/site-packages/nose/case.py\",\n> line 197, in runTest\n> self.test(*self.arg)\n> File\n>\n> \"/home/travis/sklearn_build_ubuntu/scikit-learn/sklearn/feature_extraction/tests/test_text.py\",\n> line 316, in test_bm25_no_smoothing\n> bm25_act = tr.fit_transform(X).toarray()\n> File \"/home/travis/sklearn_build_ubuntu/scikit-learn/sklearn/base.py\",\n> line 471, in fit_transform\n> return self.fit(X, **fit_params).transform(X)\n> File\n>\n> \"/home/travis/sklearn_build_ubuntu/scikit-learn/sklearn/feature_extraction/text.py\",\n> line 1045, in transform\n> binary = X.sign()\n> File \"/usr/lib/python2.7/dist-packages/scipy/sparse/base.py\", line 384,\n> in __getattr__\n> raise AttributeError(attr + \" not found\")\n> AttributeError: sign not found\n>\n>\n> Why is it complaining that lengths must be the same size as array X?\n> Lengths is a intermediate variable that I am using to get document lengths,\n> obviously it should be of size n_samples not n_features x n_samples.\n>\n> ======================================================================\n> ERROR:\n> sklearn.tests.test_common.test_non_meta_estimators('BM25Transformer',\n> <class 'sklearn.feature_extraction.text.BM25Transformer'>)\n> ----------------------------------------------------------------------\n> Traceback (most recent call last):\n> File\n>\n> \"/home/travis/build/scikit-learn/scikit-learn/testvenv/local/lib/python2.7/site-packages/nose/case.py\",\n> line 197, in runTest\n> self.test(*self.arg)\n> File\n>\n> \"/home/travis/sklearn_build_ubuntu/scikit-learn/sklearn/utils/estimator_checks.py\",\n> line 520, in check_transformer_general\n> _check_transformer(name, Transformer, X, y)\n> File\n>\n> \"/home/travis/sklearn_build_ubuntu/scikit-learn/sklearn/utils/estimator_checks.py\",\n> line 569, in _check_transformer\n> transformer.fit(X, y_)\n> File\n>\n> \"/home/travis/sklearn_build_ubuntu/scikit-learn/sklearn/feature_extraction/text.py\",\n> line 999, in fit\n> lengths = np.array(X.sum(axis=1)).reshape((n_samples,))\n> ValueError: total size of new array must be unchanged\n>\n> I'm a little overwhelmed and not sure how to start fixing this... I also\n> found some bugs in my pull request outside of anything raised by these\n> tests which I've already fixed. I would really appreciate some direction.\n>\n> Sincerely,\n> Basil Beirouti\n>\n> —\n> You are receiving this because you commented.\n> Reply to this email directly, view it on GitHub\n> <https://github.com/scikit-learn/scikit-learn/pull/6973#issuecomment-232139412>,\n> or mute the thread\n> <https://github.com/notifications/unsubscribe/AAEz62k0WyBK_BMvO_zVcCoY_0YoOyuYks5qU98agaJpZM4JI7KN>\n> .\n>\n"}}
{"actor":{"display_login":"jnothman"}}