{"action":"created","issue":{"url":"https://api.github.com/repos/freelawproject/courtlistener/issues/822","repository_url":"https://api.github.com/repos/freelawproject/courtlistener","labels_url":"https://api.github.com/repos/freelawproject/courtlistener/issues/822/labels{/name}","comments_url":"https://api.github.com/repos/freelawproject/courtlistener/issues/822/comments","events_url":"https://api.github.com/repos/freelawproject/courtlistener/issues/822/events","html_url":"https://github.com/freelawproject/courtlistener/pull/822","id":315526597,"number":822,"title":"Add feature: Recommendations for Opinons based on MoreLikeThis (WIP)","user":{"login":"mschwarzer","id":7977470,"avatar_url":"https://avatars1.githubusercontent.com/u/7977470?v=4","gravatar_id":"","url":"https://api.github.com/users/mschwarzer","html_url":"https://github.com/mschwarzer","followers_url":"https://api.github.com/users/mschwarzer/followers","following_url":"https://api.github.com/users/mschwarzer/following{/other_user}","gists_url":"https://api.github.com/users/mschwarzer/gists{/gist_id}","starred_url":"https://api.github.com/users/mschwarzer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mschwarzer/subscriptions","organizations_url":"https://api.github.com/users/mschwarzer/orgs","repos_url":"https://api.github.com/users/mschwarzer/repos","events_url":"https://api.github.com/users/mschwarzer/events{/privacy}","received_events_url":"https://api.github.com/users/mschwarzer/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-04-18T15:19:26Z","updated_at":"2018-04-20T23:31:13Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/freelawproject/courtlistener/pulls/822","html_url":"https://github.com/freelawproject/courtlistener/pull/822","diff_url":"https://github.com/freelawproject/courtlistener/pull/822.diff","patch_url":"https://github.com/freelawproject/courtlistener/pull/822.patch"},"body":"Hi all,\r\n\r\nI'd like to contribute a recommender system to CL. Apache Solr is already in use and therefore the most straight-forward approach would be to implement a recommendation features based on Solr's [MoreLikeThis](https://lucene.apache.org/solr/guide/6_6/morelikethis.html) (MLT). But before I continue with the implementation we should discuss the details.\r\n\r\nIn order to use MLT, the `solrconfig.xml` from the corresponding cores (collection1, ...) needs to be extended with a `RequestHandler` for MLT:\r\n\r\n```\r\n<requestHandler name=\"/mlt\" class=\"solr.MoreLikeThisHandler\">\r\n   <lst name=\"defaults\">\r\n     <str name=\"echoParams\">explicit</str>\r\n     <str name=\"wt\">xml</str>\r\n     <str name=\"df\">text</str>\r\n   </lst>\r\n</requestHandler>\r\n```\r\n\r\nInstead of directly querying Solr and displaying the MLT recommendations, I choose to pre-compute the recommendation and store them in the database. This is mainly because I plan to test other and more sophisticated recommender algorithms in the future. Other algorithms that generate the recommendations externally will be easier to integrate with this approach (e.g. import recommendations from file).\r\n\r\nTesting different recommender algorithms also requires some kind of A/B testing feature. Does such thing already exist for CL? Or do you already use other tracking systems which would allow a proper evaluation?\r\n\r\nThis PR is still WIP and I'm happy to receive your feedback!\r\n\r\nBest,\r\nMalte\r\n"},"comment":{"url":"https://api.github.com/repos/freelawproject/courtlistener/issues/comments/383247300","html_url":"https://github.com/freelawproject/courtlistener/pull/822#issuecomment-383247300","issue_url":"https://api.github.com/repos/freelawproject/courtlistener/issues/822","id":383247300,"user":{"login":"mlissner","id":236970,"avatar_url":"https://avatars3.githubusercontent.com/u/236970?v=4","gravatar_id":"","url":"https://api.github.com/users/mlissner","html_url":"https://github.com/mlissner","followers_url":"https://api.github.com/users/mlissner/followers","following_url":"https://api.github.com/users/mlissner/following{/other_user}","gists_url":"https://api.github.com/users/mlissner/gists{/gist_id}","starred_url":"https://api.github.com/users/mlissner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mlissner/subscriptions","organizations_url":"https://api.github.com/users/mlissner/orgs","repos_url":"https://api.github.com/users/mlissner/repos","events_url":"https://api.github.com/users/mlissner/events{/privacy}","received_events_url":"https://api.github.com/users/mlissner/received_events","type":"User","site_admin":false},"created_at":"2018-04-20T23:31:13Z","updated_at":"2018-04-20T23:31:13Z","author_association":"OWNER","body":"Apologies for the slow reply. I wanted some time to digest, and then I got busy with other things, and now here I finally am. Please don't take my delay as a lack of interest. This is very cool and if you look at https://github.com/freelawproject/courtlistener/issues/172#issue-38737184 you'll see I've been thinking about this as far back as 2010!\r\n\r\nSo, some thoughts after reading through it:\r\n\r\n1. First, the obvious: Did you test this? Does it work?\r\n\r\n1. It it's possible it'd be very powerful to have a link at the bottom of the sidebar section that said, \"See more similar docs\" or whatever, and if clicked, it'd do a MLT query that showed results in the search UI. \r\n\r\n    From there, people could tweak the MLT results to be in a specific jurisdiction, or during a specific date range, etc, right? That's potentially very powerful.\r\n\r\n1. The decision to pre-compute these is interesting. Do you know what kind of performance to expect? \r\n\r\n    You say you chose to pre-compute because:\r\n\r\n    > ...I plan to test other and more sophisticated recommender algorithms in the future. Other algorithms that generate the recommendations externally will be easier to integrate with this approach (e.g. import recommendations from file).\r\n\r\n    Can you say more about this? I assume you're imagining they'd get stored in the DB alongside the MLT results, but:\r\n\r\n    - How would the front end display this?\r\n    - Why are those easier to pre-compute?\r\n    - Why are they \"easier to integrate\"? \r\n\r\n1. Can this be modified to do recommendations for different types of things? For example, I could see this being pretty cool for RECAP documents in addition to opinions. If we go down this road, the [Django Content Types system][ct] might be useful. (It might be useful architecturally anyway, if we think we'd eventually want to enable MLT for other types of objects.)\r\n\r\n1. Does MLT have any Solr index requirements? Are we meeting them already?\r\n\r\nThat's probably plenty to discuss, but feel free to jump in the slack if you want to chat about this stuff there as well.\r\n\r\n[ct]: https://docs.djangoproject.com/en/2.0/ref/contrib/contenttypes/"}}
{"actor":{"display_login":"mlissner"}}