{"content":[{"text":"2","bbox":{"left":0.117647,"top":0.089701,"width":0.01580899999999999,"height":0.021758}},{"text":"Data","bbox":{"left":0.16513,"top":0.089701,"width":0.06731399999999998,"height":0.021758}},{"text":"2.1","bbox":{"left":0.117647,"top":0.130009,"width":0.03370899999999999,"height":0.01813200000000001}},{"text":"Overview","bbox":{"left":0.177751,"top":0.130009,"width":0.10813600000000001,"height":0.01813200000000001}},{"text":"This project was motivated by a desire to improve upon the Recommendation Systems based \n on less detailed data such as that using in the Netﬂix Challenge. A few major problems with \n the Netﬂix data is that few people actively rate movies (bias downwards for anger), all user \n and item characteristics are lost, and there is no ability for the user to indicate preference. \n This blocks oﬀ many user and item-based regressions that can be run to improve the system. \n Therefore, we searched for a more robust and descriptive dataset of ﬁlms. \n We decided to use Letterboxd.com, a small but vibrant ﬁlm rating site. movie rating \n system has additional data that can ameliorate these issues. Letterboxd is a small but grow- \n ing community of ﬁlm fanatics, where users can rate movies, post reviews, and follow/like \n other users and their reviews. Since this is mainly a diary-style rating platform for movie \n enthusiasts, one can expect less sparsity and more truthful review/rating behavior, reducing \n omission errors. In addition, a user can identify themselves by inputting their four ”favorite \n movies.” These favorites give a lot of information about the user supplied by the user him- \n self. Do they love older ﬁlms, horror ﬁlms, Japanese animation, etc. In addition, there is a \n watchlist that many users have that indicates future desire to watch movies. There is also \n more item (movie) information for each entry, such as genre tags, runtime, year and country \n of release, a delineated histogram of ratings (from 0.5 to 5 stars, in 0.5 star steps), and all \n crew and cast names. \n In summary, this data source is much more rich in characteristics for both users and \n movies, with the added bonus of some user input in their ”taste proﬁle”. A negative involves \n having to use Data Scrappers to extract this information, as Letterboxd has no workable \n API. However, all users are in a large directory sorted by activity, and the site is wonderfully \n built with easy html tags for all possible data of interest. Overall, a great data source for in- \n vestigations in rating systems. A sample proﬁle is here https://letterboxd.com/JohnnyMa/.","bbox":{"left":0.117647,"top":0.160262,"width":0.7648599999999999,"height":0.434623}},{"text":"2.2","bbox":{"left":0.117647,"top":0.621275,"width":0.03370899999999999,"height":0.018132000000000037}},{"text":"Scraping Procedure","bbox":{"left":0.177751,"top":0.621275,"width":0.227196,"height":0.018132000000000037}},{"text":"The letterboxd.com website is well adapted for webscraping. Much of the ﬁlm and user info \n stems from the base ”letterboxd.com” URL, and are uniquely identiﬁed throughout. \n The output we want to obtain from scraping is a data frame with users on the rows, ﬁlms \n on the columns, and associated ratings for each user and ﬁlm. We started by generating a list \n of users from letterboxd’s own user list, found https://letterboxd.com/people/popular/here. \n This is a list sorted by ”popularity”, a metric that Letterboxd states is dependent on amount \n of reviews, likes on reviews, and frequency of publication. Since we are taking from the most \n popular users, we are likely skewing our population towards more the more ﬁlm-obsessive \n than your ordinary ﬁlm Joe. However, this is potentially good for Recommendation systems, \n as the data will be less sparse, and these high popularity ﬁlm reviewers can be seen as ”taste \n makers” who spend a lot of time reviewing and thinking about these ﬁlms. \n Once we have a list of users, we can enter their proﬁles using their user ID and being to \n scrape user-related information. This includes their average rating, their number of ratings, \n and up to 4 of their favorite ﬁlms. The favorite ﬁlm data is kept for each of these users and","bbox":{"left":0.117647,"top":0.651527,"width":0.76482,"height":0.252227}},{"text":"2","bbox":{"left":0.495219,"top":0.935507,"width":0.00955199999999995,"height":0.015109999999999957}}],"images_bbox":[],"images_bbox_not_text_overlap":[]}