In 1885, about a half of the precincts of the Rinkyu-ji temple which included Rakushiken and a guest hall were returned to the Department of the Imperial Household and became a part of the Shugakuin Imperial Villa.
1885年（明治18年）、楽只軒（らくしけん）と客殿を含む、林丘寺境内の約半分が宮内省に返還され、修学院離宮の一部となった。