After being devoted to construction of the Togu-gosho Crown Prince's Palace, he reported to Emperor Meiji on the completion of the palace, but the Emperor said nothing except that the palace was 'too luxurious,' and this shocked him so much that he became physically weak and susceptible to illnesses.
東宮御所の建設に心血を注ぎ、完成の報告を明治天皇に行ったところ、一言「贅沢すぎる」と言われてショックを受け、病気がちとなったという。