This name was used because FUJIWARA no Yukinari, in his later years, lived in seclusion Toentei (north to Heiankyo Ichijo, west to Omiya, northern suburb of Dai-dairi [place of the Imperial Palace and government offices], west to the current Omiyadori Ichijo-agaru, Kamigyo Ward, Kyoto City) which was a mansion belonging to Imperial Prince Yoshiakira, a member of his mother's parents family) and he established Seson-ji Temple in the mansion and his descendants called themselves Sesonji and lived there.
藤原行成が晩年、母の里方の代明親王の邸宅だった桃園第(平安京一条の北、大宮の西、大内裏の北郊で現京都市上京区大宮通一条上ルの西方）に隠棲し、邸宅内に世尊寺を建立し、またその子孫が世尊寺家を名乗り、代々そこを住居としたためこの名を名乗る。