In 1704, when Tsunatoyo moved his residence to the west wing of Edo-jo Castle, Akifusa was selected from among the samurai of Kofu Domain to serve as a retainer of the Tokugawa shogunate; he was also promoted to Junior Fifth Rank, Lower Grade, and given the position of Viceroy of Echizen Province, becoming one of the Tokugawa clan's Chamberlains and having his income increased by 1500 koku.
宝永元年（1704年）の綱豊の江戸城西の丸城入と同時に甲府藩士から幕臣に編入され、従五位下越前国に叙任し、側衆になり、1500石加増。