When Tsunatoyo became the sixth Shogun, Takakazu began working for the Edo shogunate and was appointed the Maruonando kumikashira (a job whose duty was to manage the assets belonging to the Shogun's family).
綱豊が6代将軍となると直参として江戸詰めととなり、西の丸御納戸組頭に任じられた。