In 1777, he became a sobayonin; as he got promoted to Jushiinoge (Junior Fourth Rank, Lower Grade), his territory was changed to 20,000 koku of Suruga-Numazu Domain.
安永6年（1777年）には側用人となり、従四位下に昇進すると共に駿河沼津藩2万石に転封となる。