However, when he reached the end of his life, he started to lose the favor of Ieyasu due to decreased silver and gold harvest yields from mines across Japan, and was consecutively dismissed from local governor positions, starting from the governor of Mino.
しかし晩年に入ると、全国鉱山からの金銀採掘量の低下から家康の寵愛を失い、美濃代官をはじめとする代官職を次々と罷免されていくようになる。