Although he served as a jodai (keeper) of Kokura Castle in Buzen, his relationship with his older brother Tadaoki deteriorated and he therefore left his older brother and lived in seclusion in Kyoto.
豊前小倉城代も勤めたが、兄・忠興と不仲になり、兄から離れて京都で隠棲した。