He later married, renounced his religion, and lived in Nagasaki, but faced many ordeals; Yoshiaki OTOMO and Harunobu ARIMA who were his cousins hated him, a retainer of Harunobu assaulted him, and he died broken-hearted.
彼はその後結婚し、信仰も捨てて長崎で暮らしていたが、後半生はさまざまな苦難に会い、従兄弟にあたる大村喜前や有馬晴信からも疎まれ、晴信の家臣に暴行され、失意のうちにこの世を去った。