When Terunao died of illness without an heir in 1607, Yoshinao became the head of the family again, and was given the order to find and foster someone who was to be an heir among relatives; then he adopted Naouji ISSHIKI (Satte-Isshiki clan), son of Naotame SUGIURA who was his grandson from a daughter married into another family.
慶長12年（1607年）に照直が嗣子がないまま病死し、そのため、義直は再び家督に復し、嗣子となるべき血族を探し養うよう恩命を受け、外孫の杉浦直為の子の一色直氏(幸手一色氏)を養子にした。