Because of such difficulties Tsunakata lost interest in politics and retired and let his adopted son Tsunaeda KUTSUKI to take over as head of the family on July 5, 1821; he served as the guardian of his son afterward.
このような多難な中で政務への関心を失った綱方は、文政4年（1821年）6月6日に家督を養子の朽木綱条に譲って隠居し、以後はその後見を務めた。