However, the Odai's older brother Nobumoto MIZUNO who succeeded the Mizuno clan after the death of Tadamasa broke relations with the Imagawa clan who was the master of the Matsudaira family and followed Oda clan in 1544, Odai was divorced by Hirotada who worried about relations with the Imagawa family and returned to Kariya-jo Castle (Kariya City) of the Mizuno family in Mikawa Province.
しかし忠政の死後水野氏を継いだ於大の兄水野信元が1544年に松平家の主君今川氏と絶縁して織田氏に従ったため、於大は今川家との関係を慮った広忠により離縁され、実家水野家の・三河国刈谷城（刈谷市）に返された。