The Ise clan had originally served as Mandokoro Shitsuji (chief of the Mandokoro, the Administrative Board) of the Muromachi bakufu (Japanese feudal government headed by a shogun), but had gone into decline after his grandfather, Sadataka, and his father, Sadayoshi, were defeated and killed in 1563 in conflict with the Miyoshi clan.
伊勢氏は元は室町幕府の政所執事を務めた家柄であるが、祖父の貞孝と父の貞良が永禄6年（1563年）に三好氏との抗争に敗れ戦死して以来、衰退・没落していた。