During the period of Yoshiteru, they exclusively controlled the shogunate government with Nagayoshi MIYOSHI who increased in power after the decline of the Hosokawa clan, but later the Ise clan conflicted with Nagayoshi and lost their position, weakening their power.
義輝の代に、細川氏の没落後台頭した三好長慶と共に幕政を壟断したものの長慶と対立し失脚したため勢力を弱めた。