Later, during the Kanno Disturbance, there were troubles such as confrontations with the Ko clan, but during Norifusa's son, Noriaki UESUGI's time, the family became Shugo of Kozuke, Echigo and Izu Provinces.
その後、観応の擾乱では高氏と対立するなど波乱もあったが、憲房の子の上杉憲顕の時代には上野・越後国・伊豆国の守護を兼ねた。