Later he married the Imperial princess Inoe, the princess of Emperor Shomu, in 744, and suddenly he had a lot of chances to be promoted and in 762, he became Chunagon (vice-councilor of state) and he made a great achievement to put down the Rebellion of Fujiwara no Nakamaro and won the trust of Emperor Koken/Emperor Shotoku and was promoted to Dainagon (chief councilor of state) in 766.
その後天平16年（744年）聖武天皇の皇女・井上内親王（いのえのひめみこ）を妃としたことからにわかに昇進が早くなり、天平宝字6年（762年）に中納言となり、藤原仲麻呂の乱の鎮圧に功績を挙げて孝謙天皇称徳天皇の信任を受けて天平神護2年（766年）には大納言に昇進した。