In "Konjaku monogatarishu" (The Konjaku Tales, which literally means the tales from the present and the past, referring to a 31-volume collection of stories written during the late Heian period) and "Genpei seisuiki" (Rise and Fall of the Genji and the Heike), he apparently had an adulterous relationship with Empress Komyo which led to Hirotsugu finding fault and was the underlying cause of the rebellion.
『今昔物語集』『源平盛衰記』には、光明皇后と密通し、それを広嗣に見咎められたことが乱の遠因になったとしている。