It was rebuilt, but after it was destroyed again by a storm on July 9, 980 (old lunar calendar), it was never rebuilt and its remains were deserted as Ukyo (the west district of Heian-kyo) declined in population, and the security of the south area of Heian-kyo steadily deteriorated along with the desolation of the country, turning the neighborhood of Rajomon, the south of the capital city, into an area no one would dare to come close to.
再建されたが、天元3年7月9日暴風雨で再度倒壊してからは再建されず、右京の衰えと共にこの門も荒廃していき、国内の荒廃につれて平安京南部の治安は悪化の一途をたどり、洛南の羅城門周辺は夜ともなれば誰も近付かぬ荒れた一画となっていたという。