After the Edo period had started, the bakufu (Japanese feudal government headed by a shogun) placed the ban of Hideyoshi brief and pulled down the sanctuary, the main building of a Shinto shrine.
江戸時代に入ると、幕府の命により秀吉信仰が禁じられ、社殿も取り壊された。