The pressure of the anti-Kazan group remained so strong even after his death that, according to some records, they didn't allow the establishment of Kazan's tomb even after his son, Shoka WATANABE, assumed the post of karo (chief retainer) of the Tahara Domain in order to restore his family's impaired reputation (it was in April 7, 1868, immediately before the collapse of the Edo shogunate, that the shogunate allowed the rehabilitation of Kazan's honor and the construction of his tomb).
崋山に対する反崋山派の圧力はその死後も強く、また幕府の手前もあり、息子の渡辺小崋が家老に就任して家名再興を果たした後も墓を建立することが許されなかったという（江戸幕府が崋山の名誉回復と墓の建立を許可したのは、江戸幕府滅亡直前の明治元年3月15日(旧暦)（1868年4月7日）のことであった）。