Tatsuyuki Hishida's wife and fellow gang members found him collapsed and bleeding at a house in Mie prefecture, the Mainichi Shimbun said.
Hishida's hands and feet were reportedly tied, and his head appeared to have been hit with a blunt object.
The Yamaguchi-gumi, Japan's largest organised crime group, split in two in September.
Police said the incident was likely related to the split, which saw the emergence of a rival group calling itself the Kobe Yamaguchi-gumi.
Hishida was the leader of a second-tier Yamaguchi-gumi affiliate group called the Aio-kai, based in the city of Yokkaichi in Mie prefecture.
Tensions have run high between the rival groups since the split, with the Yamaguchi-gumi forced to cancel its annual Halloween party for children in Kobe this year.
A gang boss in Japan's Yamaguchi-gumi yakuza syndicate was bludgeoned to death on Sunday, according to reports.