It hit the island of Shikoku late on Thursday, bringing torrential rain and winds of up to 185 km/h (115mph).
Kyodo news agency reported at least two deaths while broadcaster NHK said at least 31 were injured.
Nangka has since weakened and been downgraded to a tropical storm, but officials have warned of continued strong winds and storms in the area.
As of Friday morning, the Japan Meterological Agency warned of high waves, gales and thunderstorms in many parts of the main island of Honshu, as the storm headed north.
It is currently moving slowly at 15km/h with a maximum sustained wind speed of 90km/h, according to the agency.
Japanese authorities urged at least 350,000 people to flee as Typhoon Nangka made landfall in southern Japan.