The benchmark Nikkei 225 spent the day in negative territory and closed down 3.55% at 16,164.16.
The Bank of Japan's closely watched Tankan report showed business sentiment sitting at plus six for the three months to March.
The previous quarter's reading came in at plus 12.
A positive reading means optimists outweighed pessimists surveyed about business conditions. Expectations were for a reading of plus eight.
Analysts said Friday's numbers marked the largest quarterly fall since the end of 2012 and that confidence could weaken further in the next three months.
"The pronounced drop in the headline index of today's Tankan survey underlines that the recent strengthening of the yen has damaged business confidence and suggests that the Bank of Japan will announce more easing later this month," said Japan economist Marcel Thieliant.
"Conditions in the sector have not been as weak since the second quarter of 2013, when the BoJ launched quantitative and qualitative easing," he added.
Beijing released data on Friday showing that manufacturing activity at China's big factories had expanded in March for the first time in nine months.
The official Purchasing Managers' Index (PMI), which measures how busy factories are, came in at 50.2 last month compared to expectations for a reading of about 49.
A PMI reading of below 50 indicates activity has contracted and anything above 50 means activity has expanded.
Hong Kong's Hang Seng index shed 1.34% to close at 20,498.92 while the Shanghai Composite ended 0.2% higher at 3,009.53.
In Australia, the benchmark S&P/ASX 200 index finished the session down 1.64% to 4,999.4, weighed down by the country's big banks and some mining firms.
Westpac and National Australia Bank both lost 2%.
Meanwhile, South Korea's benchmark Kospi index finished down 1.12% to 1,973.57.
Japan's markets suffered heavy losses on Friday after a key gauge of business confidence at big manufacturing firms came in weaker than expected.