Liam Pitchford had four match points in the fourth game against Jun Mizutani, to take the tie to a deciding rubber, but lost 8-11 11-13 11-7 12-10 11-6.
Earlier Mizutani beat Paul Drinkhall 11-8 11-9 11-4 and Pitchford lost 11-5 12-14 11-7 11-13 11-9 to Maharu Yoshimura in Kuala Lumpur, Malaysia.
Sam Walker upset Yuya Oshima 11-6 11-6 8-11 11-9 to keep England in the tie.
However, world number seven Mizutani ensured Japan reached the final where they will face China.
It was England's first semi-final in the competition since 1983 and gave them vital ranking points in the race to qualify for this summer's Rio Olympics.
England lost 3-1 to Japan in the World Team Table Tennis Championships semi-finals, but still won a bronze medal.