Liam Pitchford and Paul Drinkhall each lost in the singles and Drinkhall and Sam Walker were beaten in the doubles as China progressed 3-0.
Great Britain won bronze at the World Team Championships in March.
China, winners of 24 of the 28 Olympic table tennis golds ever awarded, play in Monday's semi-finals at 19:00 BST.
"Our results as a team are on the up so I think table tennis in Britain is in a great place. Hopefully we can keep building on that," said Drinkhall.
"We're definitely looking at Tokyo 2020. It was a tough draw here against China but I think we all believe that if that happens again in Tokyo then we'll go to beat them.
"As a team we've got great team spirit and individually we're improving a lot. I think we've got the best team spirit in table tennis."
Subscribe to the BBC Sport newsletter to get our pick of news, features and video sent to your inbox.
Great Britain have been knocked out of the men's table tennis team event in the quarter-finals by reigning Olympic champions China.