The Scots opened their series with a 26-13 win at the Toyota Stadium and face the Cherry Blossoms again in Tokyo on Saturday.
"We were just a little bit sloppy, at times," said Barclay, 29.
"If we look after the ball, go through phases and are clinical, we score tries."
A first-half penalty try, WP Nel's score in the second period and 16 points from the boot of Greig Laidlaw outweighed Shota Hosie's early try and eight points kicked by Yu Tamara for Japan.
"Lots to work on - looking after the ball in contact, accuracy," Barclay told BBC Scotland.
"We defended well but just kept giving the ball back to them. If we play like that next week, we'll probably struggle again.
"When we did have the ball, we looked quite dangerous."
Media playback is not supported on this device
Barclay admitted the humid conditions made life difficult for the tourists and added: "Hopefully, it's one of those things that you get used to after a game and next week won't impact on us quite as much."
Rory Sutherland, an early replacement for Alasdair Dickinson, described Saturday's match in Toyota City as "really tough".
"I just had to dig deep and keep battling on," he added.
"Our ball retention was pretty good. What let us down? Penalties, handling errors.
"We're not used to that style of rugby. We're not used to playing such a fast side and I thought that's where we were going to struggle.
"All in all, we stayed calm and composed and that's what won us the game."
Experienced flanker John Barclay admits Scotland have "lots to work on" ahead of next week's second Test against Japan.