Land Rover BAR came third in the first race, but endured a tough second round after a technical issue saw them slip from first to last place.
They came second in the final race, while the regatta was won by Artemis Racing who are led by Britain's two-time Olympic champion Iain Percy.
Land Rover BAR are third overall with Emirates Team New Zealand leading.
"It was one of those days. We're still third, we still feel we've got the best team and I believe we will win the World Series," said Land Rover BAR crew member David Carr after his team finished fourth out of the six boats that competed.
The Bermuda leg followed events in Portsmouth and Gothenburg this summer.
Light wind saw racing abandoned on Saturday in Bermuda, which will host the America's Cup itself in 2017, and three shorter rounds were held on Sunday.
The Bermuda leg is part of a lengthy qualification process that will count towards the 2017 America's Cup Challenger Series, the winner of which will take on Oracle in the 2017 America's Cup.
It will be the 35th staging of the oldest competition in international sport which was first raced in 1851 around the Isle of Wight.
Sir Ben Ainslie's British boat finished fourth in the third leg of the America's Cup World Series in Bermuda.