His ruling put the spill at 3.2 million barrels - the US government had estimated it at 4.09 million barrels.
It shields the oil giant from what could have been a $17.6bn fine. A final figure is expected later this month.
The case relating to the aftermath of the Deepwater Horizon drilling rig explosion was heard in New Orleans.
In his ruling the judge said BP's response to the disaster had not been grossly negligent.
However, he stuck to his earlier decision that it had been grossly negligent leading up to the explosion, in which 11 men aboard the drilling rig were killed.
BP is appealing against that decision.
These latest penalties under the US Clean Water Act would be in addition to more than $42bn BP has set aside or spent already on clean-up, compensation and fines.
The company has sold at least $39bn in assets since the spill.
In 2012, BP accepted criminal responsibility for the disaster and agreed to pay $4.5bn to the US government, thus settling its criminal liability in the spill.
These latest legal efforts have focused on the amount of civil penalties the firm must pay, both to businesses and individuals affected by the spill and to cover environmental clean-up costs.
In a short statement released on Friday, BP said that it "is continuing to review the court's decision".
US-listed shares of BP rose about 1% to $36.20 in after-hours trading.
BP faces a fine of up to $13.7bn (£9bn) after a US judge ruled that the 2010 Gulf of Mexico oil spill was smaller than initially feared.