From 20 February parking fees around the Aquatic Centre will increase in a bid to plug a financial black hole.
This will put the cost of using the facilities "out of the reach of most ordinary people", campaigners say.
The London Legacy Development Corporation, which runs the London 2012 Games site, said the charges were "consistent with other local pools".
But Dave Wardell, whose two daughters use the swimming programme at the centre, said the price hike would cost him £2,000 a year.
"Not really an Olympic legacy", said the policeman, who was recently stabbed by an armed robber in an attack that left his police dog needing emergency surgery,.
A London Legacy Development Corporation (LLDC) report said the fee increase was "intended to generate a surplus" to pay for the Queen Elizabeth Olympic Park's upkeep.
The LLDC's expected budget deficit for 2017-18 is £23m and the corporation will need to increase income fivefold to break even, according to its most recent draft budget submission.
At present, parking is free for the first hour, which includes a pick-up and drop-off point near the entrance.
Customers are then charged £1.50 for up to two hours, with a maximum charge of £15 for a 24-hour stay.
Under the new system drivers will be charged £2 an hour from 7am each day, capped at £20, and pick-ups and drop-offs will not be exempt.
A spokesperson for the London Legacy Development Corporation said: "The majority of people who use the London Aquatics Centre do not come by car and therefore will be unaffected by this change."
Olympic Park bosses are being accused of betraying legacy commitments by raising the cost of parking.