Charges of £7.50 were introduced in August following a £8m revamp.
Figures from York Museums Trust (YMT) show that visits dropped from 81,000 to 46,000 compared with the same six-month period in 2012 before it closed for refurbishment.
YMT said the charges were needed due to cuts in its council grant.
The introduction of charges brought complaints from local residents, with some  holding a protest outside the gallery.
'Fast reduction'
YMT's new chief executive, Reyahn King, said the fall in visitor numbers was "disappointing".
"We would prefer not to have to charge," she said. "But the reality of the funding situation is that was the only way we could absorb a very steep and fast reduction in our funding from the city council."
YMT said that its funding from City of York Council fell from £1.5m in 2013-14 to £600,000 in 2015/16.
It added that York residents could purchase an annual pass for £10, which allows unlimited visits to its museums.
The trust is a charity and runs 18 venues in the city, including York Castle Museum and the Yorkshire Museum.
The number of visitors to York Art Gallery has almost halved since admission fees were brought in last year.