Figures published by AHDB Dairy show the number of producers in the county, 496, is now less than half the figure recorded in 2002.
Farmers have blamed the continuing fall in the price of milk, with production costs outweighing the amount farmers receive when selling it.
January figures show there are now 9,633 dairy farmers in England and Wales - down 3.3% from 2015.
Recent government figures put the UK average farm-gate price for milk - which is the amount of cash farmers receive - at just over 24p a litre.
The November statistics show the average price is 16% lower than a year earlier, with farmers estimating current costs of 30p to produce a litre of milk.
Paul Tomkins, a dairy farmer in Melbourne, near York, said a stage could be reached where there are "no family farms left in the UK".
He said: "It's my belief that we will continue to see farms getting larger and milking a very large number of cows, as they're able to do it for the least amount of money."
David Shaw, chair of the National Farmers' Union's North East dairy board, said: "At one point we were losing three a day and I think this spring we will revert back to that as people have to make a decision about the next year."
Rishi Sunak, Conservative MP for Richmond, said the figures were "disappointing" but some progress was being made.
He said: "Supermarkets are responding to pressure and improving the labelling on their packets"
The number of dairy farms in North Yorkshire has dropped lower than 500.