Surrey Police said bottles were thrown and two people were injured, one of them an officer, in the disturbance near the main music stage at 18:15 BST.
The police officer had a tooth knocked out and was taken to hospital for treatment for facial injuries. He was later discharged.
An 18-year-old man was treated for head and facial injuries at the scene.
Police wearing riot gear arrested men aged between 18 and 36 from Sutton, Croydon, Caterham and Reigate, along with four teenagers from London and Epsom.
Video footage on Twitter shows race-goers running for cover as the police moved in.
The arrests were on suspicion of a series of offences including assault, affray and possession of drugs. The suspects remain in custody.
Police have appealed for any witnesses who have information or who may have filmed the fracas to come forward.
Earlier in the day, police made eight arrests for assaults, public order and drug offences.
Around 100,000 race-goers attended the two-day Derby Festival, which began on Friday with Ladies' Day and the Investec Oaks.
Eleven people were arrested during a fracas at Epsom Downs racecourse after Saturday's Derby.