Castres is in southern France, in the southwestern part of the Tarn department, to the north of the Montagne Noire, a mountain range in central southern France. Castres is crossed from north to south by the Agout river, a tributary of the Tarn.