The name is the plural form of borg which means "castle" or "hill/plateau suitable for a castle".
The municipality (originally the parish) is named after the old Borge farm () since the first Borge Church was built there.