The Butlers of Ormonde were an Old English dynasty. They had been very important people the southeast of Ireland since the Middle Ages. His father was Thomas Butler, Viscount Thurles. Thurles is a place in County Tipperary. James was the oldest son. 