His older brother was Thomas Taylour, 1st Marquess of Headfort and his younger brothers were Robert Taylour and Clotworthy Rowley, 1st Baron Langford.
Thomas Taylour, the older brother, held the title of 1st Marquess of Headfort. His younger siblings included Robert Taylour and Clotworthy Rowley, who was titled 1st Baron Langford.