Dufferin was born in 1826 in Florence, then the capital of the Grand Duchy of Tuscany in the Italian peninsula, with great advantages. He was educated at Eton and at Christ Church, Oxford, where he became president of the Oxford Union Society for debate, although he left Oxford after only two years without obtaining a degree. While still an Oxford undergraduate, he visited Skibbereen in County Cork to see the impact of the Irish Famine first-hand. He was appalled by what he saw, prompting him to raise money on behalf of the starving poor. In 1841, while still at school, he succeeded his father as Baron Dufferin and Claneboye in the Peerage of Ireland and in 1849 was appointed a Lord-in-Waiting to Queen Victoria. In 1850 he was additionally created Baron Claneboye, of Clandeboye in the County of Down, in the Peerage of the United Kingdom.