He was the son of Lionel Smythe, 5th Viscount Strangford (1753–1801) and Maria Eliza Philipse. In 1769, his sixteen-year-old future father left Ireland, joined the army and served during the American War of Independence. While quartered in New York in the winter of 1776 to 1777, he met and courted Maria. She was the daughter of Frederick Philipse III (1720–1785), the third and last Lord of Philipsburg Manor and a descendant of the Dutch founder of the city. At first, her father rejected Lionel, however, as Philipse was a Loyalist during the war, the New York Legislature confiscated his estate, one of the largest in the province, and Philipse changed his mind. They married in September 1779 at Trinity Church in Manhattan and they returned to the United Kingdom. Upon the withdrawal of the British troops from New York in 1783, Philipse also went to England, where he later died.