He maintained a low profile during the war, and his uncle Lord Fairfax was treated with respect and consideration despite being the only resident peer in the American colonies. Following the death of Lord Fairfax in 1781, Martin's brother Reverend Denny Martin Fairfax inherited the Northern Neck Proprietary, and Martin was given the Greenway Court estate. He took his uncle's housekeeper Mrs. Crawford as his mistress and died unmarried in 1798. Martin bequeathed his Greenway Court estate and an adjoining to his housekeeper Betsy Powers. Martin's brother Denny Fairfax was unable to properly maintain the proprietary and conveyed the remaining lands in 1797, thus terminating the Fairfax and Martin families' interests in the proprietary before it was formally dissolved in 1806. The city of Martinsburg, West Virginia, was named for Martin by his friend Adam Stephen.