His mother was Pauline Ames, the daughter of botanist Oakes Ames and artist Blanche Ames. Both of Plimpton's maternal grandparents were born with the surname Ames; his mother was the granddaughter of Medal of Honor recipient Adelbert Ames (1835-1933), an American sailor, soldier, and politician, and Oliver Ames, a US political figure and the 35th Governor of Massachusetts (1887–1890). She was also the great-granddaughter on her father's side of Oakes Ames (1804–1873), an industrialist and congressman who was implicated in the Crédit Mobilier railroad scandal of 1872; and Governor-General of New Orleans Benjamin Franklin Butler, an American lawyer and politician who represented Massachusetts in the United States House of Representatives and later served as the 33rd Governor of Massachusetts.