 Savage was the second of the four sons of Augustus Chase ("A.C.") Savage (1832–1911) and Emily Manchester (1834–1914), the founders of the Asticou Inn in Northeast Harbor, Maine.Fred was the architect of the current, 1901-built inn, the original having burned down in 1899.His first wife was Flora Lee Salisbury (1872–1960).