The Apostoli family immigrated to NYC in the 1880s from the city San Benedetto del Tronto in the Ascoli Piceno Province in the Marche region of Italy. Apostoli's mother died in child birth in the early 1920s and his father sent his other younger children back to live with relatives on the East coast and placed Freddy in the care of a Catholic orphanage in San Francisco. Apostoli attended grade school and high school in North Beach and was a lifelong friend of classmate Joe Dimaggio.