 His family later moved to Buffalo, New York.There, he played football, basketball, and baseball at St. Joseph's Collegiate Institute from which he graduated in 1940.After high school, Scherger signed with the Brooklyn Dodgers.