Rogers was born in Latrobe, Pennsylvania to James Rogers and Nancy McFeely. His father was a businessman. His mother came from a rich Pittsburgh family and also was a hospital volunteer during and after World War II. His grandfather from his mother's side, Fred McFeely, was the president of McFeely Brick, one of Latrobe's largest businesses. Rogers grew up in a large brick mansion at 737 Weldon Street in Latrobe.