Philbin attended Our Lady of Solace grammar school in the Bronx, and graduated from Cardinal Hayes High School in the Bronx in 1949 before attending the University of Notre Dame, where he graduated in 1953 with a sociology degree. He later served in the United States Navy as a supply officer, then went through a few behind-the-scenes jobs in television and radio before moving into the broadcasting arena.