After his brief call-up in 1962, Perry joined the Giants in 1963 to work mostly as a relief pitcher that year, posting a mediocre 4.03 ERA in 31 appearances. Nevertheless, in 1964 he was given the opportunity to join the starting rotation, finishing with a 2.75 ERA and a 12–11 record, both second best for the Giants that year behind Juan Marichal. In 1965 his record was 8–12, and with two full seasons as a starter, his 24–30 record attracted little national attention.