He left the Kings for a one-year contract to succeed Red Sullivan as coach of the Pittsburgh Penguins on July 2, 1969. After the Penguins ended the 1969–70 season with its first-ever playoff appearance and advanced to the semifinals, Kelly signed a five-year, $250,000 contract on May 21, 1970 to continue as coach, and also replaced Jack Riley as general manager. With the team struggling in sixth place in the NHL West Division during a stretch of winning only two of 22 contests and having failed to qualify for the postseason in 1970–71, Kelly was pressured to relinquish his general manager title back to Riley on January 29, 1972 in order to concentrate on his coaching duties. Amid a slump in which the Penguins won only two games with three draws and seven losses and slid into fifth place in the eight-team NHL West Division, Kelly was fired and replaced by Ken Schinkel on January 13, 1973.