Perry retired to his farm in Martin County, North Carolina where he grew tobacco and peanuts, but had to file for bankruptcy in 1986. He briefly worked for Fiesta Foods as a sales manager, and later in the year Limestone College in Gaffney, South Carolina chose Perry to be the college's first baseball coach. Perry was there until 1991 when he retired. In 1998, Perry was inducted into the Limestone College Athletics Hall of Fame (Class of 1998). He later moved to Spruce Pine, North Carolina.