Davenport first appeared on stage at the Savoy Theatre and then with the Shakespeare Memorial Company, before joining the English Stage Company, one of its earliest members, at the Royal Court Theatre in 1956. He began appearing in British film and television productions in supporting roles, including a walk-on in Tony Richardson's film, "Look Back in Anger" (1959). Subsequent roles included a theatre manager opposite Laurence Olivier in the film version of "The Entertainer" and a policeman in Michael Powell's "Peeping Tom" (both 1960).