Kelly was born 11 July 1929 in Dublin, Ireland, and educated at Dublin's Synge Street CBS Christian Brothers school. He began acting at the age of eight at the city's Gaiety Theatre, and trained at The Abbey School of Acting. As a backup career, he additionally trained as a draughtsman and calligrapher, and also learned watercolour art. He appeared onstage in the original production of Brendan Behan's "The Quare Fellow", and gained his first major career attention in Samuel Beckett's "Krapp's Last Tape" at the Dublin's Abbey Theatre in 1959. By then he had made his screen debut in a small part in director John Pomeroy's 1958 film noir "Dublin Nightmare".