Besides aiding Dr. Samuel Prideaux Tregelles in his investigations as a Biblical textual critic, Kelly also published, in 1860, a critical edition of the Book of Revelation, which was praised by Professor Heinrich Ewald of Göttingen. Such studies were carried on concurrently with the editing of a periodical entitled "The Prospect". He took up the editorship of "The Bible Treasury" in 1857, and continued till his death. As editor of the latter he was brought into correspondence with Dean Henry Alford, Dr Scott the lexicographer, Principal Edwards and William Sanday of Oxford, among others.