In 1883, Bush published a collection of sermons called "More Words About the Bible", a response to his colleague Heber Newton's book "Uses of the Bible". In 1885, his book "Evidence of Faith" was reviewed by "The Literary World" as "clear, simple, and unpretending", and summarized as an argument against supernatural explanations for God. According to the same journal, both works fit into the broad church movement. The "Boston Advertiser" called the latter work "the best statement of untrammeled spiritual thought" among recent books.