Montgomery himself expected that his name would live, if at all, in his hymns. Some of these, such as "Hail to the Lord's Anointed", "Prayer is the Soul's Sincere Desire", "Stand up and Bless the Lord" and the carol "Angels from the Realms of Glory", are still sung. "The Lord Is My Shepherd" is a popular hymn with many denominations, based on Psalm 23. "A Poor Wayfaring Man of Grief" has been adopted as a favourite in the Latter Day Saint movement. The earliest of his hymns dates from his days in Wath on Dearne and he added to their number over the years. The main boost came when the Rev. James Cotterill arrived at the parish church St Paul's, a chapel of ease to St Peter's, Sheffield's only parish church, in 1817.