Bridge was born in Oldbury, then in Worcestershire, in central England, the eldest son of John Bridge and his wife, Rebecca "née" Cox. In 1850, his father was appointed a vicar-choral of Rochester Cathedral. Young Bridge was admitted to the cathedral choir as a "practising boy" (that is, a probationer). The choirboys were educated by another of the vicars-choral. The régime was severe in discipline and rudimentary in curriculum, but among the alumni of the choir school of this period were future organists of four English cathedrals and of Westminster Abbey. They included Bridge's younger brother Joseph, who eventually became organist of Chester Cathedral.