Berners was born in Apley Hall, Bridgnorth, England. The Berners family were related to King Edward III of England. His father was a naval officer who was not often at home. He was looked after by his mother and grandmother who was very religious and very strict. Berners was very interested in lots of things, especially music, but his mother did nothing to help him.