De la Mare was born in Kent at 83 Maryon Road, Charlton (now part of the Royal Borough of Greenwich), partly descended from a family of French Huguenots, and was educated at St Paul's Cathedral School. He was born to James Edward de la Mare, a principal at the Bank of England, and Lucy Sophia Browning (James's second wife), daughter of Scottish naval surgeon and author Dr Colin Arrott Browning. The suggestion that Lucy was related to poet Robert Browning has been found to be incorrect. He had two brothers, Francis Arthur Edward and James Herbert, and four sisters Florence Mary, Constance Eliza, Ethel (who died in infancy), and Ada Mary. De la Mare preferred to be known as 'Jack' by his family and friends as he disliked the name Walter.