Brunner was born in April 1821 in Oxford. When he was fifteen, he began to learn architecture and surveying. In 1841, he joined the New Zealand Company in its venture to establish a settlement in the north of the South Island of New Zealand, to be called Nelson. As well as working as an apprentice surveyor and laying sections and roads for the new settlement, he explored the interior, seeking pastoral land for a growing colony. In 1846 he undertook extensive journeys with Charles Heaphy and a Māori named Kehu towards and along the West Coast.