After leaving school Brown started work aged 17 in 1921 as if just another apprentice in his family's business, David Brown & Sons (Huddersfield), cycling 6 miles to work by 7.30 a.m. This company which had been founded by his grandfather David, specialised in transmission components. While his father had no interest in cars and did not drive, his mother was a keen driver and as a small child he had ridden beside Frederick Tasker Burgess on test runs of David Brown & Sons new Valveless car but he did not learn to drive until aged 11.