Cavendish was born in Douglas, Isle of Man, the son of David also from the Isle and Adele from Yorkshire, England. He began riding Bicycle motocross (BMX) at an early age, racing at the National Sports Centre in Douglas. He said: "I was always riding a bike, getting dropped in little races." "My mum would laugh at me, and I said it was because all my mates had mountain bikes, so I asked for a mountain bike for my thirteenth birthday and got one. The very next day I went out and beat everyone." It was at that time that Cavendish met British cyclist David Millar at a race on the Isle of Man, who was an inspiration to him. Cavendish worked in a bank for two years after leaving school to earn enough money for an attempt at a professional career.