According to Ellis, he attended trials for a football career with Tranmere Rovers as a child, but chose to pursue his business career rather than football. During World War II, he was based with the Fleet Air Arm of the Royal Navy in Ceylon (now Sri Lanka), and the experience of his first trip abroad inspired him to provide low-cost foreign holidays to people of a similar economic background. Before he was 40, he had become a millionaire by pioneering package holidays to Spain, with his company Sunflight.