Sullivan was born in New York City, New York. His mother was Elizabeth F. Sullivan (née Smith) and his father was Peter Arthur Sullivan. He was of Irish descent. Sullivan used to be a boxer. He began his media work as a sportswriter for "The New York Evening Graphic".