During his minor league career, Ellis once chased a heckler in the stands with a baseball bat. He also used pills when he pitched, specifically the amphetamines Benzedrine and Dexamyl. Stressed by the pressure of his "can't-miss" status as a prospect, Ellis became addicted. He later said that he never pitched a game without using amphetamines. He eventually needed per game, or between five and twelve capsules, depending on their strength. Ellis acknowledged that he began to use cocaine in the late 1960s.