Al Hartley was born in Kearny, New Jersey, the son of Hazel Hartley and Congressman Frederick Allan Hartley, Jr. (Republican from New Jersey), co-author of the Taft-Hartley Act of 1947. He had a brother, Jack, and a sister, Lorraine. Their father, Hartley said, "encouraged me. He knew I wanted to draw from the time I could hold a crayon... My father wanted me to pursue my own dreams and never attempted to steer me in any other direction." Hartley drew for the local newspaper while still in high school, and studied at the Art Students League of New York. He began selling humorous spot illustrations to magazines, and drew a Western comic-book story about Tecumseh for the publisher Street & Smith before the U.S. joined World War II, after which he enlisted in the U.S. Army Air Corps and flew 20 missions as a B-17 Flying Fortress bomber pilot in Europe.