After high school, Jeffries attended Claremont Men's College (now Claremont McKenna College) where he graduated in 1966 with a BA in Economics. He received an MBA from Columbia Business School in 1968 and also studied at the London School of Economics. That same year Jeffries joined the management training program at Abraham & Straus, a now-defunct New York City department store. During this time Jeffries worked along with Allen Questrom (of J.C. Penney) and Millard S. Drexler (the previous CEO of Gap Inc., who now works at J. Crew).