The forward, the brother of former U's player and Manchester United manager Ron Atkinson, made 394 appearances in 12 years at the club.
In a statement Oxford United paid tribute to what it described as one of its "all-time greatest players".
Atkinson holds the club's all-time scoring record with 107 goals, and only five players have more appearances.
Friend and former teammate Ken Skeen said: "He was a great man and a great footballer who I was proud to call my best mate, even after all these years.
"I am glad I came to Oxford and got the chance to know him. He was a true gentleman who was at the heart of Oxford United for many years and he will be missed by everyone."
Atkinson started out at Aston Villa but joined his brother Ron at Oxford United during the 1959/60 season before he turned 17.
He scored the club's first ever goal in the Football League, and his 77 league goals remain a club record.
A minute's silence will be held at a forthcoming game.
Oxford United's all-time top goal scorer Graham Atkinson has died of cancer at the age of 73.