Trevor Joyce, 36, is charged with the murder of 61-year-old Justin Skrebowski in the Abingdon branch shortly after 11:20 GMT on Monday.
The Abingdon man appeared over a video link from HMP Bullingdon and spoke only to confirm his name.
He is due to return to the court for a plea and case management hearing on 7 March next year.
Updates on this story and more from Oxfordshire
Thames Valley Police said it had voluntarily referred itself to the IPCC following the stabbing.
A man accused of a fatal stabbing in a Poundland store has appeared in Oxford Crown Court.