The duo dissolved their double scull partnership in May to focus on earning spots in the larger boat.
Grainger, 40, and Wales' Thornley, 28, could miss the Games unless selectors decide to return them to the double.
The squad is to be announced on Thursday.
"Whilst we had not lost confidence in Vicky Thornley and Katherine Grainger and felt they were still a potential medal boat, they asked to be considered instead for selection to the women's eight, which is already a proven medal winning combination," said British Rowing in a statement.
"We opened up a process and gave them the opportunity. They have not been successful and we are now looking at next steps in a selection process which is on-going."
Grainger - who is Britain's most decorated female Olympian alongside swimmer Rebecca Adlington - won the double scull with Anna Watkins at London 2012 after collecting silver medals at the three previous Games.
However her hopes of defending the title appeared to be at an end after she and Thornley failed to make the podium at the European Championships, prompting British Rowing to stop work on their partnership.
British rowing legend Sir Steve Redgrave said last month that despite their struggles he still expected Grainger and 28-year-old Welshwoman Thornley to appear at Rio.
Media playback is not supported on this device
London 2012 gold medallist Katherine Grainger and former double scull partner Vicky Thornley have failed to win places in Great Britain's women's eight for the Rio Olympics.