Sutton resigned in April, ending a 14-year involvement with British Cycling.
He had been suspended by the sporting body amid claims of sexism and discrimination towards elite cyclists, although he denies the allegations.
"It was a hard few days when he first left and it was tough for the whole programme," said James.
"But we had to move on and I had to knuckle down and get on with my training, knowing it would be a journey without him but I still had to get in the best possible form before Rio."
Welsh sprinter James, 24, is particularly grateful to Sutton for the role he played in keeping her in the sport during a torrid two-year spell that saw her overcome serious knee and shoulder injuries and a cancer scare.
Having won world keirin and sprint titles in 2013, James almost quit in 2015.
"Shane was the one who always believed in me getting back," said James, who also missed London 2012 because of illness and injury.
"He said to me: 'You don't have to do this. You have already achieved a lot and you can be happy with what you've done.'
"That gave me that bit extra to want to come back. He was the one who could see it, when I couldn't. He kept me on the programme, kept me working hard."
James also admitted that Sutton's "straight-talking" could upset others but said she did not think he was a bully.
"He's always been brutally honest with me but the truth is better than lies," she added.
Media playback is not supported on this device
British Olympic cyclist Becky James believes the team will miss former technical director Shane Sutton at the Games in Rio this summer.