The 53-year-old temporarily stepped down as Wales coach in order to focus on the 2017 Lions tour of New Zealand.
Rob Howley has replaced Gatland as interim Wales coach for their autumn internationals and Six Nations.
"I've tried to stay away and allow Rob and the boys to make the decisions," Gatland told BBC Wales Sport.
He continued: "It was tough on Saturday because I thought it was a great Test match.
"I felt for the Welsh boys and there was a period with 15 minutes to go where they had England rattled and under a bit of pressure."
Media playback is not supported on this device
Gatland says England have cultivated a winning feeling in their ranks, after a 16th straight victory.
"It was a pretty critical line-out that was turned over and then, resulting from that, a scrum penalty that changed the whole momentum of the last 15 minutes," he said.
"England look like the sort of side at the moment that have got a winning habit.
"Losing is a habit but so is winning and they've been able to stay in games where sometimes they haven't played that well and have been under pressure and they are good enough to stay calm and controlled and win."
After beating England in a dramatic World Cup group game at Twickenham in 2015, Gatland praised how far Eddie Jones' team have come.
Media playback is not supported on this device
"I'm not too sure an England team of the past would be able to come back [against Wales] and you look back at 2015 in the World Cup, we put them under pressure and were able to score in the last few minutes to win the game," Gatland continued.
"But England at the moment are playing with poise and control and they can go through periods of the game where things aren't working for them but are good enough to come out the other side."
British and Irish Lions head coach Warren Gatland says it was 'tough' to watch Wales' 16-21 defeat by England in the Six Nations.