The 24-year-old, capped 60 times by Wales, has played 26 matches in the last 35 weeks for club and country.
Asked if he was tempted by Rio, North said: "Yes, obviously, but in the grand scheme of things I'm a XVs player.
"It would be awesome to play in Rio but it's a big old ask to go at the end of this sort of season with it being so long to get myself ready for sevens."
Speaking to BBC Radio Northampton he added: "For me, more for the longevity of my career, it would be better to take a few weeks away, rest up and get ready for next season."
Rugby sevens will make its debut at the Olympics this summer, with Great Britain taking a team to Brazil.
British and Irish Lion North suffered four head blows in five months last season, the final one keeping him out between 27 March and 29 August.
Since playing in the World Cup, he has struggled for domestic form this season, scoring only two tries in nine Premiership matches.
In February, North admitted Northampton's style of play "isn't really helping my game", but could be part of Warren Gatland's Wales squad for a three-Test tour against world champions New Zealand in June.
"After the summer tour, fingers crossed for selection, I've got a few weeks off when I come back and I think it's much needed both physically and mentally to step away from the club and rugby," said North.
"This season has, give or take, been about 14 months long. It takes its toll on you physically as well as mentally so it will be nice to step away from it for a bit, see some family and friends, have some good food and just chill out."
Have you added the new Top Story alerts in the BBC Sport app? Simply head to the menu in the app - and don't forget you can also add alerts for your rugby union team, cricket scores, football and more.
Northampton Saints and Wales wing George North says he was tempted to go for the 2016 Rio Olympics sevens team.