Media playback is not supported on this device
The Welsh boxer has only fought twice since winning the IBF featherweight title in May 2015, prompting criticism from his British promoter Eddie Hearn.
Selby, 29, defends his IBF featherweight title against Jonathan Victor Barros in Las Vegas on 28 January.
Asked if he would change anything, Selby said: "I don't think so, no."
Speaking to BBC Wales Sport, he added: "Al Haymon has got all the big fighters. I want the big fights, so that was the best move for me.
"With tough competition, you've got to rise to it, and to be the best you've got to beat the best.
"You've got Gary Russell Jr, the WBC champion, Carl Frampton, Abner Mares - and they are all advised by Al Haymon. I'm advised by Al Haymon so they are easy fights to make. "
Selby's fight against Barros is on the undercard of WBA champion Frampton's rematch with Leo Santa Cruz at Las Vegas' MGM Grand Arena.
Selby is eager to fight the winner of that bout, and the man from Barry in south Wales is confident his recent inactivity will have no detrimental effect.
"For some fighters it can affect them," he said. "They box, they go out, they celebrate, they end up drinking and eating loads of bad food.
"But with me, I'm straight back in the gym, working and trying to improve myself. I've been sparring, so inactivity shouldn't really affect me.
"My focus is on Jonathan Barros - he's a dangerous fighter. But say I come through that fight OK, I want to fight the winner of Santa Cruz and Carl Frampton."
Northern Ireland's Frampton has suggested Belfast's Windsor Park as a venue for a potential fight with Selby, who is open to the idea.
However, the Welshman would ultimately like to face Frampton at Cardiff's 74,500-capacity Principality Stadium.
"It is a super-fight, it's a natural fight," he said. "That is part of the reason why I am on the card so the fans can compare us and it is a natural fight for both of us.
"Ideally I'd like it in the Millennium [Principality] Stadium. I think we'll do great numbers there.
"They haven't had boxing there since Joe Calzaghe. That was a few years back but at the same time I'm happy to go to Belfast and defend my title over there."
Find out how to get into boxing with our special guide.
Lee Selby insists he has no regrets about his decision to sign with American promoter Al Haymon.