The midfielder left the Liberty Stadium to join Burnley last month in a deal that could eventually be worth £10m.
Swansea had three head coaches last season, with Paul Clement keeping them in the Premier League after Francesco Guidolin and Bob Bradley were sacked.
"It was tough with the change of managers at Swansea, it did affect us last year," said the 28-year-old.
"They probably made the right decisions with the managers in the end but it did affect us.
"Everyone has seen things in the way he [Clement] does things and how he works. Trust in him has paid off and he's shown what a good manager he can be."
Cork played under five managers during his two-and-a-half years at the Liberty Stadium, including two-time caretaker Alan Curtis.
He is relishing the opportunity to play in the Premier League under Burnley boss Sean Dyche, who has been in charge at Turf Moor since October 2012.
"It's good to come to a stable club with the manager here," Cork added. "The board and players trust him and enjoy working with him."
Managerial upheaval affected Swansea as they fought relegation last season, says their former midfielder Jack Cork.